From owner-freebsd-stable@FreeBSD.ORG Thu Nov 29 19:27:34 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0BDD16A41A; Thu, 29 Nov 2007 19:27:34 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DE6A413C447; Thu, 29 Nov 2007 19:27:33 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <474F12AE.7050808@FreeBSD.org> Date: Thu, 29 Nov 2007 20:27:42 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Alexey Popov References: <4741905E.8050300@chistydom.ru> <47419AB3.5030008@chistydom.ru> <4741A7DA.2050706@chistydom.ru> <4741DA15.9000308@FreeBSD.org> <47429DB8.7040504@chistydom.ru> <4742ADFE.40902@FreeBSD.org> <4742C46A.1060701@chistydom.ru> <47432F77.3030606@FreeBSD.org> <474339E9.4080301@FreeBSD.org> <4743629B.9090408@FreeBSD.org> <47456B71.5040205@chistydom.ru> <4745E5B3.6060200@FreeBSD.org> <47468165.5010906@chistydom.ru> <4746B21F.7050906@FreeBSD.org> <4746BD92.6000204@chistydom.ru> In-Reply-To: <4746BD92.6000204@chistydom.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Attilio Rao , freebsd-stable@freebsd.org Subject: Re: 2 x quad-core system is slower that 2 x dual core on FreeBSD X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Nov 2007 19:27:34 -0000 Alexey Popov wrote: > Hi > > Kris Kennaway wrote: >>>>> Now FreeBSD 7-STABLE ULE 8-core server without optimized PHP >>>>> realpath_cache_size (producing 2000+ lstats per request) can handle >>>>> up to ~24 rps as opposed to max. 17 rps without your patch. %sys >>>>> never grows over %user with your patch. On the server with >>>>> optimized realpath_cache_size there's no visible influence of your >>>>> patch. >>>> You said "20" before for this configuration, so I'm a bit suspicious >>>> about how seriously to treat your measurements :) >>> Sorry, my mistake. s/ULE/4BSD. >> OK, please compare ULE to ULE with and without my patch (and >> remembering to enable the sysctl), and obtain lock profiling traces in >> both cases under identical workloads & durations. That is what I need >> to proceed with this issue. > I didn't measured the exact values of requests per second on ULE with > patch and without patch, but at first glance the benefits of the patch > are similiar to 4BSD. If you need this values, I'll obtain them. > > Here you can find lock profiling results for 7-BETA3 GENERIC kernel with > SCHED_ULE running optimized PHP and unoptimized, with your patch and > without it: http://83.167.98.162/gprof/lockmgr/ > > This data was collected by th following script: > (sysctl debug.lock.prof.reset=1 > sysctl debug.lock.prof.enable=1 > sleep 60 > sysctl debug.lock.prof.enable=0 > sysctl debug.lock.prof.stats > top -d 2 -b | tail -25) > > AFAIU there's still high contention on "lockbuilder mtxpool" with patch > applied. But hopefully "lockmgr:ufs" contention which i believe produced > 80%sysCPU load is gone with your patch. Looks to me like lockmgr-related contention was reduced by 1 to 2 orders of magnitude, which is the expected result. This surely must have a measurable impact on your workload. Further lockmgr improvement will have to wait until the lockmgr replacement work proceeds. One more patch which may or may not help is: http://www.freebsd.org/~jhb/patches/namei_rwlock.patch (may also require porting since it was against an older version of 7.0-CURRENT). When I have tested this in the past it was a performance loss for reasons that I think I understand (basically, it is locally a performance improvement for the name cache but also requires a fixed lockmgr to avoid an overall performance loss), but I don't remember if I tested it in conjunction with the lockmgr patch. >> There are patches you need to enable it on woodcrest. They are in my >> p4 branch (kris-contention) but I don't have time right now to extract >> them. > I think it would be very useful because I can't see any other ways to > profile FreeBSD on the modern many-cores machines. You can extract the changeset from my branch via http://perforce.freebsd.org. Unfortunately I don't have time to do it myself. Kris