Date: Thu, 29 Nov 2007 20:27:42 +0100 From: Kris Kennaway <kris@FreeBSD.org> To: Alexey Popov <lol@chistydom.ru> Cc: Attilio Rao <attilio@freebsd.org>, freebsd-stable@freebsd.org Subject: Re: 2 x quad-core system is slower that 2 x dual core on FreeBSD Message-ID: <474F12AE.7050808@FreeBSD.org> In-Reply-To: <4746BD92.6000204@chistydom.ru> References: <4741905E.8050300@chistydom.ru> <fhs3s5$knj$1@ger.gmane.org> <47419AB3.5030008@chistydom.ru> <fhs7hp$2es$2@ger.gmane.org> <4741A7DA.2050706@chistydom.ru> <4741DA15.9000308@FreeBSD.org> <47429DB8.7040504@chistydom.ru> <4742ADFE.40902@FreeBSD.org> <4742C46A.1060701@chistydom.ru> <47432F77.3030606@FreeBSD.org> <474339E9.4080301@FreeBSD.org> <4743629B.9090408@FreeBSD.org> <47456B71.5040205@chistydom.ru> <4745E5B3.6060200@FreeBSD.org> <47468165.5010906@chistydom.ru> <4746B21F.7050906@FreeBSD.org> <4746BD92.6000204@chistydom.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Alexey Popov wrote: > Hi > > Kris Kennaway wrote: >>>>> Now FreeBSD 7-STABLE ULE 8-core server without optimized PHP >>>>> realpath_cache_size (producing 2000+ lstats per request) can handle >>>>> up to ~24 rps as opposed to max. 17 rps without your patch. %sys >>>>> never grows over %user with your patch. On the server with >>>>> optimized realpath_cache_size there's no visible influence of your >>>>> patch. >>>> You said "20" before for this configuration, so I'm a bit suspicious >>>> about how seriously to treat your measurements :) >>> Sorry, my mistake. s/ULE/4BSD. >> OK, please compare ULE to ULE with and without my patch (and >> remembering to enable the sysctl), and obtain lock profiling traces in >> both cases under identical workloads & durations. That is what I need >> to proceed with this issue. > I didn't measured the exact values of requests per second on ULE with > patch and without patch, but at first glance the benefits of the patch > are similiar to 4BSD. If you need this values, I'll obtain them. > > Here you can find lock profiling results for 7-BETA3 GENERIC kernel with > SCHED_ULE running optimized PHP and unoptimized, with your patch and > without it: http://83.167.98.162/gprof/lockmgr/ > > This data was collected by th following script: > (sysctl debug.lock.prof.reset=1 > sysctl debug.lock.prof.enable=1 > sleep 60 > sysctl debug.lock.prof.enable=0 > sysctl debug.lock.prof.stats > top -d 2 -b | tail -25) > > AFAIU there's still high contention on "lockbuilder mtxpool" with patch > applied. But hopefully "lockmgr:ufs" contention which i believe produced > 80%sysCPU load is gone with your patch. Looks to me like lockmgr-related contention was reduced by 1 to 2 orders of magnitude, which is the expected result. This surely must have a measurable impact on your workload. Further lockmgr improvement will have to wait until the lockmgr replacement work proceeds. One more patch which may or may not help is: http://www.freebsd.org/~jhb/patches/namei_rwlock.patch (may also require porting since it was against an older version of 7.0-CURRENT). When I have tested this in the past it was a performance loss for reasons that I think I understand (basically, it is locally a performance improvement for the name cache but also requires a fixed lockmgr to avoid an overall performance loss), but I don't remember if I tested it in conjunction with the lockmgr patch. >> There are patches you need to enable it on woodcrest. They are in my >> p4 branch (kris-contention) but I don't have time right now to extract >> them. > I think it would be very useful because I can't see any other ways to > profile FreeBSD on the modern many-cores machines. You can extract the changeset from my branch via http://perforce.freebsd.org. Unfortunately I don't have time to do it myself. Kris
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?474F12AE.7050808>