Date: Wed, 21 Nov 2007 14:13:19 +0300 From: Alexey Popov <lol@chistydom.ru> To: Kris Kennaway <kris@FreeBSD.org> Cc: Attilio Rao <attilio@freebsd.org>, freebsd-stable@freebsd.org Subject: Re: 2 x quad-core system is slower that 2 x dual core on FreeBSD Message-ID: <474412CF.5050200@chistydom.ru> In-Reply-To: <474339E9.4080301@FreeBSD.org> References: <4741905E.8050300@chistydom.ru> <fhs3s5$knj$1@ger.gmane.org> <47419AB3.5030008@chistydom.ru> <fhs7hp$2es$2@ger.gmane.org> <4741A7DA.2050706@chistydom.ru> <4741DA15.9000308@FreeBSD.org> <47429DB8.7040504@chistydom.ru> <4742ADFE.40902@FreeBSD.org> <4742C46A.1060701@chistydom.ru> <47432F77.3030606@FreeBSD.org> <474339E9.4080301@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi. Kris Kennaway wrote: >> In the meantime there is unfortunately not a lot that can be done, >> AFAICT. There is one hack that I will send you later but it is not >> likely to help much. I will also think about how to track down the >> cause of the contention further (the profiling trace only shows that >> it comes mostly from vget/vput but doesn't show where these are called >> from). > Actually this patch might help. It doesn't replace lockmgr but it does > fix a silly thundering herd behaviour. It probably needs some > adjustment to get it to apply cleanly (it is about 7 months old), and I > apparently stopped using it because I ran into deadlocks. It might be > stable enough to at least see how much it helps. Sorry, I didn't try you patch yet but I have other news. As mentioned in the description of your patch there is probably a scalability problem with stat() syscall on FreeBSD. The PHP code of our site consists of large amount of modules. I think this is true for many other large PHP sites. I reached out that PHP calls lstat() for every path element of each file it opens including modules. Truss output shows that PHP makes more than 2000 lstat's for one /index.php request. After investigation I found out that lstats() are called from realpath() libc function. It turned out that PHP has "realpath cache", but it's size by default is 16K which is not enough for my files. I set realpath_cache_size to 256K and now there is no that much lstat calls. Performance of 8-core machine growed in ~ 50% for me on 7-STABLE. Now it can handle 30 and more requests per seconds. I have the similiar results with 6-STABLE. Now I have not that big %sys values as it was before (see attached top output). Nevertheless, Linux with its >50 rps is still far away from FreeBSD. Linux makes that 2000+ lstat's without problem. There's still stat(), open(), gettimeofday(), close() syscalls for each include file in PHP that i can not switch off. And also it is unclear for me what to do with MySQL which happened to have the same problems for me. With best regards, Alexey Popov
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?474412CF.5050200>