From owner-freebsd-performance@FreeBSD.ORG Thu Feb 7 10:18:15 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5CD616A419; Thu, 7 Feb 2008 10:18:15 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: from blah.sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id D8C1213C46A; Thu, 7 Feb 2008 10:18:14 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: by blah.sun-fish.com (Postfix, from userid 1002) id 1EC491B10F1F; Thu, 7 Feb 2008 11:18:14 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on blah.cmotd.com X-Spam-Level: X-Spam-Status: No, score=-10.6 required=5.0 tests=ALL_TRUSTED,BAYES_00, NORMAL_HTTP_TO_IP autolearn=ham version=3.2.3 Received: from hater.haters.org (hater.cmotd.com [192.168.3.125]) by blah.sun-fish.com (Postfix) with ESMTP id 9B7101B10EE0; Thu, 7 Feb 2008 11:18:10 +0100 (CET) Message-ID: <47AADAE2.4040500@moneybookers.com> Date: Thu, 07 Feb 2008 12:18:10 +0200 From: Stefan Lambrev User-Agent: Thunderbird 2.0.0.9 (X11/20071120) MIME-Version: 1.0 To: Kris Kennaway References: <4794E6CC.1050107@moneybookers.com> <47A0B023.5020401@moneybookers.com> <47A3074A.3040409@moneybookers.com> <47A72EAB.6070602@moneybookers.com> <20080204182945.GA49276@heff.fud.org.nz> <47A780C0.2060201@moneybookers.com> <47A799A6.3070502@moneybookers.com> <47A84751.8020109@moneybookers.com> <47A8D233.8020506@FreeBSD.org> <47A8DCD6.3060209@moneybookers.com> <47A8E1F1.4040309@FreeBSD.org> <47A98CDC.2090407@moneybookers.com> <47A993D0.1060901@FreeBSD.org> <47A99736.8060809@moneybookers.com> <47A99B16.6030305@FreeBSD.org> <47A9B636.3040509@moneybookers.com> <47A9C43A.3030203@moneybookers.com> <47AA1858.3050307@FreeBSD.org> <47AA37CA.8020208@moneybookers.com> <47AA3DF9.5040109@FreeBSD.org> <47AAD2B6.6070106@moneybookers.com> In-Reply-To: <47AAD2B6.6070106@moneybookers.com> Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.91.2/5727/Thu Feb 7 10:12:33 2008 on blah.cmotd.com X-Virus-Status: Clean Cc: "freebsd-performance@freebsd.org" Subject: Re: network performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Feb 2008 10:18:15 -0000 Greetings, Stefan Lambrev wrote: > Greetings, > > Kris Kennaway wrote: >> Stefan Lambrev wrote: >>> Greetings, >>> >>> Kris Kennaway wrote: >>>> Stefan Lambrev wrote: >>>> >>>>>> I'll use again hwpmc and LOCK_PROFILING to see what's going on. >>>>>> And will try the same benchmark on quad core processor as now >>>>>> numbers of cores/cpus matter :) >>>>>> >>>>> Here are promised results - >>>>> http://89.186.204.158/lock_profiling-8.txt >>> Finally I got pmcstat working - http://89.186.204.158/hwpmc-p4.txt >>> The stats are gathered during 600kpp incoming. >>> I think that syncache or what calls MD5Transform is not SMP able, >>> and that's why outgoing 250kpps is the limit that I can't beat. >> >> It looks like the syncache is using most of the CPU time. However >> you are not hitting problems caused by lack of concurrency there. It >> does do a *lot* of work with the syncache mutex held (including >> generation of the cookie, which involves MD5) so it might be an issue >> in the future, but there are other bottlenecks in the way before that >> is your main issue. Things may be different with more CPUs. >> >> Did you compare to what happens to performance when the syncache is >> disabled? >> >> Kris >> > When I disable syncookies the server respond to more packets - from > 250kpps enabled to 320kpps when disabled. > Can I disable syncache and how? Err you answer this already :) So here are numbers in different syncache/coockies options: net.inet.tcp.syncookies: 1 & net.inet.tcp.syncookies_only: 0 input (em0) output packets errs bytes packets errs bytes colls 519748 0 31184880 252606 0 14651090 0 522321 0 31339260 251459 0 14584490 0 532242 0 31934520 242768 0 14080486 0 533484 0 32009040 249006 0 14442522 0 net.inet.tcp.syncookies: 0 & net.inet.tcp.syncookies_only: 0 input (em0) output packets errs bytes packets errs bytes colls 512100 0 30726000 316873 0 18378866 0 517952 0 31077120 315868 0 18320270 0 531862 0 31911720 318416 0 18468244 0 526125 0 31567440 315283 0 18286340 0 net.inet.tcp.syncookies: 0 & net.inet.tcp.syncookies_only: 1 input (em0) output packets errs bytes packets errs bytes colls 517257 0 31035420 315739 0 18310890 0 518328 0 31099680 316888 0 18381534 0 517168 0 31030080 316693 0 18368426 0 528273 0 31696320 315722 0 18311670 0 533153 0 31989240 314693 0 18252136 0 net.inet.tcp.syncookies: 1 & net.inet.tcp.syncookies_only: 1 input (em0) output packets errs bytes packets errs bytes colls 539069 0 32344140 282434 0 16381272 0 538075 0 32284440 281569 0 16330886 0 538329 0 32299800 281908 0 16350880 0 534111 0 32046660 280906 0 16292258 0 536204 0 32172240 281169 0 16307860 0 Increasing net.inet.tcp.syncache.hashsize to 2048 (512 default) input (em0) output packets errs bytes packets errs bytes colls 532945 0 31976700 186833 0 10835096 0 533507 0 32010420 187053 0 10850292 0 530204 0 31812240 187672 0 10883758 0 531482 0 31888920 187585 0 10879930 0 534651 0 32079060 186977 0 10845884 0 This prove that syncache have negative impact on performance ? > I'll try to increase syncache limits and will test again. > > Btw is this expected - net.inet.tcp.syncache.count: -387. > I think this number should be always > 0. > > -- Best Wishes, Stefan Lambrev ICQ# 24134177