Date: Thu, 07 Feb 2008 00:42:18 +0200 From: Stefan Lambrev <stefan.lambrev@moneybookers.com> To: Kris Kennaway <kris@FreeBSD.org> Cc: freebsd-performance@freebsd.org Subject: Re: network performance Message-ID: <47AA37CA.8020208@moneybookers.com> In-Reply-To: <47AA1858.3050307@FreeBSD.org> References: <4794E6CC.1050107@moneybookers.com> <47A0B023.5020401@moneybookers.com> <m21w7x5ilg.wl%gnn@neville-neil.com> <47A3074A.3040409@moneybookers.com> <47A72EAB.6070602@moneybookers.com> <20080204182945.GA49276@heff.fud.org.nz> <47A780C0.2060201@moneybookers.com> <47A799A6.3070502@moneybookers.com> <47A84751.8020109@moneybookers.com> <47A8D233.8020506@FreeBSD.org> <47A8DCD6.3060209@moneybookers.com> <47A8E1F1.4040309@FreeBSD.org> <47A98CDC.2090407@moneybookers.com> <47A993D0.1060901@FreeBSD.org> <47A99736.8060809@moneybookers.com> <47A99B16.6030305@FreeBSD.org> <47A9B636.3040509@moneybookers.com> <47A9C43A.3030203@moneybookers.com> <47AA1858.3050307@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Greetings, Kris Kennaway wrote: > Stefan Lambrev wrote: > >>> I'll use again hwpmc and LOCK_PROFILING to see what's going on. >>> And will try the same benchmark on quad core processor as now >>> numbers of cores/cpus matter :) >>> >> Here are promised results - http://89.186.204.158/lock_profiling-8.txt Finally I got pmcstat working - http://89.186.204.158/hwpmc-p4.txt The stats are gathered during 600kpp incoming. I think that syncache or what calls MD5Transform is not SMP able, and that's why outgoing 250kpps is the limit that I can't beat. > > Thanks. There is further work needed on the route locking, and also > you are hitting limitations of the em driver (or possibly hardware; if > you only have a single transmit queue then outbound packets from > multiple CPUs have to be serialized in the driver no matter what). > Hopefully there will be further improvements in the coming months, and > these changes will also migrate into CVS. > > If you want to start hacking things to see how much further progress > is feasible, you can apply the attached hack that nulls out all route > locking :) This should be OK as long as your routes are not changing, > although you might get some spam on the console (if this is excessive, > comment out the printfs also ;-). It may not help much though, all > the contention will probably just fall through onto the ethernet driver. I'll do tomorrow. I still did not tested how if_lagg will perform in current situation. I thing that I didn't reached the maximum of the network card today. In this configuration best values was 1,122,674 incoming packets + 210kpps outgoing with both cpu cores 100% busy. systat -ifstat showed: em0 in 63.985 MB/s 63.985 MB/s 47.465 GB out 11.674 MB/s 11.723 MB/s 12.975 GB It is good, that now I hit a bottleneck that can be upgraded, if needed :) > >> Btw I got kernel panic first time when I run sysctl >> debug.lock.prof.stats > > Yeah, it is a bit broken in 8.0 even in CVS. Also make sure not to > reset it while the CPUs are loaded :) > >> I'm still trying to get hwpmc working with my cpu's and new kernel. >> Do you have any patches Kris? >> Is it supposed to work with your sources on my CPU? >> I can fetch your latest src/lib/libpmc from from p4 if this will help :) > > It works on my systems...try with libpmc from my branch, make sure to > install the new includes first and then rebuild and reinstall libpmc > and pmcstat. I have attached a patch against the CVS libpmc which > might be easier than checking it out from p4...it relies on kernel > changes also though, which are in the kernel you already have but not > in CVS. > > Kris Yes there was a problem with the installed include file, I edit it by hand, compiled new libpmc and it works again :)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47AA37CA.8020208>