From owner-freebsd-performance@FreeBSD.ORG Sun Nov 26 09:43:25 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9411916A412 for ; Sun, 26 Nov 2006 09:43:25 +0000 (UTC) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.176.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E54343D8A for ; Sun, 26 Nov 2006 09:42:31 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.8/8.13.7) with ESMTP id kAQ9hLj3036986 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 26 Nov 2006 10:43:21 +0100 (CET) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.8/8.13.3/Submit) id kAQ9hLnV036984; Sun, 26 Nov 2006 10:43:21 +0100 (CET) Date: Sun, 26 Nov 2006 10:43:21 +0100 From: Divacky Roman To: "Matthew D. Fuller" Message-ID: <20061126094321.GA34909@stud.fit.vutbr.cz> References: <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611242027.kAOKRYZg012113@lava.sentex.ca> <20061124210305.GA49228@stud.fit.vutbr.cz> <200611242117.kAOLHuBP012313@lava.sentex.ca> <20061125083605.GA17350@stud.fit.vutbr.cz> <20061125112216.GC91673@over-yonder.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20061125112216.GC91673@over-yonder.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.57 on 147.229.176.14 Cc: freebsd-performance@freebsd.org, Mike Tancsa Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2006 09:43:25 -0000 On Sat, Nov 25, 2006 at 05:22:16AM -0600, Matthew D. Fuller wrote: > On Sat, Nov 25, 2006 at 09:36:05AM +0100 I heard the voice of > Divacky Roman, and lo! it spake thus: > > > > hm.. now I am confused. the rule is that having I586_CPU improves > > performance because optimized bzero/bcopy is included (its not > > included if you only have I686_CPU). > > Haven't we been by this before? It's not included even if you have > I586_CPU either. > > See src/sys/i386/isa/npx.c, line 432: > > #ifdef I586_CPU_XXX <-------- > ^^^ > > This has been disabled since r1.95, in 2001 (in 5-CURRENT days). > There may be SOMETHING about including I586_CPU that speeds things up, > but it ain't that. the log says: People are still having problems with i586_* on UP machines and SMP machines, so just hack it to disable them for now until it can be fixed. noone has been able to step up and fix that? for more then 5 years? huh.. I guess thats quite low hanging fruit worth shoting down.. From owner-freebsd-performance@FreeBSD.ORG Mon Nov 27 22:46:39 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2186016A416 for ; Mon, 27 Nov 2006 22:46:39 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0DE964473B for ; Mon, 27 Nov 2006 21:53:24 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kARLsNOb025072; Mon, 27 Nov 2006 16:54:23 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kARLsMC7029800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 27 Nov 2006 16:54:23 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611272154.kARLsMC7029800@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 27 Nov 2006 16:54:34 -0500 To: "Nick Pavlica" From: Mike Tancsa In-Reply-To: References: <4557CECD.2000609@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner3 X-Virus-Status: Clean Cc: freebsd-performance@freebsd.org, Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Nov 2006 22:46:39 -0000 At 02:12 PM 11/25/2006, Nick Pavlica wrote: >>I might give OpenBSD a quick try as a reference. > >Mike, > Have you done any testing on Solaris 10, or OpenSolaris? I >understand that it has a very robust IP stack. It would be Did a quick default install. Results are not so interesting since one stream livelocks the box. Basic stats at http://www.tancsa.com/blast.html If there are some OpenSolaris wizards out there who want me to tune, I am happy to retest... ---Mike From owner-freebsd-performance@FreeBSD.ORG Tue Nov 28 08:07:01 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9618016A412 for ; Tue, 28 Nov 2006 08:07:01 +0000 (UTC) (envelope-from massimo@cedoc.mo.it) Received: from mail.cedoc.mo.it (ip-147-100.sn2.eutelia.it [83.211.147.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D82343CC6 for ; Tue, 28 Nov 2006 08:06:50 +0000 (GMT) (envelope-from massimo@cedoc.mo.it) Received: from mail.cedoc.mo.it (localhost [127.0.0.1]) by mail.cedoc.mo.it (Postfix) with ESMTP id 20A1A56420; Tue, 28 Nov 2006 09:04:58 +0100 (CET) Received: from intanto (intanto.cedoc.mo.it [172.16.1.72]) by mail.cedoc.mo.it (Postfix) with SMTP id ED0C656415; Tue, 28 Nov 2006 09:04:57 +0100 (CET) Date: Tue, 28 Nov 2006 09:06:48 +0100 From: Massimo Lusetti To: Mike Tancsa Message-Id: <20061128090648.d58eb6ae.massimo@cedoc.mo.it> In-Reply-To: <200611272136.kARLaMeG029740@lava.sentex.ca> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611272136.kARLaMeG029740@lava.sentex.ca> X-Mailer: Sylpheed version 2.2.9 (GTK+ 2.10.6; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Nov 2006 08:07:01 -0000 On Mon, 27 Nov 2006 16:36:34 -0500 Mike Tancsa wrote: > OK, I added OpenBSD to the mix as well. Results are pretty crappy > with the base default install. With one stream, the box essentially > live locks. This was just with the stock kernels from the CD. The > PCIe bge nics dont work, so I cant test those. I had a look at their > errata page and there seems to be some updates to those 2 nics so if > there is interest I can try compiling in those fixes and re-testing FWIW I would definitively like to see it. But thanks for going so far.. -- Massimo From owner-freebsd-performance@FreeBSD.ORG Tue Nov 28 19:00:05 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4549416A512 for ; Tue, 28 Nov 2006 19:00:05 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D87243EAB for ; Tue, 28 Nov 2006 18:56:47 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kASIufVc053367; Tue, 28 Nov 2006 13:56:41 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kASIueZX035063 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 28 Nov 2006 13:56:40 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611281856.kASIueZX035063@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 28 Nov 2006 13:55:48 -0500 To: Massimo Lusetti From: Mike Tancsa In-Reply-To: <20061128090648.d58eb6ae.massimo@cedoc.mo.it> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611272136.kARLaMeG029740@lava.sentex.ca> <20061128090648.d58eb6ae.massimo@cedoc.mo.it> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner3 X-Virus-Status: Clean Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Nov 2006 19:00:05 -0000 At 03:06 AM 11/28/2006, Massimo Lusetti wrote: >On Mon, 27 Nov 2006 16:36:34 -0500 >Mike Tancsa wrote: > > > OK, I added OpenBSD to the mix as well. Results are pretty crappy > > with the base default install. With one stream, the box essentially > > live locks. This was just with the stock kernels from the CD. The > > PCIe bge nics dont work, so I cant test those. I had a look at their > > errata page and there seems to be some updates to those 2 nics so if > > there is interest I can try compiling in those fixes and re-testing > >FWIW I would definitively like to see it. But thanks for going so far.. I will give it another try tomorrow as I dont have much time today. However, I did one test with a new CPU. I changed the cpu from an AMD 3800 to an AMD4600. So going from a CPU with a clock speed of 2Ghz to 2.4Gz. The difference is sort of what one would expect. In the ipfw tests, (with a sample of 40 seconds) opposite path forwarding with 10 ipfw rules went from 208 Kpps to 235 Kpps or about a 10% increase as opposed to the ~ 15% increase in clock speed. Perhaps bus is more of a limiting factor that CPU but then again some the difference could be explained with normal variance. ---Mike From owner-freebsd-performance@FreeBSD.ORG Wed Nov 29 07:14:55 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BEB5816A407; Wed, 29 Nov 2006 07:14:55 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4C4B943CD8; Wed, 29 Nov 2006 07:14:03 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kARLaOYd026216; Mon, 27 Nov 2006 16:36:24 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kARLaMeG029740 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 27 Nov 2006 16:36:22 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611272136.kARLaMeG029740@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 27 Nov 2006 16:36:34 -0500 To: Massimo Lusetti From: Mike Tancsa In-Reply-To: <1164356894.4306.1.camel@massimo.datacode.it> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Nov 2006 07:14:55 -0000 At 03:28 AM 11/24/2006, Massimo Lusetti wrote: >On Thu, 2006-11-23 at 11:52 -0500, Mike Tancsa wrote: > > > I might give OpenBSD a quick try as a reference. > >That would be very interesting. OK, I added OpenBSD to the mix as well. Results are pretty crappy with the base default install. With one stream, the box essentially live locks. This was just with the stock kernels from the CD. The PCIe bge nics dont work, so I cant test those. I had a look at their errata page and there seems to be some updates to those 2 nics so if there is interest I can try compiling in those fixes and re-testing ---Mike From owner-freebsd-performance@FreeBSD.ORG Wed Nov 29 07:36:59 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E9F0F16A4D8 for ; Wed, 29 Nov 2006 07:36:59 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id D035A43D88 for ; Wed, 29 Nov 2006 07:33:57 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kAT3RoIF025051; Tue, 28 Nov 2006 22:27:50 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAT3RnJ0037112 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 28 Nov 2006 22:27:49 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611290327.kAT3RnJ0037112@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 28 Nov 2006 22:25:46 -0500 To: Massimo Lusetti From: Mike Tancsa In-Reply-To: <20061128090648.d58eb6ae.massimo@cedoc.mo.it> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611272136.kARLaMeG029740@lava.sentex.ca> <20061128090648.d58eb6ae.massimo@cedoc.mo.it> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner2 X-Virus-Status: Clean Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Nov 2006 07:37:00 -0000 At 03:06 AM 11/28/2006, Massimo Lusetti wrote: >FWIW I would definitively like to see it. But thanks for going so far.. Tried it with the patch branch. With the em nics, the box locks up with 2 streams. It works now with bge, but rates are pretty slow (220Kpps), and very slow with pf enabled. This was done with SMP. While forwarding, the box is not responsive at all. ---Mike From owner-freebsd-performance@FreeBSD.ORG Wed Nov 29 21:28:20 2006 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B476116A415; Wed, 29 Nov 2006 21:28:20 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 71EA043CAF; Wed, 29 Nov 2006 21:28:16 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kATLSJVC049584; Wed, 29 Nov 2006 16:28:19 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kATLSIMu041796 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 29 Nov 2006 16:28:18 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611292128.kATLSIMu041796@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 29 Nov 2006 16:26:17 -0500 To: freebsd-performance@FreeBSD.org From: Mike Tancsa In-Reply-To: <200611290327.kAT3RnJ0037112@lava.sentex.ca> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611272136.kARLaMeG029740@lava.sentex.ca> <20061128090648.d58eb6ae.massimo@cedoc.mo.it> <200611290327.kAT3RnJ0037112@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner3 X-Virus-Status: Clean Cc: Subject: Re: vlan forwarding performance (was Proposed em forwawrding performance) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Nov 2006 21:28:20 -0000 Did some more tests, this time using a single NIC interface in trunking mode. Strangely enough, the speed is a little faster on HEAD. Perhaps less interrupt processing ? Results in the usual place http://www.tancsa.com//blast.html ---Mike From owner-freebsd-performance@FreeBSD.ORG Wed Nov 29 21:40:31 2006 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3FDEB16A4D1 for ; Wed, 29 Nov 2006 21:40:31 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id C0C6443EAD for ; Wed, 29 Nov 2006 21:35:48 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 9904946CE4; Wed, 29 Nov 2006 16:35:44 -0500 (EST) Date: Wed, 29 Nov 2006 21:35:44 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Mike Tancsa In-Reply-To: <200611292128.kATLSIMu041796@lava.sentex.ca> Message-ID: <20061129213301.I16817@fledge.watson.org> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611272136.kARLaMeG029740@lava.sentex.ca> <20061128090648.d58eb6ae.massimo@cedoc.mo.it> <200611290327.kAT3RnJ0037112@lava.sentex.ca> <200611292128.kATLSIMu041796@lava.sentex.ca> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@FreeBSD.org Subject: Re: vlan forwarding performance (was Proposed em forwawrding performance) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Nov 2006 21:40:31 -0000 On Wed, 29 Nov 2006, Mike Tancsa wrote: > Did some more tests, this time using a single NIC interface in trunking > mode. Strangely enough, the speed is a little faster on HEAD. Perhaps less > interrupt processing ? Results in the usual place > > http://www.tancsa.com//blast.html You may want to datestamp the version of HEAD you're using in each test (assuming it changes between tests). Among other things, I set net.isr.direct=1 as the default a day or two ago, which will affect a broad range of performance characteristics. There have also been VLAN tagging related performance improvements made in HEAD that aren't present in RELENG_6, avoiding extra memory allocations and frees in storing the tags. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-performance@FreeBSD.ORG Wed Nov 29 21:51:32 2006 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 89A7316A4D4; Wed, 29 Nov 2006 21:51:32 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0561D43D67; Wed, 29 Nov 2006 21:51:07 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kATLpB8a008186; Wed, 29 Nov 2006 16:51:11 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kATLpApP041879 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 29 Nov 2006 16:51:10 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611292151.kATLpApP041879@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 29 Nov 2006 16:49:08 -0500 To: Robert Watson From: Mike Tancsa In-Reply-To: <20061129213301.I16817@fledge.watson.org> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611272136.kARLaMeG029740@lava.sentex.ca> <20061128090648.d58eb6ae.massimo@cedoc.mo.it> <200611290327.kAT3RnJ0037112@lava.sentex.ca> <200611292128.kATLSIMu041796@lava.sentex.ca> <20061129213301.I16817@fledge.watson.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@FreeBSD.org Subject: Re: vlan forwarding performance (was Proposed em forwawrding performance) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Nov 2006 21:51:32 -0000 At 04:35 PM 11/29/2006, Robert Watson wrote: >You may want to datestamp the version of HEAD you're using in each >test (assuming it changes between tests). Ahh, Good point. I have been using the sources always from Nov 24th to make comparisons as similar as possible for the tests with 7.0 >which will affect a broad range of performance >characteristics. There have also been VLAN tagging related >performance improvements made in HEAD that aren't present in >RELENG_6, avoiding extra memory allocations and frees in storing the tags. I ran the tests against RELENG_6 then as well and have posted those results. There does indeed seem to be a slight improvement in HEAD over RELENG_6. ---Mike From owner-freebsd-performance@FreeBSD.ORG Thu Nov 30 05:51:30 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7C22516A412 for ; Thu, 30 Nov 2006 05:51:30 +0000 (UTC) (envelope-from linicks@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.191]) by mx1.FreeBSD.org (Postfix) with ESMTP id B211343C9D for ; Thu, 30 Nov 2006 05:51:23 +0000 (GMT) (envelope-from linicks@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so2964290nfc for ; Wed, 29 Nov 2006 21:51:28 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=B4m9gaIMwTWszuoqAMmKM+v4ihuXR9S6osmBSrlRqeFfYBum5XALHv73MYtqLaVnzqQaPDg2RItwGhsGBAdBGiEElpBHtqzOL6VAxrQ4k7vEN3baMqX6LwlbB3ylQJF2S6N+nw5NuMnG2eFeu8H/OMk0S+x1g37LGhhM1b+Bhsw= Received: by 10.82.114.3 with SMTP id m3mr750765buc.1164865887926; Wed, 29 Nov 2006 21:51:27 -0800 (PST) Received: by 10.82.174.13 with HTTP; Wed, 29 Nov 2006 21:51:27 -0800 (PST) Message-ID: Date: Wed, 29 Nov 2006 22:51:27 -0700 From: "Nick Pavlica" To: "Mike Tancsa" In-Reply-To: <200611272154.kARLsMC7029800@lava.sentex.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4557CECD.2000609@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <200611272154.kARLsMC7029800@lava.sentex.ca> Cc: freebsd-performance@freebsd.org, Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2006 05:51:30 -0000 On 11/27/06, Mike Tancsa wrote: > At 02:12 PM 11/25/2006, Nick Pavlica wrote: > >>I might give OpenBSD a quick try as a reference. > > > >Mike, > > Have you done any testing on Solaris 10, or OpenSolaris? I > >understand that it has a very robust IP stack. It would be > > Did a quick default install. Results are not so interesting since one > stream livelocks the box. Basic stats at http://www.tancsa.com/blast.html > > If there are some OpenSolaris wizards out there who want me to tune, > I am happy to retest... > > ---Mike Mike, I'm not an OpenSolaris/Solaris expert, but was curious which build you were testing with. I have noticed various results with my testing depending on which build I was using. I have done most of my testing/learning on Solaris 10 06/06, and have played with Solaris Express B50 with good results. I did see some quirks in SolarisExpress CE or B52 at the time of this writing. Of course I patched all of these boxes before I did my testing which was mostly centered around disk I/O performance on UFS and ZFS, and some experimentation with Zones/Containers. I'm surprised that the console locked up during your tests. My limited experience with Solaris 10+ thus far has been positive in terms of performance and stability. When I have stressed my test systems, they remained responsive and seemed to have better performance than FC6 and Ubuntu6.10 when copying large files across my network. Thanks for digging in with this testing, I hope you keep at it. --Nick From owner-freebsd-performance@FreeBSD.ORG Thu Nov 30 14:17:54 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6150B16A403 for ; Thu, 30 Nov 2006 14:17:54 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D1AD843CA3 for ; Thu, 30 Nov 2006 14:17:45 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kAUEHrYW047209; Thu, 30 Nov 2006 09:17:53 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAUEHqAm046076 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 30 Nov 2006 09:17:52 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611301417.kAUEHqAm046076@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 30 Nov 2006 09:15:52 -0500 To: "Nick Pavlica" From: Mike Tancsa In-Reply-To: References: <4557CECD.2000609@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <200611272154.kARLsMC7029800@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@freebsd.org, Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2006 14:17:54 -0000 At 12:51 AM 11/30/2006, Nick Pavlica wrote: >>Did a quick default install. Results are not so interesting since one >>stream livelocks the box. Basic stats at http://www.tancsa.com/blast.html >> >>If there are some OpenSolaris wizards out there who want me to tune, >>I am happy to retest... > >Mike, > I'm not an OpenSolaris/Solaris expert, but was curious which build >you were testing with. Hi, I grabbed the latest DVD bits that were available at the time. # uname -a SunOS interlope 5.11 snv_52 i86pc i386 i86pc >SolarisExpress CE or B52 at the time of this writing. Of course I >patched all of these boxes before I did my testing which was mostly >centered around disk I/O performance on UFS and ZFS, and some >experimentation with Zones/Containers. Didnt do any patches. The only thing I did was kill off X and disable and enable ipfilter. Its quite possible there was other cruft running that I didnt know about, but like I said, this was my first exposure to OpenSolaris so I have no idea if there are things I should have set. > I'm surprised that the console >locked up during your tests. >My limited experience with Solaris 10+ >thus far has been positive in terms of performance and stability. It does recover afterwards, but pretty well all other processes stop as the CPU I guess is pegged dealing with all the interrupts. Thinking further about my tests, it doesnt really do that great of a job of simulating normal real world conditions. In the real world, the packet sizes will vary and the speeds will be all over the place. I am wondering if some of these modern nics have that in mind with their design. But then again, this is sort of the scenario when a firewal gets blasted by a high PPS attack :( >When I have stressed my test systems, they remained responsive and >seemed to have better performance than FC6 and Ubuntu6.10 when >copying large files across my network. But thats pretty different then my test setup. All the OSes I tested can do that no problem :) >Thanks for digging in with this testing, I hope you keep at it. Yeah I inadvertently slighted the NetBSD folks by leaving them out. So I guess I better give them a try as well. The part that really surprises me is the drop in performance as firewall rules are added to RELENG_6 and above. Both LINUX and RELENG_4 seem to scale well with the number of rules added but RELENG_6 takes a big drop. ---Mike From owner-freebsd-performance@FreeBSD.ORG Thu Nov 30 17:58:57 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 53A8F16A407 for ; Thu, 30 Nov 2006 17:58:57 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id DE0C343CB9 for ; Thu, 30 Nov 2006 17:58:43 +0000 (GMT) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1GpqAs-0000iD-78 for freebsd-performance@freebsd.org; Thu, 30 Nov 2006 18:58:18 +0100 Received: from 83-131-160-100.adsl.net.t-com.hr ([83.131.160.100]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Nov 2006 18:58:18 +0100 Received: from ivoras by 83-131-160-100.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Nov 2006 18:58:18 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Thu, 30 Nov 2006 18:57:48 +0100 Lines: 17 Message-ID: References: <4557CECD.2000609@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <200611272154.kARLsMC7029800@lava.sentex.ca> <200611301417.kAUEHqAm046076@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 83-131-160-100.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.8 (Windows/20061025) In-Reply-To: <200611301417.kAUEHqAm046076@lava.sentex.ca> Sender: news Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2006 17:58:57 -0000 Mike Tancsa wrote: > Yeah I inadvertently slighted the NetBSD folks by leaving them out. So > I guess I better give them a try as well. > > The part that really surprises me is the drop in performance as firewall > rules are added to RELENG_6 and above. Both LINUX and RELENG_4 seem to > scale well with the number of rules added but RELENG_6 takes a big drop. Wasn't there some important setting in ipfw you can tweak if you need lots of ipfw rules? Size of some hash table? Quick Googling found this: http://info.iet.unipi.it/~luigi/ip_dummynet/ and net.inet.ip.fw.dyn_buckets: 256. AFAIK the hash size needed to be tweaked manually in the code, and net.inet.ip.fw.dyn_buckets: 256 is listed as read-only so this might be it. Maybe mailing Luigi will help finding out... From owner-freebsd-performance@FreeBSD.ORG Thu Nov 30 18:11:19 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B2E4416A412 for ; Thu, 30 Nov 2006 18:11:19 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 38F7C43CB2 for ; Thu, 30 Nov 2006 18:11:02 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kAUI6AEW094185; Thu, 30 Nov 2006 13:06:10 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAUI68vd047209 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 30 Nov 2006 13:06:10 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611301806.kAUI68vd047209@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 30 Nov 2006 13:04:08 -0500 To: Ivan Voras , freebsd-performance@freebsd.org From: Mike Tancsa In-Reply-To: References: <4557CECD.2000609@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <200611272154.kARLsMC7029800@lava.sentex.ca> <200611301417.kAUEHqAm046076@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2006 18:11:19 -0000 At 12:57 PM 11/30/2006, Ivan Voras wrote: >Mike Tancsa wrote: > > > Yeah I inadvertently slighted the NetBSD folks by leaving them out. So > > I guess I better give them a try as well. > > > > The part that really surprises me is the drop in performance as firewall > > rules are added to RELENG_6 and above. Both LINUX and RELENG_4 seem to > > scale well with the number of rules added but RELENG_6 takes a big drop. > >Wasn't there some important setting in ipfw you can tweak if you need >lots of ipfw rules? Size of some hash table? > >Quick Googling found this: http://info.iet.unipi.it/~luigi/ip_dummynet/ >and net.inet.ip.fw.dyn_buckets: 256. AFAIK the hash size needed to be >tweaked manually in the code, and net.inet.ip.fw.dyn_buckets: 256 is >listed as read-only so this might be it. Maybe mailing Luigi will help >finding out... I was told offlist "there is additional per-packet locking overhead not seen in RELENG_4 where all processing is covered by the same spl."... ---Mike