From owner-freebsd-net@FreeBSD.ORG Fri Jul 4 09:45:22 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 74D591065674 for ; Fri, 4 Jul 2008 09:45:22 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 2B8F18FC21 for ; Fri, 4 Jul 2008 09:45:21 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-179-28.hsd1.fl.comcast.net ([76.108.179.28] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KEhnD-0002fY-N9; Fri, 04 Jul 2008 05:41:27 -0400 Message-ID: <486DF1A3.9000409@gtcomm.net> Date: Fri, 04 Jul 2008 05:47:15 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Ingo Flaschberger References: <4867420D.7090406@gtcomm.net> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> <486BC7F5.5070604@gtcomm.net> <20080703160540.W6369@delplex.bde.org> <486C7F93.7010308@gtcomm.net> <20080703195521.O6973@delplex.bde.org> <486D35A0.4000302@gtcomm.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2008 09:45:22 -0000 ngo Flaschberger wrote: > Dear Paul, > >> Opteron 2222 UP mode, no polling >> >> input (em0) output >> packets errs bytes packets errs bytes colls >> 1071020 0 66403248 2 0 404 0 > > that looks good. (but seems to be near the limit). > Yes it is , any more and errors start. >> Polling turned on provided better performance on 32 bit, but it gets >> strange errors on 64 bit.. >> Even at low pps I get small amounts of errors, and high pps same >> thing.. you would think that if >> it got errors at low pps it would get more errors at high pps but >> that isn't the case.. >> Polling on: >> packets errs bytes packets errs bytes colls >> 979736 963 60743636 1 0 226 0 >> 991838 496 61493960 1 0 178 0 >> 996125 460 61759754 1 0 178 0 >> >> >> what could cause this? > > *) kern.polling.idle_poll enabled? > *) kern.polling.user_frac ? > *) kern.polling.reg_frac ? > *) kern.polling.burst_max ? > *) kern.polling.each_burst ? I tried tons of different values for these and nothing made any significant difference. Idle polling makes a difference, allows more pps, but still errors. Without idle polling it seems PPS is limited to HZ * descriptors, or 1000 * 256 or 512 but 1000 * 1024 is the same as 512.. 4000 * 256 or 2000 * 512 works but starts erroring 600kpps (SMP right now but it happens in UP too) If anyone wants to log into the box and play with settings, recompile the kernel, etc. Let me know. Paul