From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 04:03:13 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F3F31065680 for ; Tue, 1 Jul 2008 04:03:13 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 3C16C8FC18 for ; Tue, 1 Jul 2008 04:03:13 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-179-28.hsd1.fl.comcast.net ([76.108.179.28] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KDX1t-0007Sf-0q for freebsd-net@freebsd.org; Mon, 30 Jun 2008 23:59:45 -0400 Message-ID: <4869ACFC.5020205@gtcomm.net> Date: Tue, 01 Jul 2008 00:05:16 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 CC: FreeBSD Net References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2008 04:03:13 -0000 Dual opteron 270 32 bit GENERIC KERNEL Nothing changed in sysctl except forwarding and ip forwarding Broadcom interfaces on board NIC last pid: 11557; load averages: 1.13, 0.83, 0.48 up 0+03:24:26 21:58:38 70 processes: 6 running, 46 sleeping, 18 waiting CPU states: 0.0% user, 0.0% nice, 0.2% system, 28.8% interrupt, 71.1% idle Mem: 9124K Active, 6844K Inact, 26M Wired, 9776K Buf, 1957M Free Swap: 4096M Total, 4096M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 13 root 171 ki31 0K 8K CPU1 1 203:39 98.93% idle: cpu1 14 root 171 ki31 0K 8K RUN 0 203:26 98.05% idle: cpu0 33 root -68 - 0K 8K CPU2 2 13:06 93.99% irq24: bge0 11 root 171 ki31 0K 8K CPU3 3 201:59 79.10% idle: cpu3 34 root -68 - 0K 8K WAIT 3 1:49 18.90% irq25: bge1 12 root 171 ki31 0K 8K RUN 2 190:42 4.00% idle: cpu2 input (bge0) output packets errs bytes packets errs bytes colls 410143 65619 24608586 1 0 226 0 410350 66749 24621006 1 0 178 0 409452 65382 24567126 1 0 178 0 410338 63157 24620286 1 0 178 0 410125 65777 24607446 1 0 178 0 409794 63018 24587706 1 0 178 0 408208 67566 24492486 1 0 178 0 408416 70305 24504906 1 0 178 0 407919 68339 24475206 1 0 178 0 sysctl -a | grep bge.0.stats dev.bge.0.stats.FramesDroppedDueToFilters: 0 dev.bge.0.stats.DmaWriteQueueFull: 310574781 dev.bge.0.stats.DmaWriteHighPriQueueFull: 0 dev.bge.0.stats.NoMoreRxBDs: 0 dev.bge.0.stats.InputDiscards: 20942213 dev.bge.0.stats.InputErrors: 15 dev.bge.0.stats.RecvThresholdHit: 51440366 dev.bge.0.stats.DmaReadQueueFull: 0 dev.bge.0.stats.DmaReadHighPriQueueFull: 0 dev.bge.0.stats.SendDataCompQueueFull: 0 dev.bge.0.stats.RingSetSendProdIndex: 25738223 dev.bge.0.stats.RingStatusUpdate: 51719806 dev.bge.0.stats.Interrupts: 51719806 dev.bge.0.stats.AvoidedInterrupts: 0 dev.bge.0.stats.SendThresholdHit: 0 dev.bge.0.stats.rx.Octets: 3477501149 dev.bge.0.stats.rx.Fragments: 116 dev.bge.0.stats.rx.UcastPkts: 519429709 dev.bge.0.stats.rx.MulticastPkts: 0 dev.bge.0.stats.rx.FCSErrors: 3 dev.bge.0.stats.rx.AlignmentErrors: 0 dev.bge.0.stats.rx.xonPauseFramesReceived: 0 dev.bge.0.stats.rx.xoffPauseFramesReceived: 0 dev.bge.0.stats.rx.ControlFramesReceived: 0 dev.bge.0.stats.rx.xoffStateEntered: 0 dev.bge.0.stats.rx.FramesTooLong: 0 dev.bge.0.stats.rx.Jabbers: 0 dev.bge.0.stats.rx.UndersizePkts: 0 dev.bge.0.stats.rx.inRangeLengthError: 0 dev.bge.0.stats.rx.outRangeLengthError: 0 dev.bge.0.stats.tx.Octets: 2215096864 dev.bge.0.stats.tx.Collisions: 0 dev.bge.0.stats.tx.XonSent: 0 dev.bge.0.stats.tx.XoffSent: 0 dev.bge.0.stats.tx.flowControlDone: 0 dev.bge.0.stats.tx.InternalMacTransmitErrors: 0 dev.bge.0.stats.tx.SingleCollisionFrames: 0 dev.bge.0.stats.tx.MultipleCollisionFrames: 0 dev.bge.0.stats.tx.DeferredTransmissions: 0 dev.bge.0.stats.tx.ExcessiveCollisions: 0 dev.bge.0.stats.tx.LateCollisions: 0 dev.bge.0.stats.tx.UcastPkts: 25738364 dev.bge.0.stats.tx.MulticastPkts: 0 dev.bge.0.stats.tx.BroadcastPkts: 15 dev.bge.0.stats.tx.CarrierSenseErrors: 0 dev.bge.0.stats.tx.Discards: 0 dev.bge.0.stats.tx.Errors: 0 errors..ERRORS!)(@!*() Hitting that 400k pps limit again, and this is a slower machine than my dual 2212. Going to compile 7-STABLE with options for the cpu and will report back