Date: Tue, 01 Jul 2008 00:18:45 -0400 From: Paul <paul@gtcomm.net> To: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] Message-ID: <4869B025.9080006@gtcomm.net> In-Reply-To: <4869ACFC.5020205@gtcomm.net> References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <alpine.LFD.1.10.0807010257570.19444@filebunker.xip.at> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <alpine.LFD.1.10.0807010308320.19444@filebunker.xip.at> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <ea7b9c170806302005n2a66f592h2127f87a0ba2c6d2@mail.gmail.com> <20080701033117.GH83626@cdnetworks.co.kr> <ea7b9c170806302050p2a3a5480t29923a4ac2d7c852@mail.gmail.com> <4869ACFC.5020205@gtcomm.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Dual opteron 2212, amd64 kernel, GENERIC (same setup as my 270 I just
posted except this is 64 bit and different NIC)
Intel dual port 82571 , nothing changed in sysctl except fw and fastfw
input (em0) output
packets errs bytes packets errs bytes colls
455729 69094 27343822 9 0 1412 0
455582 67566 27334938 3 0 566 0
455525 66033 27331518 3 0 566 0
456632 68384 27397938 3 0 566 0
457144 70006 27428604 4 0 776 0
453551 71307 27213226 7 0 1038 0
455282 74669 27317022 6 0 1068 0
452268 75878 27136104 4 0 744 0
455447 76942 27326838 4 0 744 0
455408 68166 27324522 5 0 922 0
456113 74600 27366798 4 0 744 0
456316 68522 27378996 5 0 922 0
455888 71665 27353322 5 0 1034 0
453448 76407 27206904 4 0 744 0
453907 79158 27234444 4 0 744 0
452921 66532 27175278 4 0 744 0
456049 60947 27363048 8 0 1216 0
em0: Excessive collisions = 0
em0: Sequence errors = 0
em0: Defer count = 0
em0: Missed Packets = 131239807
em0: Receive No Buffers = 154088217
em0: Receive Length Errors = 0
em0: Receive errors = 0
em0: Crc errors = 0
em0: Alignment errors = 0
em0: Collision/Carrier extension errors = 0
em0: RX overruns = 4986006
em0: watchdog timeouts = 0
em0: XON Rcvd = 0
em0: XON Xmtd = 0
em0: XOFF Rcvd = 0
em0: XOFF Xmtd = 0
em0: Good Packets Rcvd = 969917996
em0: Good Packets Xmtd = 98120420
em0: TSO Contexts Xmtd = 2062
em0: TSO Contexts Failed = 0
errrrrrrrrrrorss.....
79 processes: 6 running, 52 sleeping, 3 stopped, 18 waiting
CPU states: 0.0% user, 0.0% nice, 28.2% system, 0.0% interrupt, 71.8%
idle
Mem: 18M Active, 787M Inact, 241M Wired, 196K Cache, 213M Buf, 928M Free
Swap: 8192M Total, 8192M Free
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
12 root 171 ki31 0K 16K CPU2 2 348:19 99.02% idle: cpu2
13 root 171 ki31 0K 16K CPU1 1 347:02 99.02% idle: cpu1
37 root -68 - 0K 16K CPU3 3 37:48 99.02% em0 taskq
14 root 171 ki31 0K 16K RUN 0 340:08 85.94% idle: cpu0
38 root -68 - 0K 16K - 1 8:46 12.06% em1 taskq
How do you like those IDLE CPUS :) except 3 of course.. :>
Paul wrote:
> Dual opteron 270
> 32 bit GENERIC KERNEL Nothing changed in sysctl except forwarding and
> ip forwarding
> Broadcom interfaces on board NIC
>
> last pid: 11557; load averages: 1.13, 0.83,
> 0.48
> up 0+03:24:26 21:58:38
> 70 processes: 6 running, 46 sleeping, 18 waiting
> CPU states: 0.0% user, 0.0% nice, 0.2% system, 28.8% interrupt,
> 71.1% idle
> Mem: 9124K Active, 6844K Inact, 26M Wired, 9776K Buf, 1957M Free
> Swap: 4096M Total, 4096M Free
>
> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
> 13 root 171 ki31 0K 8K CPU1 1 203:39 98.93% idle: cpu1
> 14 root 171 ki31 0K 8K RUN 0 203:26 98.05% idle: cpu0
> 33 root -68 - 0K 8K CPU2 2 13:06 93.99% irq24: bge0
> 11 root 171 ki31 0K 8K CPU3 3 201:59 79.10% idle: cpu3
> 34 root -68 - 0K 8K WAIT 3 1:49 18.90% irq25: bge1
> 12 root 171 ki31 0K 8K RUN 2 190:42 4.00% idle: cpu2
>
> input (bge0) output
> packets errs bytes packets errs bytes colls
> 410143 65619 24608586 1 0 226 0
> 410350 66749 24621006 1 0 178 0
> 409452 65382 24567126 1 0 178 0
> 410338 63157 24620286 1 0 178 0
> 410125 65777 24607446 1 0 178 0
> 409794 63018 24587706 1 0 178 0
> 408208 67566 24492486 1 0 178 0
> 408416 70305 24504906 1 0 178 0
> 407919 68339 24475206 1 0 178 0
>
>
> sysctl -a | grep bge.0.stats
> dev.bge.0.stats.FramesDroppedDueToFilters: 0
> dev.bge.0.stats.DmaWriteQueueFull: 310574781
> dev.bge.0.stats.DmaWriteHighPriQueueFull: 0
> dev.bge.0.stats.NoMoreRxBDs: 0
> dev.bge.0.stats.InputDiscards: 20942213
> dev.bge.0.stats.InputErrors: 15
> dev.bge.0.stats.RecvThresholdHit: 51440366
> dev.bge.0.stats.DmaReadQueueFull: 0
> dev.bge.0.stats.DmaReadHighPriQueueFull: 0
> dev.bge.0.stats.SendDataCompQueueFull: 0
> dev.bge.0.stats.RingSetSendProdIndex: 25738223
> dev.bge.0.stats.RingStatusUpdate: 51719806
> dev.bge.0.stats.Interrupts: 51719806
> dev.bge.0.stats.AvoidedInterrupts: 0
> dev.bge.0.stats.SendThresholdHit: 0
> dev.bge.0.stats.rx.Octets: 3477501149
> dev.bge.0.stats.rx.Fragments: 116
> dev.bge.0.stats.rx.UcastPkts: 519429709
> dev.bge.0.stats.rx.MulticastPkts: 0
> dev.bge.0.stats.rx.FCSErrors: 3
> dev.bge.0.stats.rx.AlignmentErrors: 0
> dev.bge.0.stats.rx.xonPauseFramesReceived: 0
> dev.bge.0.stats.rx.xoffPauseFramesReceived: 0
> dev.bge.0.stats.rx.ControlFramesReceived: 0
> dev.bge.0.stats.rx.xoffStateEntered: 0
> dev.bge.0.stats.rx.FramesTooLong: 0
> dev.bge.0.stats.rx.Jabbers: 0
> dev.bge.0.stats.rx.UndersizePkts: 0
> dev.bge.0.stats.rx.inRangeLengthError: 0
> dev.bge.0.stats.rx.outRangeLengthError: 0
> dev.bge.0.stats.tx.Octets: 2215096864
> dev.bge.0.stats.tx.Collisions: 0
> dev.bge.0.stats.tx.XonSent: 0
> dev.bge.0.stats.tx.XoffSent: 0
> dev.bge.0.stats.tx.flowControlDone: 0
> dev.bge.0.stats.tx.InternalMacTransmitErrors: 0
> dev.bge.0.stats.tx.SingleCollisionFrames: 0
> dev.bge.0.stats.tx.MultipleCollisionFrames: 0
> dev.bge.0.stats.tx.DeferredTransmissions: 0
> dev.bge.0.stats.tx.ExcessiveCollisions: 0
> dev.bge.0.stats.tx.LateCollisions: 0
> dev.bge.0.stats.tx.UcastPkts: 25738364
> dev.bge.0.stats.tx.MulticastPkts: 0
> dev.bge.0.stats.tx.BroadcastPkts: 15
> dev.bge.0.stats.tx.CarrierSenseErrors: 0
> dev.bge.0.stats.tx.Discards: 0
> dev.bge.0.stats.tx.Errors: 0
>
> errors..ERRORS!)(@!*()
>
> Hitting that 400k pps limit again, and this is a slower machine than
> my dual 2212.
> Going to compile 7-STABLE with options for the cpu and will report back
>
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4869B025.9080006>