Date: Sat, 12 Mar 2011 21:17:28 +0100 From: Vlad Galu <dudu@dudu.ro> To: Arnaud Lacombe <lacombar@gmail.com> Cc: freebsd-net@freebsd.org Subject: Re: bge(4) on RELENG_8 mbuf cluster starvation Message-ID: <AANLkTi=rWobA40UtCTSeOzEz65TMw8vfCcxtMWBBme%2Bu@mail.gmail.com> In-Reply-To: <AANLkTimfh3OdXOe1JFo5u6JypcLrcWKv2WpSu8Uv-tgv@mail.gmail.com> References: <AANLkTimSs48ftRv8oh1wTwMEpgN1Ny3B1ahzfS=AbML_@mail.gmail.com> <AANLkTimfh3OdXOe1JFo5u6JypcLrcWKv2WpSu8Uv-tgv@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Mar 12, 2011 at 8:53 PM, Arnaud Lacombe <lacombar@gmail.com> wrote: > Hi, > > On Sat, Mar 12, 2011 at 4:03 AM, Vlad Galu <dudu@dudu.ro> wrote: > > Hi folks, > > > > On a fairly busy recent (r219010) RELENG_8 machine I keep getting > > -- cut here -- > > 1096/1454/2550 mbufs in use (current/cache/total) > > 1035/731/1766/262144 mbuf clusters in use (current/cache/total/max) > > 1035/202 mbuf+clusters out of packet secondary zone in use > (current/cache) > > 0/117/117/12800 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > > 2344K/2293K/4637K bytes allocated to network (current/cache/total) > > 0/70128196/37726935 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > > ^^^^^^^^^^^^^^^^^^^^^ > > -- and here -- > > > > kern.ipc.nmbclusters is set to 131072. Other settings: > no, netstat(8) says 262144. > > Heh, you're right, I forgot I'd doubled it a while ago. Wrote that from the top of my head. > Maybe can you include $(sysctl dev.bge) ? Might be useful. > > - Arnaud > Sure: -- cut here -- dev.bge.0.%desc: Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x004101 dev.bge.0.%driver: bge dev.bge.0.%location: slot=0 function=0 dev.bge.0.%pnpinfo: vendor=0x14e4 device=0x1659 subvendor=0x1014 subdevice=0x02c6 class=0x020000 dev.bge.0.%parent: pci4 dev.bge.0.forced_collapse: 2 dev.bge.0.forced_udpcsum: 0 dev.bge.0.stats.FramesDroppedDueToFilters: 0 dev.bge.0.stats.DmaWriteQueueFull: 0 dev.bge.0.stats.DmaWriteHighPriQueueFull: 0 dev.bge.0.stats.NoMoreRxBDs: 0 dev.bge.0.stats.InputDiscards: 0 dev.bge.0.stats.InputErrors: 0 dev.bge.0.stats.RecvThresholdHit: 0 dev.bge.0.stats.rx.ifHCInOctets: 68037590571 dev.bge.0.stats.rx.Fragments: 0 dev.bge.0.stats.rx.UnicastPkts: 85983136 dev.bge.0.stats.rx.MulticastPkts: 0 dev.bge.0.stats.rx.BroadcastPkts: 1 dev.bge.0.stats.rx.FCSErrors: 0 dev.bge.0.stats.rx.AlignmentErrors: 0 dev.bge.0.stats.rx.xonPauseFramesReceived: 0 dev.bge.0.stats.rx.xoffPauseFramesReceived: 0 dev.bge.0.stats.rx.ControlFramesReceived: 0 dev.bge.0.stats.rx.xoffStateEntered: 0 dev.bge.0.stats.rx.FramesTooLong: 0 dev.bge.0.stats.rx.Jabbers: 0 dev.bge.0.stats.rx.UndersizePkts: 0 dev.bge.0.stats.tx.ifHCOutOctets: 91957314255 dev.bge.0.stats.tx.Collisions: 0 dev.bge.0.stats.tx.XonSent: 0 dev.bge.0.stats.tx.XoffSent: 0 dev.bge.0.stats.tx.InternalMacTransmitErrors: 0 dev.bge.0.stats.tx.SingleCollisionFrames: 0 dev.bge.0.stats.tx.MultipleCollisionFrames: 0 dev.bge.0.stats.tx.DeferredTransmissions: 0 dev.bge.0.stats.tx.ExcessiveCollisions: 0 dev.bge.0.stats.tx.LateCollisions: 0 dev.bge.0.stats.tx.UnicastPkts: 624045221 dev.bge.0.stats.tx.MulticastPkts: 10841 dev.bge.0.stats.tx.BroadcastPkts: 1121 dev.bge.1.%desc: Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x004101 dev.bge.1.%driver: bge dev.bge.1.%location: slot=0 function=0 dev.bge.1.%pnpinfo: vendor=0x14e4 device=0x1659 subvendor=0x1014 subdevice=0x02c6 class=0x020000 dev.bge.1.%parent: pci5 dev.bge.1.forced_collapse: 2 dev.bge.1.forced_udpcsum: 0 dev.bge.1.stats.FramesDroppedDueToFilters: 0 dev.bge.1.stats.DmaWriteQueueFull: 0 dev.bge.1.stats.DmaWriteHighPriQueueFull: 0 dev.bge.1.stats.NoMoreRxBDs: 680050 dev.bge.1.stats.InputDiscards: 228755931 dev.bge.1.stats.InputErrors: 49080818 dev.bge.1.stats.RecvThresholdHit: 0 dev.bge.1.stats.rx.ifHCInOctets: 2095148839247 dev.bge.1.stats.rx.Fragments: 47887706 dev.bge.1.stats.rx.UnicastPkts: 32672557601 dev.bge.1.stats.rx.MulticastPkts: 1218 dev.bge.1.stats.rx.BroadcastPkts: 2 dev.bge.1.stats.rx.FCSErrors: 2822217 dev.bge.1.stats.rx.AlignmentErrors: 0 dev.bge.1.stats.rx.xonPauseFramesReceived: 0 dev.bge.1.stats.rx.xoffPauseFramesReceived: 0 dev.bge.1.stats.rx.ControlFramesReceived: 0 dev.bge.1.stats.rx.xoffStateEntered: 0 dev.bge.1.stats.rx.FramesTooLong: 0 dev.bge.1.stats.rx.Jabbers: 0 dev.bge.1.stats.rx.UndersizePkts: 0 dev.bge.1.stats.tx.ifHCOutOctets: 48751515826 dev.bge.1.stats.tx.Collisions: 0 dev.bge.1.stats.tx.XonSent: 0 dev.bge.1.stats.tx.XoffSent: 0 dev.bge.1.stats.tx.InternalMacTransmitErrors: 0 dev.bge.1.stats.tx.SingleCollisionFrames: 0 dev.bge.1.stats.tx.MultipleCollisionFrames: 0 dev.bge.1.stats.tx.DeferredTransmissions: 0 dev.bge.1.stats.tx.ExcessiveCollisions: 0 dev.bge.1.stats.tx.LateCollisions: 0 dev.bge.1.stats.tx.UnicastPkts: 281039183 dev.bge.1.stats.tx.MulticastPkts: 0 dev.bge.1.stats.tx.BroadcastPkts: 1153 -- and here -- And now, that I remembered about this as well: -- cut here -- Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll bge1 1500 <Link#2> 00:11:25:22:0d:ed 32321767025 278517070 37726837 281068216 0 0 -- and here -- The colo provider changed my cable a couple of times so I'd not blame it on that. Unfortunately, I don't have access to the port statistics on the switch. Running netstat with -w1 yields between 0 and 4 errors/second. -- Good, fast & cheap. Pick any two.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTi=rWobA40UtCTSeOzEz65TMw8vfCcxtMWBBme%2Bu>