Date: Tue, 9 Mar 2010 14:04:57 -0800 From: "David Christensen" <davidch@broadcom.com> To: "pyunyh@gmail.com" <pyunyh@gmail.com> Cc: Ian FREISLICH <ianf@clue.co.za>, "current@freebsd.org" <current@freebsd.org>, David Christensen <davidch@freebsd.org> Subject: RE: dev.bce.X.com_no_buffers increasing and packet loss Message-ID: <5D267A3F22FD854F8F48B3D2B52381933AF90EEDA7@IRVEXCHCCR01.corp.ad.broadcom.com> In-Reply-To: <20100309214012.GQ1311@michelle.cdnetworks.com> References: <20100305210435.GF14818@michelle.cdnetworks.com> <20100305184046.GD14818@michelle.cdnetworks.com> <20100305175639.GB14818@michelle.cdnetworks.com> <E1NnVaT-0003Ft-3p@clue.co.za> <E1Nnc4d-0003mB-6e@clue.co.za> <E1Nne0Q-0003uZ-OR@clue.co.za> <E1Noulp-0007Rc-Ro@clue.co.za> <20100309212139.GO1311@michelle.cdnetworks.com> <5D267A3F22FD854F8F48B3D2B52381933AF90EED69@IRVEXCHCCR01.corp.ad.broadcom.com> <20100309214012.GQ1311@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > > It's been running for about 1:23 on the patched driver.=20 > I'm still=20 > > > > seeing the com_no_buffers increase: > > > >=20 > > > > [firewall2.jnb1] ~ # sysctl dev.bce |grep com_no_buffers > > > > dev.bce.0.com_no_buffers: 5642 > > > > dev.bce.1.com_no_buffers: 497 > > > > dev.bce.2.com_no_buffers: 6260612 > > > > dev.bce.3.com_no_buffers: 4871338 > > > >=20 > > >=20 > > > Still have no idea why these counters are increasing here. > > > Actually the counter is read from scratch pad of completion=20 > > > processor. The datasheet does not even document the counter. > > > Maybe david know better what's happening here(CCed). > > >=20 > > > > Interupt rate is down now, at about 3500 per second per=20 > interface. > > > >=20 > > > > Interestingly setting net.inet.ip.fastforwarding=3D0 reduces CPU=20 > > > > consumption from 25% to 9% and less packet loss. > >=20 > > The com_no_buffers statistic comes from firmware and indicates how=20 > > many times a valid frame was received but could not be=20 > placed into the=20 > > receive chain because there were no available RX buffers. The=20 > > firmware will then drop the frame but that dropped frame won't be=20 > > reflected in any of the hardware based statistics. > >=20 >=20 > Yeah, but the question is why bce(4) has no available RX buffers. > The system has a lot of available mbufs so I don't see the=20 > root cause here. What's the traffic look like? Jumbo, standard, short frames? Any=20 good ideas on profiling the code? I haven't figured out how to use the CPU TSC but there is a free running timer on the device that might be usable to calculate where the driver's time is spent. Dave=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5D267A3F22FD854F8F48B3D2B52381933AF90EEDA7>