Date: Tue, 31 Mar 2015 19:01:52 -0700 From: hiren panchasara <hiren@strugglingcoder.info> To: freebsd-net@freebsd.org Cc: jfv@freebsd.org, erj@freebsd.org, nitroboost@gmail.com Subject: Re: em(4): difference between missed_packets and rx_overrun Message-ID: <20150401020152.GM10892@strugglingcoder.info> In-Reply-To: <20150327173334.GB39674@strugglingcoder.info> References: <20150326200853.GA19536@strugglingcoder.info> <20150327173334.GB39674@strugglingcoder.info>
next in thread | previous in thread | raw e-mail | index | archive | help
--kunpHVz1op/+13PW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 03/27/15 at 10:33P, hiren panchasara wrote: > + jfv, erj from Intel. >=20 > On 03/26/15 at 01:08P, hiren panchasara wrote: > > This is what we are seeing on em(4) 82574L chipset running stable/10: > >=20 > > dev.em.0.mac_stats.missed_packets: 1441927 > > dev.em.0.interrupts.rx_overrun: 153 > >=20 > > From the datasheet: > > http://www.intel.com/content/www/us/en/ethernet-controllers/82574l-gbe-= controller-datasheet.html > >=20 > > 10.2.7.4 Missed Packets Count - MPC (0x04010; R) > > Counts the number of missed packets. Packets are missed when the receive > > FIFO has insufficient space to store the incoming packet. This could be > > caused because of too few buffers allocated, or because there is > > insufficient bandwidth on the IO bus. Events setting this counter > > cause RXO, the receiver overrun interrupt, to be set. This register > > does not increment if receives are not enabled. > >=20 > > 10.2.4.1 Interrupt Cause Read Register - ICR (0x000C0; RC/WC) > > RXO Receiver Overrun > > Set on receive data FIFO overrun. Could be caused either because > > there are no available buffers or because PCIe receive bandwidth is > > inadequate. > >=20 > > So, first one is a count and another one is an interrupt. Are these 2 > > related? Both seem to be happen when on card FIFO gets full. We see no > > evidence of RX queue on the host being full based on > > dev.em.0.mac_stats.recv_no_buff. > >=20 > > Many a times we see missed_packets increasing without rx_overrun > > changing. > >=20 > > The spec says there is a 40KB buffer on card which seems to be used by > > both RX and TX? Is is split between them for 20KB each? OR is it > > possible that when we are doing high rate TX, we use up that buffer and > > RX suffers from that? So, I found some seemingly useful things in the spec: 10.2.12 Diagnostic Register Descriptions I tried to read on-chip FIFO head/tail to make a sense of what we have there when we see mpc (missed packet count) increase. In em_update_stats_counters() whenever I see mpc increasing, I go print following registers. fhead =3D E1000_READ_REG(&adapter->hw, E1000_RDFH); ftail =3D E1000_READ_REG(&adapter->hw, E1000_RDFT); fheads =3D E1000_READ_REG(&adapter->hw, E1000_RDFHS); ftails =3D E1000_READ_REG(&adapter->hw, E1000_RDFTS); fcount =3D E1000_READ_REG(&adapter->hw, E1000_RDFPC); But I do not see any pattern here. Some times I see a large differenec between head and tail and still no mpc increase and sometimes mpc++ when head/tail are following eachother closely. Also, fcount E1000_RDFPC which is supposed to be "The number of received packets currently in the RX FIFO." is always 0 OR a moderately low number. That doesn't make sense to me. If FIFO is not full why do we get so many missed packets? cheers, Hiren --kunpHVz1op/+13PW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQF8BAEBCgBmBQJVG1GPXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lLSYH/27M2X+5Kj3YCgPH5kEWMyts +kO7dOrPPLa1VOR1qC++M1SE3+0N9m7TSSAXe0NLretU1ln+v+2G94xAOoRh7IXl p+07fO/zMgqwsKVjzsahkSC7c8L/EdXgWd5kT5XKtuV6PyONgXUVTcaFFwDgJ74v g/w6CgsqVURKgH+n/uteG77VCZkrKzQ/HIvUMKOnjXw53HmbwdeAF3SmH7W2Hslq EeyYdH3lmxxWjiQW3Yb/JpXP6Ig2JiElsvjrsPLopKkVk9TKhigWQT5RDjOHtTmc HVz0kCRVhXUQbG5DDGlShnfig+TVJPmAmpmGxgkCgHdR38BEY+KX1peGICISZoU= =2bW7 -----END PGP SIGNATURE----- --kunpHVz1op/+13PW--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150401020152.GM10892>