Date: Tue, 18 Aug 2009 16:35:04 -0600 From: Manish Vachharajani <manishv@lineratesystems.com> To: Barney Cordoba <barney_cordoba@yahoo.com> Cc: freebsd-net@freebsd.org Subject: Re: Dropped vs. missed packets in the ixgbe driver Message-ID: <5bc218350908181535o7c5275dfn2f6647454cfac804@mail.gmail.com> In-Reply-To: <373149.52091.qm@web63907.mail.re1.yahoo.com> References: <5bc218350908171524m5a46c3dbm3e6af625c51370d0@mail.gmail.com> <373149.52091.qm@web63907.mail.re1.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Indeed the debugging info is also interesting. However, I'd like to get some data from netstat when the driver drops frames. When looking at the bge driver source, it appears that all input drops at the NIC are reported as input errors. It appears that the intel drivers (e1000, ixgb, and ixgbe) don't report that number outside of the debug and stats printfs at all, and this seems broken. What I want to know is if I have just missed where these are reported. So, in a nutshell, the question is: should these drivers be reporting miss events as input errors in the ifnet struct as the bge driver does, or as drops in the ifnet struct, was there some conscious decision not to report miss events anywhere outside the debug and stats info, or am I just being silly and not seeing where the numbers are reported? Also, don't worry on the performance front, we are also looking at the driver in FreeBSD 8.0 :) which supports RSS to help performance scaling, though we have some interesting data there that I'll post about once I confirm that the numbers are indeed correct and not a tuning or setup problem. Manish > --- On Mon, 8/17/09, Manish Vachharajani <manishv@lineratesystems.com> wr= ote: > >> From: Manish Vachharajani <manishv@lineratesystems.com> >> Subject: Dropped vs. missed packets in the ixgbe driver >> To: freebsd-net@freebsd.org >> Date: Monday, August 17, 2009, 6:24 PM >> I've been doing some performance >> testing on freebsd 7.2 and noticed >> that the ixgbe driver does not report missed packets as >> dropped when >> queried via netstat -id (the ixgbe driver in the 8.0 >> beta has a >> similar issue).=A0 A missed packet is a packet that was >> correctly >> received by the NIC but because it was out of descriptors >> and internal >> memory, the packet had to be dropped by the NIC itself -- a >> hardware >> register counts such events.=A0 Instead the driver only >> reports drops >> that are due to the driver itself, i.e., the NIC DMAed the >> packet to >> memory but the driver had to drop something because it was >> out of >> mbufs.=A0 Is the miss count reported elsewhere (besides >> via a kernel >> printf from the driver)?=A0 At the end of the email I >> give a stats dump >> from the driver and from netstat, note the number of >> packets dropped >> is 0 as reported by netstat but the Missed field in the >> dmesg output >> shows many missed packets. >> >> >From my perspective, it is disconcerting to see >> performance >> degradation on the link, along with TCP ack retransmits, >> packet >> reordering, etc. (on a point-to-point link with no switch >> in between) >> but then see no drops reported by netstat because the >> driver didn't >> drop the packet, the NIC did.=A0 The fix should be >> straight-forward and >> I'll gladly make a patch assuming that it is indeed a bug >> and not a >> conscious design choice. >> >> Here is the relevant netstat output >> >> Name=A0 =A0 Mtu Network >> =A0=A0=A0Address >> =A0 =A0 Ipkts Ierrs=A0 =A0 Opkts >> Oerrs=A0 Coll Drop >> ix0=A0 =A0 1500 <Link#1> >> 00:30:48:94:60:ec=A0 =A0 =A0 =A0 0 >> =A0=A0=A00=A0 =A0 =A0 =A0 1 >> =A00=A0 =A0=A0=A00=A0 =A0 0 >> ix0=A0 =A0 1500 192.168.105.0 192.168.105.2 >> =A0 =A0 =A0 =A0 =A0 0 >> =A0=A0=A0-=A0 =A0 =A0 =A0 0 >> =A0-=A0 =A0=A0=A0-=A0 =A0 - >> ix1=A0 =A0 1500 <Link#2> >> 00:30:48:94:60:ed=A0 =A0 =A0 11M >> =A0=A0=A00=A0 =A0=A0=A06.1M >> =A00=A0 =A0=A0=A00=A0 =A0 0 >> ix1=A0 =A0 1500 >> 192.168.5.0=A0=A0=A0192.168.5.2 >> =A0 =A0 =A0 10M=A0 =A0=A0=A0- >> =A0=A0=A06.1M >> =A0-=A0 =A0=A0=A0-=A0 =A0 - >> >> And here is the dmesg output after doing a sysctl >> dev.ix.1.stats=3D1 >> >> ix1: Std Mbuf Failed =3D 0 >> ix1: Missed Packets =3D 413872 >> ix1: Receive length errors =3D 0 >> ix1: Crc errors =3D 0 >> ix1: Driver dropped packets =3D 0 >> ix1: watchdog timeouts =3D 0 >> ix1: XON Rcvd =3D 616428212235 >> ix1: XON Xmtd =3D 0 >> ix1: XOFF Rcvd =3D 616428212235 >> ix1: XOFF Xmtd =3D 0 >> ix1: Total Packets Rcvd =3D 12424533 >> ix1: Good Packets Rcvd =3D 12010661 >> ix1: Good Packets Xmtd =3D 6419128 >> ix1: TSO Transmissions =3D 0 >> >> Manish > > the debug sysctl show more interesting info. Don't get too excited > about doing performance testing with that driver. Its not designed > to be any higher in performance than any of the other intel drivers. > > Barney > > > > --=20 Manish Vachharajani Founder LineRate Systems manishv@lineratesystems.com (609)635-9531 M
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5bc218350908181535o7c5275dfn2f6647454cfac804>