Date: Thu, 20 Mar 2014 15:00:07 -0700 From: Jack Vogel <jfvogel@gmail.com> To: Christopher Forgeron <csforgeron@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Garrett Wollman <wollman@hergotha.csail.mit.edu> Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <CAFOYbckqw1wT41a-_3FMs6-KNVMV319nODtQ2F09eDRZavFPTg@mail.gmail.com> In-Reply-To: <CAB2_NwCjQ6qtx0SbEERVGs2Y_5pae-g=UbFEwkzmWYGoTuoP%2BQ@mail.gmail.com> References: <CAB2_NwDG=gB1WCJ7JKTHpkJCrvPuAhipkn%2BvPyT%2BxXzOBrTGkg@mail.gmail.com> <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <CAB2_NwAOmPtZjB03pdDiTK2OvQgqk-tYf83Jq4Ukt9jnZA8CNA@mail.gmail.com> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> <CAB2_NwCjQ6qtx0SbEERVGs2Y_5pae-g=UbFEwkzmWYGoTuoP%2BQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
What he's saying is that the driver should not be using 9K mbuf clusters, I thought this had been changed but I see the code in HEAD is still using the larger clusters when you up the mtu. I will put it on my list to change with the next update to HEAD. What version of ixgbe are you using? Jack On Thu, Mar 20, 2014 at 2:34 PM, Christopher Forgeron <csforgeron@gmail.com>wrote: > I have found this: > > http://lists.freebsd.org/pipermail/freebsd-net/2013-October/036955.html > > I think what you're saying is that; > - a MTU of 9000 doesn't need to equal a 9k mbuf / jumbo cluster > - modern NIC drivers can gather 9000 bytes of data from various memory > locations > - The fact that I'm seeing 9k jumbo clusters is showing me that my driver > is trying to allocate 9k of contiguous space, and it's failing. > > Please correct me if I'm off here, I'd love to understand more. > > > On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < > wollman@hergotha.csail.mit.edu> wrote: > > > In article > > <CAB2_NwAOmPtZjB03pdDiTK2OvQgqk-tYf83Jq4Ukt9jnZA8CNA@mail.gmail.com>, > > csforgeron@gmail.com writes: > > > > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) > > > > This is going to screw you. You need to make sure that no NIC driver > > ever allocates 9k jumbo pages -- unless you are using one of those > > mythical drivers that can't do scatter/gather DMA on receive, which > > you don't appear to be. > > > > These failures occur when the driver is trying to replenish its > > receive queue, but is unable to allocate three *physically* contiguous > > pages of RAM to construct the 9k jumbo cluster (of which the remaining > > 3k is simply wasted). This happens on any moderately active server, > > once physical memory gets checkerboarded with active single pages, > > particularly with ZFS where those pages are wired in kernel memory and > > so can't be evicted. > > > > -GAWollman > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFOYbckqw1wT41a-_3FMs6-KNVMV319nODtQ2F09eDRZavFPTg>