Date: Fri, 27 Jul 2018 15:18:43 -0700 From: John-Mark Gurney <jmg@funkthat.com> To: Ryan Moeller <ryan@ixsystems.com> Cc: freebsd-net@freebsd.org Subject: Re: 9k jumbo clusters Message-ID: <20180727221843.GZ2884@funkthat.com> In-Reply-To: <EBDE6EDD-D875-43D8-8D65-1F1344A6B817@ixsystems.com> References: <EBDE6EDD-D875-43D8-8D65-1F1344A6B817@ixsystems.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Ryan Moeller wrote this message on Fri, Jul 27, 2018 at 12:45 -0700: > There is a long-standing issue with 9k mbuf jumbo clusters in FreeBSD. > For example: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183381 > https://lists.freebsd.org/pipermail/freebsd-net/2013-March/034890.html > > This comment suggests the 16k pool does not have the fragmentation problem: > https://reviews.freebsd.org/D11560#239462 > I???m curious whether that has been confirmed. > > Is anyone working on the pathological case with 9k jumbo clusters in the > physical memory allocator? There was an interesting discussion started a > few years ago but I???m not sure what ever came of it: > http://docs.freebsd.org/cgi/mid.cgi?21225.20047.947384.390241 > > I have seen some work in the direction of avoiding larger than page size > jumbo clusters in 12-CURRENT. Many existing drivers avoid the 9k cluster > size already. The code for larger cluster sizes in iflib is #ifdef'd out > so it maxes out at the page size jumbo clusters until "CONTIGMALLOC_WORKS" > (apparently it doesn't). > > With all the changes due to iflib, is there any chance some of this will > get MFC'd to address the serious problem that remains in 11-STABLE? > > Otherwise, would it be feasible to disable the use of the 9k cluster pool > in at least some of the popular NIC drivers as a solution for the stable > branches? > > Finally, I have studied some of the driver code in 11-STABLE and posted the > gist of my notes in relation to this problem. If anyone spots a mistake or > has something else to contribute, comments on the gist would be greatly > appreciated! > https://gist.github.com/freqlabs/eba9b755f17a223260246becfbb150a1 Drivers need to be fixed to use 4k pages instead of cluster. I really hope no one is using a card that can't do 4k pages, or if they are, then they should get a real card that can do scatter/gather on 4k pages for jumbo frames.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180727221843.GZ2884>