Date: Thu, 13 Feb 2014 10:36:24 -0800 From: Adrian Chadd <adrian@freebsd.org> To: Garrett Wollman <wollman@bimajority.org> Cc: FreeBSD Net <freebsd-net@freebsd.org>, John-Mark Gurney <jmg@funkthat.com> Subject: Re: Use of contiguous physical memory in cxgbe driver Message-ID: <CAJ-Vmo=odbLSWZ46NrFYNxuvx9Mht5T4hbwCfCp0xt91oWP9fg@mail.gmail.com> In-Reply-To: <21245.2981.568824.19800@hergotha.csail.mit.edu> References: <21216.22944.314697.179039@hergotha.csail.mit.edu> <201402111348.52135.jhb@freebsd.org> <CAJ-VmonCdNQPUCQwm0OhqQ3Kt_7x6-g-JwGVZQfzWTgrDYfmqw@mail.gmail.com> <201402121446.19278.jhb@freebsd.org> <21244.20212.423983.960018@hergotha.csail.mit.edu> <20140213075651.GY34851@funkthat.com> <21245.1163.754141.154430@hergotha.csail.mit.edu> <20140213180736.GA34851@funkthat.com> <21245.2981.568824.19800@hergotha.csail.mit.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On 13 February 2014 10:15, Garrett Wollman <wollman@bimajority.org> wrote: > <<On Thu, 13 Feb 2014 10:07:36 -0800, John-Mark Gurney <jmg@funkthat.com> said: > >>> [I wrote:] >>> It's very easy to get to that state if the driver insists on getting >>> three physically contiguous pages (which is what it takes to allocate >>> a single 9k cluster) > >> Well, if you're using a cheap NIC that can't do scatter/gather DMA, >> then you get what you pay for... > > No, I'm using expensive 10G NICs, like the Chelsio ones mentioned in > the subject header. They have no trouble doing s/g (subject to some > reasonable limits on chain length). The problem is when the drivers > insist on physical contiguity anyway. > > I finally gave up and hacked the cxgbe driver never to use anything > longer than a physical page. Have you tried it at 40G? ISTR np@ commenting on there being benefits to larger (up to 64K) allocations for the 40G T5 NICs that do packet -> buffer packing on receive side. -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=odbLSWZ46NrFYNxuvx9Mht5T4hbwCfCp0xt91oWP9fg>