Date: Sat, 4 May 2013 13:47:20 -0700 From: Jack Vogel <jfvogel@gmail.com> To: Richard Sharpe <realrichardsharpe@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Adrian Chadd <adrian@freebsd.org> Subject: Re: Is there any way to limit the amount of data in an mbuf chain submitted to a driver? Message-ID: <CAFOYbcmUj24-bT29o_bu0ecRAfPud-tUZmUc1GW0SXZm0rSToQ@mail.gmail.com> In-Reply-To: <CAFOYbcmZMW=-7Mwz9mwJLeM3Ju%2BF8_AsXFAPqCa8%2BuuRWq3xsg@mail.gmail.com> References: <CACyXjPwC5LRb7DT82n6PMbawceER3_nHko9c9tvrdQqceLiPww@mail.gmail.com> <CAJ-Vmon_5eyXMP5UOsVVBP8UgKQLw5HLMO1NgswoGb-zF=2wtg@mail.gmail.com> <CACyXjPzu3fXpo0i5YcdVBFye%2BRFTPUye=fgZ%2BycTkkiEmcRh%2BQ@mail.gmail.com> <CAFOYbcmZMW=-7Mwz9mwJLeM3Ju%2BF8_AsXFAPqCa8%2BuuRWq3xsg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Yes, I checked: #define IXGBE_TSO_SIZE 262140 So, the driver is not limiting you to 64K assuming you are using a version of recent vintage. Jack On Sat, May 4, 2013 at 1:41 PM, Jack Vogel <jfvogel@gmail.com> wrote: > If you don't use TSO you will hurt your TX performance significantly from > the tests that I've run. What exactly is the device you are using, I don'= t > have the source in front of me now, but I'm almost sure that the limit is > not 64K but 256K, or are you using some ancient version of the driver? > > Jack > > > > On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe < > realrichardsharpe@gmail.com> wrote: > >> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd <adrian@freebsd.org> wrote= : >> > On 4 May 2013 06:52, Richard Sharpe <realrichardsharpe@gmail.com> >> wrote: >> >> Hi folks, >> >> >> >> I understand better why I am seeing EINVAL intermittently when sendin= g >> >> data from Samba via SMB2. >> >> >> >> The ixgbe driver, for TSO reasons, limits the amount of data that can >> >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger >> >> than that. >> >> >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is less >> >> than 64kiB of space available, so that is all TCP etc can put into th= e >> >> socket in one chain of mbufs. However, every now and then there is >> >> more than 65535 bytes available in the socket buffers, and we have an >> >> SMB packet that is larger than 65535 bytes, and we get hit. >> >> >> >> To confirm this I am going to set SO_SNDBUF back to the default of >> >> 65536 and test again. My repros are very reliable. >> >> >> >> However, I wondered if my only way around this if I want to continue >> >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf >> >> chains in the driver? >> > >> > Hm, is this is a problem without TSO? >> >> We are using the card without TSO, so I am thinking of changing that >> limit to 131072 and retesting. >> >> I am currently testing with SO_SNDBUF=3D32768 and have not hit the probl= em. >> >> > Is the problem that the NIC can't handle a frame that big, or a buffer >> that big? >> > Ie - if you handed the hardware two descriptors of 64k each, for the >> > same IP datagram, will it complain? >> >> I can't find any documentation, but it seems that with TSO it cannot >> handle a frame that big. Actually, since we are not using TSO, there >> really should not be a problem with larger frames. >> >> > Or do you need to break it up into two separate IP datagrams, facing >> > the driver, with a maximum size of 64k each? >> >> Not sure, but it looks like we need to do that. >> >> >> -- >> Regards, >> Richard Sharpe >> (=A6=F3=A5H=B8=D1=BC~=A1H=B0=DF=A6=B3=A7=F9=B1d=A1C--=B1=E4=BE=DE) >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFOYbcmUj24-bT29o_bu0ecRAfPud-tUZmUc1GW0SXZm0rSToQ>