Date: Sat, 9 May 2015 21:16:57 -0300 From: Christopher Forgeron <csforgeron@gmail.com> To: Mark Schouten <mark@tuxis.nl> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Frequent hickups on the networking layer Message-ID: <CAB2_NwAR11OTM5N%2BS4A4om9Bfat%2BGbdBHMNJZ_Zg7EpmaJ5cKQ@mail.gmail.com> In-Reply-To: <554A2D3D.3060408@tuxis.nl> References: <137094161.27589033.1430255162390.JavaMail.root@uoguelph.ca> <5540889A.5030904@tuxis.nl> <21824.58754.452182.195043@hergotha.csail.mit.edu> <554A2D3D.3060408@tuxis.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
Mark, did switching to a MTU of 1500 ever help? I'm currently reliving a problem with this - I'm down to a MTU of 4000, but I still see jumbo pages being allocated - I believe it's my iSCSI setup (using 4k block size, which means the packet is bigger than 4k), but I'm not sure where it's all coming from yet. I'm on 10.1 RELEASE fyi. I'm going to patch my network devs to not use MJUM9BYTES and see if that has an effect. For me, the problem all started again once I really started putting storage load on the FreeBSD machines. At times, I'm seeing 7 Gbits on the 10 Gbit adapters. Oh, and there are gremlins in the new ctld / iscsi as well. I'll get into that later, but if a heavily loaded iscsi target goes down, when it reboots, the reconnect storm from all the iscsi machines kernel panics the FreeBSD iscsi target host. My machine looped through three boot-start-panic loops before I caught it and put it into single-user mode. Starting ctld manually seems to make everything okay. On Wed, May 6, 2015 at 12:03 PM, Mark Schouten <mark@tuxis.nl> wrote: > Hi, > > On 04/29/2015 04:06 PM, Garrett Wollman wrote: > > If you're using one of the drivers that has this problem, then yes, >> keeping your layer-2 MTU/MRU below 4096 will probably cause it to use >> 4k (page-sized) clusters instead, which are perfectly safe. >> >> As a side note, at least on the hardware I have to support, Infiniband >> is limited to 4k MTU -- so I have one "jumbo" network with 4k frames >> (that's bridged to IB) and one with 9k frames (that everything else >> uses). >> > > So I was thinking, a customer of mine runs mostly the same setup, and has > no issues at all. The only difference, MTU of 1500 vs MTU of 9000. > > I also created a graph in munin, graphing the number of mbuf_jumbo > requests and failures. I find that when lots of writes occur to the > iscsi-layer, the number of failed requests grow, and so so the number of > errors on the ethernet interface. See attached images. My customer is also > not suffering from crashing ctld-daemons, which crashes every other minute > in my setup. > > So tonight I'm going to switch to an MTU of 1500, I'll let you know if > that helped. > > > Regards, > > Mark Schouten > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAB2_NwAR11OTM5N%2BS4A4om9Bfat%2BGbdBHMNJZ_Zg7EpmaJ5cKQ>