Date: Sun, 10 May 2015 06:41:38 +0200 From: "Mark Schouten" <mark@tuxis.nl> To: Christopher Forgeron <csforgeron@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Frequent hickups on the networking layer Message-ID: <680E11E4-A7AE-4334-B903-8127D0622CE8@tuxis.nl> In-Reply-To: <CAB2_NwAR11OTM5N%2BS4A4om9Bfat%2BGbdBHMNJZ_Zg7EpmaJ5cKQ@mail.gmail.com> References: <137094161.27589033.1430255162390.JavaMail.root@uoguelph.ca> <5540889A.5030904@tuxis.nl> <21824.58754.452182.195043@hergotha.csail.mit.edu> <554A2D3D.3060408@tuxis.nl> <CAB2_NwAR11OTM5N%2BS4A4om9Bfat%2BGbdBHMNJZ_Zg7EpmaJ5cKQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, Yes, it did. I see no mbuf errors anymore, no Ethernet errors. Ctld does not= crash anymore, it kept running after lowering the mtu to 1500. I am using vlans and the weirdest thing when lowering the mtu was that every= thing went crazy when I only lowered the mtu for the vlan interface. Ctld wo= uld not start completely, pings started taking several hundred milliseconds.= It just wouldn't work anymore. Only when I lowered all interfaces to 1500, s= tuff was ok and has been since. I do see a lot of jumbo page allocations during backups at night, which migh= t be nfsd or ctld, but it's not causing any issues. I've learned, for now: FreeBSD and jumbo frames is a no-go.=20 Hope it helps for you too. Regards, --=20 Mark Schouten Tuxis Internet Engineering mark@tuxis.nl / 0318 200208 > On 10 May 2015, at 02:17, "Christopher Forgeron" <csforgeron@gmail.com> wr= ote: >=20 > Mark, did switching to a MTU of 1500 ever help? >=20 > I'm currently reliving a problem with this - I'm down to a MTU of 4000, bu= t I still see jumbo pages being allocated - I believe it's my iSCSI setup (u= sing 4k block size, which means the packet is bigger than 4k), but I'm not s= ure where it's all coming from yet. >=20 > I'm on 10.1 RELEASE fyi.=20 >=20 > I'm going to patch my network devs to not use MJUM9BYTES and see if that h= as an effect. >=20 > For me, the problem all started again once I really started putting storag= e load on the FreeBSD machines. At times, I'm seeing 7 Gbits on the 10 Gbit a= dapters.=20 >=20 > Oh, and there are gremlins in the new ctld / iscsi as well. I'll get into t= hat later, but if a heavily loaded iscsi target goes down, when it reboots, t= he reconnect storm from all the iscsi machines kernel panics the FreeBSD isc= si target host. My machine looped through three boot-start-panic loops befo= re I caught it and put it into single-user mode. Starting ctld manually seem= s to make everything okay.=20 >=20 >=20 >=20 >=20 >> On Wed, May 6, 2015 at 12:03 PM, Mark Schouten <mark@tuxis.nl> wrote: >> Hi, >>=20 >> On 04/29/2015 04:06 PM, Garrett Wollman wrote: >>=20 >>> If you're using one of the drivers that has this problem, then yes, >>> keeping your layer-2 MTU/MRU below 4096 will probably cause it to use >>> 4k (page-sized) clusters instead, which are perfectly safe. >>>=20 >>> As a side note, at least on the hardware I have to support, Infiniband >>> is limited to 4k MTU -- so I have one "jumbo" network with 4k frames >>> (that's bridged to IB) and one with 9k frames (that everything else >>> uses). >>=20 >> So I was thinking, a customer of mine runs mostly the same setup, and has= no issues at all. The only difference, MTU of 1500 vs MTU of 9000. >>=20 >> I also created a graph in munin, graphing the number of mbuf_jumbo reques= ts and failures. I find that when lots of writes occur to the iscsi-layer, t= he number of failed requests grow, and so so the number of errors on the eth= ernet interface. See attached images. My customer is also not suffering from= crashing ctld-daemons, which crashes every other minute in my setup. >>=20 >> So tonight I'm going to switch to an MTU of 1500, I'll let you know if th= at helped. >>=20 >>=20 >> Regards, >>=20 >> Mark Schouten >>=20 >>=20 >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?680E11E4-A7AE-4334-B903-8127D0622CE8>