Date: Sun, 25 Jun 2017 11:32:00 -0400 From: Ryan Stone <rysto32@gmail.com> To: Ben RUBSON <ben.rubson@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org>, "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org> Subject: Re: mbuf_jumbo_9k & iSCSI failing Message-ID: <CAFMmRNwbEwn=TmTAd56rViDV5nDXq_hPmTp-cDwmVqu1XYm=fA@mail.gmail.com> In-Reply-To: <CAFMmRNzo=xB8XF6SFD%2BwksmBYjRZ_peYjiPBCXNVyqP%2BdxnujQ@mail.gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <DB3PR05MB089A5789A0A619FA8B7CA36C36C0@DB3PR05MB089.eurprd05.prod.outlook.com> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <DB3PR05MB089011A41EF87A40C7AC741C36E0@DB3PR05MB089.eurprd05.prod.outlook.com> <F19B51C7-7DDD-4FAB-9091-0B7C8A7CE649@gmail.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <CAFMmRNzo=xB8XF6SFD%2BwksmBYjRZ_peYjiPBCXNVyqP%2BdxnujQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Having looking at the original email more closely, I see that you showed an mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf clusters increase while you are far below the zone's limit means that you're definitely running into the bug I'm describing, and this bug could plausibly cause the iSCSI errors that you describe. The issue is that the newer version of the driver tries to allocate a single buffer to accommodate an MTU-sized packet. Over time, however, memory will become fragmented and eventually it can become impossible to allocate a 9k physically contiguous buffer. When this happens the driver is unable to allocate buffers to receive packets and is forced to drop them. Presumably, if iSCSI suffers too many packet drops it will terminate the connection. The older version of the driver limited itself to page-sized buffers, so it was immune to issues with memory fragmentation.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFMmRNwbEwn=TmTAd56rViDV5nDXq_hPmTp-cDwmVqu1XYm=fA>