Date: Mon, 26 Jun 2017 15:44:58 +0200 From: Julien Cigar <julien@perdition.city> To: "Andrey V. Elsukov" <bu7cher@yandex.ru> Cc: Ryan Stone <rysto32@gmail.com>, Ben RUBSON <ben.rubson@gmail.com>, FreeBSD Net <freebsd-net@freebsd.org>, "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org> Subject: Re: mbuf_jumbo_9k & iSCSI failing Message-ID: <20170626134458.GT43966@mordor.lan> In-Reply-To: <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru> References: <DB3PR05MB089A5789A0A619FA8B7CA36C36C0@DB3PR05MB089.eurprd05.prod.outlook.com> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <DB3PR05MB089011A41EF87A40C7AC741C36E0@DB3PR05MB089.eurprd05.prod.outlook.com> <F19B51C7-7DDD-4FAB-9091-0B7C8A7CE649@gmail.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> <CAFMmRNzo=xB8XF6SFD%2BwksmBYjRZ_peYjiPBCXNVyqP%2BdxnujQ@mail.gmail.com> <CAFMmRNwbEwn=TmTAd56rViDV5nDXq_hPmTp-cDwmVqu1XYm=fA@mail.gmail.com> <64abec26-e310-d66d-93ae-3536914ddd84@yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
--TgCXP+xznsSrEyty Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 26, 2017 at 04:13:33PM +0300, Andrey V. Elsukov wrote: > On 25.06.2017 18:32, Ryan Stone wrote: > > Having looking at the original email more closely, I see that you showe= d an > > mlxen interface with a 9020 MTU. Seeing allocation failures of 9k mbuf > > clusters increase while you are far below the zone's limit means that > > you're definitely running into the bug I'm describing, and this bug cou= ld > > plausibly cause the iSCSI errors that you describe. > >=20 > > The issue is that the newer version of the driver tries to allocate a > > single buffer to accommodate an MTU-sized packet. Over time, however, > > memory will become fragmented and eventually it can become impossible to > > allocate a 9k physically contiguous buffer. When this happens the driv= er > > is unable to allocate buffers to receive packets and is forced to drop > > them. Presumably, if iSCSI suffers too many packet drops it will termi= nate > > the connection. The older version of the driver limited itself to > > page-sized buffers, so it was immune to issues with memory fragmentatio= n. >=20 > I think it is not mlxen specific problem, we have the same symptoms with > ixgbe(4) driver too. To avoid the problem we have patches that are > disable using of 9k mbufs, and instead only use 4k mbufs. I had the same issue on a lightly loaded HP DL20 machine (BCM5720=20 chipsets), 8GB of RAM, running 10.3. Problem usually happens within 30 days with 9k jumbo clusters allocation failure. >=20 > --=20 > WBR, Andrey V. Elsukov >=20 --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --TgCXP+xznsSrEyty Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEE7vn2l0to0nV7EWolsrs3EKIEI8AFAllRD9cACgkQsrs3EKIE I8Dj9w//XKLEOkjmTdf9HKiS5Dhe7nLJyFP5RFVXqSU4gx9b6oxh9jHKbBU+10iK v+yxFye/LtQKDabaOvwiGhMhhcYt2zVLWAKecpPwSxFE4KisW1KUw+PkzMXVoEP9 d9MXC3iPbTG3vNBFqZtK+VaXNQc3BZ6ZhgGMjO85Mbn2bp841kEtJROfPLvpYPfF yDcCZOAvD/ALzTprxRWzFZbRCl7TLJRUFCLHIGScm4B/QXbzdb/uqw9U265DQneO cDvig+wddfrC8DI6nhUhkv/o6CvN4pqIm66UZRCGyfni12MnMfXPyKP+ohrZPPGR btrUAzb0lBBM6E1Vmpi37IHERKR22wRsUkB52//ffJwHmmziR8ytM6Rns3V9xgkJ Qf0+PidI5fsqltf47IM47iXgwT04+FSWvZ+aUOv67nRPttAQdoZehXodU/ECnb22 jRMTIISu6p9Jo6ihwoMqFXGqJdOHAumLLu4uzXpVTcgSa6Qk6ei868PxBLo5EoEJ o2MXuz97EnlAfhfpo9zE0uzHnqao8QOrdQgz8CEBv3+0lozuTYGO5hgx+4O8dpl4 531pADsW6zlqTbCfTNJAEZeaGGG/fI8YmosQ4G1zdXL793O21QOwlkk7KXV9HX8U 9M179eg5NssUB2tBTXXqYAdo8iZf/c3staTo5DWj8qEtn78GIVk= =erJV -----END PGP SIGNATURE----- --TgCXP+xznsSrEyty--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170626134458.GT43966>