From owner-freebsd-net@freebsd.org Sun Jun 25 16:56:49 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2FB0D9571C; Sun, 25 Jun 2017 16:56:49 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x22e.google.com (mail-wr0-x22e.google.com [IPv6:2a00:1450:400c:c0c::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5623376166; Sun, 25 Jun 2017 16:56:49 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x22e.google.com with SMTP id 77so124109801wrb.1; Sun, 25 Jun 2017 09:56:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3FL9L7x3JW4a1K9f2uyaxkqNrwNaS2oDAJKRyxGJF3s=; b=rVN+UHhTK5C1k0MsRtveBnlLadwcmlKbZ393jeaHwLbDmePZ1uE07tdb9gywHyLJqS qMe4d1jpjHznV4Ape8xyWHVgvJicKXdGOIIECgmnk6MDb85B4C1dK5qPIZ49ix97eGhG PtPONyMcEyD4PUHH3QwHC8JKeZJLut3eepdmoPxihKpYFF1jsaLCttgBp7+Y5c+8MSJd D1V1GeTfw1V71X9+M1beX9M1MaHhzq0vf2jD1QgX8+5DZXS1aV0k4Oef4mvn9O4I1vVl trAGHZ0T3NnSDCzDqGXLGBiu15v3nTELAjl6QN9R001IzWxCaJyhfLIiWk1L4Azs9Jvf D3gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3FL9L7x3JW4a1K9f2uyaxkqNrwNaS2oDAJKRyxGJF3s=; b=qGJeObOA7sHXZz6O63CsXaa4zTG56G0LvIliYgKPagQn8r23vdM1TTMC1Y2XMPv3tm gw2w2NvC2WhTbDB5jdvqm34TmlMLN+n55R+Ih9latd9mKEFm0vdKsLne3fWVKgsoHG4E 5VMhijRpkfcySGHG3nsAwwZADqjCf/U4mockEpGVtVpG9YVEzcVP/KF+S8h5MiULb6Vs 96/0iHVkXYKDSh4Z4bHWHI55WlU4LHKmDeuo/9Sibwhmfkv/0fBab1Cdc4Ck8qFyDXhw JGzKRkNV8/TdqLEi8G1boeRTyg/Na8Z8wtISf5D3lS+npSPCl1SgfwHA4O8yDC4jAo3C hWCw== X-Gm-Message-State: AKS2vOwvEi/bsm3JwNMotn+nmuz54683rTtA4wE8ZogwzA3HIxoU1RbT zkV1+cL+JEGvlw== X-Received: by 10.223.176.253 with SMTP id j58mr10795933wra.65.1498409807388; Sun, 25 Jun 2017 09:56:47 -0700 (PDT) Received: from ben.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id m26sm14755941wrm.4.2017.06.25.09.56.46 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 25 Jun 2017 09:56:46 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mbuf_jumbo_9k & iSCSI failing From: Ben RUBSON In-Reply-To: Date: Sun, 25 Jun 2017 18:56:45 +0200 Cc: FreeBSD Net , "freebsd-scsi@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <14CB3F50-0426-48BD-838C-943B6D15FEB9@gmail.com> References: <486A6DA0-54C8-40DF-8437-F6E382DA01A8@gmail.com> <6a31ef00-5f7a-d36e-d5e6-0414e8b813c7@selasky.org> <613AFD8E-72B2-4E3F-9C70-1D1E43109B8A@gmail.com> <2c9a9c2652a74d8eb4b34f5a32c7ad5c@AM5PR0502MB2916.eurprd05.prod.outlook.com> <52A2608C-A57E-4E75-A952-F4776BA23CA4@gmail.com> <9B507AA6-40FE-4B8D-853F-2A9422A2DF67@gmail.com> To: Ryan Stone X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2017 16:56:49 -0000 > On 25 Jun 2017, at 17:32, Ryan Stone wrote: >=20 > Having looking at the original email more closely, I see that you = showed an mlxen interface with a 9020 MTU. Seeing allocation failures = of 9k mbuf clusters increase while you are far below the zone's limit = means that you're definitely running into the bug I'm describing, and = this bug could plausibly cause the iSCSI errors that you describe. >=20 > The issue is that the newer version of the driver tries to allocate a = single buffer to accommodate an MTU-sized packet. Over time, however, = memory will become fragmented and eventually it can become impossible to = allocate a 9k physically contiguous buffer. When this happens the = driver is unable to allocate buffers to receive packets and is forced to = drop them. Presumably, if iSCSI suffers too many packet drops it will = terminate the connection. The older version of the driver limited = itself to page-sized buffers, so it was immune to issues with memory = fragmentation. Thank you for your explanation Ryan. You say "over time", and you're right, I have to wait several days (here = 88) before the problem occurs. Strange however that in 2500MB free memory system is unable to find 9k = physically contiguous. But we never know :) Let's then wait for your patch ! (and reboot for now) Many thx ! Ben=