From owner-freebsd-net@FreeBSD.ORG Thu Mar 20 21:22:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33C1465B for ; Thu, 20 Mar 2014 21:22:48 +0000 (UTC) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D25926A3 for ; Thu, 20 Mar 2014 21:22:47 +0000 (UTC) Received: by mail-qg0-f50.google.com with SMTP id q108so4385878qgd.9 for ; Thu, 20 Mar 2014 14:22:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Gyx+UwU3f8EhNHE+wAk73s4QM6ZroZMnOco5Uis+G2k=; b=S2nKIgnwrWshUTwiigtalPwlUjOfhiHZL13lYyAOktTf1STFGrn4d83OFsoEiR9Ioi uW/K3UXH1v5f9eTInnRukXINN5TkbmzI4Tr5fbiw8AcTrZLbiHR0pKgMwwsFX1FKHv0L 1BEFe2qfakhp3Uv4MrkFOBmUUMZoJnIAi47WM7fIkRLwRTat82bxB3dP0tixivazzh2f l4PFG9Jvgsqq4jXHkJxB6Gh70Qi6auL5RL5QscNxLaMBwVk8iJ1IKm57YD+c9EFrIgid 8VWzzYCbQ4P3eDauhhrlxUA5e6X5qO/1YvZ6lLMvCzuPconsxXbPG2VINRTwUdooqmEm u8IA== MIME-Version: 1.0 X-Received: by 10.224.123.7 with SMTP id n7mr982734qar.18.1395350566609; Thu, 20 Mar 2014 14:22:46 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 20 Mar 2014 14:22:46 -0700 (PDT) In-Reply-To: <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> References: <1159309884.25490921.1395282576806.JavaMail.root@uoguelph.ca> <201403202113.s2KLD7GB085085@hergotha.csail.mit.edu> Date: Thu, 20 Mar 2014 18:22:46 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Garrett Wollman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 21:22:48 -0000 Any recommendations on what to do? I'm experimenting with disabling TSO right now, but it's too early to tell if it fixes my problem. On my 9.2 box, we don't see this number climbing. With TSO off on 10.0, I also see the number is not climbing. I'd appreciate any links you may have so I can read up on this. Thanks for the comment. On Thu, Mar 20, 2014 at 6:13 PM, Garrett Wollman < wollman@hergotha.csail.mit.edu> wrote: > In article > , > csforgeron@gmail.com writes: > > >50/27433/0 requests for jumbo clusters denied (4k/9k/16k) > > This is going to screw you. You need to make sure that no NIC driver > ever allocates 9k jumbo pages -- unless you are using one of those > mythical drivers that can't do scatter/gather DMA on receive, which > you don't appear to be. > > These failures occur when the driver is trying to replenish its > receive queue, but is unable to allocate three *physically* contiguous > pages of RAM to construct the 9k jumbo cluster (of which the remaining > 3k is simply wasted). This happens on any moderately active server, > once physical memory gets checkerboarded with active single pages, > particularly with ZFS where those pages are wired in kernel memory and > so can't be evicted. > > -GAWollman >