From owner-freebsd-net@FreeBSD.ORG Tue Nov 8 18:53:58 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E78B106566C for ; Tue, 8 Nov 2011 18:53:58 +0000 (UTC) (envelope-from nitroboost@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id A496E8FC15 for ; Tue, 8 Nov 2011 18:53:57 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so980272bkb.13 for ; Tue, 08 Nov 2011 10:53:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=zKPOz4Dp0bDLMcd8/6Fmpa3Euez/R7kDd0qF57Tt0vA=; b=XocLTqYbJiv/HgG5Dck1QjbS0cZM640ctlzCi5+fqZJmMR85ruFIpHur9V+kY5pKxv VPD9UfkAaKqI69/HlylYojAwhr49aJZv1VTys0T2lGT6H3PRlfNI+islCsTeQ4tri0Vw v39UcmMo+VvZ9uxBgXJ7JUNiRTaC0A28v5VEk= MIME-Version: 1.0 Received: by 10.182.13.1 with SMTP id d1mr10380396obc.74.1320778436019; Tue, 08 Nov 2011 10:53:56 -0800 (PST) Received: by 10.182.30.164 with HTTP; Tue, 8 Nov 2011 10:53:55 -0800 (PST) In-Reply-To: <4EB96511.50701@gmail.com> References: <4E8F157A.40702@sentex.net> <4E8F51D4.1060509@sentex.net> <4EA7E203.3020306@sepehrs.com> <4EA80818.3030504@sentex.net> <4EA80F88.4000400@hotplug.ru> <4EA82715.2000404@gmail.com> <4EA8FA40.7010504@hotplug.ru> <4EA91836.2040508@gmail.com> <4EA959EE.2070806@hotplug.ru> <4EAD116A.8090006@gmail.com> <4EAE58A2.9040803@gmail.com> <4EB96511.50701@gmail.com> Date: Tue, 8 Nov 2011 11:53:55 -0700 Message-ID: From: Jason Wolfe To: Hooman Fazaeli Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: pyunyh@gmail.com, freebsd-net@freebsd.org, Jack Vogel , Emil Muratov Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2011 18:53:58 -0000 On Tue, Nov 8, 2011 at 10:21 AM, Hooman Fazaeli wrote: > I have allocated more time to the problem and guess I can explain what > your problem is. > > With MSIX disabled, the driver uses fast interrupt handler (em_irq_fast) > which calls rx/tx task and then checks for link status change. This > implies that rx/tx task is executed with every link state change. This is > not efficient, as it is a waste of time to start transmission when link is > down. > However, it has the effect that after a temporary link loss > (active->inactive->active), > _start is executed and transmission continues normally. The value of > link_toggles (3) > clearly indicates that you had such a transition when the problem occured. > > With MSIX enabled, the link task (em_handle_link) does _not_ triggers > _start when the link changes state from inactive to active (which it > should). > If if_snd quickly fills up during a temporary link loss, transmission is > stopped forever and the driver never recovers from that state. > > The last patch should have reduced the frequency of the problem > but it assumes every IFQ_ENQUEUE is followed by a if_start which > is not a true assumption. > > If you are willing to test, I can prepare another patch for you to fix > the issue in a different and more reliable way. > > Hooman, Thanks again for the assist, it sounds like this may also be why we see a bit higher latency with MSI-X disabled on this chipset. I'm happy to test any patches as I have a handful of boxes set aside to 'research' this issue. Hopefully the testing here helps along any patches to the tree for others benefit also. Jason