From owner-freebsd-net@FreeBSD.ORG Thu Aug 16 21:37:00 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 180FA1065672 for ; Thu, 16 Aug 2012 21:37:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id E0B118FC1A for ; Thu, 16 Aug 2012 21:36:59 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5079AB91A; Thu, 16 Aug 2012 17:36:59 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Date: Thu, 16 Aug 2012 17:36:47 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <201208160835.53303.jhb@freebsd.org> In-Reply-To: <201208160835.53303.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201208161736.47250.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 16 Aug 2012 17:36:59 -0400 (EDT) Cc: Jack Vogel , Vijay Singh Subject: Re: ixgbe rx & tx locks X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2012 21:37:00 -0000 On Thursday, August 16, 2012 8:35:53 am John Baldwin wrote: > On Monday, August 13, 2012 6:17:53 pm Jack Vogel wrote: > > After looking at the code again I think this is just what's happening, > > mq_start > > will schedule the task when it can't get the lock (due to the interrupt > > already > > holding it presumeably), so you get the tasklet code in contention with the > > interrupt. > > > > Anyone with a clever notion of how to do things better? > > Are you queueing a task that does both RX and TX? That was a bug I fixed > in igb that caused out-of-order packet processing for RX with igb. igb > uses a smaller task for when it's start routine fails that only tries to > restart transmission but doesn't do fullblown interrupt handling. > > Here's a possible patch (compiles, not run-tested) to implement this for > ixgbe: > > http://www.FreeBSD.org/~jhb/patches/ixgbe_txq_task.patch Actually, it looks like recent changes to the watchdog handling have reintroduced a variant of this bug. They schedule the full RX/TX task to run anytime the timer fires and there are pending TX completions that are in progress. This seems a bit odd as in the normal case you will get a TX completion interrupt once there is something to do. OTOH, it adds another instance of duplicate RX processing which is much worse than what this is trying to fix. I've updated this patch to disable that bit of the watchdog handler in both igb and ixgbe: http://www.FreeBSD.org/~jhb/patches/ixgbe_txq_task2.patch -- John Baldwin