From owner-freebsd-current@FreeBSD.ORG Mon Apr 12 16:26:08 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B82A9106566B; Mon, 12 Apr 2010 16:26:08 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 23A028FC1D; Mon, 12 Apr 2010 16:26:07 +0000 (UTC) Received: by wyb28 with SMTP id 28so1015726wyb.13 for ; Mon, 12 Apr 2010 09:26:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:cc:content-type; bh=/hAMDKafnzBSXEmpq9tk4lvIxYjkYutCXHzbwV0NbBI=; b=CEo0YdoXdCJRLOayIJ9PR4tDPWvoNmpQDa15vymjvfOQR7TEhx8mieqYknkCrdiMTH 1Qdu/GRnMEsthjKA1iMPUI9vGW6t6epgpTna5rGI7nv0uPvfAhvziwwnNHdIVh+Q81Uy ebkbmnBpRN0L+eoA98n2RiCN3B0mK4oFo73b0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=oCixnuvZt84AFUPwdfNjUf6UsN8wZLgGHqR2YD//vjgmXUvG8tEbh7h0IvwGlO8piD nd9TzAKEUKvkQqwIYMCj2vR9KmqsthC0s5UKfV2+Pf1Ty5reD3tP5Xs/eKpmIe4b1XQ6 dxteROShOr/k2syxtHxN2bS+TtBvYrij4xeRY= MIME-Version: 1.0 Received: by 10.216.11.8 with HTTP; Mon, 12 Apr 2010 09:26:06 -0700 (PDT) In-Reply-To: <201004121052.42350.jhb@freebsd.org> References: <201004081313.o38DD4JM041821@lava.sentex.ca> <201004091900.o39J0b0k051687@lava.sentex.ca> <201004121052.42350.jhb@freebsd.org> Date: Mon, 12 Apr 2010 09:26:06 -0700 Received: by 10.216.172.73 with SMTP id s51mr2653413wel.113.1271089566999; Mon, 12 Apr 2010 09:26:06 -0700 (PDT) Message-ID: From: Jack Vogel To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: pyunyh@gmail.com, freebsd-current@freebsd.org, Mike Tancsa Subject: Re: LOR on em in HEAD ( was Re: em driver regression X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2010 16:26:08 -0000 On Mon, Apr 12, 2010 at 7:52 AM, John Baldwin wrote: > On Friday 09 April 2010 3:09:24 pm Jack Vogel wrote: > > Someone else also pointed this out. I'm dubious about its claim. > > This happens because there is an RX lock taken in rxeof, its held > > thru the call into the stack, it then encounters another lock there > > and hence this complaint. I've had the RX hold as it is for a long > > while and would rather not have to give it up, can someone look > > at it and advise? > > I've seen it happen with igb. I suspect it is a transitive lock order. > That > is, you probably never have the UDP lock acquired before an em/igb RX lock. > However, if you have an em/igb adapter TX lock held when you acquire an > em/igb > RX lock in one place, and in if_start() you acquire the TX lock while the > UDP > lock is held, that can trigger the LOR. Specifically, those two paths > would > give you these two orders: > > TX -> RX > UDP -> TX > > which implies the order > > UDP -> RX > > (lock order relationsips are transitive, just like a > b and b > c implies > a > c). > > However, I haven't been able to track down what the raw orders are that > might > lead to this transitive order. Attilio added some sysctls to dump all the > raw > lock orders in one of the debug.witness sysctls. You can also try > hardcoding > the 'RX -> UDP' order using WITNESS_DEFINEORDER() before any of the em/igb > RX/TX locks are acquired to see what different LOR is triggered. If that > LOR > looks valid then you can keep hardcoding valid orders until you find the > invalid one. > > Do you think releasing the RX lock before the stack entry would get rid of the problem? Other ideas? Jack