From owner-freebsd-stable@FreeBSD.ORG Fri Jan 16 14:16:48 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C33F106564A; Fri, 16 Jan 2009 14:16:48 +0000 (UTC) (envelope-from rwatson@FreeBSD.ogr) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 27FE98FC1B; Fri, 16 Jan 2009 14:16:48 +0000 (UTC) (envelope-from rwatson@FreeBSD.ogr) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id D532446B42; Fri, 16 Jan 2009 09:16:47 -0500 (EST) Date: Fri, 16 Jan 2009 14:16:47 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Pete French In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-stable@freebsd.org, drosih@rpi.edu, dchagin@freebsd.org, rblayzor.bulk@inoc.net Subject: Re: Big problems with 7.1 locking up :-( X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2009 14:16:48 -0000 On Fri, 16 Jan 2009, Pete French wrote: >> I rather feared as much. Let's run down the path of "perhaps there's a >> problem with the new UDP locking code" for a bit and see where it takes us. >> Is it possible to run those boxes with WITNESS -- I believe that the fact >> that "show alllocks" is failing is because WITNESS isn't present. > > Yes, I can do that. The only reason I wasn't running with WITNESS is that it > didn't lock up when I added the BREAK_TO_DEBUGGER so I was seeing if a > simple GENERIC kernel would lock up when I added that. I will go back and > add WITNESS when you tell me theres nothing more we can get out of this lock > up (recompiling will involve restarting the machine so I loose the 'boekn to > debugger' state). Should I add anything else ? Skip spinlocks ? Invariants ? > >> The other thing we can do is revert UDP to using purely write locks -- the >> risk there is that it might change the timing but not actually resolve the >> bug, so if we can analyze it a bit using WITNESS first that would be >> useful. > > Yes, I will run with WITNESS and anything else you might want. Is there > anything else you, or anyone else, wants from this kernel ? It may take > another day to lock up when I've restarted it unfortunately. If you do INVARIANTS + WITNESS + WITNESS_SKIPSPIN, that should be good. WITNESS does a number of things, including tracking (and being judgemental about) lock order. One nice side effect of that tracking is that we keep track of a lot more lock state explicitly, so DDB's "show allocks", "show locks", etc, commands can build on that. "show lockedvnods" works without WITNESS, though, so your results so far suggest this is likely not related to vnode locking. Robert N M Watson Computer Laboratory University of Cambridge