From owner-freebsd-current@FreeBSD.ORG Wed Jan 30 21:52:43 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D1DD16A479 for ; Wed, 30 Jan 2008 21:52:43 +0000 (UTC) (envelope-from swhetzel@gmail.com) Received: from rn-out-0910.google.com (rn-out-0910.google.com [64.233.170.188]) by mx1.freebsd.org (Postfix) with ESMTP id CFD6513C448 for ; Wed, 30 Jan 2008 21:52:42 +0000 (UTC) (envelope-from swhetzel@gmail.com) Received: by rn-out-0910.google.com with SMTP id s42so191115rnb.13 for ; Wed, 30 Jan 2008 13:52:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=L2ujFmfiYAeT7XH8UjLdhVOpaYsMH+ErPzFF0YReGVM=; b=GdRDcbN//luIQAbKnF64wdPnNfCv4ZnYr9wIB2op8FRCQgb22xGYO2CQBf7HUka8Cmb1oGBohVSiyw3dmh4D3U9as90ZMpz3NMT54QRie5362OVJZ0qDguFKhov+R+ydSlcjROYGW74t6pNEcr37ZwVpa6Ioa7hsCYy6gWxkyfA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=PbqEldk32oI/U7mIdLJ1xSR1H+TRCjRUUqrVg6Ys8kdvXMAFZGoSHP0SZpw5B1LK2Ks0M4GcsX/NE0me/Yt+l211eC+XC0vK1ql5VzgmVo5FTl0gfyh2zEf4kM9pU1hIjn5LMkXsbF8YvCa3bc17KTGb8Wmr7uE9ortfFcqFHDE= Received: by 10.150.181.11 with SMTP id d11mr324154ybf.3.1201729961823; Wed, 30 Jan 2008 13:52:41 -0800 (PST) Received: by 10.150.203.18 with HTTP; Wed, 30 Jan 2008 13:52:41 -0800 (PST) Message-ID: <790a9fff0801301352xa91a69ci3f08488dfcfc982@mail.gmail.com> Date: Wed, 30 Jan 2008 15:52:41 -0600 From: "Scot Hetzel" To: "Attilio Rao" In-Reply-To: <3bbf2fe10801300707u3fd121c0k199605c2f0be6cbf@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10801250000k5852c2f2j5d1897c900096818@mail.gmail.com> <3bbf2fe10801260918k56c1a73hf28ff154eca0b74c@mail.gmail.com> <479BBDAA.6000008@FreeBSD.org> <3bbf2fe10801261657x7d7c9de4q71adeaf3a2dd8159@mail.gmail.com> <479C0B5B.9030709@FreeBSD.org> <3bbf2fe10801270642m5ec609d8xb29add77ced36d8a@mail.gmail.com> <479FA3E8.10606@FreeBSD.org> <3bbf2fe10801291411v302dd33at54ebe538397e8fac@mail.gmail.com> <20080130130820.GA88429@comp.chem.msu.su> <3bbf2fe10801300707u3fd121c0k199605c2f0be6cbf@mail.gmail.com> Cc: Kostik Belousov , Yar Tikhiy , Doug Barton , freebsd-current@freebsd.org Subject: Re: panic: System call lstat returning with 1 locks held X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2008 21:52:43 -0000 On 1/30/08, Attilio Rao wrote: > 2008/1/30, Yar Tikhiy : > > On Tue, Jan 29, 2008 at 11:11:13PM +0100, Attilio Rao wrote: > > > > > > I'm committing my WITNESS patch now to perforce so that other people > > > can hopefully stress-test it before to be committed. > > > > Do you think that that patch is applicable in my case? I.e., shall > > I use it to get more debug info on my panics? > > > > If so, where is the patched file in the depot? > > Sorry but I had to delay the operation so far. > In the end, a suitable patch is located here: > http://www.freebsd.org/~attilio/witness_lockmgr.diff > > I tried it and it alredy reported 4 LORs just when booting the kernel :) > So I would expect reasonably LOR cascades with this patch. > > If you all 3 (Scot, Yar and Doug) could try and test it I would > appreciate a lot. > Reading back to Doug's and Yar's messages regarding the NTFS filesystem, I noticed that I am also mounting NTFS filesystems at boot time. I disabled the mounting of the NTFS filesystems. When 'cd /usr/ports ; find . -print' or '/usr/local/etc/cvsup/update.sh' is run, the panic doesn't occur. But when I mount the NTFS filesystem, and rerun the above commands, they cause the lstat panic. Even though these commands are not touching the NTFS filesystems. Also mounting/unmounting a NTFS filesystem will cause a panic. I applied the above patch to sources that were checked out about 2 hrs ago. Rebuilt/installed kernel and rebooted. If I don't mount a NTFS filesystem then the kernel doesn't panic when the above commands are run. But when the NTFS filesystem is mounted, the following lock order reversal occurs: lock order reversal: 1st 0xffffff0023285288 pseudofs (pseudofs) @ kern/vfs_subr.c:2061 2nd 0xffffff00232f2ca0 vfslock (vfslock) @ kern/vfs_subr.c:364 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a witness_checkorder() at witness_checkorder+0x606 _lockmgr() at _lockmgr+0x4cb vfs_busy() at vfs_busy+0xdf vfs_donmount() at vfs_donmount+0x9aa nmount() at nmount+0xa4 syscall() at syscall+0x1ce Xfast_syscall() at Xfast_syscall+0xab --- syscall (378, FreeBSD ELF64, nmount), rip = 0x80079a57c, rsp = 0x7fffffffe8 28, rbp = 0x65a9d0 --- lock order reversal: 1st 0xffffff002347f668 ntfs (ntfs) @ kern/vfs_subr.c:2061 2nd 0xffffff00232f2650 vfslock (vfslock) @ kern/vfs_subr.c:364 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a witness_checkorder() at witness_checkorder+0x606 _lockmgr() at _lockmgr+0x4cb vfs_busy() at vfs_busy+0xdf vfs_donmount() at vfs_donmount+0x9aa nmount() at nmount+0xa4 syscall() at syscall+0x1ce Xfast_syscall() at Xfast_syscall+0xab --- syscall (378, FreeBSD ELF64, nmount), rip = 0x80079a57c, rsp = 0x7fffffffe8 28, rbp = 0x65ad80 --- Instead of getting the lstat panic, I am now getting the following panic when /usr/local/etc/cvsup/update.sh ran: Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x8:0xffffffff80301051 stack pointer = 0x10:0xffffffffd6bb0100 frame pointer = 0x10:0xffffffffd6bb0190 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 1243 (cvsup) panic: Assertion !mtx_owned(&w_mtx) failed at ../../../kern/subr_witness.c:959 cpuid = 0 Uptime: 11m14s Physical memory: 2031 MB Dumping 325 MB: 310 294 278 262 246 230 214 198 182 166 150 134 118 102 86 70 54 38 22 6 #0 doadump () at pcpu.h:194 194 __asm __volatile("movq %%gs:0,%0" : "=r" (td)); Scot