From owner-freebsd-stable@FreeBSD.ORG Sat Dec 30 23:38:16 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DEDC416A416 for ; Sat, 30 Dec 2006 23:38:15 +0000 (UTC) (envelope-from sven@dmv.com) Received: from smtp-gw-cl-d.dmv.com (smtp-gw-cl-d.dmv.com [216.240.97.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7E98913C457 for ; Sat, 30 Dec 2006 23:38:15 +0000 (UTC) (envelope-from sven@dmv.com) Received: from mail-gw-cl-b.dmv.com (mail-gw-cl-b.dmv.com [216.240.97.39]) by smtp-gw-cl-d.dmv.com (8.12.10/8.12.10) with ESMTP id kBUN1MJX059699; Sat, 30 Dec 2006 18:01:22 -0500 (EST) (envelope-from sven@dmv.com) Received: from [67.62.150.139] (static-67-62-150-139.dsl.cavtel.net [67.62.150.139]) (authenticated bits=0) by mail-gw-cl-b.dmv.com (8.12.9/8.12.9) with ESMTP id kBUN1J8s004420; Sat, 30 Dec 2006 18:01:22 -0500 (EST) (envelope-from sven@dmv.com) Message-ID: <4596F06D.30004@dmv.com> Date: Sat, 30 Dec 2006 18:04:13 -0500 From: Sven Willenberger User-Agent: Thunderbird 1.5.0.7 (X11/20060919) MIME-Version: 1.0 To: Sven Willenberger References: <20061205.004323.78708386.hrs@allbsd.org> <20061204160949.GM35681@deviant.kiev.zoral.com.ua> <20061205.123805.59655403.hrs@allbsd.org> <1166194879.6317.11.camel@lanshark.dmv.com> <20061215181548.GA58555@xor.obsecurity.org> <1166209936.6317.21.camel@lanshark.dmv.com> <20061215192958.GA86926@xor.obsecurity.org> <20061215212040.GG23698@deviant.kiev.zoral.com.ua> <1166463200.11562.5.camel@lanshark.dmv.com> In-Reply-To: <1166463200.11562.5.camel@lanshark.dmv.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.48 on 216.240.97.42 X-Scanned-By: MIMEDefang 2.48 on 216.240.97.39 Cc: Kostik Belousov , stable@freebsd.org, Kris Kennaway Subject: Re: Not panic in nfsd (Re: panic in nfsd on 6.2-RC1) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 23:38:16 -0000 Sven Willenberger presumably uttered the following on 12/18/06 12:33: > On Fri, 2006-12-15 at 23:20 +0200, Kostik Belousov wrote: >> On Fri, Dec 15, 2006 at 02:29:58PM -0500, Kris Kennaway wrote: > > <> > >>> >>>> FWIW, I do see the following appearing in the /var/log/messages: >>>> ufs_rename: fvp == tvp (can't happen) >>>> about once or twice a day, but cannot correlate those to lockup. Now >>>> that I have enabled the options mentioned above in the kernel, I am >>>> seeing some LOR issues: >>>> >>>> kernel: lock order reversal: >>>> kernel: 1st 0xffffff00c3bab200 kqueue (kqueue) @ /usr/src/sys/kern/kern_event.c:1547 >>>> kernel: 2nd 0xffffff0005bb6078 struct mount mtx (struct mount mtx) @ /usr/src/sys/ufs/ufs/ufs_vnops.c:138 >>> OK, this is interesting, so let's proceed from here. >>> >>> Kris >> Try this. >> >> Index: ufs/ufs/ufs_vnops.c >> =================================================================== >> RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/ufs_vnops.c,v >> retrieving revision 1.283 >> diff -u -r1.283 ufs_vnops.c >> --- ufs/ufs/ufs_vnops.c 6 Nov 2006 13:42:09 -0000 1.283 >> +++ ufs/ufs/ufs_vnops.c 15 Dec 2006 21:19:51 -0000 >> @@ -133,19 +133,15 @@ >> { >> struct inode *ip; >> struct timespec ts; >> - int mnt_locked; >> >> ip = VTOI(vp); >> - mnt_locked = 0; >> - if ((vp->v_mount->mnt_flag & MNT_RDONLY) != 0) { >> - VI_LOCK(vp); >> + VI_LOCK(vp); >> + if ((vp->v_mount->mnt_flag & MNT_RDONLY) != 0) >> goto out; >> + if ((ip->i_flag & (IN_ACCESS | IN_CHANGE | IN_UPDATE)) == 0) { >> + VI_UNLOCK(vp); >> + return; >> } >> - MNT_ILOCK(vp->v_mount); /* For reading of mnt_kern_flags. */ >> - mnt_locked = 1; >> - VI_LOCK(vp); >> - if ((ip->i_flag & (IN_ACCESS | IN_CHANGE | IN_UPDATE)) == 0) >> - goto out_unl; >> >> if ((vp->v_type == VBLK || vp->v_type == VCHR) && !DOINGSOFTDEP(vp)) >> ip->i_flag |= IN_LAZYMOD; >> @@ -172,10 +168,7 @@ >> >> out: >> ip->i_flag &= ~(IN_ACCESS | IN_CHANGE | IN_UPDATE); >> - out_unl: >> VI_UNLOCK(vp); >> - if (mnt_locked) >> - MNT_IUNLOCK(vp->v_mount); >> } >> >> /* > > > Patch applied cleanly (offset 6 lines), make buildworld, make kernel, > reboot, make installworld, etc. > > kernel: lock order reversal: > kernel: 1st 0xffffff00b9181800 kqueue (kqueue) @ /usr/src/sys/kern/kern_event.c:1547 > kernel: 2nd 0xffffff00c16030d0 vnode interlock (vnode interlock) @ /usr/src/sys/ufs/ufs/ufs_vnops.c:132 > > > > _______________________________________________ Having enabled witness and ddb, etc I cannot get this LOR to trigger anymore, but the machine is still locking up. I finally managed to get a piece of what was appearing on the console which is the following (copied by hand by an onsite tech so there may be a typo here and there): --------cut-------------- bge_intr() at loge_intr+0x84a ithread_loop() at ithread_loop+0x14c fork_exit() at fork_exit+0xbb fork_trampoline() at fork_trampoline+0xee --- trap 0, rip-0, rsp-0xffffffffb371ad00, rbp-0 --- Fatal trap 12: page fault while in Kernel Mode cupid=1, apic id=01 fault virtual address - 0x28 fault code - supervisor write, page not present instruction pointer - 0x8:0xffffffff801dae1a stack pointer - 0x10:0xffffffffb371ab70 frame pointer - 0x10:0xffffffffb371abd0 code segment - base 0x0, limit 0xfffff, type 0x1b - DPL 0, pres 1, long 1, def32 0, gram 1 processor eflags=interrupt enabled, resume, IOPL=0 current process=28 (irq 24:bge0) trap number=12 panic: page fault cupid=1 Uptime - 4d10h52m36s Dumping 4031MB (2 chunks) chunk0: 1MB (156 pages)... ok chunk1: 4031MB (1031920) ----------cut----------------- For some reason, by the time it reboots, there is no dump file available (even though it is enabled in rc.conf and there is more than enough room in /var/crash to hold it). Sven