From owner-freebsd-current@FreeBSD.ORG Fri Feb 1 18:42:00 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B14B16A419 for ; Fri, 1 Feb 2008 18:42:00 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id 959A613C457 for ; Fri, 1 Feb 2008 18:41:59 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by fg-out-1718.google.com with SMTP id 16so1158223fgg.35 for ; Fri, 01 Feb 2008 10:41:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=jE48ZQrz3pxlifUzYTV0vN6LwvU/MT1OEhSaugmaGzM=; b=AOKPwhJLkdjqTv0dHrsmCEtWYkx4jevTy8A/eN1dW8ye4C5nhOhJBeY2gKIgKCUirJ1oeexYmY9f+Yfln5FAcbySnOV2uEzQOAD8M5E+mWZ3HkOuS44xK121CARVotZ79E88lVeOGjvRJESq4ljikwZujBb/DtjTlkcPPoO5suw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=lhUH+rnzdosR0gX/PdAUgryYK+mnzmO/iP4lBRDzB5XISONtDrpleQ937wtM8pcGRu6Di3BUOSSpYRD7SdU7bb5yiMa4pU4jNw1dZczDpWZIgrup9IBVUgTn8PnshTvt4k3bx7BJ5pRRwxeuNoJu6Ci7hdP6FOQEBlv9+SOxPbw= Received: by 10.86.25.17 with SMTP id 17mr3471635fgy.15.1201891318338; Fri, 01 Feb 2008 10:41:58 -0800 (PST) Received: by 10.86.28.19 with HTTP; Fri, 1 Feb 2008 10:41:58 -0800 (PST) Message-ID: <3bbf2fe10802011041t28e419c9n5f0f6f34d6450184@mail.gmail.com> Date: Fri, 1 Feb 2008 19:41:58 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Yar Tikhiy" In-Reply-To: <20080201145051.GE79881@comp.chem.msu.su> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3bbf2fe10801270642m5ec609d8xb29add77ced36d8a@mail.gmail.com> <3bbf2fe10801291411v302dd33at54ebe538397e8fac@mail.gmail.com> <20080130130820.GA88429@comp.chem.msu.su> <3bbf2fe10801300707u3fd121c0k199605c2f0be6cbf@mail.gmail.com> <790a9fff0801301352xa91a69ci3f08488dfcfc982@mail.gmail.com> <3bbf2fe10801310243tddedfeckbc4c94be87f0a4ca@mail.gmail.com> <20080131130210.GA37090@comp.chem.msu.su> <3bbf2fe10801310504j486924bdm86e0436597a42b09@mail.gmail.com> <790a9fff0801312241s346068b6s40fcae71ebbf546@mail.gmail.com> <20080201145051.GE79881@comp.chem.msu.su> X-Google-Sender-Auth: f67361e37fe5273e Cc: Kostik Belousov , Scot Hetzel , freebsd-current@freebsd.org, Doug Barton Subject: Re: panic: System call lstat returning with 1 locks held X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Feb 2008 18:42:00 -0000 2008/2/1, Yar Tikhiy : > On Fri, Feb 01, 2008 at 12:41:30AM -0600, Scot Hetzel wrote: > > On 1/31/08, Attilio Rao wrote: > > > 2008/1/31, Yar Tikhiy : > > > > > > > On Thu, Jan 31, 2008 at 11:43:10AM +0100, Attilio Rao wrote: > > > > > > > > > > The assertion failing should not happen now. > > > > > Could you please hand-add a check in _lockmgr_disown() > > > > > (kern/kern_lock.c) in order to check for the panicstr before to call > > > > > WITNESS? I cannot access to perforce now and produce a suitable diff, > > > > > so you can just do this by hand: > > > > > > > > > > if (lkp->lk_lockholder == td) { > > > > > if (panicstr != NULL) > > > > > WITNESS_UNLOCK(&lkp->lk_object, LOP_EXCLUSIVE, file, line); > > > > > td->td_locks--; > > > > > } > > > > > > > > > > > > Shouldn't the test for panicstr be inverse: `panicstr == NULL'? > > > > I guess we shouldn't call WITNESS when panicing, should we? > > > > Sorry if I got it wrong. > > > > > > > > > Weee, you are right, sorry! > > > > > > > > I added this change to kern/kern_lock.c, but I'm still getting this > > panic after mounting the ntfs filesystem, and using cvsup to update > > the local mirror: > > > > Fatal trap 9: general protection fault while in kernel mode > > cpuid = 0; apic id = 00 > > instruction pointer = 0x8:0xffffffff80301051 > > stack pointer = 0x10:0xffffffffd43b9100 > > frame pointer = 0x10:0xffffffffd43b9190 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags = resume, IOPL = 0 > > current process = 1229 (cvsup) > > panic: Assertion !mtx_owned(&w_mtx) failed at ../../../kern/subr_witness.c:959 > > cpuid = 0 > > Uptime: 4m38s > > Physical memory: 2031 MB > > Dumping 324 MB: 309 293 277 261 245 229 213 197 181 165 149 133 117 > > 101 85 69 53 37 21 5 > > > FWIW, the same panic happens in my case, too. In addition, reported > are a number of LORs I haven't seen before. The relevant kernel > message log from the serial console is attached. Thanks! > > -- > Yar > > [...] > WARNING: WITNESS option enabled, expect reduced performance. > GEOM_LABEL: Label for provider ad0s1 is ntfs/SYSTEM. > GEOM_LABEL: Label for provider ad0s2 is ntfs/STORE. > lock order reversal: > 1st 0xc2ecfe28 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2061 > 2nd 0xc2fbead4 devfsmount (devfsmount) @ /usr/src/sys/fs/devfs/devfs_vnops.c:20 > 1 > KDB: stack backtrace: > db_trace_self_wrapper(c0b0fa49,d3cecbbc,c07a2c5e,c0b11f90,c2fbead4,...) at db_tr > ace_self_wrapper+0x26 > kdb_backtrace(c0b11f90,c2fbead4,c0b0343a,c0b0343a,c0b0347b,...) at kdb_backtrace > +0x29 > witness_checkorder(c2fbead4,9,c0b0347b,c9,c7,...) at witness_checkorder+0x6de > _sx_xlock(c2fbead4,0,c0b0347b,c9,c2fbead4,...) at _sx_xlock+0x7d > devfs_allocv(c2fbd700,c2fc3000,d3cecc28,c2d0fcc0,c0b17d73,...) at devfs_allocv+0 > x144 > devfs_root(c2fc3000,2,c0e1c118,c2d0fcc0,ca,...) at devfs_root+0x51 > set_rootvnode(c0e1c100,0,c0b17d73,5ed,c07e0150,...) at set_rootvnode+0x2b > vfs_mountroot(c0dc9d50,4,c0b078ec,260,0,...) at vfs_mountroot+0x356 > start_init(0,d3cecd38,c0b091cd,30c,c2d0dab0,...) at start_init+0x65 > fork_exit(c0732530,0,d3cecd38) at fork_exit+0xb8 > fork_trampoline() at fork_trampoline+0x8 > --- trap 0, eip = 0, esp = 0xd3cecd70, ebp = 0 --- > Trying to mount root from ufs:/dev/ad0s3a > WARNING: / was not properly dismounted > lock order reversal: > 1st 0xc2ecf9e8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2061 > 2nd 0xc2fc3000 vfslock (vfslock) @ /usr/src/sys/kern/vfs_subr.c:364 > KDB: stack backtrace: > db_trace_self_wrapper(c0b0fa49,d3cec9e0,c07a2c5e,c0b11f90,c2fc3000,...) at db_tr > ace_self_wrapper+0x26 > kdb_backtrace(c0b11f90,c2fc3000,c0b17e71,c0b17e71,c0b1840e,...) at kdb_backtrace > +0x29 > witness_checkorder(c2fc3000,1,c0b1840e,16c,151,...) at witness_checkorder+0x6de > _lockmgr(c2fc3000,2001,c2fc3030,c0b1840e,16c,...) at _lockmgr+0x174 > vfs_busy(c2fc3000,0,0,c2d0fcc0,d3cecb58,...) at vfs_busy+0x198 > lookup(d3cecb44,c0b17b21,c6,bf,c2ce142c,...) at lookup+0x7c4 > namei(d3cecb44,c2d0fd54,c0bc54a4,c0b17d73,c2fc3030,...) at namei+0x34b > kern_unlink(c2d0fcc0,c0b181b0,1,628,0,...) at kern_unlink+0x40 > vfs_mountroot_try(c0b1836a,c0b0660d,c0aff4ee,1,c07e0150,...) at vfs_mountroot_tr > y+0x470 > vfs_mountroot(c0dc9d50,4,c0b078ec,260,0,...) at vfs_mountroot+0x418 > start_init(0,d3cecd38,c0b091cd,30c,c2d0dab0,...) at start_init+0x65 > fork_exit(c0732530,0,d3cecd38) at fork_exit+0xb8 > fork_trampoline() at fork_trampoline+0x8 > --- trap 0, eip = 0, esp = 0xd3cecd70, ebp = 0 --- > lock order reversal: > 1st 0xc2d13044 user map (user map) @ /usr/src/sys/vm/vm_map.c:3111 > 2nd 0xc2ecf7c8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2061 > KDB: stack backtrace: > db_trace_self_wrapper(c0b0fa49,d3cec9cc,c07a2c5e,c0b11f90,c2ecf7c8,...) at db_tr > ace_self_wrapper+0x26 > kdb_backtrace(c0b11f90,c2ecf7c8,c0b06ec0,c0b06ec0,c0b1840e,...) at kdb_backtrace > +0x29 > witness_checkorder(c2ecf7c8,1,c0b1840e,80d,c0dd52f4,...) at witness_checkorder+0 > x6de > _lockmgr(c2ecf7c8,3041,c2ecf7f8,c0b1840e,80d,...) at _lockmgr+0x174 > ffs_lock(d3ceca78,c075b6bd,c0dd52f4,3041,c2ecf770,...) at ffs_lock+0x8a > VOP_LOCK1_APV(c0bdfbc0,d3ceca78,c0b0660b,3,c2ecf7f8,...) at VOP_LOCK1_APV+0xa5 > _vn_lock(c2ecf770,3041,c0b1840e,80d,0,...) at _vn_lock+0xf2 > vget(c2ecf770,3041,c2d0fcc0,4a9,c1460600,...) at vget+0x109 > vnode_pager_lock(c1460480,0,c0b2ec90,127,d3cecbe8,...) at vnode_pager_lock+0x1ad > vm_fault(c2d13000,80d2000,2,8,80d2000,...) at vm_fault+0x1df > trap_pfault(5,0,c0b3ce02,2c8,c2d0dab0,...) at trap_pfault+0x118 > trap(d3cecd38) at trap+0x267 > calltrap() at calltrap+0x6 > --- trap 0xc, eip = 0x80480e5, esp = 0xbfbfeef0, ebp = 0xbfbfef10 --- > Enter full pathname of shell or RETURN for /bin/sh: > # > # mount -r /usr > # mount -r /var > # mount /ntfs > # > # umount /ntfs > lock order reversal: > 1st 0xc2fece28 ntfs (ntfs) @ /usr/src/sys/kern/vfs_subr.c:2361 > 2nd 0xc2fbbe24 ntnode (ntnode) @ /usr/src/sys/modules/ntfs/../../fs/ntfs/ntfs_s > ubr.c:361 > KDB: stack backtrace: > db_trace_self_wrapper(c0b0fa49,d614ea38,c07a2c5e,c0b11f90,c2fbbe24,...) at db_tr > ace_self_wrapper+0x26 > kdb_backtrace(c0b11f90,c2fbbe24,c301f540,c301f540,c301f6aa,...) at kdb_backtrace > +0x29 > witness_checkorder(c2fbbe24,9,c301f6aa,169,167,...) at witness_checkorder+0x6de > _lockmgr(c2fbbe24,2002,c2fbbe54,c301f6aa,169,...) at _lockmgr+0x448 > ntfs_ntget(c2fbbe00,c3020320,d614eab4,c2fbbe00,c3020320,...) at ntfs_ntget+0x5d > ntfs_reclaim(d614eb04,c0b3ef3d,0,c2fecdd0,c3021aa0,...) at ntfs_reclaim+0x3b > VOP_RECLAIM_APV(c3020320,d614eb04,c3021aa0,0,0,...) at VOP_RECLAIM_APV+0xc5 > vgonel(c2fece58,0,c0b1840e,95b,1,...) at vgonel+0x1cf > vflush(c2fc2538,0,1,c3021aa0,c0e1bfe0,...) at vflush+0x367 > ntfs_unmount(c2fc2538,8000000,c3021aa0,4f0,4da,...) at ntfs_unmount+0x4f > dounmount(c2fc2538,8000000,c3021aa0,482,8,...) at dounmount+0x426 > unmount(c3021aa0,d614ecfc,8,d614ed38,c0bbeef0,...) at unmount+0x2e0 > syscall(d614ed38) at syscall+0x2b3 > > Xint0x80_syscall() at Xint0x80_syscall+0x20 > > --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280c62db, esp = 0xbfbfe6ac, eb > p = 0xbfbfe768 --- > kernel trap 12 with interrupts disabled > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 0; apic id = 00 > > fault virtual address = 0xdeadc0ee > fault code = supervisor read, page not present > instruction pointer = 0x20:0xc07a0676 > stack pointer = 0x28:0xd614e9a0 > frame pointer = 0x28:0xd614e9a4 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, def32 1, gran 1 > > processor eflags = resume, IOPL = 0 > > current process = 43 (umount) > [thread pid 43 tid 100052 ] > Stopped at isitmychild+0x6: movl 0x10(%eax),%ecx > db> panic: Assertion !mtx_owned(&w_mtx) failed at /usr/src/sys/kern/subr_witness > .c:959 > cpuid = 0 > Uptime: 2m14s > Cannot dump. No dump device defined. > Automatic reboot in 15 seconds - press a key on the console to abort It would be suitable for you to add DDB to your kernel config and see a backtrace for it? Attilio -- Peace can only be achieved by understanding - A. Einstein