From owner-freebsd-stable@FreeBSD.ORG Thu Feb 25 08:14:05 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D57DC106566C for ; Thu, 25 Feb 2010 08:14:05 +0000 (UTC) (envelope-from jholland@fastsoft.com) Received: from hq-es.fastsoft.com (hq-es.fastsoft.com [38.102.243.86]) by mx1.freebsd.org (Postfix) with ESMTP id BDDC38FC19 for ; Thu, 25 Feb 2010 08:14:05 +0000 (UTC) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Thu, 25 Feb 2010 00:14:04 -0800 Message-ID: In-Reply-To: <20100225073317.GA82327@icarus.home.lan> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: vfs deadlock during panic? Thread-Index: Acq17P2HqgDeHy/bQtC5+FwnOKe37gAArmzg References: <20100225073317.GA82327@icarus.home.lan> From: "Jake Holland" To: Subject: RE: vfs deadlock during panic? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 08:14:05 -0000 > All the backtraces indicate UFS snapshots are being used, as in=20 > you're doing dump -L or mksnap_ffs. Is that the case? Not to my knowledge, but I'm not sure of everything that's happening during startup. The 3 I listed last time appear in the serial console, usually within 10s of the login prompt's first appearance after reboot. If I do more file operations, I typically see more such messages, but not every time. For example, I tried running umount on my /var/crash (mounted on /dev/mfid0s3d) and it gave me this: lock order reversal: 1st 0xffffff00104337e8 ufs (ufs) @ kern/vfs_mount.c:1204 2nd 0xffffff001045aa58 devfs (devfs) @ ufs/ffs/ffs_vfsops.c:1194 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e __lockmgr_args() at __lockmgr_args+0xd31 vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 ffs_flushfiles() at ffs_flushfiles+0xc5 softdep_flushfiles() at softdep_flushfiles+0x63 ffs_unmount() at ffs_unmount+0x2e1 dounmount() at dounmount+0x2e6 unmount() at unmount+0x27e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (22, FreeBSD ELF64, unmount), rip =3D 0x80069f4bc, rsp =3D 0x7fffffffe2f 8, rbp =3D 0 --- But I got nothing when I ran mount -a afterwards. Then I ran vi on /etc/ports.conf and saw this: lock order reversal: 1st 0xffffff81983bd5b8 bufwait (bufwait) @ kern/vfs_bio.c:2559 2nd 0xffffff000bd2ac00 dirhash (dirhash) @ ufs/ufs/ufs_dirhash.c:285 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 ufsdirhash_acquire() at ufsdirhash_acquire+0x44 ufsdirhash_add() at ufsdirhash_add+0x19 ufs_direnter() at ufs_direnter+0x88b ufs_makeinode() at ufs_makeinode+0x2a7 VOP_CREATE_APV() at VOP_CREATE_APV+0xb3 vn_open_cred() at vn_open_cred+0x482 kern_openat() at kern_openat+0x179 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (5, FreeBSD ELF64, open), rip =3D 0x800ce075c, rsp =3D 0x7fffffffe0d8, r bp =3D 0xf --- But only the first time. Subsequently re-opening the same or another few arbitrary text files with vi did not produce any repeats. And on some occasions, nothing came up even on the first run of vi, though usually the first vi does give a similar stack.