Date: Fri, 27 Sep 2013 10:58:14 +0200 From: "Ronald Klop" <ronald-freebsd8@klop.yi.org> To: freebsd-stable@freebsd.org Subject: Re: lock order reversal in 10-alpha2 Message-ID: <op.w32a7cpg8527sy@212-182-167-131.ip.telfort.nl> In-Reply-To: <52450239.7010100@ShaneWare.Biz> References: <52450239.7010100@ShaneWare.Biz>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 27 Sep 2013 05:57:45 +0200, Shane Ambler <FreeBSD@shaneware.biz> wrote: > After booting from a 10-alpha2 disk I am seeing "lock order reversal" > messages show up from time to time. Current logs have 35 entries. FreeBSD 10-ALPHA is still being build with kernel option WITNESS on. This gives more diagnostics of internal state of the kernel. A lot of these LORs (Lock Order Reverals) should be fixed some day, but are harmless for the continues working of the machine. If you compile FreeBSD 9 with WITNESS on, you will some LORs also. Ronald. > > The machine normally is running 9.1 from zfs root and I have setup a > separate disk (eSATA case connected through backplane port to onboard > SATA port) that I have installed 10-alpha amd64 onto a ufs partition to > test port building with. I started by building 10 alpha1 and installing > onto the new disk. I have since done svn up (last revision is 255868) > then rebuilt and installed kernel and world while running 10 and still > see these messages. > > I mentioned the existing 9.1 on zfs which I am not importing while > running 10 from ufs as I noticed zfs mentioned in one of the entries. > > Initially I built with an empty src.conf but the last build I used the > following - > > WITH_BSD_GREP=yes > WITH_CLANG_EXTRAS=yes > WITH_CTF=yes > WITHOUT_LIB32=yes > WITH_LLDB=yes > > Hardware is ASUS P8H61-M LE/USB3 corei5 8GB RAM nvidia GT520 > > I can provide full copy of log/messages or dmesg if required. > > A few samples -- > > messages:Sep 26 02:01:27 leader kernel: lock order reversal: > messages-Sep 26 02:01:27 leader kernel: 1st 0xfffffe01eebd07f8 bufwait > (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3059 > messages-Sep 26 02:01:27 leader kernel: 2nd 0xfffff800122f8200 dirhash > (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:284 > messages-Sep 26 02:01:27 leader kernel: KDB: stack backtrace: > messages-Sep 26 02:01:27 leader kernel: db_trace_self_wrapper() at > db_trace_self_wrapper+0x2b/frame 0xfffffe0238a8f270 > messages-Sep 26 02:01:27 leader kernel: kdb_backtrace() at > kdb_backtrace+0x39/frame 0xfffffe0238a8f320 > messages-Sep 26 02:01:27 leader kernel: witness_checkorder() at > witness_checkorder+0xd23/frame 0xfffffe0238a8f3b0 > messages-Sep 26 02:01:27 leader kernel: _sx_xlock() at > _sx_xlock+0x75/frame 0xfffffe0238a8f3f0 > messages-Sep 26 02:01:27 leader kernel: ufsdirhash_add() at > ufsdirhash_add+0x3b/frame 0xfffffe0238a8f430 > messages-Sep 26 02:01:27 leader kernel: ufs_direnter() at > ufs_direnter+0x688/frame 0xfffffe0238a8f4f0 > messages-Sep 26 02:01:27 leader kernel: ufs_makeinode() at > ufs_makeinode+0x573/frame 0xfffffe0238a8f6b0 > messages-Sep 26 02:01:27 leader kernel: VOP_CREATE_APV() at > VOP_CREATE_APV+0xea/frame 0xfffffe0238a8f6e0 > messages-Sep 26 02:01:27 leader kernel: vn_open_cred() at > vn_open_cred+0x300/frame 0xfffffe0238a8f830 > messages-Sep 26 02:01:27 leader kernel: kern_openat() at > kern_openat+0x261/frame 0xfffffe0238a8f9a0 > messages-Sep 26 02:01:27 leader kernel: amd64_syscall() at > amd64_syscall+0x265/frame 0xfffffe0238a8fab0 > messages-Sep 26 02:01:27 leader kernel: Xfast_syscall() at > Xfast_syscall+0xfb/frame 0xfffffe0238a8fab0 > messages-Sep 26 02:01:27 leader kernel: --- syscall (5, FreeBSD ELF64, > sys_open), rip = 0x80185baca, rsp = 0x7fffffffd168, rbp = 0x7fffffffd1a0 > --- > > messages.0:Sep 23 10:08:11 leader kernel: lock order reversal: > messages.0-Sep 23 10:08:11 leader kernel: 1st 0xfffff801ba2e65f0 ufs > (ufs) @ /usr/src/sys/kern/vfs_syscalls.c:3435 > messages.0-Sep 23 10:08:11 leader kernel: 2nd 0xfffffe01ef93c1c0 bufwait > (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:262 > messages.0-Sep 23 10:08:11 leader kernel: 3rd 0xfffff801ba2e6240 ufs > (ufs) @ /usr/src/sys/kern/vfs_subr.c:2099 > messages.0-Sep 23 10:08:11 leader kernel: KDB: stack backtrace: > messages.0-Sep 23 10:08:11 leader kernel: db_trace_self_wrapper() at > db_trace_self_wrapper+0x2b/frame 0xfffffe02397c2300 > messages.0-Sep 23 10:08:11 leader kernel: kdb_backtrace() at > kdb_backtrace+0x39/frame 0xfffffe02397c23b0 > messages.0-Sep 23 10:08:11 leader kernel: witness_checkorder() at > witness_checkorder+0xd23/frame 0xfffffe02397c2440 > messages.0-Sep 23 10:08:11 leader kernel: __lockmgr_args() at > __lockmgr_args+0x6f2/frame 0xfffffe02397c2570 > messages.0-Sep 23 10:08:11 leader kernel: ffs_lock() at > ffs_lock+0x84/frame 0xfffffe02397c25c0 > messages.0-Sep 23 10:08:11 leader kernel: VOP_LOCK1_APV() at > VOP_LOCK1_APV+0xf5/frame 0xfffffe02397c25f0 > messages.0-Sep 23 10:08:11 leader kernel: _vn_lock() at > _vn_lock+0xab/frame 0xfffffe02397c2660 > messages.0-Sep 23 10:08:11 leader kernel: vget() at vget+0x70/frame > 0xfffffe02397c26b0 > messages.0-Sep 23 10:08:11 leader kernel: vfs_hash_get() at > vfs_hash_get+0xf5/frame 0xfffffe02397c2700 > messages.0-Sep 23 10:08:11 leader kernel: ffs_vgetf() at > ffs_vgetf+0x41/frame 0xfffffe02397c2790 > messages.0-Sep 23 10:08:11 leader kernel: softdep_sync_buf() at > softdep_sync_buf+0x8fa/frame 0xfffffe02397c2840 > messages.0-Sep 23 10:08:11 leader kernel: ffs_syncvnode() at > ffs_syncvnode+0x258/frame 0xfffffe02397c28c0 > messages.0-Sep 23 10:08:11 leader kernel: ffs_fsync() at > ffs_fsync+0x20/frame 0xfffffe02397c28f0 > messages.0-Sep 23 10:08:11 leader kernel: VOP_FSYNC_APV() at > VOP_FSYNC_APV+0xf0/frame 0xfffffe02397c2920 > messages.0-Sep 23 10:08:11 leader kernel: sys_fsync() at > sys_fsync+0x156/frame 0xfffffe02397c29a0 > messages.0-Sep 23 10:08:11 leader kernel: amd64_syscall() at > amd64_syscall+0x265/frame 0xfffffe02397c2ab0 > messages.0-Sep 23 10:08:11 leader kernel: Xfast_syscall() at > Xfast_syscall+0xfb/frame 0xfffffe02397c2ab0 > messages.0-Sep 23 10:08:11 leader kernel: --- syscall (95, FreeBSD > ELF64, sys_fsync), rip = 0x8029a41fa, rsp = 0x7fffffffcf28, rbp = > 0x7fffffffcf40 --- > > messages.0:Sep 23 10:08:11 leader kernel: lock order reversal: > messages.0-Sep 23 10:08:11 leader kernel: 1st 0xfffff801ba2e65f0 ufs > (ufs) @ /usr/src/sys/kern/vfs_syscalls.c:3435 > messages.0-Sep 23 10:08:11 leader kernel: 2nd 0xfffffe01ef93c1c0 bufwait > (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:262 > messages.0-Sep 23 10:08:11 leader kernel: 3rd 0xfffff801ba2e6240 ufs > (ufs) @ /usr/src/sys/kern/vfs_subr.c:2099 > messages.0-Sep 23 10:08:11 leader kernel: KDB: stack backtrace: > messages.0-Sep 23 10:08:11 leader kernel: db_trace_self_wrapper() at > db_trace_self_wrapper+0x2b/frame 0xfffffe02397c2300 > messages.0-Sep 23 10:08:11 leader kernel: kdb_backtrace() at > kdb_backtrace+0x39/frame 0xfffffe02397c23b0 > messages.0-Sep 23 10:08:11 leader kernel: witness_checkorder() at > witness_checkorder+0xd23/frame 0xfffffe02397c2440 > messages.0-Sep 23 10:08:11 leader kernel: __lockmgr_args() at > __lockmgr_args+0x6f2/frame 0xfffffe02397c2570 > messages.0-Sep 23 10:08:11 leader kernel: ffs_lock() at > ffs_lock+0x84/frame 0xfffffe02397c25c0 > messages.0-Sep 23 10:08:11 leader kernel: VOP_LOCK1_APV() at > VOP_LOCK1_APV+0xf5/frame 0xfffffe02397c25f0 > messages.0-Sep 23 10:08:11 leader kernel: _vn_lock() at > _vn_lock+0xab/frame 0xfffffe02397c2660 > messages.0-Sep 23 10:08:11 leader kernel: vget() at vget+0x70/frame > 0xfffffe02397c26b0 > messages.0-Sep 23 10:08:11 leader kernel: vfs_hash_get() at > vfs_hash_get+0xf5/frame 0xfffffe02397c2700 > messages.0-Sep 23 10:08:11 leader kernel: ffs_vgetf() at > ffs_vgetf+0x41/frame 0xfffffe02397c2790 > messages.0-Sep 23 10:08:11 leader kernel: softdep_sync_buf() at > softdep_sync_buf+0x8fa/frame 0xfffffe02397c2840 > messages.0-Sep 23 10:08:11 leader kernel: ffs_syncvnode() at > ffs_syncvnode+0x258/frame 0xfffffe02397c28c0 > messages.0-Sep 23 10:08:11 leader kernel: ffs_fsync() at > ffs_fsync+0x20/frame 0xfffffe02397c28f0 > messages.0-Sep 23 10:08:11 leader kernel: VOP_FSYNC_APV() at > VOP_FSYNC_APV+0xf0/frame 0xfffffe02397c2920 > messages.0-Sep 23 10:08:11 leader kernel: sys_fsync() at > sys_fsync+0x156/frame 0xfffffe02397c29a0 > messages.0-Sep 23 10:08:11 leader kernel: amd64_syscall() at > amd64_syscall+0x265/frame 0xfffffe02397c2ab0 > messages.0-Sep 23 10:08:11 leader kernel: Xfast_syscall() at > Xfast_syscall+0xfb/frame 0xfffffe02397c2ab0 > messages.0-Sep 23 10:08:11 leader kernel: --- syscall (95, FreeBSD > ELF64, sys_fsync), rip = 0x8029a41fa, rsp = 0x7fffffffcf28, rbp = > 0x7fffffffcf40 --- > > messages.0:Sep 23 10:36:02 leader kernel: lock order reversal: > messages.0-Sep 23 10:36:02 leader kernel: 1st 0xfffff801ba9be240 zfs > (zfs) @ /usr/src/sys/kern/vfs_mount.c:1237 > messages.0-Sep 23 10:36:02 leader kernel: 2nd 0xfffff801babab7c8 syncer > (syncer) @ /usr/src/sys/kern/vfs_subr.c:2210 > messages.0-Sep 23 10:36:02 leader kernel: KDB: stack backtrace: > messages.0-Sep 23 10:36:02 leader kernel: db_trace_self_wrapper() at > db_trace_self_wrapper+0x2b/frame 0xfffffe02397ef460 > messages.0-Sep 23 10:36:02 leader kernel: kdb_backtrace() at > kdb_backtrace+0x39/frame 0xfffffe02397ef510 > messages.0-Sep 23 10:36:02 leader kernel: witness_checkorder() at > witness_checkorder+0xd23/frame 0xfffffe02397ef5a0 > messages.0-Sep 23 10:36:02 leader kernel: __lockmgr_args() at > __lockmgr_args+0x6f2/frame 0xfffffe02397ef6d0 > messages.0-Sep 23 10:36:02 leader kernel: vop_stdlock() at > vop_stdlock+0x3c/frame 0xfffffe02397ef6f0 > messages.0-Sep 23 10:36:02 leader kernel: VOP_LOCK1_APV() at > VOP_LOCK1_APV+0xf5/frame 0xfffffe02397ef720 > messages.0-Sep 23 10:36:02 leader kernel: _vn_lock() at > _vn_lock+0xab/frame 0xfffffe02397ef790 > messages.0-Sep 23 10:36:02 leader kernel: vputx() at vputx+0x208/frame > 0xfffffe02397ef7f0 > messages.0-Sep 23 10:36:02 leader kernel: dounmount() at > dounmount+0x327/frame 0xfffffe02397ef870 > messages.0-Sep 23 10:36:02 leader kernel: sys_unmount() at > sys_unmount+0x356/frame 0xfffffe02397ef9a0 > messages.0-Sep 23 10:36:02 leader kernel: amd64_syscall() at > amd64_syscall+0x265/frame 0xfffffe02397efab0 > messages.0-Sep 23 10:36:02 leader kernel: Xfast_syscall() at > Xfast_syscall+0xfb/frame 0xfffffe02397efab0 > messages.0-Sep 23 10:36:02 leader kernel: --- syscall (22, FreeBSD > ELF64, sys_unmount), rip = 0x80191f24a, rsp = 0x7fffffffc3d8, rbp = > 0x7fffffffc860 --- > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?op.w32a7cpg8527sy>