Date: Thu, 12 Sep 2013 10:36:43 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Jimmy Olgeni <olgeni@olgeni.com> Cc: freebsd-stable@freebsd.org Subject: Re: Possible kqueue related issue on STABLE/RC. Message-ID: <20130912073643.GM41229@kib.kiev.ua> In-Reply-To: <alpine.BSF.2.00.1309112316160.2547@olgeni.olgeni> References: <alpine.BSF.2.00.1309111705460.89324@olgeni.olgeni> <20130911171913.GG41229@kib.kiev.ua> <alpine.BSF.2.00.1309112316160.2547@olgeni.olgeni>
next in thread | previous in thread | raw e-mail | index | archive | help
--dhq2JOS40XFo2g0l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 11, 2013 at 11:18:34PM +0200, Jimmy Olgeni wrote: >=20 > Hi, >=20 > On Wed, 11 Sep 2013, Konstantin Belousov wrote: >=20 > > Also, do you have all options listed at > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/ke= rneldebug-deadlocks.html > > enabled ? >=20 > This time I tried with clang + these options and I got something more=20 > interesting. All works fine until the lock violation below: Clang is, well, not relevant there. >=20 > acquiring duplicate lock of same type: "os.lock_mtx" > 1st os.lock_mtx @ nvidia_os.c:748 > 2nd os.lock_mtx @ nvidia_os.c:748 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff8360e= 1e2f0 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff8360e1e3a0 > witness_checkorder() at witness_checkorder+0xc0a/frame 0xffffff8360e1e420 > _mtx_lock_flags() at _mtx_lock_flags+0x74/frame 0xffffff8360e1e460 > os_acquire_spinlock() at os_acquire_spinlock+0x17/frame 0xffffff8360e1e470 > _nv012281rm() at _nv012281rm+0x9/frame 0xffffff800cfadec0 > lock order reversal: > 1st 0xfffffe003603c098 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1240 > 2nd 0xfffffe003603b848 syncer (syncer) @ /usr/src/sys/kern/vfs_subr.c:2= 335 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff8360b= f3660 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff8360bf3710 > witness_checkorder() at witness_checkorder+0xc0a/frame 0xffffff8360bf3790 > __lockmgr_args() at __lockmgr_args+0x744/frame 0xffffff8360bf38b0 > vop_stdlock() at vop_stdlock+0x3c/frame 0xffffff8360bf38d0 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0xbe/frame 0xffffff8360bf3900 > _vn_lock() at _vn_lock+0x63/frame 0xffffff8360bf3960 > vputx() at vputx+0x34b/frame 0xffffff8360bf39c0 > dounmount() at dounmount+0x282/frame 0xffffff8360bf3a30 > sys_unmount() at sys_unmount+0x3a6/frame 0xffffff8360bf3b20 > amd64_syscall() at amd64_syscall+0x259/frame 0xffffff8360bf3c30 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff8360bf3c30 > --- syscall (22, FreeBSD ELF64, sys_unmount), rip =3D 0x801918a7c, rsp = =3D 0x7fffffffbf18, rbp =3D 0x802818800 --- > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff83611= df6b0 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff83611df760 > assert_vop_elocked() at assert_vop_elocked+0x6a/frame 0xffffff83611df790 > fifo_open() at fifo_open+0x38/frame 0xffffff83611df810 > VOP_OPEN_APV() at VOP_OPEN_APV+0xd1/frame 0xffffff83611df840 > vn_open_cred() at vn_open_cred+0x532/frame 0xffffff83611df9b0 > kern_openat() at kern_openat+0x1c1/frame 0xffffff83611dfb20 > amd64_syscall() at amd64_syscall+0x259/frame 0xffffff83611dfc30 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff83611dfc30 >=20 >=20 > Here it goes: >=20 >=20 > --- syscall (5, FreeBSD ELF64, sys_open), rip =3D 0x800db3d3c, rsp =3D 0x= 7fffffffc968, rbp =3D 0 --- > fifo_open: 0xfffffe0063251cd0 is not exclusive locked but should be > KDB: enter: lock violation >=20 > 0xfffffe0063251cd0: tag zfs, type VFIFO > usecount 2, writecount 0, refcount 2 mountedhere 0xfffffe005048cc00 > flags (VI_ACTIVE) > lock type zfs: SHARED (count 1) > #0 0xffffffff808b22fa at __lockmgr_args+0xeba > #1 0xffffffff8095df4c at vop_stdlock+0x3c > #2 0xffffffff80d7f6ae at VOP_LOCK1_APV+0xbe > #3 0xffffffff80980153 at _vn_lock+0x63 > #4 0xffffffff8096e321 at vget+0xa1 > #5 0xffffffff80959921 at cache_lookup_times+0x591 > #6 0xffffffff8095aa7d at vfs_cache_lookup+0x9d > #7 0xffffffff80d7c881 at VOP_LOOKUP_APV+0xd1 > #8 0xffffffff80963aff at lookup+0x6bf > #9 0xffffffff80963029 at namei+0x589 > #10 0xffffffff8097fa0f at vn_open_cred+0x27f > #11 0xffffffff80978031 at kern_openat+0x1c1 > #12 0xffffffff80ccce49 at amd64_syscall+0x259 > #13 0xffffffff80cb5e9b at Xfast_syscall+0xfb > , fifo with 0 readers and 1 writers It seems you edited the kernel output, at least rearranging large blocks of text. I tried to interpret what I see in a useful way. Might be, your issue is that some filesystems do not care about proper locking mode for the fifos. UFS carefully disables shared locking for VFIFO, but it seems ZFS is not. I can propose the following band-aid, which could help you. I have no idea is it the same issue as the kqueue panic. diff --git a/sys/kern/vfs_vnops.c b/sys/kern/vfs_vnops.c index c53030a..00bd998 100644 --- a/sys/kern/vfs_vnops.c +++ b/sys/kern/vfs_vnops.c @@ -267,6 +267,8 @@ vn_open_vnode(struct vnode *vp, int fmode, struct ucred= *cred, return (error); } } + if (vp->v_type =3D=3D VFIFO && VOP_ISLOCKED(vp) !=3D LK_EXCLUSIVE) + vn_lock(vp, LK_UPGRADE | LK_RETRY); if ((error =3D VOP_OPEN(vp, fmode, cred, td, fp)) !=3D 0) return (error); =20 @@ -358,7 +360,7 @@ vn_close(vp, flags, file_cred, td) struct mount *mp; int error, lock_flags; =20 - if (!(flags & FWRITE) && vp->v_mount !=3D NULL && + if (vp->v_type !=3D VFIFO && !(flags & FWRITE) && vp->v_mount !=3D NULL && vp->v_mount->mnt_kern_flag & MNTK_EXTENDED_SHARED) lock_flags =3D LK_SHARED; else --dhq2JOS40XFo2g0l Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQIcBAEBAgAGBQJSMW8KAAoJEJDCuSvBvK1B2UUP/2i9sQ15vwor1+Uiiz/W9IFt 2II3kbht/Mtexoxd24OU0SDor9owVrVzpimWbS+h5BL3SGfD8+y3PQSmf64Rv5qZ 5jOr4XwTpn27YEVLE/ep8nJthuClYgK8e/5TojlroYOVe/O3Ib4AggrWJ6IqLKvH HjUFFqdTQ4jIr8KtwPJCzoz0uQyGzSL+3HEA8wZKoMON1wDxjh4oMZuYjk10EnSA oVZbpJCgohwRo3jjwSSiFPvCUrJNrICK7uEjUK89sE37bFwf9NaQLBINm5EOO6sy V726Zfsi8YT5cuoZpgbzljw8WZgbcsi+MGOi9fOav3GOQZ+oorijuDQaF+dRNQVw oD4pm/X1LDPH9OvJ81KtA8d2iL88dzdBkbQ0yUQm9639f2UfSWqo6JgMvNCajob2 RWXKMgI0aUk5Ogk4eGo/fZEe/idKsi02uCXD3SA83CO2bODQmAq6sdHgqa5/HNR8 W5BQOmdapjr8EUw3K5BnJHr/W+KEw6Makk2KxEYJKQ8JXWFxijCjRKblxmOif0g4 yLrtts9LBu4heWHg3/rm1qAwFDN2uTldvuNgSwoP3hf1gmx8UKJbiv/Wk3C11Yi9 JYoq23DOKlwEI4Kxs2b7Ba2tTRMNhiX0o6TRg92idbM9TfEqcabIaa2gLy+z5MB0 b24TvhONzstOW134hPdz =7S8r -----END PGP SIGNATURE----- --dhq2JOS40XFo2g0l--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130912073643.GM41229>