Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 27 Jul 2014 16:44:42 +0200
From:      Harald Schmalzbauer <h.schmalzbauer@omnilan.de>
To:        Konstantin Belousov <kostikbel@gmail.com>
Cc:        FreeBSD Stable <freebsd-stable@freebsd.org>
Subject:   Re: panic/lock on 9.3-RELEASE with nullfs/nfs/zfs combination
Message-ID:  <53D5105A.6040204@omnilan.de>
In-Reply-To: <20140726114839.GF93733@kib.kiev.ua>
References:  <53D12973.3010805@omnilan.de> <20140724165917.GT93733@kib.kiev.ua> <53D1503B.2030200@omnilan.de> <20140724193048.GU93733@kib.kiev.ua> <53D2006C.7090207@omnilan.de> <20140725151448.GY93733@kib.kiev.ua> <53D380B2.2080700@omnilan.de> <20140726114839.GF93733@kib.kiev.ua>

next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig09804AF5039DD5A9FD7ABF27
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

 Bez=C3=BCglich Konstantin Belousov's Nachricht vom 26.07.2014 13:48
(localtime):
> On Sat, Jul 26, 2014 at 12:19:30PM +0200, Harald Schmalzbauer wrote:
>> vnode vnode 0xfffffe006cb86bd0: 0xfffffe006cb86bd0: tag null, type VDI=
R
>> tag null, type VDIR
>> usecount 1, writecount 0, refcount 1 mountedhere 0
>> usecount 1, writecount 0, refcount 1 mountedhere 0
>> flags (VV_ROOT|VI_ACTIVE)
>> flags (VV_ROOT|VI_ACTIVE)
>> v_object 0xfffffe006c8d30e8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
>> v_object 0xfffffe006c8d30e8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
>> lock type zfs: EXCL by thread 0xfffffe0033498920 (pid 1668)
>> lock type zfs: EXCL by thread 0xfffffe0033498920 (pid 1668)
>> vp=3D0xfffffe006cb86bd0, lowervp=3D0xfffffe001dbc53f0
>> vp=3D0xfffffe006cb86bd0, lowervp=3D0xfffffe001dbc53f0
>>
> It is useful, but still requires more work to get to the issue. From th=
e
> data you posted, I see that the problem was already reported sometime
> this winter. We did not come to any conclusion that time.
>
> Please, do the following. First, apply this debugging patch and obtain
> the same data as you did right now. Hopefully, the assert below would
> trigger.
>
> diff --git a/sys/fs/nullfs/null_vnops.c b/sys/fs/nullfs/null_vnops.c
> index 70402e3..bc9772c 100644
> --- a/sys/fs/nullfs/null_vnops.c
> +++ b/sys/fs/nullfs/null_vnops.c
> @@ -372,6 +372,10 @@ null_lookup(struct vop_lookup_args *ap)
>  	 */
>  	ldvp =3D NULLVPTOLOWERVP(dvp);
>  	vp =3D lvp =3D NULL;
> +	KASSERT((ldvp->v_vflag & VV_ROOT) =3D=3D 0 ||
> +	    ((dvp->v_vflag & VV_ROOT) !=3D 0 && (flags & ISDOTDOT) =3D=3D 0),=

> +	    ("ldvp %p fl %#x dvp %p fl %#x flags %#x", ldvp, ldvp->v_vflag,
> +	     dvp, dvp->v_vflag, flags));
>  	error =3D VOP_LOOKUP(ldvp, &lvp, cnp);
>  	if (error =3D=3D EJUSTRETURN && (flags & ISLASTCN) &&
>  	    (dvp->v_mount->mnt_flag & MNT_RDONLY) &&

I can't emphasize how much I hate my lousy C skills... Got "format '%#x'
expects type 'unsigned int', but argument 3 has type 'u_long'
[-Wformat]" from the compiler and found out that I can't read/understand
any part of your patch; not even the syntax :-(

After reading printf(3) I change the 3rd additional line into
'("ldvp %p fl %#lx dvp %p fl %#lx flags %#x", ldvp, ldvp->v_vflag,'
=E2=80=A6 A blind man may sometimes hit the mark.

Here's the console output with an additional (ufs/devfs) LOR prepended:
1st 0xfffffe014bc09098 ufs (ufs) @
/usr/local/share/deploy-tools/RELENG_9_3/src/sys/kern/vfs_mount.c:851
2nd 0xfffffe014bd97098 devfs (devfs) @
/usr/local/share/deploy-tools/RELENG_9_3/src/sys/kern/vfs_subr.c:2225
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame
0xffffff82e4f7c280
kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffff82e4f7c340
_witness_debugger() at _witness_debugger+0x2c/frame 0xffffff82e4f7c360
witness_checkorder() at witness_checkorder+0x875/frame 0xffffff82e4f7c420=

__lockmgr_args() at __lockmgr_args+0x1197/frame 0xffffff82e4f7c500
vop_stdlock() at vop_stdlock+0x39/frame 0xffffff82e4f7c520
VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe3/frame 0xffffff82e4f7c550
_vn_lock() at _vn_lock+0x57/frame 0xffffff82e4f7c5b0
vget() at vget+0x7b/frame 0xffffff82e4f7c600
devfs_allocv() at devfs_allocv+0x13e/frame 0xffffff82e4f7c660
devfs_root() at devfs_root+0x4d/frame 0xffffff82e4f7c6a0
vfs_donmount() at vfs_donmount+0xb89/frame 0xffffff82e4f7c990
sys_nmount() at sys_nmount+0x66/frame 0xffffff82e4f7c9d0
amd64_syscall() at amd64_syscall+0x318/frame 0xffffff82e4f7caf0
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff82e4f7caf0
--- syscall (378, FreeBSD ELF64, sys_nmount), rip =3D 0x800a94ecc, rsp =3D=

0x7fffffffdc08, rbp =3D 0x7fffffffdc20 ---

Here's the panic:
panic: ldvp 0xfffffe001bafb000 fl 0x1 dvp 0xfffffe01b4e5a9d8 fl 0 flags
0x120e144
cpuid =3D 1
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame
0xffffff82e4e88f80
kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffff82e4e89040
panic() at panic+0x1cd/frame 0xffffff82e4e89140
null_lookup() at null_lookup+0xa6/frame 0xffffff82e4e891c0
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x102/frame 0xffffff82e4e891f0
lookup() at lookup+0x32f/frame 0xffffff82e4e89290
namei() at namei+0x3df/frame 0xffffff82e4e89340
vn_open_cred() at vn_open_cred+0x1e2/frame 0xffffff82e4e894b0
vop_stdvptocnp() at vop_stdvptocnp+0x1af/frame 0xffffff82e4e897e0
null_vptocnp() at null_vptocnp+0xf5/frame 0xffffff82e4e89850
VOP_VPTOCNP_APV() at VOP_VPTOCNP_APV+0x105/frame 0xffffff82e4e89880
vn_vptocnp_locked() at vn_vptocnp_locked+0x15b/frame 0xffffff82e4e89910
vn_fullpath1() at vn_fullpath1+0x100/frame 0xffffff82e4e89970
kern___getcwd() at kern___getcwd+0xd4/frame 0xffffff82e4e899d0
amd64_syscall() at amd64_syscall+0x318/frame 0xffffff82e4e89af0
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff82e4e89af0
--- syscall (326, FreeBSD ELF64, sys___getcwd), rip =3D 0x8011a191c, rsp =
=3D
0x7fffffffe658, rbp =3D 0x801873400 ---
KDB: enter: panic
[ thread pid 1676 tid 100824 ]
Stopped at kdb_enter+0x3b: movq $0,0x6455e2(%rip)
db> show alllocks
Process 1676 (vim) thread 0xfffffe0036fb8490 (100824)
shared lockmgr zfs (zfs) r =3D 0 (0xfffffe001bafb098) locked @
/usr/local/share/deploy-tools/RELENG_9_3/src/sys/fs/nullfs/null_vnops.c:6=
24
db> show mount
=E2=80=A6
0xfffffe0007b03338
/zfs/netshares/deployment/pub/FreeBSD/OmniLAN/ports/ports on
/.JAILfbsm/usr/ports (nullfs)
=E2=80=A6
db> show mount 0xfffffe0007b03338
0xfffffe0007b03338
/zfs/netshares/deployment/pub/FreeBSD/OmniLAN/ports/ports on
/.JAILfbsm/usr/ports (nullfs)
mnt_flag =3D LOCAL
mnt_kern_flag =3D EXTENDED_SHARED, SHARED_WRITES, LOOKUP_EXCL_DOTDOT,
MPSAFE, LOOKUP_SHARED
mnt_opt =3D rw, fstype, fspath, target, errmsg
mnt_stat =3D { version=3D537068824 type=3D222 flags=3D0x000000001000111c
bsize=3D512 iosize=3D131072 blocks=3D6314028651 bfree=3D6309830121
bavail=3D6309830121 files=3D6310125501 ffree=3D6309830121 syncwrites=3D0
asyncwrites=3D0 syncreads=3D0 asyncreads=3D0 namemax=3D255 owner=3D0
fsid=3D[687931140, 41] }
mnt_cred =3D { uid=3D0 ruid=3D0 }
mnt_ref =3D 56
mnt_gen =3D 1
mnt_nvnodelistsize =3D 56
mnt_activevnodelistsize =3D 5
mnt_writeopcount =3D 0
mnt_maxsymlinklen =3D 0
mnt_iosize_max =3D 65536
mnt_hashseed =3D 308101864
mnt_secondary_writes =3D 0
mnt_secondary_accwrites =3D 0
mnt_gjprovider =3D NULL


List of active vnodes
vnode vnode 0xfffffe01b4e5a9d8: 0xfffffe01b4e5a9d8: tag null, type VDIR
tag null, type VDIR
usecount 2, writecount 0, refcount 2 mountedhere 0
usecount 2, writecount 0, refcount 2 mountedhere 0
flags (VI_ACTIVE)
flags (VI_ACTIVE)
v_object 0xfffffe014bbf4828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bbf4828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: SHARED (count 1)
lock type zfs: SHARED (count 1)
vp=3D0xfffffe01b4e5a9d8, lowervp=3D0xfffffe001bafb000
vp=3D0xfffffe01b4e5a9d8, lowervp=3D0xfffffe001bafb000
vnode vnode 0xfffffe014bf26000: 0xfffffe014bf26000: tag null, type VDIR
tag null, type VDIR
usecount 3, writecount 0, refcount 3 mountedhere 0
usecount 3, writecount 0, refcount 3 mountedhere 0
flags (VI_ACTIVE)
flags (VI_ACTIVE)
v_object 0xfffffe01b42ae488 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae488 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bf26000, lowervp=3D0xfffffe014bf261f8
vp=3D0xfffffe014bf26000, lowervp=3D0xfffffe014bf261f8
vnode vnode 0xfffffe014bd633f0: 0xfffffe014bd633f0: tag null, type VDIR
tag null, type VDIR
usecount 1, writecount 0, refcount 1 mountedhere 0xfffffe001bb46000
usecount 1, writecount 0, refcount 1 mountedhere 0xfffffe001bb46000
flags (VI_ACTIVE)
flags (VI_ACTIVE)
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bd633f0, lowervp=3D0xfffffe014bd635e8
vp=3D0xfffffe014bd633f0, lowervp=3D0xfffffe014bd635e8
vnode vnode 0xfffffe014bc08000: 0xfffffe014bc08000: tag syncer, type VNON=

tag syncer, type VNON
usecount 1, writecount 0, refcount 1 mountedhere 0
usecount 1, writecount 0, refcount 1 mountedhere 0
flags (VI_ACTIVE)
flags (VI_ACTIVE)
lock type syncer: UNLOCKED
lock type syncer: UNLOCKED
vnode vnode 0xfffffe014bc03dc8: 0xfffffe014bc03dc8: tag null, type VDIR
tag null, type VDIR
usecount 1, writecount 0, refcount 1 mountedhere 0
usecount 1, writecount 0, refcount 1 mountedhere 0
flags (VV_ROOT|VI_ACTIVE)
flags (VV_ROOT|VI_ACTIVE)
v_object 0xfffffe014bbf4828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bbf4828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: SHARED (count 1)
lock type zfs: SHARED (count 1)
vp=3D0xfffffe014bc03dc8, lowervp=3D0xfffffe001bafb000
vp=3D0xfffffe014bc03dc8, lowervp=3D0xfffffe001bafb000


List of inactive vnodes
vnode vnode 0xfffffe014bd975e8: 0xfffffe014bd975e8: tag null, type VDIR
tag null, type VDIR
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bbf4740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bbf4740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bd975e8, lowervp=3D0xfffffe014b5c43f0
vp=3D0xfffffe014bd975e8, lowervp=3D0xfffffe014b5c43f0
vnode vnode 0xfffffe014bfe1bd0: 0xfffffe014bfe1bd0: tag null, type VDIR
tag null, type VDIR
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ca828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ca828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bfe1bd0, lowervp=3D0xfffffe014bfe1dc8
vp=3D0xfffffe014bfe1bd0, lowervp=3D0xfffffe014bfe1dc8
vnode vnode 0xfffffe014bf27000: 0xfffffe014bf27000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bf27000, lowervp=3D0xfffffe014bf271f8
vp=3D0xfffffe014bf27000, lowervp=3D0xfffffe014bf271f8
vnode vnode 0xfffffe014bf26bd0: 0xfffffe014bf26bd0: tag null, type VDIR
tag null, type VDIR
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014be0a3a0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014be0a3a0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bf26bd0, lowervp=3D0xfffffe014bf26dc8
vp=3D0xfffffe014bf26bd0, lowervp=3D0xfffffe014bf26dc8
vnode vnode 0xfffffe014bf267e0: 0xfffffe014bf267e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bf267e0, lowervp=3D0xfffffe014bf269d8
vp=3D0xfffffe014bf267e0, lowervp=3D0xfffffe014bf269d8
vnode vnode 0xfffffe014bf263f0: 0xfffffe014bf263f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bf263f0, lowervp=3D0xfffffe014bf265e8
vp=3D0xfffffe014bf263f0, lowervp=3D0xfffffe014bf265e8
vnode vnode 0xfffffe014bf5fdc8: 0xfffffe014bf5fdc8: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae1d0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae1d0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bf5fdc8, lowervp=3D0xfffffe014bf5fbd0
vp=3D0xfffffe014bf5fdc8, lowervp=3D0xfffffe014bf5fbd0
vnode vnode 0xfffffe014beeddc8: 0xfffffe014beeddc8: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf700e8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf700e8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014beeddc8, lowervp=3D0xfffffe014bd981f8
vp=3D0xfffffe014beeddc8, lowervp=3D0xfffffe014bd981f8
vnode vnode 0xfffffe014be037e0: 0xfffffe014be037e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf9c3a0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf9c3a0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014be037e0, lowervp=3D0xfffffe014bf5f9d8
vp=3D0xfffffe014be037e0, lowervp=3D0xfffffe014bf5f9d8
vnode vnode 0xfffffe014bc0b1f8: 0xfffffe014bc0b1f8: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf9b9f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf9b9f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bc0b1f8, lowervp=3D0xfffffe014be8c9d8
vp=3D0xfffffe014bc0b1f8, lowervp=3D0xfffffe014be8c9d8
vnode vnode 0xfffffe014be8c1f8: 0xfffffe014be8c1f8: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b4036cb0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b4036cb0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014be8c1f8, lowervp=3D0xfffffe014be8c7e0
vp=3D0xfffffe014be8c1f8, lowervp=3D0xfffffe014be8c7e0
vnode vnode 0xfffffe01b423b3f0: 0xfffffe01b423b3f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf70740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf70740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b423b3f0, lowervp=3D0xfffffe014be8c000
vp=3D0xfffffe01b423b3f0, lowervp=3D0xfffffe014be8c000
vnode vnode 0xfffffe014bfe21f8: 0xfffffe014bfe21f8: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bedecb0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bedecb0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014bfe21f8, lowervp=3D0xfffffe014bfe2000
vp=3D0xfffffe014bfe21f8, lowervp=3D0xfffffe014bfe2000
vnode vnode 0xfffffe014b45a5e8: 0xfffffe014b45a5e8: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b4036bc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b4036bc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b45a5e8, lowervp=3D0xfffffe0036f2bdc8
vp=3D0xfffffe014b45a5e8, lowervp=3D0xfffffe0036f2bdc8
vnode vnode 0xfffffe00366a0dc8: 0xfffffe00366a0dc8: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf15828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf15828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe00366a0dc8, lowervp=3D0xfffffe00366147e0
vp=3D0xfffffe00366a0dc8, lowervp=3D0xfffffe00366147e0
vnode vnode 0xfffffe0036614dc8: 0xfffffe0036614dc8: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42441d0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42441d0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe0036614dc8, lowervp=3D0xfffffe0036614bd0
vp=3D0xfffffe0036614dc8, lowervp=3D0xfffffe0036614bd0
vnode vnode 0xfffffe014b602bd0: 0xfffffe014b602bd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bdd39f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bdd39f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b602bd0, lowervp=3D0xfffffe014bfe25e8
vp=3D0xfffffe014b602bd0, lowervp=3D0xfffffe014bfe25e8
vnode vnode 0xfffffe014b6027e0: 0xfffffe014b6027e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf08ae0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf08ae0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b6027e0, lowervp=3D0xfffffe014b6029d8
vp=3D0xfffffe014b6027e0, lowervp=3D0xfffffe014b6029d8
vnode vnode 0xfffffe014b6023f0: 0xfffffe014b6023f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf6f740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf6f740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b6023f0, lowervp=3D0xfffffe014b6025e8
vp=3D0xfffffe014b6023f0, lowervp=3D0xfffffe014b6025e8
vnode vnode 0xfffffe014b602000: 0xfffffe014b602000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014befc9f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014befc9f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b602000, lowervp=3D0xfffffe014b6021f8
vp=3D0xfffffe014b602000, lowervp=3D0xfffffe014b6021f8
vnode vnode 0xfffffe01b42b3bd0: 0xfffffe01b42b3bd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b4244740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b4244740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b42b3bd0, lowervp=3D0xfffffe01b42b3dc8
vp=3D0xfffffe01b42b3bd0, lowervp=3D0xfffffe01b42b3dc8
vnode vnode 0xfffffe01b42b37e0: 0xfffffe01b42b37e0: tag null, type VREG
tag null, type VREG [279/1886]
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf15740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf15740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b42b37e0, lowervp=3D0xfffffe01b42b39d8
vp=3D0xfffffe01b42b37e0, lowervp=3D0xfffffe01b42b39d8
vnode vnode 0xfffffe01b42b33f0: 0xfffffe01b42b33f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bd21910 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bd21910 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b42b33f0, lowervp=3D0xfffffe01b42b35e8
vp=3D0xfffffe01b42b33f0, lowervp=3D0xfffffe01b42b35e8
vnode vnode 0xfffffe01b42b3000: 0xfffffe01b42b3000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf9dae0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf9dae0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b42b3000, lowervp=3D0xfffffe01b42b31f8
vp=3D0xfffffe01b42b3000, lowervp=3D0xfffffe01b42b31f8
vnode vnode 0xfffffe01b428bbd0: 0xfffffe01b428bbd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae658 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae658 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b428bbd0, lowervp=3D0xfffffe01b423cdc8
vp=3D0xfffffe01b428bbd0, lowervp=3D0xfffffe01b423cdc8
vnode vnode 0xfffffe01b428b7e0: 0xfffffe01b428b7e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf09ae0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf09ae0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b428b7e0, lowervp=3D0xfffffe01b428b9d8
vp=3D0xfffffe01b428b7e0, lowervp=3D0xfffffe01b428b9d8
vnode vnode 0xfffffe01b428b3f0: 0xfffffe01b428b3f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bef72b8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bef72b8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b428b3f0, lowervp=3D0xfffffe01b428b5e8
vp=3D0xfffffe01b428b3f0, lowervp=3D0xfffffe01b428b5e8
vnode vnode 0xfffffe01b428b000: 0xfffffe01b428b000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b4246000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b4246000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b428b000, lowervp=3D0xfffffe01b428b1f8
vp=3D0xfffffe01b428b000, lowervp=3D0xfffffe01b428b1f8
vnode vnode 0xfffffe01b428abd0: 0xfffffe01b428abd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b4244488 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b4244488 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b428abd0, lowervp=3D0xfffffe01b428adc8
vp=3D0xfffffe01b428abd0, lowervp=3D0xfffffe01b428adc8
vnode vnode 0xfffffe01b428a7e0: 0xfffffe01b428a7e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae910 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae910 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b428a7e0, lowervp=3D0xfffffe01b428a9d8
vp=3D0xfffffe01b428a7e0, lowervp=3D0xfffffe01b428a9d8
vnode vnode 0xfffffe01b428a3f0: 0xfffffe01b428a3f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe0007a31000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe0007a31000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b428a3f0, lowervp=3D0xfffffe01b428a5e8
vp=3D0xfffffe01b428a3f0, lowervp=3D0xfffffe01b428a5e8
vnode vnode 0xfffffe01b428a000: 0xfffffe01b428a000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b4244570 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b4244570 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b428a000, lowervp=3D0xfffffe01b428a1f8
vp=3D0xfffffe01b428a000, lowervp=3D0xfffffe01b428a1f8
vnode vnode 0xfffffe01b427fbd0: 0xfffffe01b427fbd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe0007a30bc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe0007a30bc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b427fbd0, lowervp=3D0xfffffe014b602dc8
vp=3D0xfffffe01b427fbd0, lowervp=3D0xfffffe014b602dc8
vnode vnode 0xfffffe01b427f7e0: 0xfffffe01b427f7e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42472b8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42472b8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b427f7e0, lowervp=3D0xfffffe01b427f9d8
vp=3D0xfffffe01b427f7e0, lowervp=3D0xfffffe01b427f9d8
vnode vnode 0xfffffe01b427f3f0: 0xfffffe01b427f3f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae0e8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae0e8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b427f3f0, lowervp=3D0xfffffe01b427f5e8
vp=3D0xfffffe01b427f3f0, lowervp=3D0xfffffe01b427f5e8
vnode vnode 0xfffffe01b427f000: 0xfffffe01b427f000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42473a0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42473a0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b427f000, lowervp=3D0xfffffe01b427f1f8
vp=3D0xfffffe01b427f000, lowervp=3D0xfffffe01b427f1f8
vnode vnode 0xfffffe01b427ebd0: 0xfffffe01b427ebd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf9cd98 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf9cd98 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b427ebd0, lowervp=3D0xfffffe01b427edc8
vp=3D0xfffffe01b427ebd0, lowervp=3D0xfffffe01b427edc8
vnode vnode 0xfffffe01b427e7e0: 0xfffffe01b427e7e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b427e7e0, lowervp=3D0xfffffe01b427e9d8
vp=3D0xfffffe01b427e7e0, lowervp=3D0xfffffe01b427e9d8
vnode vnode 0xfffffe01b427e3f0: 0xfffffe01b427e3f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae570 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae570 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b427e3f0, lowervp=3D0xfffffe01b427e5e8
vp=3D0xfffffe01b427e3f0, lowervp=3D0xfffffe01b427e5e8
vnode vnode 0xfffffe01b427e000: 0xfffffe01b427e000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae9f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae9f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b427e000, lowervp=3D0xfffffe01b427e1f8
vp=3D0xfffffe01b427e000, lowervp=3D0xfffffe01b427e1f8
vnode vnode 0xfffffe014b5cbbd0: 0xfffffe014b5cbbd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014be0a570 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014be0a570 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b5cbbd0, lowervp=3D0xfffffe01b428bdc8
vp=3D0xfffffe014b5cbbd0, lowervp=3D0xfffffe01b428bdc8
vnode vnode 0xfffffe014b5cb7e0: 0xfffffe014b5cb7e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bf6f910 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bf6f910 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b5cb7e0, lowervp=3D0xfffffe014b5cb9d8
vp=3D0xfffffe014b5cb7e0, lowervp=3D0xfffffe014b5cb9d8
vnode vnode 0xfffffe014b5cb3f0: 0xfffffe014b5cb3f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bedebc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bedebc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b5cb3f0, lowervp=3D0xfffffe014b5cb5e8
vp=3D0xfffffe014b5cb3f0, lowervp=3D0xfffffe014b5cb5e8
vnode vnode 0xfffffe014b5cb000: 0xfffffe014b5cb000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42aebc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42aebc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b5cb000, lowervp=3D0xfffffe014b5cb1f8
vp=3D0xfffffe014b5cb000, lowervp=3D0xfffffe014b5cb1f8
vnode vnode 0xfffffe014b5cabd0: 0xfffffe014b5cabd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae3a0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae3a0 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b5cabd0, lowervp=3D0xfffffe014b5cadc8
vp=3D0xfffffe014b5cabd0, lowervp=3D0xfffffe014b5cadc8
vnode vnode 0xfffffe014b5ca7e0: 0xfffffe014b5ca7e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe01b42ae2b8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe01b42ae2b8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b5ca7e0, lowervp=3D0xfffffe014b5ca9d8
vp=3D0xfffffe014b5ca7e0, lowervp=3D0xfffffe014b5ca9d8
vnode vnode 0xfffffe014b5ca3f0: 0xfffffe014b5ca3f0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bede9f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bede9f8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b5ca3f0, lowervp=3D0xfffffe014b5ca5e8
vp=3D0xfffffe014b5ca3f0, lowervp=3D0xfffffe014b5ca5e8
vnode vnode 0xfffffe014b5ca000: 0xfffffe014b5ca000: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bede000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bede000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe014b5ca000, lowervp=3D0xfffffe014b5ca1f8
vp=3D0xfffffe014b5ca000, lowervp=3D0xfffffe014b5ca1f8
vnode vnode 0xfffffe01b42b4bd0: 0xfffffe01b42b4bd0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bef7000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bef7000 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b42b4bd0, lowervp=3D0xfffffe01b427fdc8
vp=3D0xfffffe01b42b4bd0, lowervp=3D0xfffffe01b427fdc8
vnode vnode 0xfffffe01b42b47e0: 0xfffffe01b42b47e0: tag null, type VREG
tag null, type VREG
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014befdbc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014befdbc8 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b42b47e0, lowervp=3D0xfffffe01b42b49d8
vp=3D0xfffffe01b42b47e0, lowervp=3D0xfffffe01b42b49d8
vnode vnode 0xfffffe01b4e5abd0: 0xfffffe01b4e5abd0: tag null, type VDIR
tag null, type VDIR
usecount 0, writecount 0, refcount 0 mountedhere 0
usecount 0, writecount 0, refcount 0 mountedhere 0
flags (VI_FREE)
flags (VI_FREE)
v_object 0xfffffe014bbf4740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
v_object 0xfffffe014bbf4740 ref 0 pages 0 cleanbuf 0 dirtybuf 0
lock type zfs: UNLOCKED
lock type zfs: UNLOCKED
vp=3D0xfffffe01b4e5abd0, lowervp=3D0xfffffe014b5c43f0
vp=3D0xfffffe01b4e5abd0, lowervp=3D0xfffffe014b5c43f0
db>


> After that, you could try the patch which I posted at winter, but which=

> was not tested. I do not know whether it is of any help, and I do need
> the debugging information with the patch above before I can make any
> conclusions.
>
> diff --git a/sys/fs/nullfs/null_subr.c b/sys/fs/nullfs/null_subr.c
> index fa6c4af..3f74579 100644
> --- a/sys/fs/nullfs/null_subr.c
> +++ b/sys/fs/nullfs/null_subr.c
> @@ -251,6 +251,7 @@ null_nodeget(mp, lowervp, vpp)
>  	vp->v_type =3D lowervp->v_type;
>  	vp->v_data =3D xp;
>  	vp->v_vnlock =3D lowervp->v_vnlock;
> +	vp->v_vflag =3D lowervp->v_vflag & VV_ROOT;
>  	error =3D insmntque1(vp, mp, null_insmntque_dtr, xp);
>  	if (error !=3D 0)
>  		return (error);

I tested this without the ldvp-debug patch and besides the same
ufs/devds LOR nothing else happens with my usual simple end limited test.=


Thanks so far, let me know if you want more debug output/testings!

-Harry


--------------enig09804AF5039DD5A9FD7ABF27
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (FreeBSD)

iEYEARECAAYFAlPVEGAACgkQLDqVQ9VXb8i6WgCfSw7IXgJVxJTdHSMEPhLwVz+V
/ToAoMLFCL+I2xG/b18y3qGWvXd9CJ4E
=v3mV
-----END PGP SIGNATURE-----

--------------enig09804AF5039DD5A9FD7ABF27--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53D5105A.6040204>