Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 29 Dec 2015 13:14:45 -0700
From:      Alan Somers <asomers@freebsd.org>
To:        NGie Cooper <yaneurabeya@gmail.com>
Cc:        Florian Ermisch <0xf10e@fsfe.org>, FreeBSD CURRENT <freebsd-current@freebsd.org>,  "will@freebsd.org" <will@freebsd.org>
Subject:   Re: Stack backtrace on `zpool export`
Message-ID:  <CAOtMX2j7HesuyceH-TEKqAQarLjBaMnQHejspbBN0GcgZg5dGw@mail.gmail.com>
In-Reply-To: <3B435E9A-F93A-407E-AB54-C7B98291269B@gmail.com>
References:  <5B1F1E9E-7651-4C1D-807F-A875EDEF8705@fsfe.org> <3B435E9A-F93A-407E-AB54-C7B98291269B@gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Dec 29, 2015 at 12:54 PM, NGie Cooper <yaneurabeya@gmail.com> wrote=
:
>
>> On Dec 29, 2015, at 11:48, Florian Ermisch <0xf10e@fsfe.org> wrote:
>>
>> Hi *,
>>
>> since I've upgraded my laptop from 10.2-RELEASE to 11-CURRENT (r292536, =
now r292755) I see this stack backtrace when a zpool is exported:
>>
>> Dec 27 18:44:02 fuchi-cyber220 kernel: lock order reversal:
>> Dec 27 18:44:02 fuchi-cyber220 kernel: 1st 0xfffff800c7472418 zfs (zfs) =
@ /usr/src/sys/kern/vfs_mount.c:1224
>> Dec 27 18:44:02 fuchi-cyber220 kernel: 2nd 0xfffff800c73fad50 zfs_gfs (z=
fs_gfs) @ /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/gfs.c:494
>> Dec 27 18:44:02 fuchi-cyber220 kernel: stack backtrace:
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #0 0xffffffff80a7d6f0 at witness_=
debugger+0x70
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #1 0xffffffff80a7d5f1 at witness_=
checkorder+0xe71
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #2 0xffffffff809fedcb at __lockmg=
r_args+0xd3b
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #3 0xffffffff80ac55ec at vop_stdl=
ock+0x3c
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #4 0xffffffff80fbb220 at VOP_LOCK=
1_APV+0x100
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #5 0xffffffff80ae653a at _vn_lock=
+0x9a
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #6 0xffffffff8209db13 at gfs_file=
_create+0x73
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #7 0xffffffff8209dbbd at gfs_dir_=
create+0x1d
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #8 0xffffffff821649e7 at zfsctl_m=
knode_snapdir+0x47
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #9 0xffffffff8209e135 at gfs_dir_=
lookup+0x185
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #10 0xffffffff8209e61d at gfs_vop=
_lookup+0x1d
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #11 0xffffffff82163a05 at zfsctl_=
root_lookup+0xf5
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #12 0xffffffff821648a3 at zfsctl_=
umount_snapshots+0x83
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #13 0xffffffff8217d5ab at zfs_umo=
unt+0x7b
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #14 0xffffffff80acf0b0 at dounmou=
nt+0x530
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #15 0xffffffff80aceaed at sys_unm=
ount+0x35d
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #16 0xffffffff80e6e13b at amd64_s=
yscall+0x2db
>> Dec 27 18:44:02 fuchi-cyber220 kernel: #17 0xffffffff80e4dd8b at Xfast_s=
yscall+0xfb
>>
>> (From /var/log/messages)
>>
>> First I've only seen this on the console at shutdown or reboot (after sy=
nc I think) but later found I can reproduce it by exporting a zpool.
>> While the only pools I can trigger this without a shutdown/reboot are co=
nnected via USB(3) I still see it just before poweroff at shutdown when the=
 system's root pool is synced.
>>
>> When I try to export a zpool under heavy load (`make -C /use/src -j 4 bu=
ildworld` on a 2 core CPU w/ HT) the system locks up completely. I don't th=
ink it's related to memory pressure as I haven't seen swap being used durin=
g a buildworld (with 8 gigs of RAM).
>>
>> Has anyone else seen this?
>
> Please file a bug for this issue and assign it to freebsd-fs. It doesn=E2=
=80=99t look familiar (there might be a lock ordering issue with either zfs=
 or vfs+zfs).
> Thanks!
> -NGie

Adding Will@.  I think he was working on this LOR at one point.
-Alan



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2j7HesuyceH-TEKqAQarLjBaMnQHejspbBN0GcgZg5dGw>