Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 26 Jan 2018 04:29:47 -0800
From:      David Wolfskill <david@catwhisker.org>
To:        current@freebsd.org
Subject:   Panic on shutdown @r328436: "Unholding 6 with cnt = -559038242"
Message-ID:  <20180126122947.GA1287@albert.catwhisker.org>

next in thread | raw e-mail | index | archive | help

--7ULH7gtHJxM9ukDW
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

This is on my "build machine" (laptop is still building updated ports
for today, so I don't know yet whether or not it encounters this.)=20

I had performed a source-based update from r328393 to r328436,
rebooted, performed "make delete-old-libs", and all seemed well.

I then issued "sudo shutdown -p now", and serial console shows:

FreeBSD/amd64 (freebeast.catwhisker.org) (ttyu0)

login: Jan 26 12:11:03 Stopping sshd.
Waiting for PIDS: 681.
Stopping rsyncd.
Waiting for PIDS: 652.
Stopping powerd.
Waiting for PIDS: 636.
Stopping ntpd.
Waiting for PIDS: 633, 633.
Stopping lpd.
Waiting for PIDS: 610.
Stopping lockd.
WaitingWARNING: autofs_unmount: vflush failed with error 16
 for PIDS: 592.
Stopping statd.
Waiting for PIDS: 589.
Stopping nfsd.
WaitinlJan 26 12:11:05 ock ordefreebeast syslogr reversal:
 1st 0xfffff800692cd490 filed: exiting on sidesc structure (filedesc struct=
ure) @ /usr/src/sys/kern/sys_generic.c:1567
 2nd 0xfffff8006973f9a0 devfs (devfs) @ /usr/src/gnal 15
sys/kern/vfs_vnops.c:1526
stack backtrace:
#0 0xffffffff80b2c2e3 at witness_debugger+0x73
#1 0xffffffff80b2c164 at witness_checkorder+0xe34
#2 0xffffffff80a9ca21 at lockmgr_lock_fast_path+0x1b1
#3 0xffffffff810f8ca9 at VOP_LOCK1_APV+0xd9
#4 0xffffffff80ba7746 at _vn_lock+0x66
#5 0xffffffff80ba654b at vn_poll+0x3b
#6 0xffffffff80992f4d at devfs_poll_f+0xcd
#7 0xffffffff80b33255 at kern_poll+0x385
#8 0xffffffff80b32ec0 at sys_poll+0x50
#9 0xffffffff80f7a37b at amd64_syscall+0x79b
#10 0xffffffff80f569c8 at fast_syscall_common+0xfc
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `bufdaemon' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...=20
Syncing disks, vnodes remaining... 5 5 lock order reversal:
 1st 0xfffff80007927240 syncer (syncer) @ /usr/src/sys/kern/vfs_subr.c:2157
 2nd 0xfffff80007570d50 devfs (devfs) @ /usr/src/sys/ufs/ffs/ffs_vfsops.c:1=
583
stack backtrace:
#0 0xffffffff80b2c2e3 at witness_debugger+0x73
#1 0xffffffff80b2c164 at witness_checkorder+0xe34
#2 0xffffffff80a9ca21 at lockmgr_lock_fast_path+0x1b1
#3 0xffffffff810f8ca9 at VOP_LOCK1_APV+0xd9
#4 0xffffffff80ba7746 at _vn_lock+0x66
#5 0xffffffff80dc1dac at ffs_sync+0x2cc
#6 0xffffffff80b9ce8f at sync_fsync+0xff
#7 0xffffffff810f7c19 at VOP_FSYNC_APV+0xd9
#8 0xffffffff80b9acc4 at sched_sync+0x284
#9 0xffffffff80a88414 at fork_exit+0x84
#10 0xffffffff80f56e1e at fork_trampoline+0xe
5 3 2 2 1 1 1 1 1 0 0 0 0 0 done
All buffers synced.
lock order reversal:
 1st 0xfffff800079c57c8 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1280
 2nd 0xfffff8000795e418 devfs (devfs) @ /usr/src/sys/ufs/ffs/ffs_vfsops.c:1=
371
stack backtrace:
#0 0xffffffff80b2c2e3 at witness_debugger+0x73
#1 0xffffffff80b2c164 at witness_checkorder+0xe34
#2 0xffffffff80a9ca21 at lockmgr_lock_fast_path+0x1b1
#3 0xffffffff810f8ca9 at VOP_LOCK1_APV+0xd9
#4 0xffffffff80ba7746 at _vn_lock+0x66
#5 0xffffffff80dbef63 at ffs_flushfiles+0x93
#6 0xffffffff80da24f2 at softdep_flushfiles+0x82
#7 0xffffffff80dc15f7 at ffs_unmount+0x77
#8 0xffffffff80b8e2c9 at dounmount+0x519
#9 0xffffffff80b9800b at vfs_unmountall+0x6b
#10 0xffffffff80b73d25 at bufshutdown+0x3a5
#11 0xffffffff80ac754a at kern_reboot+0x1da
#12 0xffffffff80ac7312 at sys_reboot+0x3c2
#13 0xffffffff80f7a37b at amd64_syscall+0x79b
#14 0xffffffff80f569c8 at fast_syscall_common+0xfc
Swap device [file] removed.
Uptime: 1m44s
(ada0:ahcich0:0:0:0): spin-down
(ada1:ahcich2:0:0:0): spin-down
(ada2:ahcich3:0:0:0): spin-down
(ada3:ahcich4:0:0:0): spin-down
panic: Unholding 6 with cnt =3D -559038242
cpuid =3D 3
time =3D 1516968697
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0000428=
8c0
vpanic() at vpanic+0x18d/frame 0xfffffe0000428920
panic() at panic+0x43/frame 0xfffffe0000428980
dadiskgonecb() at dadiskgonecb+0x42/frame 0xfffffe00004289a0
g_disk_providergone() at g_disk_providergone+0x25/frame 0xfffffe00004289d0
g_destroy_provider() at g_destroy_provider+0xae/frame 0xfffffe00004289f0
g_wither_washer() at g_wither_washer+0x87/frame 0xfffffe0000428a30
g_run_events() at g_run_events+0x3ca/frame 0xfffffe0000428a70
fork_exit() at fork_exit+0x84/frame 0xfffffe0000428ab0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000428ab0
--- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 ---
KDB: enter: panic
[ thread pid 13 tid 100044 ]
Stopped at      kdb_enter+0x3b: movq    $0,kdb_why
db>=20


As noted, this is a build machine, and it was to be powered off for
the rest of the day anyway, so I don't need to get it up & running
immediately: I can poke at the ddb prompt, given some clues.

When running head, the system does not use ZFS (only UFS2+SU -- not
SUJ -- & tmpfs).

Same system had completed a source-based update for stable/11 from
r328392 to r328429 earlier today without incident (using a different
slice of the boot drive).

Peace,
david
--=20
David H. Wolfskill				david@catwhisker.org
"unfortunately, no trust!=E2=80=9D -- well, of course!  You reap what you s=
ow.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.

--7ULH7gtHJxM9ukDW
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQGTBAEBCgB9FiEEzLfO+ReoAfQwZNd7FTnMQKBJ7hcFAlprHztfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEND
QjdDRUY5MTdBODAxRjQzMDY0RDc3QjE1MzlDQzQwQTA0OUVFMTcACgkQFTnMQKBJ
7hfl0QgAkIRYZ8zKVQUwM80iXVpTV8FeAj6Y24EHtEEolm8IaAfrYEAqklibp84X
izxlp9raYgLNZuPKBqFBgQFVCkiZMypGfOOzVfDuetkYHXhxzJg6SVmJW8hc3gvg
Cj1IqcsbPEucmfBZJmmF6juDVsPw0zEQkLeVCp58PKmZ8W/ziy3GLOwfYZakqkCn
0zIxJZL/FwNESDVl532qVDlc9k8yQ0RbajnIlMAHsvHY0b5ROI5TJgBSeokj0QXi
swr3wc05Gb35XTEG7UxzSGD/FvUainNL0opTFN649o7xickDFeO9XijzAYDzyJjr
IzBAmfbeHPw/MhiQr7Z852e1qyDhLA==
=4NS6
-----END PGP SIGNATURE-----

--7ULH7gtHJxM9ukDW--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180126122947.GA1287>