Date: Wed, 10 Dec 2014 21:48:18 +0100 From: Walter Hop <freebsd@spam.lifeforms.nl> To: freebsd-stable@freebsd.org Subject: Re: System hang on shutdown when running freebsd-update Message-ID: <7FE045BA-246F-460F-81F5-CFC312072A92@spam.lifeforms.nl> In-Reply-To: <54885894.2060006@club-internet.fr> References: <548846F8.4080208@club-internet.fr> <20141210133658.GA12721@ozzmosis.com> <54885894.2060006@club-internet.fr>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 10 Dec 2014, at 15:28, Juan Ram=F3n Molina Menor = <listjm@club-internet.fr> wrote: >=20 >> A FreeBSD 10.1-REL amd64 VM here also hangs after /sbin/reboot is = run, >> after installing updates with freebsd-update. The VM runs under >> VirtualBox on an Ubuntu 14.04 host. >>=20 >> On hard-resetting the VM, it appears the filesystem (UFS) wasn't = flushed: >>=20 >> Dec 11 00:29:52 vbox-freebsd kernel: WARNING: / was not properly = dismounted >=20 > Yes, I forgot that, same here on bare metal. I=92d expect it to happen again, since the update touched /sbin/init, = and that seems to be a way to get reboot/unmount hanging with = UFS2+softupdates. (See my minimal example in = https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D195458 = <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D195458>) A workaround to prevent the hang/fsck is to first disable softupdates on = / before doing the freebsd-update. We updated around 40 boxes with this = one weird trick, and we did not have the problem. I=92ve tried if the hang is apparent in a 11-CURRENT snapshot too, and = it is. (Tried the iso snapshot available today, which is r273635). = WITNESS indicates a lock order reversal, which looks related to me: root@current:~ # chflags noschg /sbin/init root@current:~ # cp -Rp /sbin/init /sbin/init2 lock order reversal: 1st 0xfffffe007b842fa0 bufwait (bufwait) @ = /usr/src/sys/kern/vfs_bio.c:3093 2nd 0xfffff80002b9ea00 dirhash (dirhash) @ = /usr/src/sys/ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe000025c270 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe000025c320 witness_checkorder() at witness_checkorder+0xdad/frame = 0xfffffe000025c3b0 _sx_xlock() at _sx_xlock+0x75/frame 0xfffffe000025c3f0 ufsdirhash_add() at ufsdirhash_add+0x3a/frame 0xfffffe000025c430 ufs_direnter() at ufs_direnter+0x6a0/frame 0xfffffe000025c4f0 ufs_makeinode() at ufs_makeinode+0x560/frame 0xfffffe000025c6a0 VOP_CREATE_APV() at VOP_CREATE_APV+0xf1/frame 0xfffffe000025c6d0 vn_open_cred() at vn_open_cred+0x29d/frame 0xfffffe000025c820 kern_openat() at kern_openat+0x26f/frame 0xfffffe000025c9a0 amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe000025cab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe000025cab0 --- syscall (5, FreeBSD ELF64, sys_open), rip =3D 0x80094f01a, rsp =3D = 0x7fffffffe958, rbp =3D 0x7fffffffe9c0 --- Screenshot is here: http://lf.ms/current-r273635-hang-1.png = <http://lf.ms/current-r273635-hang-1.png>; Finally, when rebooting, another lock order reversal appears and the = system hangs. I don=92t have a text log of this, so I=92ll copy the first few lines: Syncing disks, vnodes remaining=851 0 0 done All buffers synced. lock order reversal: 1st 0xfffff80002e65d50 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1223 2nd 0xfffff80002e665f0 devfs (devfs) @ = /usr/src/sys/kern/vfs_subr.c:2144 Screenshot is here: http://lf.ms/current-r273635-hang-2.png = <http://lf.ms/current-r273635-hang-2.png>; I don=92t have kernel hacking experience, but these source files look = awfully related to the parts that we are having problems with. I would really love some research into this and possibly an errata for = 10.1. What can we do to make this actionable? WH --=20 Walter Hop | PGP key: https://lifeforms.nl/pgp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7FE045BA-246F-460F-81F5-CFC312072A92>