Date: Fri, 28 Feb 2014 01:28:04 GMT From: Frank Hoeflich <frank.hoeflich@tidalscale.com> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/187126: panic: vm_page_unwire: page XXX's wire count is 0 on NFS client Message-ID: <201402280128.s1S1S4vK052079@cgiserv.freebsd.org> Resent-Message-ID: <201402280130.s1S1U0dI010398@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 187126
>Category: kern
>Synopsis: panic: vm_page_unwire: page XXX's wire count is 0 on NFS client
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Fri Feb 28 01:30:00 UTC 2014
>Closed-Date:
>Last-Modified:
>Originator: Frank Hoeflich
>Release: stable/10
>Organization:
TidalScale Inc.
>Environment:
NFS/DHCP client:
node0:~ $ uname -a
FreeBSD node0.tsvirt.pod 10.0-STABLE FreeBSD 10.0-STABLE #0 bb452b1(current): Fri Feb 21 12:25:36 PST 2014 tsdev@coral:/usr/home/tsdev/obj/usr/src/sys/TS amd64
NFS/DHCP server:
coral:~ $ uname -a
FreeBSD coral 10.0-ALPHA5 FreeBSD 10.0-ALPHA5 #0 r256092: Sun Oct 6 22:30:23 UTC 2013 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
>Description:
In preparation for a customer alpha release, I had created two VMs on VMware Fusion. One acted as a DHCP and NFS server; the other, node0, was a DHCP and NFS client. After booting node0 as root I attached to a process on it using gdb. After debugging I typed `shell reboot', node0 synced and went down, then got this panic. Saved the dump and got this bt from kgdb after coming back up:
(kgdb) #0 doadump (textdump=0) at pcpu.h:219
#1 0xffffffff80345ffe in db_dump (dummy=<value optimized out>, dummy2=0,
dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:543
#2 0xffffffff80345a9d in db_command (cmd_table=<value optimized out>)
at /usr/src/sys/ddb/db_command.c:449
#3 0xffffffff80345814 in db_command_loop ()
at /usr/src/sys/ddb/db_command.c:502
#4 0xffffffff80348260 in db_trap (type=<value optimized out>, code=0)
at /usr/src/sys/ddb/db_main.c:231
#5 0xffffffff80900f73 in kdb_trap (type=3, code=0, tf=<value optimized out>)
at /usr/src/sys/kern/subr_kdb.c:656
#6 0xffffffff80cde7bb in trap (frame=0xfffffe0171bfe270)
at /usr/src/sys/amd64/amd64/trap.c:579
#7 0xffffffff80cc3192 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:232
#8 0xffffffff8090072e in kdb_enter (why=0xffffffff80f54b6f "panic",
msg=<value optimized out>) at cpufunc.h:63
#9 0xffffffff808c6b76 in vpanic (fmt=<value optimized out>,
ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:752
#10 0xffffffff808c6be3 in panic (fmt=<value optimized out>)
at /usr/src/sys/kern/kern_shutdown.c:688
#11 0xffffffff80b687a9 in vm_page_unwire (m=<value optimized out>,
activate=<value optimized out>) at /usr/src/sys/vm/vm_page.c:2381
#12 0xffffffff80959bd6 in vfs_vmio_release (bp=0xfffffe0171c74298)
at /usr/src/sys/kern/vfs_bio.c:1859
#13 0xffffffff80957f98 in brelse (bp=0xfffffe0171c74298)
at /usr/src/sys/kern/vfs_bio.c:1712
#14 0xffffffff80970e7a in flushbuflist (bufv=<value optimized out>,
flags=<value optimized out>, bo=0xfffff800046430d0, slpflag=96,
slptimeo=0) at /usr/src/sys/kern/vfs_subr.c:1383
#15 0xffffffff80970b3a in bufobj_invalbuf (bo=0xfffff800046430d0, flags=1,
slpflag=0, slptimeo=0) at /usr/src/sys/kern/vfs_subr.c:1259
#16 0xffffffff80973e0b in vgonel (vp=0xfffff80004643000)
at /usr/src/sys/kern/vfs_subr.c:1320
#17 0xffffffff80973646 in vflush (mp=0xfffff80004409330, rootrefs=1, flags=2,
td=0xfffff80004dc1000) at /usr/src/sys/kern/vfs_subr.c:2568
#18 0xffffffff807f5155 in nfs_unmount (mp=0xfffff80004409330,
mntflags=<value optimized out>)
at /usr/src/sys/fs/nfsclient/nfs_clvfsops.c:1503
#19 0xffffffff8096b2bf in dounmount (mp=0xfffff80004409330, flags=524288,
td=0xfffff80004dc1000) at /usr/src/sys/kern/vfs_mount.c:1324
#20 0xffffffff80974931 in vfs_unmountall ()
at /usr/src/sys/kern/vfs_subr.c:3396
#21 0xffffffff808c62a8 in kern_reboot (howto=0)
at /usr/src/sys/kern/kern_shutdown.c:435
#22 0xffffffff808c5cc8 in sys_reboot (td=<value optimized out>,
uap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:196
#23 0xffffffff80cdf483 in amd64_syscall (td=0xfffff80004dc1000, traced=0)
at subr_syscall.c:134
#24 0xffffffff80cc347b in Xfast_syscall ()
at /usr/src/sys/amd64/amd64/exception.S:391
#25 0x000000080085e12c in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language: auto; currently minimal
(kgdb)
>How-To-Repeat:
1. Boot NFS client.
2. On client, find a process to which you may attach:
node0:~ $ ps -ax | grep cron
1136 - Is 0:00.01 /usr/sbin/cron -s
1212 0 R+ 0:00.00 grep cron
3. Attach to it: gdb /usr/sbin/cron 1136
4. At the gdb prompt, type `shell reboot'.
This has worked on the first try for me a few times now (and hasn't failed yet).
>Fix:
I do not have a patch. This resembles
kern/181590: [vm] [panic] amd(8) related vm_page_unwire panics
kern/182661: [PANIC] 9.2-RELEASE vm_page_unwire: page n's wire count is zero
which were reported against msdosfs+tmpfs and nullfs respectively. 182661 has been closed and it's possible that similarly patching the NFS vnops would fix it. To my knowledge I'm the first to report it against NFS.
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201402280128.s1S1S4vK052079>