Date: Fri, 17 Mar 2017 21:31:43 +0000 From: Rick Macklem <rmacklem@uoguelph.ca> To: Larry Rosenman <ler@lerctr.org>, "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Cc: "freebsd-current@FreeBSD.org" <freebsd-current@FreeBSD.org> Subject: Re: crash: umount_nfs: Current Message-ID: <YTXPR01MB01892D3066B805F0B1BBC288DD390@YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM> In-Reply-To: <A5DCCD5C-F6DE-4C54-A9A6-2A3729808E67@lerctr.org> References: <20170316024433.qiujcewz5bclbgq5@borg.lerctr.org> <YTXPR01MB0189ABA592F02FBB9FFC2C9DDD260@YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM>, <A5DCCD5C-F6DE-4C54-A9A6-2A3729808E67@lerctr.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Oops, yea I see that now. In the krpc it is very hard to tell when the data= structures (that hold the mutexes) can be safely free'd. Code like xprt_unregister() g= et called asynchronously and lock a mutex as soon as called. (The crash fixed by r313735 was a prematurely free'd mutex that xprt_unregi= ster() used, but on the server side.) I'll look at the code as see if I can figure out how to delay freeing the s= tructure without leaving it allocated "forever". (I'll admit I've been tempted to just never= free it, since the memory leak this would cause would be small enough nothing would really= notice it. And, of course for your case of shutdown, it would be harmless to just = not free it.) rick ________________________________________ From: Larry Rosenman <ler@lerctr.org> Sent: Thursday, March 16, 2017 7:46:51 PM To: Rick Macklem; freebsd-fs@freebsd.org Cc: freebsd-current@FreeBSD.org Subject: Re: crash: umount_nfs: Current Err, I=92m at r315289=85. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 On 3/16/17, 5:51 PM, "Rick Macklem" <rmacklem@uoguelph.ca> wrote: I believe the cause of this crash was fixed by a recent commit to head r313735 (which was MFC'd to stable/11 and stable/10). rick ________________________________________ From: owner-freebsd-current@freebsd.org <owner-freebsd-current@freebsd.= org> on behalf of Larry Rosenman <ler@lerctr.org> Sent: Wednesday, March 15, 2017 10:44:33 PM To: freebsd-fs@freebsd.org Cc: freebsd-current@FreeBSD.org Subject: crash: umount_nfs: Current Recent current, playing with new FreeNAS Corral, client is FreeBSD -CUR= RENT. Lovely crash: borg.lerctr.org dumped core - see /var/crash/vmcore.1 Wed Mar 15 21:38:53 CDT 2017 FreeBSD borg.lerctr.org 12.0-CURRENT FreeBSD 12.0-CURRENT #11 r315289: = Tue Mar 14 20:55:36 CDT 2017 root@borg.lerctr.org:/usr/obj/usr/src/sys/= VT-LER amd64 panic: general protection fault GNU gdb (GDB) 7.12.1 [GDB v7.12.1 for FreeBSD] Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl= .html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copyi= ng" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd12.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /boot/kernel/kernel...Reading symbols from /usr/li= b/debug//boot/kernel/kernel.debug...done. done. Unread portion of the kernel message buffer: Fatal trap 9: general protection fault while in kernel mode cpuid =3D 1; apic id =3D 21 instruction pointer =3D 0x20:0xffffffff80a327ae stack pointer =3D 0x28:0xfffffe535ebb2800 frame pointer =3D 0x28:0xfffffe535ebb2830 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 3172 (umount) trap number =3D 9 panic: general protection fault cpuid =3D 1 time =3D 1489631515 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe535= ebb2440 vpanic() at vpanic+0x19c/frame 0xfffffe535ebb24c0 panic() at panic+0x43/frame 0xfffffe535ebb2520 trap_fatal() at trap_fatal+0x322/frame 0xfffffe535ebb2570 trap() at trap+0x5e/frame 0xfffffe535ebb2730 calltrap() at calltrap+0x8/frame 0xfffffe535ebb2730 --- trap 0x9, rip =3D 0xffffffff80a327ae, rsp =3D 0xfffffe535ebb2800, r= bp =3D 0xfffffe535ebb2830 --- __mtx_lock_flags() at __mtx_lock_flags+0x3e/frame 0xfffffe535ebb2830 xprt_unregister() at xprt_unregister+0x28/frame 0xfffffe535ebb2850 clnt_reconnect_destroy() at clnt_reconnect_destroy+0x38/frame 0xfffffe5= 35ebb2880 nfs_unmount() at nfs_unmount+0x182/frame 0xfffffe535ebb28d0 dounmount() at dounmount+0x5c1/frame 0xfffffe535ebb2950 sys_unmount() at sys_unmount+0x30f/frame 0xfffffe535ebb2a70 amd64_syscall() at amd64_syscall+0x55a/frame 0xfffffe535ebb2bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe535ebb2bf0 --- syscall (22, FreeBSD ELF64, sys_unmount), rip =3D 0x800872b9a, rsp = =3D 0x7fffffffde88, rbp =3D 0x7fffffffe3c0 --- Uptime: 2h43m8s Dumping 5744 out of 131005 MB:..1%..11%..21%..31%..41%..51%..61%..71%..= 81%..91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/li= b/debug//boot/kernel/zfs.ko.debug...done. done. Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from= /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Reading symbols from /boot/kernel/linux.ko...Reading symbols from /usr/= lib/debug//boot/kernel/linux.ko.debug...done. done. Reading symbols from /boot/kernel/linux_common.ko...Reading symbols fro= m /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. done. Reading symbols from /boot/kernel/if_lagg.ko...Reading symbols from /us= r/lib/debug//boot/kernel/if_lagg.ko.debug...done. done. Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from /u= sr/lib/debug//boot/kernel/coretemp.ko.debug...done. done. Reading symbols from /boot/kernel/aesni.ko...Reading symbols from /usr/= lib/debug//boot/kernel/aesni.ko.debug...done. done. Reading symbols from /boot/kernel/filemon.ko...Reading symbols from /us= r/lib/debug//boot/kernel/filemon.ko.debug...done. done. Reading symbols from /boot/kernel/fuse.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/fuse.ko.debug...done. done. Reading symbols from /boot/kernel/ichsmb.ko...Reading symbols from /usr= /lib/debug//boot/kernel/ichsmb.ko.debug...done. done. Reading symbols from /boot/kernel/smbus.ko...Reading symbols from /usr/= lib/debug//boot/kernel/smbus.ko.debug...done. done. Reading symbols from /boot/kernel/ichwd.ko...Reading symbols from /usr/= lib/debug//boot/kernel/ichwd.ko.debug...done. done. Reading symbols from /boot/kernel/cpuctl.ko...Reading symbols from /usr= /lib/debug//boot/kernel/cpuctl.ko.debug...done. done. Reading symbols from /boot/kernel/cryptodev.ko...Reading symbols from /= usr/lib/debug//boot/kernel/cryptodev.ko.debug...done. done. Reading symbols from /boot/kernel/dtraceall.ko...Reading symbols from /= usr/lib/debug//boot/kernel/dtraceall.ko.debug...done. done. Reading symbols from /boot/kernel/profile.ko...Reading symbols from /us= r/lib/debug//boot/kernel/profile.ko.debug...done. done. Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from /usr= /lib/debug//boot/kernel/dtrace.ko.debug...done. done. Reading symbols from /boot/kernel/systrace_freebsd32.ko...Reading symbo= ls from /usr/lib/debug//boot/kernel/systrace_freebsd32.ko.debug...done. done. Reading symbols from /boot/kernel/systrace.ko...Reading symbols from /u= sr/lib/debug//boot/kernel/systrace.ko.debug...done. done. Reading symbols from /boot/kernel/sdt.ko...Reading symbols from /usr/li= b/debug//boot/kernel/sdt.ko.debug...done. done. Reading symbols from /boot/kernel/fasttrap.ko...Reading symbols from /u= sr/lib/debug//boot/kernel/fasttrap.ko.debug...done. done. Reading symbols from /boot/kernel/fbt.ko...Reading symbols from /usr/li= b/debug//boot/kernel/fbt.ko.debug...done. done. Reading symbols from /boot/kernel/dtnfscl.ko...Reading symbols from /us= r/lib/debug//boot/kernel/dtnfscl.ko.debug...done. done. Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols from /u= sr/lib/debug//boot/kernel/dtmalloc.ko.debug...done. done. Reading symbols from /boot/modules/vboxdrv.ko...(no debugging symbols f= ound)...done. Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/ipmi.ko.debug...done. done. Reading symbols from /boot/kernel/ipmi_linux.ko...Reading symbols from = /usr/lib/debug//boot/kernel/ipmi_linux.ko.debug...done. done. Reading symbols from /boot/kernel/hwpmc.ko...Reading symbols from /usr/= lib/debug//boot/kernel/hwpmc.ko.debug...done. done. Reading symbols from /boot/kernel/mfip.ko...Reading symbols from /usr/l= ib/debug//boot/kernel/mfip.ko.debug...done. done. Reading symbols from /boot/kernel/ums.ko...Reading symbols from /usr/li= b/debug//boot/kernel/ums.ko.debug...done. done. Reading symbols from /boot/modules/vboxnetflt.ko...(no debugging symbol= s found)...done. Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /u= sr/lib/debug//boot/kernel/netgraph.ko.debug...done. done. Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from /u= sr/lib/debug//boot/kernel/ng_ether.ko.debug...done. done. Reading symbols from /boot/modules/vboxnetadp.ko...(no debugging symbol= s found)...done. Reading symbols from /boot/kernel/linux64.ko...Reading symbols from /us= r/lib/debug//boot/kernel/linux64.ko.debug...done. done. Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /= usr/lib/debug//boot/kernel/linprocfs.ko.debug...done. done. Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/= lib/debug//boot/kernel/tmpfs.ko.debug...done. done. __curthread () at ./machine/pcpu.h:232 232 __asm("movq %%gs:%1,%0" : "=3Dr" (td) (kgdb) #0 __curthread () at ./machine/pcpu.h:232 #1 doadump (textdump=3D1) at /usr/src/sys/kern/kern_shutdown.c:318 #2 0xffffffff80a52c15 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:386 #3 0xffffffff80a53206 in vpanic (fmt=3D<optimized out>, ap=3D0xfffffe5= 35ebb2500) at /usr/src/sys/kern/kern_shutdown.c:779 #4 0xffffffff80a53253 in panic (fmt=3D<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:710 #5 0xffffffff80ecd5c2 in trap_fatal (frame=3D0xfffffe535ebb2740, eva= =3D0) at /usr/src/sys/amd64/amd64/trap.c:801 #6 0xffffffff80eccb8e in trap (frame=3D0xfffffe535ebb2740) at /usr/src/sys/amd64/amd64/trap.c:197 #7 <signal handler called> #8 __mtx_lock_flags (c=3D0xdeadc0dedeadc0f6, opts=3D0, file=3D0xffffffff8147e721 "/usr/src/sys/rpc/svc.c", line=3D380) at /usr/src/sys/kern/kern_mutex.c:239 #9 0xffffffff80cc14b8 in xprt_unregister (xprt=3D0xfffff8022e0fce00) at /usr/src/sys/rpc/svc.c:380 #10 0xffffffff80cbc058 in clnt_reconnect_destroy (cl=3D0xfffff80146fa39= 00) at /usr/src/sys/rpc/clnt_rc.c:500 #11 0xffffffff80969972 in nfs_unmount (mp=3D<optimized out>, mntflags=3D<optimized out>) at /usr/src/sys/fs/nfsclient/nfs_clvfso= ps.c:1704 #12 0xffffffff80b0b761 in dounmount (mp=3D0xdeadc0dedeadc0f6, flags=3D1= 34217728, td=3D0xfffff8018f2eb000) at /usr/src/sys/kern/vfs_mount.c:1388 #13 0xffffffff80b0b11f in sys_unmount (td=3D0xfffff8018f2eb000, uap=3D0xfffffe535ebb2b70) at /usr/src/sys/kern/vfs_mount.c:1215 #14 0xffffffff80ecdfea in syscallenter (td=3D0xfffff8018f2eb000, sa=3D<optimized out>) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:135 #15 amd64_syscall (td=3D0xfffff8018f2eb000, traced=3D0) at /usr/src/sys/amd64/amd64/trap.c:902 #16 <signal handler called> Can't read data for section '.eh_frame' in file '/' (kgdb) vmcore / source IS available. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 _______________________________________________ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.o= rg"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YTXPR01MB01892D3066B805F0B1BBC288DD390>