From owner-freebsd-bugs@FreeBSD.ORG Tue Mar 5 16:40:02 2013 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 37793F18 for ; Tue, 5 Mar 2013 16:40:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 1B501E61 for ; Tue, 5 Mar 2013 16:40:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r25Ge1YD038228 for ; Tue, 5 Mar 2013 16:40:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r25Ge1r9038227; Tue, 5 Mar 2013 16:40:01 GMT (envelope-from gnats) Date: Tue, 5 Mar 2013 16:40:01 GMT Message-Id: <201303051640.r25Ge1r9038227@freefall.freebsd.org> To: freebsd-bugs@FreeBSD.org Cc: From: Rasmus Skaarup Subject: Re: kern/176636: Periodical crashes with 9.1-R X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Rasmus Skaarup List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Mar 2013 16:40:02 -0000 The following reply was made to PR kern/176636; it has been noted by GNATS. From: Rasmus Skaarup To: Andriy Gapon Cc: bug-followup@FreeBSD.org Subject: Re: kern/176636: Periodical crashes with 9.1-R Date: Tue, 5 Mar 2013 17:38:13 +0100 # kgdb /boot/kernel/kernel /home/crash/vmcore.3 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain = conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for = details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: vm_page_free: freeing busy page 0xfffffe00d5d2db38 cpuid =3D 1 KDB: stack backtrace: #0 0xffffffff809208a6 at kdb_backtrace+0x66 #1 0xffffffff808ea8be at panic+0x1ce #2 0xffffffff80b50923 at vm_page_free_toq+0x273 #3 0xffffffff816bd5ba at zfs_freebsd_read+0x62a #4 0xffffffff8099113d at vn_rdwr+0x1ad #5 0xffffffff8095c9dd at kern_sendfile+0xdad #6 0xffffffff8095d12c at do_sendfile+0xdc #7 0xffffffff80bd7ae6 at amd64_syscall+0x546 #8 0xffffffff80bc3447 at Xfast_syscall+0xf7 Uptime: 17m54s Dumping 1026 out of 5100 = MB:..2%..11%..21%..32%..41%..52%..61%..71%..82%..91% Reading symbols from /boot/modules/virtio.ko...Reading symbols from = /boot/modules/virtio.ko.symbols...done. done. Loaded symbols for /boot/modules/virtio.ko Reading symbols from /boot/modules/virtio_pci.ko...Reading symbols from = /boot/modules/virtio_pci.ko.symbols...done. done. Loaded symbols for /boot/modules/virtio_pci.ko Reading symbols from /boot/modules/virtio_blk.ko...Reading symbols from = /boot/modules/virtio_blk.ko.symbols...done. done. Loaded symbols for /boot/modules/virtio_blk.ko Reading symbols from /boot/modules/if_vtnet.ko...Reading symbols from = /boot/modules/if_vtnet.ko.symbols...done. done. Loaded symbols for /boot/modules/if_vtnet.ko Reading symbols from /boot/modules/virtio_balloon.ko...Reading symbols = from /boot/modules/virtio_balloon.ko.symbols...done. done. Loaded symbols for /boot/modules/virtio_balloon.ko Reading symbols from /boot/kernel/zfs.ko...Reading symbols from = /boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from = /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from = /boot/kernel/ipfw.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipfw.ko Reading symbols from /boot/kernel/ipdivert.ko...Reading symbols from = /boot/kernel/ipdivert.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipdivert.ko #0 doadump (textdump=3DVariable "textdump" is not available. ) at pcpu.h:224 224 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump (textdump=3DVariable "textdump" is not available. ) at pcpu.h:224 #1 0xffffffff808ea3a1 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:448 #2 0xffffffff808ea897 in panic (fmt=3D0x1
) at /usr/src/sys/kern/kern_shutdown.c:636 #3 0xffffffff80b50923 in vm_page_free_toq (m=3D0xfffffe00d5d2db38) at /usr/src/sys/vm/vm_page.c:1862 #4 0xffffffff816bd5ba in zfs_freebsd_read (ap=3DVariable "ap" is not = available. ) at = /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/= zfs_vnops.c:476 #5 0xffffffff8099113d in vn_rdwr (rw=3DUIO_READ, vp=3D0xfffffe01058db780,= =20 base=3D0x0, len=3DVariable "len" is not available. ) at vnode_if.h:384 #6 0xffffffff8095c9dd in kern_sendfile (td=3D0xfffffe00986bf8e0,=20 uap=3D0xffffff8162323bb0, hdr_uio=3D0xfffffe002cf6da00, trl_uio=3D0x0,= compat=3D0) at /usr/src/sys/kern/uipc_syscalls.c:2115 #7 0xffffffff8095d12c in do_sendfile (td=3D0xfffffe00986bf8e0,=20 uap=3D0xffffff8162323bb0, compat=3D0) at = /usr/src/sys/kern/uipc_syscalls.c:1799 #8 0xffffffff80bd7ae6 in amd64_syscall (td=3D0xfffffe00986bf8e0, = traced=3D0) at subr_syscall.c:135 #9 0xffffffff80bc3447 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 #10 0x0000000801feb5cc in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) quit [root@dentredje ~]# kgdb /boot/kernel/kernell /home/crash/vmcore.3 kernel kernel.symbols =20 [root@dentredje ~]# kgdb /boot/kernel/kernell /home/crash/vmcore.3 kernel kernel.symbols =20 [root@dentredje ~]# kgdb /boot/kernel/kernel /home/crash/vmcore.3 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain = conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for = details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: vm_page_free: freeing busy page 0xfffffe00d5d2db38 cpuid =3D 1 KDB: stack backtrace: #0 0xffffffff809208a6 at kdb_backtrace+0x66 #1 0xffffffff808ea8be at panic+0x1ce #2 0xffffffff80b50923 at vm_page_free_toq+0x273 #3 0xffffffff816bd5ba at zfs_freebsd_read+0x62a #4 0xffffffff8099113d at vn_rdwr+0x1ad #5 0xffffffff8095c9dd at kern_sendfile+0xdad #6 0xffffffff8095d12c at do_sendfile+0xdc #7 0xffffffff80bd7ae6 at amd64_syscall+0x546 #8 0xffffffff80bc3447 at Xfast_syscall+0xf7 Uptime: 17m54s Dumping 1026 out of 5100 = MB:..2%..11%..21%..32%..41%..52%..61%..71%..82%..91% Reading symbols from /boot/modules/virtio.ko...Reading symbols from = /boot/modules/virtio.ko.symbols...done. done. Loaded symbols for /boot/modules/virtio.ko Reading symbols from /boot/modules/virtio_pci.ko...Reading symbols from = /boot/modules/virtio_pci.ko.symbols...done. done. Loaded symbols for /boot/modules/virtio_pci.ko Reading symbols from /boot/modules/virtio_blk.ko...Reading symbols from = /boot/modules/virtio_blk.ko.symbols...done. done. Loaded symbols for /boot/modules/virtio_blk.ko Reading symbols from /boot/modules/if_vtnet.ko...Reading symbols from = /boot/modules/if_vtnet.ko.symbols...done. done. Loaded symbols for /boot/modules/if_vtnet.ko Reading symbols from /boot/modules/virtio_balloon.ko...Reading symbols = from /boot/modules/virtio_balloon.ko.symbols...done. done. Loaded symbols for /boot/modules/virtio_balloon.ko Reading symbols from /boot/kernel/zfs.ko...Reading symbols from = /boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from = /boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from = /boot/kernel/ipfw.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipfw.ko Reading symbols from /boot/kernel/ipdivert.ko...Reading symbols from = /boot/kernel/ipdivert.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipdivert.ko #0 doadump (textdump=3DVariable "textdump" is not available. ) at pcpu.h:224 224 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump (textdump=3DVariable "textdump" is not available. ) at pcpu.h:224 #1 0xffffffff808ea3a1 in kern_reboot (howto=3D260) at = /usr/src/sys/kern/kern_shutdown.c:448 #2 0xffffffff808ea897 in panic (fmt=3D0x1
) = at /usr/src/sys/kern/kern_shutdown.c:636 #3 0xffffffff80b50923 in vm_page_free_toq (m=3D0xfffffe00d5d2db38) at = /usr/src/sys/vm/vm_page.c:1862 #4 0xffffffff816bd5ba in zfs_freebsd_read (ap=3DVariable "ap" is not = available. ) at = /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/= zfs_vnops.c:476 #5 0xffffffff8099113d in vn_rdwr (rw=3DUIO_READ, vp=3D0xfffffe01058db780,= base=3D0x0, len=3DVariable "len" is not available. ) at vnode_if.h:384 #6 0xffffffff8095c9dd in kern_sendfile (td=3D0xfffffe00986bf8e0, = uap=3D0xffffff8162323bb0, hdr_uio=3D0xfffffe002cf6da00, trl_uio=3D0x0,=20= compat=3D0) at /usr/src/sys/kern/uipc_syscalls.c:2115 #7 0xffffffff8095d12c in do_sendfile (td=3D0xfffffe00986bf8e0, = uap=3D0xffffff8162323bb0, compat=3D0) at /usr/src/sys/kern/uipc_syscalls.c:1799 #8 0xffffffff80bd7ae6 in amd64_syscall (td=3D0xfffffe00986bf8e0, = traced=3D0) at subr_syscall.c:135 #9 0xffffffff80bc3447 in Xfast_syscall () at = /usr/src/sys/amd64/amd64/exception.S:387 #10 0x0000000801feb5cc in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) up 6 #6 0xffffffff8095c9dd in kern_sendfile (td=3D0xfffffe00986bf8e0, = uap=3D0xffffff8162323bb0, hdr_uio=3D0xfffffe002cf6da00, trl_uio=3D0x0,=20= compat=3D0) at /usr/src/sys/kern/uipc_syscalls.c:2115 warning: Source file is more recent than executable. 2115 error =3D vn_rdwr(UIO_READ, vp, = NULL, MAXBSIZE, (kgdb) frame 6 #6 0xffffffff8095c9dd in kern_sendfile (td=3D0xfffffe00986bf8e0, = uap=3D0xffffff8162323bb0, hdr_uio=3D0xfffffe002cf6da00, trl_uio=3D0x0,=20= compat=3D0) at /usr/src/sys/kern/uipc_syscalls.c:2115 2115 error =3D vn_rdwr(UIO_READ, vp, = NULL, MAXBSIZE, (kgdb) print pg $1 =3D (struct vm_page *) 0xfffffe00d5d2db38 (kgdb) info reg rax 0x0 0 rbx 0x0 0 rcx 0x0 0 rdx 0x0 0 rsi 0x0 0 rdi 0x0 0 rbp 0xffffff8162323ab0 0xffffff8162323ab0 rsp 0xffffff8162323910 0xffffff8162323910 r8 0x0 0 r9 0x0 0 r10 0x0 0 r11 0x0 0 r12 0xfffffe00d5d2db38 -2195435889864 r13 0xfffffe002c731600 -2198277515776 r14 0xffffff8162323bb0 -543813387344 r15 0xfffffe000e2ea3a0 -2198785317984 rip 0xffffffff8095c9dd 0xffffffff8095c9dd = eflags 0x0 0 cs 0x0 0 ss 0x0 0 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 (kgdb)=20 Best regards, Rasmus skaarup On 05/03/2013, at 11.55, Andriy Gapon wrote: >=20 >=20 > Please try the following patch. >=20 > --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > @@ -543,7 +543,9 @@ mappedread_sf(vnode_t *vp, int nbytes, uio_t *uio) > vm_page_io_finish(pp); > vm_page_lock(pp); > if (error) { > - vm_page_free(pp); > + if (pp->wire_count =3D=3D 0 && pp->valid = =3D=3D 0 && > + pp->busy =3D=3D 0 && !(pp->oflags & = VPO_BUSY)) > + vm_page_free(pp); > } else { > pp->valid =3D VM_PAGE_BITS_ALL; > vm_page_activate(pp); >=20 > Additionally, if possible, I would like to see the following = information from > "Crash 3": > - value of 'pg' in kern_sendfile frame > - output of 'info reg' command in zfs_freebsd_read frame >=20 > Thank you. > --=20 > Andriy Gapon >=20