From owner-freebsd-fs@FreeBSD.ORG Wed Jul 29 20:15:14 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0E941065672; Wed, 29 Jul 2009 20:15:14 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id 2E3728FC18; Wed, 29 Jul 2009 20:15:14 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from c83-253-252-234.bredband.comhem.se ([83.253.252.234]:51115 helo=mx.exscape.org) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.68) (envelope-from ) id 1MWFYM-0000YR-3p; Wed, 29 Jul 2009 22:15:12 +0200 Received: from [192.168.1.5] (macbookpro [192.168.1.5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx.exscape.org (Postfix) with ESMTPSA id 2B3ED4429A; Wed, 29 Jul 2009 22:15:10 +0200 (CEST) Message-Id: <16B40A2B-A1B5-4528-8721-6D352E7D5419@exscape.org> From: Thomas Backman To: Andriy Gapon In-Reply-To: <4A708455.5070304@freebsd.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Wed, 29 Jul 2009 22:15:06 +0200 References: <20090727072503.GA52309@jpru.ffm.jpru.de> <4A6E06E6.9030300@mail.zedat.fu-berlin.de> <4A6EC9E2.5070200@icyb.net.ua> <20090729084723.GD1586@garage.freebsd.pl> <4A7030B6.8010205@icyb.net.ua> <97D5950F-4E4D-4446-AC22-92679135868D@exscape.org> <4A7048A9.4020507@icyb.net.ua> <52AA86CB-6C06-4370-BA73-CE19175467D0@exscape.org> <4A705299.8060504@icyb.net.ua> <4A7054E1.5060402@icyb.net.ua> <5918824D-A67C-43E6-8685-7B72A52B9CAE@exscape.org> <4A705E50.8070307@icyb.net.ua> <4A70728C.7020004@freebsd.org> <6D47A34B-0753-4CED-BF3D-C505B37748FC@exscape.org> <4A708455.5070304@freebsd.org> X-Mailer: Apple Mail (2.935.3) X-Originating-IP: 83.253.252.234 X-Scan-Result: No virus found in message 1MWFYM-0000YR-3p. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1MWFYM-0000YR-3p bc9334e9662bd9b399117d24cd624934 Cc: freebsd-fs@freebsd.org, FreeBSD current , Pawel Jakub Dawidek Subject: Re: zfs: Fatal trap 12: page fault while in kernel mode X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jul 2009 20:15:15 -0000 On Jul 29, 2009, at 19:18, Andriy Gapon wrote: > > Thanks a lot again! > > Could you please try the following change? > In sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c, in > function > zfs_inactive() insert the following line: > vrecycle(vp, curthread); > before the following line: > zfs_znode_free(zp); > > This is in "if (zp->z_dbuf == NULL)" branch. > > I hope that this should work in concert with the patch that Pawel > has posted. > > P.S. > Also Pawel has told me that adding 'CFLAGS+=-DDEBUG=1' to sys/ > modules/zfs/Makefile > should enable additional debugging checks (ASSERTs) in ZFS code. > > -- > Andriy Gapon Better backtraces: Without your vrecycle() addition, and with the -DDEBUG=1 one (note to self: core.txt.32): Unread portion of the kernel message buffer: panic: solaris assert: ((zp)->z_vnode) == ((void *)0), file: /usr/src/ sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/ zfs_znode.c, line: 1043 cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a panic() at panic+0x182 zfs_znode_free() at zfs_znode_free+0xef zfs_freebsd_inactive() at zfs_freebsd_inactive+0x1a VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0x4a vinactive() at vinactive+0x6a vput() at vput+0x1c6 dounmount() at dounmount+0x4af unmount() at unmount+0x3c8 syscall() at syscall+0x28f Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (22, FreeBSD ELF64, unmount), rip = 0x80104e9ec, rsp = 0x7fffffffaa98, rbp = 0x801223300 --- KDB: enter: panic panic: from debugger cpuid = 0 Uptime: 1m5s Physical memory: 2034 MB Dumping 1405 MB: ... #11 0xffffffff8033a9cb in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:558 #12 0xffffffff80aed21f in zfs_znode_free () from /boot/kernel/zfs.ko #13 0xffffffff80b10a9a in zfs_freebsd_inactive () from /boot/kernel/ zfs.ko #14 0xffffffff805c5b5a in VOP_INACTIVE_APV (vop=0xffffffff80b88220, a=0xffffff00401b9a48) at vnode_if.c:1863 #15 0xffffffff803c6aaa in vinactive (vp=0xffffff004038c3b0, td=0xffffff0040031000) at vnode_if.h:807 #16 0xffffffff803cbf26 in vput (vp=0xffffff004038c3b0) at /usr/src/sys/kern/vfs_subr.c:2257 #17 0xffffffff803c57ef in dounmount (mp=0xffffff0001cea8d0, flags=0, td=Variable "td" is not available. ) at /usr/src/sys/kern/vfs_mount.c:1333 #18 0xffffffff803c5df8 in unmount (td=0xffffff0013adfab0, uap=0xffffff803ead0bf0) at /usr/src/sys/kern/vfs_mount.c:1174 #19 0xffffffff805980bf in syscall (frame=0xffffff803ead0c80) at /usr/src/sys/amd64/amd64/trap.c:984 #20 0xffffffff8057e2c1 in Xfast_syscall () at /usr/src/sys/amd64/ amd64/exception.S:373 #21 0x000000080104e9ec in ?? () Previous frame inner to this frame (corrupt stack?) --------------------------- WITH the vrecycle() and -DDEBUG=1: kernel trap 9 with interrupts disabled Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x20:0xffffffff80342b99 stack pointer = 0x28:0xffffff803eaf8910 frame pointer = 0x28:0xffffff803eaf8970 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 1414 (zpool) panic: from debugger cpuid = 0 KDB: stack backtrace: Uptime: 1m16s Physical memory: 2034 MB Dumping 1407 MB: ... #9 0xffffffff805986aa in trap (frame=0xffffff803eaf8860) at /usr/src/ sys/amd64/amd64/trap.c:639 #10 0xffffffff8057dfe7 in calltrap () at /usr/src/sys/amd64/amd64/ exception.S:224 #11 0xffffffff80342b99 in _sx_xlock_hard (sx=0xffffff0044136251, tid=18446742975340199936, opts=Variable "opts" is not available. ) at /usr/src/sys/kern/kern_sx.c:575 #12 0xffffffff8034350e in _sx_xlock (sx=Variable "sx" is not available. ) at sx.h:155 #13 0xffffffff80aed172 in zfs_znode_free () from /boot/kernel/zfs.ko #14 0xffffffff80b10a8a in zfs_freebsd_inactive () from /boot/kernel/ zfs.ko #15 0xffffffff805c5b5a in VOP_INACTIVE_APV (vop=0xffffff0044136251, a=0xffffff0015b9cd38) at vnode_if.c:1863 #16 0xffffffff803c6aaa in vinactive (vp=0xffffff00443dc588, td=0xffffff0044136001) at vnode_if.h:807 #17 0xffffffff803cbf26 in vput (vp=0xffffff00443dc588) at /usr/src/sys/ kern/vfs_subr.c:2257 #18 0xffffffff803c57ef in dounmount (mp=0xffffff0001cc38d0, flags=0, td=Variable "td" is not available. ) at /usr/src/sys/kern/vfs_mount.c:1333 #19 0xffffffff803c5df8 in unmount (td=0xffffff004415c000, uap=0xffffff803eaf8bf0) at /usr/src/sys/kern/vfs_mount.c:1174 #20 0xffffffff805980bf in syscall (frame=0xffffff803eaf8c80) at /usr/ src/sys/amd64/amd64/trap.c:984 #21 0xffffffff8057e2c1 in Xfast_syscall () at /usr/src/sys/amd64/amd64/ exception.S:373 #22 0x000000080104e9ec in ?? () Previous frame inner to this frame (corrupt stack?) Time to sleep. I only have the kernel.debug for the latter panic, by the way, but at a quick glance they appear to be the same except for the panic line...? Regards, Thomas