From owner-freebsd-current@FreeBSD.ORG Thu May 21 08:52:34 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4CB5E106566B; Thu, 21 May 2009 08:52:34 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id CE2C78FC23; Thu, 21 May 2009 08:52:33 +0000 (UTC) (envelope-from serenity@exscape.org) Received: from c83-253-252-234.bredband.comhem.se ([83.253.252.234]:39204 helo=mx.exscape.org) by ch-smtp01.sth.basefarm.net with esmtp (Exim 4.69) (envelope-from ) id 1M73lq-0000fW-3m; Thu, 21 May 2009 10:37:06 +0200 Received: from [192.168.1.5] (macbookpro [192.168.1.5]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx.exscape.org (Postfix) with ESMTPSA id 29B38335EC; Thu, 21 May 2009 10:36:54 +0200 (CEST) Message-Id: From: Thomas Backman To: Kip Macy In-Reply-To: <3c1674c90905181815r49a5bbe2u5d4a73e4f91f89a6@mail.gmail.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Thu, 21 May 2009 10:36:54 +0200 References: <20090518145614.GF82547@egr.msu.edu> <3c1674c90905181659g1d20f0f1w3f623966ae4440ec@mail.gmail.com> <3c1674c90905181800x469d3cabx5df959d3585b4b5a@mail.gmail.com> <040ce5cee10b3b92fd127bbce83f4c77.squirrel@webmail.lerctr.org> <3c1674c90905181815r49a5bbe2u5d4a73e4f91f89a6@mail.gmail.com> X-Mailer: Apple Mail (2.935.3) X-Originating-IP: 83.253.252.234 X-Scan-Result: No virus found in message 1M73lq-0000fW-3m. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1M73lq-0000fW-3m 409936a4669b1a76da0e6a730d04844e Cc: current@freebsd.org Subject: Re: Fatal trap 12: page fault panic with recent kernel with ZFS X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 May 2009 08:52:34 -0000 On May 19, 2009, at 03:15 AM, Kip Macy wrote: > > Please update to r192360 and let me know if that helps. > > > Thanks, > Kip Hey, I (not the OP) am still having trouble with this. The panics appear to be gone since I set arc_min to 30M and arc_max to 100M, though (2GB RAM). I get/got them during a full zfs send -R | zfs recv -Fvd of a ~3.3GB pool. The only modification I've done to the source tree is the libzfs_sendrecv patch here: http://lists.freebsd.org/pipermail/freebsd-current/2009-May/006814.html FreeBSD chaos.exscape.org 8.0-CURRENT FreeBSD 8.0-CURRENT #1: Wed May 20 21:19:29 CEST 2009 root@chaos.exscape.org:/usr/obj/usr/src/sys/ DTRACE amd64 (GENERIC + the 3 DTrace lines added) Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x0 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff8085e89a stack pointer = 0x28:0xffffff803ea4e4d0 frame pointer = 0x28:0xffffff803ea4e590 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1281 (zfs) vmstat -s from core.txt shows 388955 pages wired down - I take it that's 388955*4 kiB or almost 1.5GB out of 2GB. 110685 pages are still free, if that's relevant (I'm not sure why the crash occurs, but I guess it might be). Backtrace: #0 doadump () at pcpu.h:223 223 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:223 #1 0xffffffff80576089 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:420 #2 0xffffffff805764dc in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:576 #3 0xffffffff801d5a97 in db_panic (addr=Variable "addr" is not available. ) at /usr/src/sys/ddb/db_command.c:478 #4 0xffffffff801d5ea1 in db_command (last_cmdp=0xffffffff80bd7c20, cmd_table=Variable "cmd_table" is not available. ) at /usr/src/sys/ddb/db_command.c:445 #5 0xffffffff801d60f0 in db_command_loop () at /usr/src/sys/ddb/db_command.c:498 #6 0xffffffff801d8089 in db_trap (type=Variable "type" is not available. ) at /usr/src/sys/ddb/db_main.c:229 #7 0xffffffff805a6bb5 in kdb_trap (type=12, code=0, tf=0xffffff803ea4e420) at /usr/src/sys/kern/subr_kdb.c:534 #8 0xffffffff808603bd in trap_fatal (frame=0xffffff803ea4e420, eva=Variable "eva" is not available. ) at /usr/src/sys/amd64/amd64/trap.c:847 #9 0xffffffff80860794 in trap_pfault (frame=0xffffff803ea4e420, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:768 #10 0xffffffff80861283 in trap (frame=0xffffff803ea4e420) at /usr/src/sys/amd64/amd64/trap.c:494 #11 0xffffffff8083ae57 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:223 #12 0xffffffff8085e89a in bzero () at /usr/src/sys/amd64/amd64/ support.S:64 #13 0xffffffff80e8aab1 in dbuf_read () from /boot/kernel/zfs.ko #14 0xffffffff80e8a12e in dmu_buf_rele () from /boot/kernel/zfs.ko #15 0xffffff0002e34460 in ?? () #16 0x0000000000000000 in ?? () #17 0x0000000000000000 in ?? () #18 0x000000003ea4e570 in ?? () #19 0xffffff003533c2d0 in ?? () #20 0xffffff803ea4e588 in ?? () #21 0xffffff8004619240 in ?? () #22 0xffffff00415591c0 in ?? () #23 0x0000000000000000 in ?? () #24 0x0000000000000000 in ?? () #25 0x000000044153f300 in ?? () #26 0x0000000000000000 in ?? () #27 0xffffff0002e34460 in ?? () #28 0x0000000000000005 in ?? () #29 0xffffff004153f300 in ?? () #30 0x0000000000000000 in ?? () #31 0x000000000000da01 in ?? () #32 0xffffff803ea4e5c0 in ?? () #33 0xffffffff80e963ea in dmu_tx_check_ioerr () from /boot/kernel/zfs.ko Regards, Thomas PS. I have the full core.txt if it'd help. DS.