From owner-freebsd-stable@FreeBSD.ORG Tue Dec 26 23:19:30 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B45D216A403 for ; Tue, 26 Dec 2006 23:19:29 +0000 (UTC) (envelope-from geoffwa@cs.rmit.edu.au) Received: from its-mu-mail1.its.rmit.edu.au (its-mu-mail1.its.rmit.edu.au [131.170.1.11]) by mx1.freebsd.org (Postfix) with ESMTP id 5DB6C13C463 for ; Tue, 26 Dec 2006 23:19:29 +0000 (UTC) (envelope-from geoffwa@cs.rmit.edu.au) Received: from wombat.cs.rmit.edu.au (wombat.cs.rmit.edu.au [131.170.24.41]) by its-mu-mail1.its.rmit.edu.au (8.13.7/8.13.7/mail1) with ESMTP id kBQMsEiq002266 for ; Wed, 27 Dec 2006 09:54:14 +1100 (EST) Received: from goanna.cs.rmit.edu.au (root@goanna.cs.rmit.edu.au [131.170.24.40]) by wombat.cs.rmit.edu.au (8.12.10/8.12.10/cshub) with ESMTP id kBQMsDXn010656 for ; Wed, 27 Dec 2006 09:54:14 +1100 (EST) Received: from goanna.cs.rmit.edu.au (geoffwa@localhost [127.0.0.1]) by goanna.cs.rmit.edu.au (8.13.4+Sun/8.13.4/csnode) with ESMTP id kBQMsBmt009372 for ; Wed, 27 Dec 2006 09:54:13 +1100 (EST) Received: (from geoffwa@localhost) by goanna.cs.rmit.edu.au (8.13.4+Sun/8.13.3/Submit) id kBQMsB1o009371 for stable@freebsd.org; Wed, 27 Dec 2006 09:54:11 +1100 (EST) Date: Wed, 27 Dec 2006 09:54:11 +1100 From: Geoffrey Giesemann To: stable@freebsd.org Message-ID: <20061226225411.GA8706@cs.rmit.edu.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.11 X-Scanned-By: MIMEDefang 2.44 Cc: Subject: Background fsck causes kernel panic X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Dec 2006 23:19:30 -0000 Running a fairly recent RELENG_6 SMP kernel, I had a nasty experience where filesystem corruption on a pair of RAID5 volumes on an arcmsr would cause the machine to panic during the background fsck. This resulted in a merry crash-reboot-crash cycle until I booted on to an install CD, ran a foreground fsck and fixed the errors. Given that this machine *is* a file server I'm rather eager to make sure the problem doesn't happen again. I'm not familiar with the code at all, but all three panics look like they're happening in soft-update sections: Unread portion of the kernel message buffer: panic: softdep_setup_freeblocks: inode busy cpuid = 0 Uptime: 4h58m59s Dumping 1023 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 1023MB (261872 pages) 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:165 165 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); (kgdb) where #0 doadump () at pcpu.h:165 #1 0xc061e541 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc061e934 in panic (fmt=0xc0905fc5 "softdep_setup_freeblocks: inode busy") at /usr/src/sys/kern/kern_shutdown.c:565 #3 0xc07bb7f9 in softdep_setup_freeblocks (ip=0xc6002c60, length=Unhandled dwarf expression opcode 0x93 ) at /usr/src/sys/ufs/ffs/ffs_softdep.c:2271 #4 0xc07ae85d in ffs_truncate (vp=0xc5fff000, length=0, flags=3072, cred=0x0, td=0xc531b180) at /usr/src/sys/ufs/ffs/ffs_inode.c:278 #5 0xc07d0abe in ufs_inactive (ap=0x0) at /usr/src/sys/ufs/ufs/ufs_inode.c:126 #6 0xc08b364e in VOP_INACTIVE_APV (vop=0x0, a=0x0) at vnode_if.c:1535 #7 0xc068faf2 in vinactive (vp=0xc5fff000, td=0x0) at vnode_if.h:795 #8 0xc068f81c in vput (vp=0xc5fff000) at /usr/src/sys/kern/vfs_subr.c:2158 #9 0xc0697539 in kern_unlink (td=0xc531b180, path=0xbfbfe4b0
, pathseg=UIO_USERSPACE) at /usr/src/sys/kern/vfs_syscalls.c:1722 #10 0xc0697322 in unlink (td=0x0, uap=0x0) at /usr/src/sys/kern/vfs_syscalls.c:1658 #11 0xc089d6f0 in syscall (frame= {tf_fs = -1078001605, tf_es = 135528507, tf_ds = -1078001605, tf_edi = 1, tf_esi = 135534336, tf_ebp = -1077942072, tf_isp = -412762780, tf_ebx = 135461640, tf_edx = 0, tf_ecx = 5, tf_eax = 10, tf_trapno = 0, tf_err = 2, tf_eip = 674363207, tf_cs = 51, tf_eflags = 642, tf_esp = -1077944196, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:983 #12 0xc088501f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #13 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) I've filed a PR about it (which has two more backtraces): http://www.freebsd.org/cgi/query-pr.cgi?pr=107206 Can anyone suggest further action? --Geoff