From owner-freebsd-current@FreeBSD.ORG Sat Jun 25 10:31:13 2005 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9524E16A41C for ; Sat, 25 Jun 2005 10:31:13 +0000 (GMT) (envelope-from morganw@chemikals.org) Received: from ms-smtp-04-eri0.southeast.rr.com (ms-smtp-04-lbl.southeast.rr.com [24.25.9.103]) by mx1.FreeBSD.org (Postfix) with ESMTP id 49D8B43D4C for ; Sat, 25 Jun 2005 10:31:12 +0000 (GMT) (envelope-from morganw@chemikals.org) Received: from volatile.chemikals.org (cpe-024-211-118-154.sc.res.rr.com [24.211.118.154]) by ms-smtp-04-eri0.southeast.rr.com (8.12.10/8.12.7) with ESMTP id j5PAVAL4000415 for ; Sat, 25 Jun 2005 06:31:10 -0400 (EDT) Received: from volatile.chemikals.org (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.13.3/8.13.3) with ESMTP id j5PAV9Ap036815 for ; Sat, 25 Jun 2005 06:31:09 -0400 (EDT) (envelope-from morganw@chemikals.org) Received: from localhost (morganw@localhost) by volatile.chemikals.org (8.13.3/8.13.3/Submit) with ESMTP id j5PAV8BJ036812 for ; Sat, 25 Jun 2005 06:31:08 -0400 (EDT) (envelope-from morganw@chemikals.org) X-Authentication-Warning: volatile.chemikals.org: morganw owned process doing -bs Date: Sat, 25 Jun 2005 06:31:07 -0400 (EDT) From: Wesley Morgan To: current@freebsd.org Message-ID: <20050625062259.G36786@volatile.chemikals.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: Symantec AntiVirus Scan Engine Cc: Subject: processes hanging in ufs/vfs X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jun 2005 10:31:13 -0000 The strange panics I was getting last week seem to have stopped, but now I am seeing what appears to be deadlocked processes trying to read or write to the same file. The processes are, of course, unkillable and prevent a clean reboot. I was running with the "knlist_locking" patch but that no longer compiles, and backing out the recent kqueue patch doesn't work either. Traces from ddb are below. 1001 859 856 0 96 0 23444 22672 vnode LL+ p9 2:17.00 tf (tf-50b7) 1001 878 876 0 76 0 1344 672 vnode L+ pb 0:01.50 tail -n 10000 -f lothos.2005.25.log 1001 969 967 0 -4 0 1344 672 ufs D+ pf 0:01.63 tail -n 10000 -f lothos.2005.25.log Stopped at kdb_enter+0x30: leave db> trace 969 Tracing pid 969 tid 100152 td 0xc210e480 sched_switch(c210e480,0,1,9c8da765,31d15dff) at sched_switch+0x160 mi_switch(1,0,c210e480,0,c210e480) at mi_switch+0x1d9 sleepq_switch(c20c8c08,d5eee9b8,c0514567,c20c8c08,50) at sleepq_switch+0xee sleepq_wait(c20c8c08,50,c0692182,0,c210e480) at sleepq_wait+0x11 msleep(c20c8c08,c06dae08,50,c0692182,0) at msleep+0x3a7 acquire(d5eeea18,40,70000,c06522ff,c06cee40) at acquire+0x8e lockmgr(c20c8c08,3001,c20c8c2c,c210e480,d5eeea40) at lockmgr+0x196 vop_stdlock(d5eeea98,0,c06c44e0,d5eeea98,d5eeea50) at vop_stdlock+0x2f VOP_LOCK_APV(c06c4a20,d5eeea98,d5eeea68,c0664b84,d5eeea98) at VOP_LOCK_APV+0x54 ffs_lock(d5eeea98,120,1001,c20c8bb0,d5eeeab4) at ffs_lock+0x19 VOP_LOCK_APV(c06c44e0,d5eeea98,cb704bd0,202122,d5eeeacc) at VOP_LOCK_APV+0x54 vn_lock(c20c8bb0,1001,c210e480,0,d5eeeb10) at vn_lock+0x13c filt_vfsread(c1a0b154,0,c210e300,0,0) at filt_vfsread+0x4a kqueue_scan(c2157e80,1,d5eeecbc,0,d5eeebe8) at kqueue_scan+0x590 kern_kevent(c210e480,4,0,1,d5eeecbc) at kern_kevent+0x1a0 kevent(c210e480,d5eeed04,18,d5eeed2c,c0533cd6) at kevent+0x7a syscall(804003b,2816003b,bfbf003b,bfbfe160,1) at syscall+0x370 Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (363, FreeBSD ELF32, kevent), eip = 0x280ca27f, esp = 0xbfbfe12c, ebp = 0xbfbfe1d8 --- db> trace 859 Tracing pid 859 tid 100120 td 0xc1f79180 sched_switch(c1f79180,0,1,6842a435,914c421e) at sched_switch+0x160 mi_switch(1,0,c1d18280,c06dfda0,c210a000) at mi_switch+0x1d9 turnstile_wait(c210a000,c210e480,0,c210a000,d5e67b08) at turnstile_wait+0x297 _mtx_lock_sleep(c210a000,c1f79180,0,0,0) at _mtx_lock_sleep+0xb4 knote(c210a030,6,0,456136,456136) at knote+0x61 VOP_WRITE_APV(c06c44e0,d5e67bf8,c1f79180,d5e67bec,c07e2860) at VOP_WRITE_APV+0x1 6a vn_write(c1a11a68,d5e67c6c,c20dbe00,0,c1f79180) at vn_write+0x240 dofilewrite(c1f79180,c1a11a68,4,833d000,32) at dofilewrite+0xcb write(c1f79180,d5e67d04,c,0,d5e67d10) at write+0x69 syscall(857003b,ffff003b,bfbf003b,833d000,2837d558) at syscall+0x370 Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (4, FreeBSD ELF32, write), eip = 0x282d9bdf, esp = 0xbfbfe04c, ebp = 0xbfbfe068 --- db> trace 898 Tracing pid 898 tid 100132 td 0xc20bb180 sched_switch(c20bb180,0,1,4c8ded05,6b06e864) at sched_switch+0x160 mi_switch(1,0,0,c20bb180,c06e0d24) at mi_switch+0x1d9 sleepq_switch(c06e0d24,0,d5eb0b3c,c04d7e60,c06e0d24) at sleepq_switch+0xee sleepq_wait_sig(c06e0d24,c06e0d00,c0690b85,101,0) at sleepq_wait_sig+0x12 cv_wait_sig(c06e0d24,c06e0d00,d5eb0b78,14,c1644c00) at cv_wait_sig+0x1d0 kern_select(c20bb180,14,bfbfe140,bfbfe0c0,0) at kern_select+0x62d select(c20bb180,d5eb0d04,14,0,d5eb0d10) at select+0x6a syscall(80a003b,3b,bfbf003b,0,10) at syscall+0x370 Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (93, FreeBSD ELF32, select), eip = 0x28182f6f, esp = 0xbfbfe08c, ebp = 0xbfbfe1d8 --- -- Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread!