From owner-freebsd-stable@FreeBSD.ORG Tue Dec 13 21:33:20 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10A0E1065670 for ; Tue, 13 Dec 2011 21:33:20 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id A3DF98FC0C for ; Tue, 13 Dec 2011 21:33:19 +0000 (UTC) Received: by wgbdr11 with SMTP id dr11so204309wgb.31 for ; Tue, 13 Dec 2011 13:33:18 -0800 (PST) Received: by 10.227.198.142 with SMTP id eo14mr80010wbb.28.1323810681943; Tue, 13 Dec 2011 13:11:21 -0800 (PST) Received: from [10.254.254.77] ([95.165.122.0]) by mx.google.com with ESMTPS id fw16sm435458wbb.13.2011.12.13.13.11.20 (version=SSLv3 cipher=OTHER); Tue, 13 Dec 2011 13:11:21 -0800 (PST) Message-ID: <4EE7BF77.5000504@zonov.org> Date: Wed, 14 Dec 2011 01:11:19 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.24) Gecko/20100228 Thunderbird/2.0.0.24 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: directory listing hangs in "ufs" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 21:33:20 -0000 Hi, I've got STABLE-8 (r221983) with mongodb-1.8.1 installed on it. A couple days ago I observed that listing of mongodb directory stuck in a few minutes in "ufs" state. I've run it again with ktrace and got following (kdump -R): 91324 ls 0.000003 CALL lstat(0x32c199c8,0x32c19950) 91324 ls 0.000003 NAMI "base.1" 91324 ls 21.357255 STRU struct stat {dev=116, ino=45125633, mode=-rw------- , nlink=1, uid=922, gid=922, rdev=180226648, atime=1323709877, stime=1323776461, ctime=1323776461, birthtime=1314798592, size=134217728, blksize=16384, blocks=262304, flags=0x0 } 91324 ls 0.000014 RET lstat 0 kgdb backtrace of this process was looked like this: Thread 297 (Thread 100372): #0 sched_switch (td=0xffffff0095c008c0, newtd=0xffffff000357b8c0, flags=) at /usr/src/sys/kern/sched_ule.c:1866 #1 0xffffffff80406696 in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:449 #2 0xffffffff8043c072 in sleepq_wait (wchan=0xffffff0103aaf7f8, pri=80) at /usr/src/sys/kern/subr_sleepqueue.c:609 #3 0xffffffff803e4a5a in __lockmgr_args (lk=0xffffff0103aaf7f8, flags=2097408, ilk=0xffffff0103aaf820, wmesg=) at /usr/src/sys/kern/kern_lock.c:220 #5 0xffffffff8061239c in ffs_lock (ap=0xffffff84867fc550) at lockmgr.h:94 #5 0xffffffff806d2462 in VOP_LOCK1_APV (vop=0xffffffff80921fe0, a=0xffffff84867fc550) at vnode_if.c:1988 #6 0xffffffff804a58b7 in _vn_lock (vp=0xffffff0103aaf760, flags=2097152, file=0xffffffff80736e70 "/usr/src/sys/kern/vfs_subr.c", line=2137) at vnode_if.h:859 #7 0xffffffff80498bc0 in vget (vp=0xffffff0103aaf760, flags=2097408, td=0xffffff0095c008c0) at /usr/src/sys/kern/vfs_subr.c:2137 #8 0xffffffff804845f4 in cache_lookup (dvp=0xffffff0095675b10, vpp=0xffffff84867fc910, cnp=0xffffff84867fc938) at /usr/src/sys/kern/vfs_cache.c:587 #9 0xffffffff80484a30 in vfs_cache_lookup (ap=) at /usr/src/sys/kern/vfs_cache.c:905 #10 0xffffffff806d2e7c in VOP_LOOKUP_APV (vop=0xffffffff80922820, a=0xffffff84867fc790) at vnode_if.c:123 #11 0xffffffff8048bc80 in lookup (ndp=0xffffff84867fc8e0) at vnode_if.h:54 #12 0xffffffff8048cf0e in namei (ndp=0xffffff84867fc8e0) at /usr/src/sys/kern/vfs_lookup.c:269 #13 0xffffffff8049c972 in kern_statat_vnhook (td=0xffffff0095c008c0, flag=) at /usr/src/sys/kern/vfs_syscalls.c:2346 #14 0xffffffff8049cbb5 in kern_statat (td=) at /usr/src/sys/kern/vfs_syscalls.c:2327 #15 0xffffffff8049cc7a in lstat (td=) at /usr/src/sys/kern/vfs_syscalls.c:2390 #16 0xffffffff8043e7dd in syscallenter (td=0xffffff0095c008c0, sa=0xffffff84867fcbb0) at /usr/src/sys/kern/subr_trap.c:326 #17 0xffffffff8066a5eb in syscall (frame=0xffffff84867fcc50) at /usr/src/sys/amd64/amd64/trap.c:916 #18 0xffffffff806517f2 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:384 #19 0x000000003298f75c in ?? () The very first idea was to turn off name caching (set debug.vfscache to 0), but it didn't help. The second idea was to reboot, but it didn't help too. This directory locks fine. It has 10 files and 1 empty directory. Have you any ideas what is going on? or how to catch the problem? -- Andrey Zonov