From owner-freebsd-fs@FreeBSD.ORG Wed Sep 29 15:03:53 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2CF31065674 for ; Wed, 29 Sep 2010 15:03:53 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 450D88FC14 for ; Wed, 29 Sep 2010 15:03:52 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA29301; Wed, 29 Sep 2010 18:03:48 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4CA35553.1080804@icyb.net.ua> Date: Wed, 29 Sep 2010 18:03:47 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: Karl Pielorz References: <201009291024.o8TAOnph013730@higson.cam.lispworks.com> <8CF1F1F15531907E2F8DC2A2@HexaDeca64.dmpriest.net.uk> In-Reply-To: <8CF1F1F15531907E2F8DC2A2@HexaDeca64.dmpriest.net.uk> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: FreeBSD 8.1-R/amd64 - zfs 'hangs' - help tracing? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 15:03:54 -0000 on 29/09/2010 13:30 Karl Pielorz said the following: > --On 29 September 2010 11:24 +0100 Martin Simmons wrote: > >>> I saw a while ago a command under 8.1 to get 'more info' for these stuck >>> processes, but can't for the life of me remember it? >> >> Maybe procstat -k -k $pid is what you are looking for (i.e. a kernel >> backtrace)? Use -a instead of $pid to get all processes. > > Yup, that's it - thanks! > > Having run it I get: > > procstat -k -k 1927 (PID 1927 is the 'ls' that's locked up) > > PID TID COMM TDNAME KSTACK > 1927 100206 ls - mi_switch+0x16f sleepq_wait+0x42 > _cv_wait+0x111 zio_wait+0x61 dbuf_read+0x39a dnode_hold_impl+0xe7 > dmu_bonus_hold+0x2a zfs_zget+0x227 zfs_dirent_lock+0x4e3 zfs_dirlook+0x69 > zfs_lookup+0x1f0 zfs_freebsd_lookup+0x81 vfs_cache_lookup+0xf0 VOP_LOOKUP_APV+0x40 > lookup+0x40a namei+0x52b kern_statat_vnhook+0x8f kern_statat+0x15 > > > Which will hopefully mean something more to someone here than it does me at the > moment ;) This looks like the process is stuck waiting for I/O completion. Can't tell whether it's an I/O problem, or perhaps the I/O operation has long completed but wakeup from it was lost... -- Andriy Gapon