From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 10:55:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7BE201065678 for ; Tue, 2 Aug 2011 10:55:44 +0000 (UTC) (envelope-from seanrees@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 37B488FC22 for ; Tue, 2 Aug 2011 10:55:43 +0000 (UTC) Received: by vxg33 with SMTP id 33so6772420vxg.13 for ; Tue, 02 Aug 2011 03:55:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=khoX/ChfFBRZI3Qt95h/yzl5ykJN/PW6uH/IAWwSmZU=; b=e7LjdBEbO8A2dKbi7oTCiVAg/EBcsqjDoGZxJDBSffwtAkpA/HaIZwp4Abu5Ew6zkZ uwNUW/me4DI0hIGXsVLK0lIP5Yz6avW8u+UnQqpdVcF3JFPFdUWghBgEeOGgHr0GJitO 5HVHN4fs0uPnVkuax1j30RM14N/XNJ9kDucVU= MIME-Version: 1.0 Received: by 10.52.71.1 with SMTP id q1mr5168895vdu.380.1312282543498; Tue, 02 Aug 2011 03:55:43 -0700 (PDT) Received: by 10.52.168.68 with HTTP; Tue, 2 Aug 2011 03:55:43 -0700 (PDT) In-Reply-To: <4E37CD13.1070402@digsys.bg> References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> Date: Tue, 2 Aug 2011 11:55:43 +0100 Message-ID: From: "seanrees@gmail.com" To: Daniel Kalchev Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 10:55:44 -0000 On Tue, Aug 2, 2011 at 11:10 AM, Daniel Kalchev wrote: >> If it is a limitation in ZFS it would be nice to know that, perhaps it >> truly, really is a bug that can be avoided (or it's inherent in the way = ZFS >> handles such things) > > It is possible =A0that there is not enough memory in ARC to cache that la= rge > directory. > > Other than that, perhaps in ZFS it would be easier to prune the unused > directory entries, than it is in UFS. It looks like this is not implement= ed. > > Another reason might be some FreeBSD specific implementation issue for > fstatfs. > > In any case, the data available is not sufficient. More information would > help, like how much RAM this system has, how much ARC uses, some ARC stat= s. Which sysctl's would you like? I grabbed these to start: kstat.zfs.misc.arcstats.size: 118859656 kstat.zfs.misc.arcstats.hdr_size: 3764416 kstat.zfs.misc.arcstats.data_size: 53514240 kstat.zfs.misc.arcstats.other_size: 61581000 kstat.zfs.misc.arcstats.hits: 46762467 kstat.zfs.misc.arcstats.misses: 16999907 The machine has 2GB of memory. > What made me wonder is .. how exactly the kernel and zpool disagree on zp= ool > version? What is the pool version in fact? % dmesg | grep ZFS ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is pres= ent; to enable, add "vfs.zfs.prefetch_disable=3D0" to /boot/loader.c= onf. ZFS filesystem version 5 ZFS storage pool version 28 % zpool get version tank NAME PROPERTY VALUE SOURCE tank version 15 local % zpool upgrade tank This system is currently running ZFS pool version 15. Pool 'tank' is already formatted using the current version. Sean