From owner-freebsd-questions Thu Jan 2 8:58:43 2003 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6FDE837B413 for ; Thu, 2 Jan 2003 08:58:42 -0800 (PST) Received: from mta5.snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2576A43EC5 for ; Thu, 2 Jan 2003 08:58:42 -0800 (PST) (envelope-from mbsd@pacbell.net) Received: from atlas ([64.160.45.11]) by mta5.snfc21.pbi.net (iPlanet Messaging Server 5.1 HotFix 1.6 (built Oct 18 2002)) with ESMTP id <0H8300IW2J5TA9@mta5.snfc21.pbi.net> for freebsd-questions@FreeBSD.ORG; Thu, 02 Jan 2003 08:58:42 -0800 (PST) Date: Thu, 02 Jan 2003 08:58:41 -0800 (PST) From: =?ISO-8859-1?Q?Mikko_Ty=F6l=E4j=E4rvi?= Subject: Re: Lots of files in a directory In-reply-to: <200301020955300340.09F341B4@home.24cl.com> X-X-Sender: mikko@atlas.home To: MikeM Cc: freebsd-questions@FreeBSD.ORG Message-id: <20030102084356.R18514-100000@atlas.home> MIME-version: 1.0 Content-type: TEXT/PLAIN; charset=US-ASCII Content-transfer-encoding: 7BIT Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 2 Jan 2003, MikeM wrote: > On 1/2/03 at 3:13 PM Roman Neuhauser wrote: > > |# MyRaQ@mgm51.com / 2003-01-02 09:01:27 -0500: > |> I'm considering setting up my server as a mirror site for the freedb.org > |> lookup database. Unfortunately, I've seem to have run into a stumbling > |> block. The server app requires over 250,000 files in a single > directory. > |> Each file is about 2k in size. It can be argued that the application is poorly designed... [...] > I already have UFS_DIRHASH specified in my kernel config file. :-( Maybe you can tweak some sysctls: atlas% sysctl -a | grep dirhash vfs.ufs.dirhash_minsize: 2560 vfs.ufs.dirhash_maxmem: 2097152 vfs.ufs.dirhash_mem: 795239 vfs.ufs.dirhash_docheck: 0 Try playing with with "vfs.ufs.dirhash_maxmem". I suppose it has to be at least as large as the size of the Directory From Hell. Also remember that any application that lists all of those files sequentially (as opposed to accessing a single one by name) will be very slow, no matter what. And don't forget that "ls" and shell expansion of "*" will not only read through all entries, but also sort them... It would probably be better to patch the dang server to use a multi-level directory structure instead. $.02, /Mikko To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message