From owner-freebsd-questions@FreeBSD.ORG Sun Nov 9 08:45:49 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBBE51065670 for ; Sun, 9 Nov 2008 08:45:48 +0000 (UTC) (envelope-from no-spam@people.net.au) Received: from smtp.ade.people.net.au (smtp.ade.people.net.au [218.214.228.98]) by mx1.freebsd.org (Postfix) with SMTP id 877298FC19 for ; Sun, 9 Nov 2008 08:45:47 +0000 (UTC) (envelope-from no-spam@people.net.au) Received: (qmail 22070 invoked from network); 9 Nov 2008 08:19:06 -0000 Received: from unknown (HELO bigblack) (218.214.176.70) by smtp.ade.people.net.au with SMTP; 9 Nov 2008 08:19:06 -0000 From: Ian To: Jeremy Chadwick Date: Sun, 9 Nov 2008 18:48:24 +1030 User-Agent: KMail/1.9.9 References: <50261.1226194851@people.net.au> <20081109024046.GB27423@icarus.home.lan> In-Reply-To: <20081109024046.GB27423@icarus.home.lan> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3823548.elkEv6MmAj"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200811091848.32453.no-spam@people.net.au> Cc: freebsd-questions@freebsd.org Subject: Re: UFS2 limits X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Nov 2008 08:45:50 -0000 --nextPart3823548.elkEv6MmAj Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sun, 9 Nov 2008 13:10:46 Jeremy Chadwick wrote: > On Sun, Nov 09, 2008 at 01:40:51AM +0000, no-spam@people.net.au wrote: > > Hi, > > I have a FreeBSD server that has about 10,500 subdirectories within a > > single directory. > > This number will keep rising and I assume UFS2 has a limit to the number > > of sub-directories in a single directory - can anyone tell me what it i= s? > > As far as I know, there is no such limit on the number of files/dirs > inside of a directory. > > I don't want to change the topic of discussion, but I *highly* recommend > you ***stop*** whatever it is you're doing that is creating such a > directory structure. Software which has to iterate through that > directory using opendir() and readdir() will get slower and slower as > time goes on. > > If this is something you've written or have control over or can work > with engineers in regards to, I recommend you change your directory > naming scheme to have separate subdirectories with the first 2 or 3 > letters of the directory you wish to create. E.g.: > > /some/place/00/00ilikezeros/* > /some/place/01/01binaryheaven/* > /some/place/aa/aardvarks/* > /some/place/ab/abuse/* > /some/place/ac/actuary/* > ... > /some/place/xy/xylophones/* > > You get the point. > > Traversing this structure is much more efficient, and requires very > little code change on your part. Those who run nameservers that host > many zones, for example, use this structure to ensure the daemon doesn't > take 32498231 years to start up. > > > What about ZFS? > > > > At some point I'll have to re-arrange things so that I have a deeper > > directory structure, just wondering when I'll hit the limit so I can pl= an > > in advance :-) > > What baffles me is why you're looking at this problem from a " "how can > the filesystem solve this engineering mistake I made for me" standpoint, > rather than "how can I solve this engineering mistake I made so that it > doesn't impact the filesystem". Very strange. Sometimes looking at > things in a different light makes all the difference. > > Hope this helps. > > P.S. -- I hope this mail makes it to you, because your From line is > no-spam@people.net.au (I'll be surprised if your account name really is > that!). Thanks for that Jeremy. I didn't invent this structure, but I daresay I can= =20 either modify it or get the original writer to do that. I never really gave= =20 it a thought before now - it was the system I was given to work with and it= 's=20 worked fine so far, except when I try to list the contents of the directory= -=20 that takes ages! All the folders are 7 digit numbers and we are up to approx. 0010500 entrie= s=20 (ie subdirs) so far.=20 I guess it will just be a matter of experimenting to find the optimum numbe= r=20 of sub-sub-directories per sub-directory :-/ Oh, and yes, that email address does work - I use it for mailing lists and= =20 other stuff where I'm likely to get spammed - it's ironic really :-) Cheers, =2D-=20 Ian gpg key: http://home.swiftdsl.com.au/~imoore/no-spam.asc --nextPart3823548.elkEv6MmAj Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEABECAAYFAkkWnNgACgkQPUlnmbKkJ6A8NACdHUthkAgwXsoNxCPI83/teghn K7oAniCLqzB+DI80S2wO9f0/nfw9Qesb =YdI7 -----END PGP SIGNATURE----- --nextPart3823548.elkEv6MmAj--