Date: Sun, 9 Nov 2008 18:48:24 +1030 From: Ian <no-spam@people.net.au> To: Jeremy Chadwick <koitsu@freebsd.org> Cc: freebsd-questions@freebsd.org Subject: Re: UFS2 limits Message-ID: <200811091848.32453.no-spam@people.net.au> In-Reply-To: <20081109024046.GB27423@icarus.home.lan> References: <50261.1226194851@people.net.au> <20081109024046.GB27423@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
--nextPart3823548.elkEv6MmAj Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sun, 9 Nov 2008 13:10:46 Jeremy Chadwick wrote: > On Sun, Nov 09, 2008 at 01:40:51AM +0000, no-spam@people.net.au wrote: > > Hi, > > I have a FreeBSD server that has about 10,500 subdirectories within a > > single directory. > > This number will keep rising and I assume UFS2 has a limit to the number > > of sub-directories in a single directory - can anyone tell me what it i= s? > > As far as I know, there is no such limit on the number of files/dirs > inside of a directory. > > I don't want to change the topic of discussion, but I *highly* recommend > you ***stop*** whatever it is you're doing that is creating such a > directory structure. Software which has to iterate through that > directory using opendir() and readdir() will get slower and slower as > time goes on. > > If this is something you've written or have control over or can work > with engineers in regards to, I recommend you change your directory > naming scheme to have separate subdirectories with the first 2 or 3 > letters of the directory you wish to create. E.g.: > > /some/place/00/00ilikezeros/* > /some/place/01/01binaryheaven/* > /some/place/aa/aardvarks/* > /some/place/ab/abuse/* > /some/place/ac/actuary/* > ... > /some/place/xy/xylophones/* > > You get the point. > > Traversing this structure is much more efficient, and requires very > little code change on your part. Those who run nameservers that host > many zones, for example, use this structure to ensure the daemon doesn't > take 32498231 years to start up. > > > What about ZFS? > > > > At some point I'll have to re-arrange things so that I have a deeper > > directory structure, just wondering when I'll hit the limit so I can pl= an > > in advance :-) > > What baffles me is why you're looking at this problem from a " "how can > the filesystem solve this engineering mistake I made for me" standpoint, > rather than "how can I solve this engineering mistake I made so that it > doesn't impact the filesystem". Very strange. Sometimes looking at > things in a different light makes all the difference. > > Hope this helps. > > P.S. -- I hope this mail makes it to you, because your From line is > no-spam@people.net.au (I'll be surprised if your account name really is > that!). Thanks for that Jeremy. I didn't invent this structure, but I daresay I can= =20 either modify it or get the original writer to do that. I never really gave= =20 it a thought before now - it was the system I was given to work with and it= 's=20 worked fine so far, except when I try to list the contents of the directory= -=20 that takes ages! All the folders are 7 digit numbers and we are up to approx. 0010500 entrie= s=20 (ie subdirs) so far.=20 I guess it will just be a matter of experimenting to find the optimum numbe= r=20 of sub-sub-directories per sub-directory :-/ Oh, and yes, that email address does work - I use it for mailing lists and= =20 other stuff where I'm likely to get spammed - it's ironic really :-) Cheers, =2D-=20 Ian gpg key: http://home.swiftdsl.com.au/~imoore/no-spam.asc --nextPart3823548.elkEv6MmAj Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEABECAAYFAkkWnNgACgkQPUlnmbKkJ6A8NACdHUthkAgwXsoNxCPI83/teghn K7oAniCLqzB+DI80S2wO9f0/nfw9Qesb =YdI7 -----END PGP SIGNATURE----- --nextPart3823548.elkEv6MmAj--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200811091848.32453.no-spam>