Date: Fri, 07 Jun 2002 11:07:24 -0400 From: Bill Moran <wmoran@potentialtech.com> To: Darren Pilgrim <dmp@pantherdragon.org> Cc: freebsd-questions@freebsd.org Subject: Re: How many files in a directory is "a large number"? Message-ID: <3D00CC2C.6010203@potentialtech.com> References: <3D008B3C.A5944FF7@pantherdragon.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Darren Pilgrim wrote: > I'm building a file archive, and I know that when you have a large > number of files in directory, that lookups get really slow. I've only > ever heard people say "a large number." No real qualification. How > many files can one have before the slow downs become readily apparent? > Thousands? Tens of thousands? Millions? I need to know in order to > set my archive with as few segmentations in the collections as possible > without running into excessively slow lookups. I seem to remember someone quoting the number 20,000. Hopefully I'm not leading you astray with that number, because I can't remember where I heard/found it. On the flip side, I have some personal experience with directories with lots of files ... I have a client who has ~100,000 files on the shared portion of their fileserver. While I've argued and pushed to keep things organized in directories, there are still a few directories here and there that have 6000-7000 files in them. These directories are unmanagable for the most part. Directly accessing a file in them is fast enough, but if you want a directory listing over the network (SMB) it takes forever ( 15+ seconds just to bring up the list in explorer and how long does it take to scroll through the files to find the one you want?) If you're going to have some method to directly access the file, then the 20,000 is probably ok, but if folks are going to be browsing these directories, I would consider 2000-3000 a more practical maximum. Hope this helps, and I hope I'm remembering correctly with the 20,000 figure. -- Bill Moran Potential Technology http://www.potentialtech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3D00CC2C.6010203>