Date: Sat, 25 Jul 2009 18:29:18 -0400 (EDT) From: vogelke+unix@pobox.com (Karl Vogel) To: freebsd-questions@freebsd.org Subject: Re: limit to number of files seen by ls? Message-ID: <20090725222918.AC51DB7E0@kev.msw.wpafb.af.mil> In-Reply-To: <200907231025.49916.mel.flynn%2Bfbsd.questions@mailing.thruhere.net> (message from Mel Flynn on Thu, 23 Jul 2009 10:25:49 -0800)
next in thread | previous in thread | raw e-mail | index | archive | help
>> On Thursday 23 July 2009 09:41:26 Karl Vogel wrote: K> Every version of Unix I've ever used had an upper limit on the size of K> the argument list you could pass to a program, so it won't just be "ls" K> that's affected here. That's why I use 1,000 as a rule of thumb for the K> maximum number of files I put in a directory. >> On Thu, 23 Jul 2009 10:25:49 -0800, >> Mel Flynn <mel.flynn+fbsd.questions@mailing.thruhere.net> said: M> That arbitrary number works simply because kern.argmax default has been M> raised somewhere in 6.x (before it was 64kB). That arbitrary number has worked very nicely for me for 20 years under Solaris, Linux, and several BSD variants. The main reason I stick with 1000 is because directories are read linearly unless you're using something like ReiserFS, and I get impatient waiting for more than that number of filenames to be sorted when using ls. M> And MAXNAMLEN in sys/dirent.h is 255. That's the maximum length of a single filename in a directory. Since I keep my filenames much shorter, I don't have a problem. M> Knowing your way around maximum arguments length through xargs as M> suggested in this thread is much better solution then trying to exercise M> control over directory sizes, which may or not be under your control in M> the first place. Xargs is very useful, but it's not a substitute for poor design, and it's not something you can drop into any existing pipeline without a little thought first. If your application is trying to create hundreds of thousands or millions of files in any one directory, or you're creating lots of 200-character filenames from hell, then your design is a poor match for most varieties of Unix; small directories perform better than enormous ones, and lots of commonly-used scripts and programs will fall over when handed zillion-file argument lists. I'm sure the latest version of <insert-cool-OS-or-filesystem-here> fixes all these objections, but not everyone gets to run the latest and greatest. Don't fight your filesystem, and it won't fight you. -- Karl Vogel I don't speak for the USAF or my company Birds of a feather flock together and usually crap on your car.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090725222918.AC51DB7E0>