From owner-freebsd-current@FreeBSD.ORG Wed Apr 21 06:43:05 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BECF916A4CE for ; Wed, 21 Apr 2004 06:43:05 -0700 (PDT) Received: from otter3.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B05143D1F for ; Wed, 21 Apr 2004 06:43:05 -0700 (PDT) (envelope-from anderson@centtech.com) Received: from centtech.com (neutrino.centtech.com [10.177.171.220]) by otter3.centtech.com (8.12.3/8.12.3) with ESMTP id i3LDh4E8082032 for ; Wed, 21 Apr 2004 08:43:04 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <40867A5D.9010600@centtech.com> Date: Wed, 21 Apr 2004 08:42:53 -0500 From: Eric Anderson User-Agent: Mozilla Thunderbird 0.5 (X11/20040406) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Directories with 2million files X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Apr 2004 13:43:05 -0000 First, let me say that I am impressed (but not shocked) - FreeBSD quietly handled my building of a directory with 2055476 files in it. I'm not sure if there is a limit to this number, but at least we know it works to 2million. I'm running 5.2.1-RELEASE. However, several tools seem to choke on that many files - mainly ls and du. Find works just fine. Here's what my directory looks like (from the parent): drwxr-xr-x 2 anderson anderson 50919936 Apr 21 08:25 data and when I cd into that directory, and do an ls: $ ls -al | wc -l ls: fts_read: Cannot allocate memory 0 Watching memory usage, it goes up to about 515Mb, and runs out of memory (can't swap it), and then dies. (I only have 768Mb in this machine). du does the exact same thing. find, however, works fine (and is very fast!): $ time find . | wc -l 2055476 real 0m3.589s user 0m2.501s sys 0m1.073s I'd work on some patches, but I'm not worth much when it comes to C/C++. If someone has some patches, or code to try, let me know - I'd be more than willing to test, possibly even give out an account on the machine. Eric -- ------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Today is the tomorrow you worried about yesterday. ------------------------------------------------------------------