From owner-freebsd-questions Thu Jan 2 9:39: 4 2003 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 13DAF37B42C for ; Thu, 2 Jan 2003 09:38:58 -0800 (PST) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 17EF843EB2 for ; Thu, 2 Jan 2003 09:38:58 -0800 (PST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.6/8.12.6) id h02HcqeU023015; Thu, 2 Jan 2003 11:38:52 -0600 (CST) (envelope-from dan) Date: Thu, 2 Jan 2003 11:38:51 -0600 From: Dan Nelson To: MikeM Cc: freebsd-questions@FreeBSD.ORG Subject: Re: Lots of files in a directory Message-ID: <20030102173851.GG24622@dan.emsphone.com> References: <200301020901270548.09C1C68B@sentry.24cl.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200301020901270548.09C1C68B@sentry.24cl.com> X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.3i Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Jan 02), MikeM said: > It was a surprise to me (but probably not to those on this list) that > the file system does not handle that many files in an expeditious > manner (I'm being kind here). > > Is there anything I can do so that the file system works faster with > such a large number of files? I'm looking for an increase in the > area of 5 to 1. For example, the command "rm -rf misc" where "misc" > is the directory containing the 250,000 files takes a couple of hours > to run. If "misc" is my current working directory, and I type "rm *" > I get the message that there are too many arguments being passed into > rm. In addition to the other suggestions, enabling softupdates will make file creation and deletion much faster (your rm -rf will speed up ~100x, for example). If you're doing file operations directly on known filenames, then the dirhash suggestions will really help. If freedb is doing directory scans or wildcard globbing, then no filesystem will save you :) -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message