From owner-freebsd-questions Fri Jan 3 6:25:26 2003 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DA9ED37B401 for ; Fri, 3 Jan 2003 06:25:24 -0800 (PST) Received: from sentry.24cl.com (174.113.sn.ct.dsl.thebiz.net [216.238.113.174]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0008F43EC5 for ; Fri, 3 Jan 2003 06:25:23 -0800 (PST) (envelope-from myraq@mgm51.com) Received: from ntmm (unknown [63.119.50.193]) by sentry.24cl.com (Postfix) with ESMTP id EB77B29496 for ; Fri, 3 Jan 2003 09:25:22 -0500 (EST) Message-ID: <200301030925230109.03BFB685@home.24cl.com> In-Reply-To: <20030103044538.GB3132@gothmog.gr> References: <20030102084356.R18514-100000@atlas.home> <200301021213290839.0A719772@home.24cl.com> <200301020901270548.09C1C68B@sentry.24cl.com> <20030103044538.GB3132@gothmog.gr> X-Mailer: Calypso Version 3.30.00.00 (1) Date: Fri, 03 Jan 2003 09:25:23 -0500 Reply-To: myraq@mgm51.com From: "MikeM" To: freebsd-questions@FreeBSD.ORG Subject: Re: Lots of files in a directory Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 1/3/03 at 6:45 AM Giorgos Keramidas wrote: |On 2003-01-02 09:01, MikeM wrote: |> I'm considering setting up my server as a mirror site for the freedb.org |> lookup database. Unfortunately, I've seem to have run into a stumbling |> block. The server app requires over 250,000 files in a single directory. |> Each file is about 2k in size. | |On 2003-01-02 12:13, MikeM wrote: |> On 1/2/03 at 8:58 AM Mikko Ty?l?j?rvi wrote: |> > It can be argued that the application is poorly designed... |> |> There's no arguing. It *is* poorly designed. I would have used a |> hash to spread the files across multiple directories. |> Unfortunately, I didn't design it. | |You are probably hitting the limit of fragments that are allowed on a |filesystem and "time optimization" becomes "space optimization". Are |you getting entries in your system logs like the following? | | optimization changed from TIME to SPACE ============= I just grep'd /var/log/* and the word "optimization" did not appear. To you and others who have helped me, here's what I'm doing. - tomorrow I am upgrading from 4.5 to 4.7 - once that proves stable, I'll increase the DIRHASH value appropriately. - once that proves stable, I'll "tunefs -n enable /usr" to turn on soft dependencies I'll report back with my experiences with the new configuration so that the archive remains complete. Thanks to everyone who helped. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message