From owner-freebsd-current@FreeBSD.ORG Mon Apr 12 14:26:50 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BC16F16A4CE; Mon, 12 Apr 2004 14:26:50 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 40D2043D5F; Mon, 12 Apr 2004 14:26:50 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i3CLQHPq017571; Mon, 12 Apr 2004 17:26:17 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i3CLQGDE017568; Mon, 12 Apr 2004 17:26:17 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Mon, 12 Apr 2004 17:26:16 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Jaye Mathisen In-Reply-To: <20040411003729.GB60706@backmaster.cdsnet.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: current@freebsd.org Subject: Re: Monumentally slow directory performance. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2004 21:26:50 -0000 On Sat, 10 Apr 2004, Jaye Mathisen wrote: > I know. if I start off with an empty dir, it goes really really fast > until I get around 2000-2500 or so. And then it just gets slower and > slower. > > If it's samba, I have no idea what. I'm running a current samba, pretty > much the default config. On large files I see 12-13MB/second. > > If my switch supported jumbo frames I'm sure it'd be even faster. > > I'll do some of the tests from the command prompt, see if I see the same > thing. I use some fairly large directory sizes with FreeBSD 4.x pretty frequently (1 million plus files), and the largest single problem I have is with tools that want to sort directory output. For example, I find that ls with the default arguments will burn several seconds sorting, whereas reading and listing the directory without sorting will be a small fraction of that. I think it would be useful to try and reproduce your performance problems using hand-crafted C to time inserts, listing, and removals, to avoid the possibility that applications are performing O(NlogN) or worse operations on the list. How long are the filenames you're using? Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research > > > On Sat, Apr 10, 2004 at 11:38:35AM -0400, Brian F. Feldman wrote: > > Jaye Mathisen wrote: > > > > > > > > > Running a 5.2 supped as of 4/9. > > > > > > I have the UFS_DIRHASH stuff compiled into my kernel. Is there some other > > > frob to actually "turn" it on? > > > > > > I run softupdates, and moving files from my local box over gig ethernet to a directory > > > with 5000 files in it is moving about 1 file every 6-7 seconds. > > > > > > If I move the same files to an empty directory, they move so fast I can' even > > > read the names, which is more "normal". > > > > > > Even directories with 2500 or so files seem really, really, really, really slow. > > > at least for adding/removing. > > > > > > My FS is UFS2, on a 3ware RAID controller. I would love any ideas. smbd > > > is burning up 60-70% of the CPU, I've tried kernels with and without all > > > the witness/diagnostic and all that stuff. > > > > Burning up 60-70% of the CPU? Just writing a file every 6-7 seconds to a > > pretty small directory? > > > > {"/home/green"}$ ls Mail/current | wc -l > > 64109 > > {"/home/green"}$ date; touch Mail/current/rand$RANDOM;\ > > > date; touch Mail/current/rand$RANDOM; date > > Sat Apr 10 11:37:55 EDT 2004 > > Sat Apr 10 11:37:55 EDT 2004 > > Sat Apr 10 11:37:55 EDT 2004 > > > > You should be getting "many" a second. Samba must be doing something wrong/ > > stuck on something. > > > > -- > > Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ > > <> green@FreeBSD.org \ The Power to Serve! \ > > Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\ > > > > > > > > !DSPAM:407814fe446531253994025! > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >