Date: Thu, 02 Jan 2003 09:01:27 -0500 From: "MikeM" <MyRaQ@mgm51.com> To: freebsd-questions@FreeBSD.ORG Subject: Lots of files in a directory Message-ID: <200301020901270548.09C1C68B@sentry.24cl.com>
next in thread | raw e-mail | index | archive | help
I'm considering setting up my server as a mirror site for the freedb.org
lookup database. Unfortunately, I've seem to have run into a stumbling
block. The server app requires over 250,000 files in a single directory.
Each file is about 2k in size.
It was a surprise to me (but probably not to those on this list) that the
file system does not handle that many files in an expeditious manner (I'm
being kind here).
Is there anything I can do so that the file system works faster with such a
large number of files? I'm looking for an increase in the area of 5 to 1.
For example, the command "rm -rf misc" where "misc" is the directory
containing the 250,000 files takes a couple of hours to run. If "misc" is
my current working directory, and I type "rm *" I get the message that
there are too many arguments being passed into rm.
Some details:
FreeBSD 4.5, dmesg below.
Filesystem containing the files: /dev/da0s1f on /usr (ufs, local, with
quotas)
Copyright (c) 1992-2002 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 4.5-RELEASE-p20 #12: Sat Sep 28 11:03:59 EDT 2002
[snip]:/usr/obj/usr/src/sys/STABLE4FW
Timecounter "i8254" frequency 1193182 Hz
CPU: Pentium III/Pentium III Xeon/Celeron (797.48-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x686 Stepping = 6
Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MC
A,CMOV,PAT,PSE36,MMX,FXSR,SSE>
real memory = 805224448 (786352K bytes)
avail memory = 778473472 (760228K bytes)
Preloaded elf kernel "kernel" at 0xc0468000.
Pentium Pro MTRR support enabled
md0: Malloc disk
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <ServerWorks NB6635 3.0LE host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
pci0: <S3 Savage 4 graphics accelerator> at 1.0
fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0x2200-0x223f mem
0xfea00000-0xfeafffff,0xfeb7f000-0xfeb7ffff irq 10 at device 2.0 on pci0
fxp0: Ethernet address 00:06:29:1f:27:61
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
isab0: <ServerWorks IB6566 PCI to ISA bridge> at device 15.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <ServerWorks ROSB4 ATA33 controller> port 0x700-0x70f at device
15.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ohci0: <OHCI (generic) USB controller> mem 0xfeb7e000-0xfeb7efff irq 7 at
device 15.2 on pci0
usb0: OHCI version 1.0, legacy support
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: (unknown) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pcib1: <ServerWorks NB6635 3.0LE host to PCI bridge> on motherboard
pci1: <PCI bus> on pcib1
ahc0: <Adaptec aic7892 Ultra160 SCSI adapter> port 0x2300-0x23ff mem
0xeffff000-0xefffffff irq 9 at device 3.0 on pci1
aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/255 SCBs
orm0: <Option ROM> at iomem 0xc0000-0xc9fff on isa0
fdc0: direction bit not set
fdc0: cmd 3 failed at out byte 1 of 3
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A, console
sio1: configured irq 3 not in bitmap of probed irqs 0
ppc0: parallel port not found.
IP packet filtering initialized, divert disabled, rule-based forwarding
disabled, default to deny, logging limited to 10 packets/entry by default
acd0: CDROM <CRN-8241B> at ata0-master using PIO4
Waiting 15 seconds for SCSI devices to settle
pass2 at ahc0 bus 0 target 8 lun 0
pass2: <IBM FTlV1 S2 0> Fixed Processor SCSI-2 device
pass2: 3.300MB/s transfers
da0 at ahc0 bus 0 target 0 lun 0
da0: <IBM-PSG DDYS-T18350M M S9AA> Fixed Direct Access SCSI-3 device
da0: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged Queueing
Enabled
da0: 17357MB (35548320 512 byte sectors: 255H 63S/T 2212C)
da1 at ahc0 bus 0 target 1 lun 0
da1: <IBM-ESXS ST336605LC !# B243> Fixed Direct Access SCSI-3 device
da1: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged Queueing
Enabled
da1: 34715MB (71096640 512 byte sectors: 255H 63S/T 4425C)
Mounting root from ufs:/dev/da0s1a
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200301020901270548.09C1C68B>