Date: Fri, 21 Nov 2003 14:57:41 -0500 From: "Marco Greene (Home)" <mgreene@aci.on.ca> To: "'List, FreeBSD Questions'" <freebsd-questions@freebsd.org> Subject: Slow creating lots of files... Message-ID: <000901c3b069$bb19f100$f6c8a8c0@pepstep>
next in thread | raw e-mail | index | archive | help
Doing some further research into my NetBackup slow restore = performance...I have noticed that if I create a lot of small files....it can take about = 10 times as long as creating one big file of the same size. With a sample set of 500MB... Creating 500MB with of text files (about 30,000 of them) takes about 10 minutes Creating 1 500 MB file takes about 1 minute. Restoring 1 500MB file takes about 1 minute or so...a little more to = allow the robot to mount the tape/position the tape etc. Restoring the 30,000 files with NetBackup takes about 20 minutes Restoring the 30,000 files to an alternate location takes about 40 = minutes Now interms of backing this all up...well 2.5GB of it takes about 8 = minutes. My hardware platform is a Compaq DL360 with dual PIII/933MHz CPUs (only = one configured with the kernel to date). Two 36GB (10K RPM) drives configured with Hardware RAID1 1 GB RAM. (The OS is configured with a 2 GB swap space.)=20 Here is the FreeBSD Slice... # /dev/idad0s1c: type: ESDI disk: idad0s1 label: flags: bytes/sector: 512 sectors/track: 32 tracks/cylinder: 255 sectors/cylinder: 8160 cylinders: 8320 sectors/unit: 67891200 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # milliseconds track-to-track seek: 0 # milliseconds drivedata: 0 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 2097152 0 4.2BSD 2048 16384 89 # (Cyl. 0 - = 257*) b: 4194304 2097152 swap # (Cyl. 257*- = 771*) c: 67891200 0 unused 0 0 # (Cyl. 0 - = 8319) e: 8388608 6291456 4.2BSD 2048 16384 89 # (Cyl. 771*- 1799*) f: 49152000 14680064 4.2BSD 2048 16384 89 # (Cyl. 1799*- 7822*) g: 4059136 63832064 4.2BSD 2048 16384 90 # (Cyl. 7822*- 8319*) mislog01 # df -h Filesystem Size Used Avail Capacity Mounted on /dev/idad0s1a 1008M 50M 878M 5% / /dev/idad0s1g 1.9G 4.0K 1.8G 0% /home /dev/idad0s1e 3.9G 976M 2.7G 26% /usr /dev/idad0s1f 23G 2.5G 19G 12% /var procfs 4.0K 4.0K 0B 100% /proc Have any of you seen this before? Is there some tuning that I can do?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?000901c3b069$bb19f100$f6c8a8c0>