Date: Mon, 04 Oct 1999 13:32:45 +0100 From: Alan Judge <Alan.Judge@indigo.ie> To: Jeff Lynch <jeff@mercury.jorsm.com> Cc: freebsd-isp@freebsd.org Subject: Re: NetApp NFS & FreeBSD Message-ID: <19991004123247.873D915404@hub.freebsd.org> In-Reply-To: Message from Jeff Lynch dated Friday at 16:15.
next in thread | raw e-mail | index | archive | help
You'll find the two scripts below. Not much to them and I'd be very
wary of making any sort of serious deductions based on something so
simple, particularly since I made no attempt to take other load off
the filer or to think about caching effects, single process vs
multiple process effects, or lots of other things. There are other
benchmarks out there that might be useful, depending on the
application you are actually interested in running.
Just out of interest, I've also run the two scripts on local disk (a
CCD stripe across two Seagate Cheetahs, with soft-updates enabled).
Results are:
Create lots of small files:
perl ~judgea/f.pl 2.60s user 28.82s system 7% cpu 6:45.43 total
Create same file lots of times:
perl ~judgea/f2.pl 1.60s user 14.09s system 94% cpu 16.651 total
Interesting, that the lots of files test is slower than the filer even
with soft-updates, whereas a straight sequential I/O test with iozone
gets 35-38MB/s on the same file system (vs 10Mb/s for the filer,
mostly limited by ethernet). However, you can really see the effect
of local caching and so on in the single file test, since most of the
work is just in memory, as opposed to being written out over NFS.
I've also appended another script that I use sometimes, which more
accurately mimics our high-load cases. It creates and deletes lots of
random files of various sizes and can be tweaked to change the
distributions to whatever you like. For this test, the filer under
load is a little slower than local CCD stripe.
--
Alan
f.pl:
#!/usr/bin/perl
foreach $i (0..99) {
$f = sprintf("%02d", $i);
mkdir($f, 0700);
}
$string = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXX\n" x 1500;
$nloops = 50000;
$size = 20000;
foreach $i (0..$nloops)
{
$fn = sprintf("%02d/%05d", $i%100, $i);
open(F, ">$fn") || die "$!: open $fn";
syswrite(F, $string, $size);
close(F);
}
f2.pl:
#!/usr/bin/perl
$string = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXX\n" x 300;
$nloops = 50000;
$size = 20000;
foreach $i (0..$nloops)
{
open(F, ">myfile") || die "$!: open $fn";
syswrite(F, $string, $size);
close(F);
}
file-test.pl:
#!/usr/bin/perl
$nloops = 100000;
$debug = 0;
$maxf = 1000000;
$ndir = 69;
$string = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n" x 1500;
print "File test V2. Fewer subdirs. Higher delete rate.\n";
print "nloops = $nloops, maxf = $maxf, ndir = $ndir\n";
# Quick hack script to test creating and deleting files.
# Step 1. Create sub dirs.
foreach $i (0..$ndir)
{
$d = sprintf("%02d", $i);
mkdir($d, 0777);
}
# Step 2. Loop.
print "Start step 2\n" if ($debug);
$start = time();
while($nloops-- > 0) {
if ((rand(1.0) > 0.51) && $#flist > 0) {
if (rand(1.0) > 0.5) {
# Delete most recent file
$f = shift(@flist);
&Unlink($f);
print "Array now: ", join(":", @flist), "\n" if ($debug);
} else {
# Delete random file
$fn = int(rand($#flist));
$f = $flist[$fn];
splice(@flist, $fn, 1);
&Unlink($f);
print "Array now: ", join(":", @flist), "\n" if ($debug);
}
} else {
# Create
do {
$f = int(rand($maxf));
} until (! -f &fname($f));
&mkfile($f);
push(@flist, $f);
print "Array now: ", join(":", @flist), "\n" if ($debug);
}
}
# End.
($user,$system,$cuser,$csystem) = times;
print "Clock time = ", time()-$start, "\n";
print "utime $user stime $system cutime $cuser csys $csystem\n";
print "left over files: ", $#flist, "\n";
sub fname {
local($fn) = @_[0];
sprintf("%02d/%d", $fn%$ndir, $fn);
}
sub Unlink {
local($f) = @_[0];
local($fn) = &fname($f);
print "Unlinking $f -> $fn\n" if ($debug);
unlink($fn) || die "$!: unlink";
}
sub mkfile {
local($f) = @_[0];
local($fn) = &fname($f);
local($size);
if (rand(1.0) > 0.1) {
# Size is 0..8K
$size = int(rand(8192));
} else {
$size = int(rand(102400));
}
print "Creating $fn at size $size\n" if ($debug);
open(F, ">$fn") || die "$!: open $fn";
syswrite(F, $string, $size);
close(F);
}
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19991004123247.873D915404>