Date: Tue, 03 May 2005 02:05:52 -0500 From: Jonathan Noack <noackjr@alumni.rice.edu> To: Petri Helenius <pete@he.iki.fi> Cc: Eric Anderson <anderson@centtech.com> Subject: Re: Very low disk performance on 5.x Message-ID: <427722D0.8050200@alumni.rice.edu> In-Reply-To: <42770026.80901@he.iki.fi> References: <19879.1115061648@critter.freebsd.dk> <20050502214208.M87351@fledge.watson.org> <42770026.80901@he.iki.fi>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigAAE6D0C55B37F0B412973BCF
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
On 05/02/05 23:37, Petri Helenius wrote:
> Robert Watson wrote:
>> The next thing that would be quite nice to measure is the rate of I/O
>> transactions per second we can get to the disk using the disk device
>> directly, with a minimal transaction size. I have a vague
>> recollection that you have to be careful in Linux because their
>> character device nodes for disk devices are buffered, and you really
>> want unbuffered I/O.
>
> I noticed that changing vfs.read_max from the default 8 to 16 has a
> dramatic effect on sequential read performance. Increasing it further
> did not have measurable effect.
Wow. Setting vfs.read_max=16 resulted in a 50+% performance boost when
sequentially reading from my 4-disk RAID5 (ida(4)) via the filesystem.
Write performance was unchanged. Raw device read performance was also
unchanged (~40 MB/s). With vfs.read_max>8, reading through the
filesystem is *faster* than reading from the raw device!
Note that these tests were run in multi-user on an idle 5.4-RC4 machine
with 1 GB of RAM and quad P3 Xeons. The machine is currently using ULE.
These tests were repeated 3 times with results in the +-0.1MB/s range.
$ sysctl vfs.read_max
vfs.read_max: 8
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 185.571466 secs (35315774 bytes/sec)
3m5.58s real 0.16s user 1m0.89s sys
$ sysctl vfs.read_max=16
vfs.read_max: 8 -> 16
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 118.846665 secs (55143323 bytes/sec)
1m58.85s real 0.21s user 1m3.25s sys
$ sysctl vfs.read_max=24
vfs.read_max: 16 -> 24
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 107.360600 secs (61042878 bytes/sec)
1m47.36s real 0.14s user 1m3.27s sys
$ sysctl vfs.read_max=32
vfs.read_max: 24 -> 32
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 102.288822 secs (64069562 bytes/sec)
1m42.29s real 0.20s user 1m4.52s sys
--
Jonathan Noack | noackjr@alumni.rice.edu | OpenPGP: 0x991D8195
--------------enigAAE6D0C55B37F0B412973BCF
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (FreeBSD)
iD8DBQFCdyLVUFz01pkdgZURAlDKAJwP9Q85zVvLODRJrTC25BYo+tdCeQCfYQmu
OCQNhEc3+DOVFk90zEmoliw=
=WR+k
-----END PGP SIGNATURE-----
--------------enigAAE6D0C55B37F0B412973BCF--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?427722D0.8050200>