Date: Sun, 25 Jun 2006 16:41:14 +1000 From: Peter Jeremy <peterjeremy@optushome.com.au> To: Nikolas Britton <nikolas.britton@gmail.com> Cc: freebsd-stable@freebsd.org Subject: Re: Effects of changing tar's -b option. Message-ID: <20060625064114.GF747@turion.vk2pj.dyndns.org> In-Reply-To: <ef10de9a0606242012j63039323w7b1bb63f1e325bb3@mail.gmail.com> References: <ef10de9a0606242012j63039323w7b1bb63f1e325bb3@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--jTMWTj4UTAEmbWeb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, 2006-Jun-24 20:12:56 -0700, Nikolas Britton wrote: >Test Setup: >250 50MB files (13068252KB) >dd if=3D/dev/random of=3Dtestfile bs=3D1m count=3D50 >Ethernet mtu=3D6500 >Transferred files were wiped after every test with 'rm -r *'. > >Test: >hostB: nc -4l port | tar xpbf n - >hostA: date; tar cbf n - . | nc hostB port; date > >Test Results: >seconds =3D n >645sec. =3D 1024 >670sec. =3D 512 >546sec. =3D 256 >503sec. =3D 128 >500sec. =3D 128 (control) >515sec. =3D 96 >508sec. =3D 64 >501sec. =3D 20 (default) > >Conclusions: Make your own. I don't think that's so unexpected. tar doesn't use multiple buffers so filling and emptying the buffer is done serially. Once the buffer exceeds the space in the pipe buffer and the local TCP send buffer, then the write from the hostA tar is delayed until the TCP buffer can drain. At the same time, the read from the hostB tar is blocked waiting for data from the network. Optimal throughput will depend on maximally overlapping the file reads on hostB with the network traffic and file writes on hostB. This, in turn, means you want to be able to hold at least a full buffer of data in the intervening processes and kernel buffers. Assuming that you aren't network bandwidth limited, you should look at increasing net.inet.tcp.sendspace and maybe net.inet.tcp.recvspace, or using an intervening program on hostA that does its own re-buffering. --=20 Peter Jeremy --jTMWTj4UTAEmbWeb Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEnjAI/opHv/APuIcRAsuBAJ4+byjnPI45lBohjlYEGFNEzPV3TQCfchD1 W1GahgtbkahJJiObj4OkSIg= =ABUZ -----END PGP SIGNATURE----- --jTMWTj4UTAEmbWeb--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060625064114.GF747>