From owner-freebsd-stable@FreeBSD.ORG Fri Aug 13 21:44:35 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0ABB41065694 for ; Fri, 13 Aug 2010 21:44:35 +0000 (UTC) (envelope-from rsmith@xs4all.nl) Received: from smtp-vbr13.xs4all.nl (smtp-vbr13.xs4all.nl [194.109.24.33]) by mx1.freebsd.org (Postfix) with ESMTP id 6DF2C8FC13 for ; Fri, 13 Aug 2010 21:44:34 +0000 (UTC) Received: from slackbox.erewhon.net (slackbox.xs4all.nl [213.84.242.160]) by smtp-vbr13.xs4all.nl (8.13.8/8.13.8) with ESMTP id o7DLW5Gi019080; Fri, 13 Aug 2010 23:32:06 +0200 (CEST) (envelope-from rsmith@xs4all.nl) Received: by slackbox.erewhon.net (Postfix, from userid 1001) id 49E9EBABC; Fri, 13 Aug 2010 23:32:05 +0200 (CEST) Date: Fri, 13 Aug 2010 23:32:05 +0200 From: Roland Smith To: Kevin Oberman Message-ID: <20100813213205.GB29150@slackbox.erewhon.net> References: <20100813160109.8BDDA1CC3A@ptavv.es.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="61jdw2sOBCFtR2d/" Content-Disposition: inline In-Reply-To: <20100813160109.8BDDA1CC3A@ptavv.es.net> X-GPG-Fingerprint: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 X-GPG-Key: http://www.xs4all.nl/~rsmith/pubkey.txt X-GPG-Notice: If this message is not signed, don't assume I sent it! User-Agent: Mutt/1.5.20 (2009-06-14) X-Virus-Scanned: by XS4ALL Virus Scanner Cc: stable@freebsd.org Subject: Re: Inconsistent IO performance X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2010 21:44:35 -0000 --61jdw2sOBCFtR2d/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 13, 2010 at 09:01:09AM -0700, Kevin Oberman wrote: > For some time I have seen very odd issues with IO performance on > 8-Stable. Going back to November of last year when 8.0 was released, I > see variations of up to 22% in identical operations. This is not a > degradation as the performance moves up and down. >=20 > This is a very simplistic case. I have two identical disks (Fujitsu 80G) > on a ThinkPad T43 with a 2 GHz CPU and 2G RAM. I run the command: > dd bs=3D516096 if=3D/dev/ad0 of=3D/dev/ad2 Why are you using this peculiar block size? > Note the dramatic differences even on the same kernel. For the December > 6 kernel, for example, I see a maximum of 23,676,086 and a minimum of > just 18,304,565. ???? Both figures seem quite low to me? I cannot exactly reproduce your test, because I don't have an empty second disk handy, but doing dd if=3D/dev/zero bs=3D1m count=3D100 of=3D/tmp/foo yields the following writing speed on 8.1-RELEASE amd64,=20 WDC WD5001ABYS SATA harddisk @ 7200 rpm.: 1) 87263174 bytes/sec 2) 87878728 bytes/sec 3) 86397125 bytes/sec 4) 86550094 bytes/sec 5) 86524741 bytes/sec Th maximum variation in write speed is (87878728-86397125)/86397125*100% =3D 1.7%, which doesn't seem that much to me. This is in multi-user, with X11 running but on an otherwise idling machine, and with filesystem overhead to boot. Still the numbers are a lot higher th= an yours, which puzzles me.=20 Trying only reading does yield very inconsistent results because of caching= , I think; dd if=3D/tmp/foo bs=3D1m count=3D100 of=3D/dev/null 1) 1454216957 bytes/sec 2) 1003691691 bytes/sec 3) 1429956761 bytes/sec 4) 2324794646 bytes/sec 5) 1804563681 bytes/sec This is a (2324794646-1003691691)/1003691691*100% =3D 132% difference. OTOH, your data set should be big enough to negate caching effects, I guess. :-) What this does show is that writing seems to be the bottleneck. If I both read from and write to a file (on the same disk & partition); dd if=3D/tmp/foo bs=3D1m count=3D100 of=3D/tmp/bar gives 1) 85161534 bytes/sec 2) 84978770 bytes/sec 3) 87966613 bytes/sec 4) 83036312 bytes/sec 5) 86536879 bytes/sec This is a (87966613-83036312)/83036312*100% =3D 5.9% difference between lar= gest and smallest. The speed seems to be dictated by the writing. > Can anyone explain what might be causing such a dramatic difference? Maybe there is a hardware component here? Are both disks on the same controller? Or if not are both controllers using the same interrupt line? You should have a look at 'systat -vmstat' with dd running in the background. That might give a clue as to where the bottleneck is. Roland --=20 R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) --61jdw2sOBCFtR2d/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkxludUACgkQEnfvsMMhpyUS6wCgl1EvoHBOiJuooNp08uwo8+9P /T8AoIVwv/qwQTvAkVOU3nPvXN/h2y06 =nDr3 -----END PGP SIGNATURE----- --61jdw2sOBCFtR2d/--