Date: Sat, 2 Jan 2016 01:28:01 -0500 From: Allan Jude <allanjude@freebsd.org> To: "Jonathan T. Looney" <jtl@freebsd.org>, Bruce Evans <brde@optusnet.com.au> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r292955 - head/lib/libmd Message-ID: <56876DF1.4030807@freebsd.org> In-Reply-To: <56857911.5010205@freebsd.org> References: <201512301804.tBUI4oGp065466@repo.freebsd.org> <20151231115651.R995@besplex.bde.org> <20151231143314.Y1520@besplex.bde.org> <5684D606.3080609@freebsd.org> <D2AADE36.4E8D4%jlooney@juniper.net> <56857911.5010205@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --o512q8QPWFHJtvrgCfHRr1eoILvnNqKbC Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-12-31 13:50, Allan Jude wrote: > On 2015-12-31 13:32, Jonathan T. Looney wrote: >> On 12/31/15, 2:15 AM, "Allan Jude" <allanjude@freebsd.org> wrote: >> >>> It seems these problems also slow things down, a lot: >>> >>> # time md5 /media/md5test/bigdata >>> MD5 (/media/md5test/bigdata) =3D 6afad0bf5d8318093e943229be05be67 >>> 4.310u 3.476s 0:07.79 99.8% 20+167k 0+0io 0pf+0w >>> # time env LD_PRELOAD=3D/usr/obj/media/svn/md5/head/tmp/lib/libmd.so >>> /usr/obj/media/svn/md5/head/sbin/md5/md5 /media/md5test/bigdata >>> MD5 (/media/md5test/bigdata) =3D 6afad0bf5d8318093e943229be05be67 >>> 4.133u 0.354s 0:04.49 99.7% 20+167k 1+0io 0pf+0w >>> >>> (file is fully cached in ZFS ARC, dd reads it at 11GB/s) >>> >>> Will investigate more tomorrow. >> >> md5 will be slower than dd due to the extra processing it needs to do = to >> generate the hash. I suspect that explains the difference you're seein= g >> between those utilities. >> >> Jonathan >> >> >> >=20 > Sorry, you missed my point here. >=20 > I replaced MDXFile() with the implementation included in my earlier > email. Using the newer libmd with that code, cut the time to md5 the > SAME data down a lot. I need to do a more scientific test on a box that= > isn't doing other stuff still though. >=20 > The comment about dd doing 11GB/s, was just to clarify that I wasn't > reading the file from disk, which would introduce other variables. >=20 I found the cause of my bogus benchmark, the world on my test machine was just old enough to be missing jmg@'s bufsize patch. Now the difference is about 1 second on a 2GB file, so ignore my foolishness. --=20 Allan Jude --o512q8QPWFHJtvrgCfHRr1eoILvnNqKbC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJWh230AAoJEBmVNT4SmAt+DvoQAJ47k8w+2fxCLPHv6XwZxFzh o3Fy0W4OVh8DdZPh1i2Q38aTkQTZrN0YYAe+8kbQ2S2Rv8ALA+Ezy+VV9U0qtc37 ZIi958iISK9f0Z+JZ7OmU3Esw804f8yYCLoheyc73cK9tjgtc5Lqk7mQ6UeTo8fp oRt5GAj6DHa06LEGCqbIInxuqpKLBBybAD4rkRo8W5X3xLjnVg12QHSDRftcZM/2 sdodS53pP/lJVVm/Y4oML6dg8umAKIx772CN5c+cQAFXQkzOAwT3Xctxw7/Y7CjK vEBZ2AtpwSDFbkAJtuxPMp6c+DysuCea/QbpqmpxPOe4YfM/bYcquek++rslArLB +q/M90Judv4wtCUYg2hpM/AGSWbSEtluExUvldHZ1hR7kUD49qXy4zKmPyWDdsDA Q5SbgY/susvQf8MjwZye44Aj07CRuCgl1ssePv1LYmURD2RCP4kdwBn60I3ivv97 JfPSS0xEtfQCqGtewGDnXeaeDC/ocAcXM24Y+s3qdGyf2H90gGUUDGj8PuW79/ey QVy8dVhKesQZ0S793Gka9Zv8nIskuF+M5XYhN2QIZx+aQG7gWN2guaxnEqB/1Giy uU8YpOXhNQbz5LGCTCkF94Bz4l3csgInZbUXalklm/P8iJWODxryBDNrCHW4KDFp F0cOfSIfi6U2ejRdF79n =XYxK -----END PGP SIGNATURE----- --o512q8QPWFHJtvrgCfHRr1eoILvnNqKbC--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?56876DF1.4030807>