Date: Mon, 26 Dec 2011 16:09:15 -0800 From: Steve Kargl <sgk@troutmask.apl.washington.edu> To: Doug Barton <dougb@FreeBSD.org> Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, Marius Strobl <marius@alchemy.franken.de> Subject: Re: svn commit: r228857 - in head/usr.bin: . csup Message-ID: <20111227000915.GA80162@troutmask.apl.washington.edu> In-Reply-To: <4EF902EB.5050009@FreeBSD.org> References: <201112241216.pBOCGd1H012696@svn.freebsd.org> <4EF645D2.8080407@FreeBSD.org> <20111226102820.GT90831@alchemy.franken.de> <4EF8DC5B.9070404@FreeBSD.org> <20111226225136.GA79882@troutmask.apl.washington.edu> <4EF902EB.5050009@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Dec 26, 2011 at 03:27:39PM -0800, Doug Barton wrote:
> On 12/26/2011 14:51, Steve Kargl wrote:
> > On Mon, Dec 26, 2011 at 12:43:07PM -0800, Doug Barton wrote:
> >> On 12/26/2011 02:28, Marius Strobl wrote:
> >>> On Sat, Dec 24, 2011 at 01:36:18PM -0800, Doug Barton wrote:
> >>>> On 12/24/2011 04:16, Marius Strobl wrote:
> >>>>> On FreeBSD just use the MD5 implementation of libmd rather than that of
> >>>>> libcrypto so we don't need to relinquish csup when world is built without
> >>>>> OpenSSL.
> >>>>
> >>>> Did you benchmark this at all? I agree that keeping csup available
> >>>> absent openssl is a good goal, but csup is a prototypical "tool that
> >>>> does the same thing many thousands of times" so even tiny regressions
> >>>> could add up to a large cost in wall clock time.
> >>>
> >>> Well, in a real world test updating the same base on an amd64 machine
> >>> connected to the Internet
> >>
> >> Adding a network connection to the test is almost certainly going to
> >> obscure the results beyond utility.
> >
> > Given that the majority of FreeBSD users will be pulling code
> > from the internet, this seems to be the most relevant test.
>
> Sorry if I wasn't clear. The change was to how the md5 portion of csup
> is linked. In order to isolate the effects of that change you have to
> remove everything that isn't related to that change.
>
> But this is regression testing 101, so I'm sure that you know that already.
If 99% of the usage of csup goes over the internet and 95% of the
execution time (as determined by a profiler) involves dealing with
network, then worrying about libmd vs libcrypto is a waste of time.
If you're concerned about performance, then find the bottlenecks
for that most common usage pattern. Micro-optimizing a synthetic
usage case is a waste of time.
> >> The appropriate way to test this
> >> would be to create a binary out of the md5 routine in csup, and link it
> >> alternately with libcrypto and libmd. Then for each version run it
> >> against the src tree (or ports, either way) 10 times. Discard the first
> >> and last, and then plot the results with ministat.
> >
> > The proper way to test the libmd vs libcrypto versions of
> > the md5 routines is to use a profiler.
>
> That'll give you a good view of where the performance bottlenecks are if
> it turns out that libmd is actually slower, sure. But the interesting
> question in terms of this change is the effect on wall clock time, since
> that's what users are going to see.
>
> > Of course, one might ask the question on how the use of
> > libmd effects the majority of FreeBSD users (ie., not FreeBSD
> > developers). Does the majority run csup hourly? Daily?
> > Weekly?
>
> For those that use csup, I imagine that they use it at least daily. But
> that's not the point.
>
> > For a utility seldomly run be the majority of FreeBSD
> > users, Doug, you seem to be wasting Marius's time.
>
> How often it's used isn't really relevant to whether or not introducing
> a pessimization is worth it. In any case I didn't ask him to back it
> out, I only asked to have it be an option if it turns out that libmd is
> slower.
Yes, I know you did not ask him to back out his change. You asked
him if he measured the impact on performance with that implication
that he should run some performance test. Marius ran additional
tests (wasting his time) to answer your question. Your response
was essentially, "well, your really need to do the test this way
(ie., no internet)", with the obvious implication of "please go
do your test again."
> I understand that what you're really trying to do here is to take a shot
> at me relative to my assertion that profiled libs should be off by
> default. If you're going to respond in kind to every message I send it's
> going to get boring really quick.
Nope. I'm concerned that your wasting valuable developer time. If you
were really concerned with the performance, I suspect that you know
how to do the tests you have asked of Marius. Now, let's read his
commit message:
On FreeBSD just use the MD5 implementation of libmd rather than
that of libcrypto so we don't need to relinquish csup when world
is built without OpenSSL.
His change actually allows FreeBSD users to use csup if they are
in a situation where WITHOUT_CRYPTO and/or WITHOUT_OPENSSL is
required. Yes, I know you want him to waste his time to come
up with the perfect patch with Makefile magic to find libcrypto
and fall back to libmd. If you're really concerned about
performance I'm fairly certain that Marius would be willing
to review your Makefile magic patch.
--
Steve
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111227000915.GA80162>