Date: Mon, 04 Aug 2008 17:45:27 +0300 From: "V.Chukharev" <chukharev@mail.ru> To: "Doug Barton" <dougb@freebsd.org> Cc: Dirk Meyer <dirk.meyer@dinoex.sub.org>, freebsd-ports@freebsd.org Subject: Re: Mk/bsd.openssl.mk optimization Message-ID: <op.ufc9w5ny0g54sc@localhost> In-Reply-To: <48969F94.2060100@FreeBSD.org> References: <op.ueqfl1xy0g54sc@localhost> <08GlBzRclM@dmeyer.dinoex.sub.org> <op.ueumske80g54sc@localhost> <op.ueuwd8rs0g54sc@localhost> <488A0B2E.9070403@FreeBSD.org> <op.ue17xeex0g54sc@localhost> <op.ue2j4tv60g54sc@localhost> <488F79F8.1030105@FreeBSD.org> <op.ue3brjk0pcmiy5@localhost> <48969F94.2060100@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 04 Aug 2008 09:20:04 +0300, Doug Barton <dougb@freebsd.org> wrote:
> Vladimir Chukharev wrote:
>> On Tue, 29 Jul 2008 23:13:44 +0300, Doug Barton <dougb@freebsd.org>
>> wrote:
>>
>>> V.Chukharev wrote:
>>>> Another patch, just one line. It can be applied independently
>>>> from the patch for bsd.port.subdir.mk.
>>> I use: grep -l "@comment ORIGIN:${1}$" $pdb/*/+CONTENTS for
>>> similar purposes in portmaster with no complaints so far. Is
>>> there a reason you need to do the complicated thing?
>>
>> I am a bit worried by the size of * expansion.
>
> I'm starting to wonder if you've actually tested and/or benchmarked
Not very deeply, you are right. Actually I hoped that when I
demonstrate the 10 time reduced time of index generation with
a bad patch, maintainer or someone else would jump in and
make a real fix for the problem. When that did not happen,
I tried to prepare a patch which does not break things. It's
not ready yet (repeat: I knew practically nothing about make
and bsd.*.mk when starting!).
> this stuff. I've run tests of the construction above for 5,000
> directories which is way more ports than a user would ever have
> installed. Rerunning this contrived example:
Ok, that means that the limit is between 5000 and 6721:
$ ls /var/db/pkg/*/+* | wc
bash: /bin/ls: Argument list too long
0 0 0
$ time find /var/db/pkg -type f | wc -l
6721
real 0m0.151s
user 0m0.013s
sys 0m0.091s
Is 5000 more then a user would EVER have? So, NEVER more than 4 times
what I have now? Hm... It's possible that you are right. But I
didn't want to bet on it and used find.
> 101$ time grep -l doug /home/dougb/testglob/*/file
> /home/dougb/testglob/4785/file
> real 0m0.718s
> user 0m0.026s
> sys 0m0.690s
>
> time find /home/dougb/testglob/* -type f -name file -exec grep -l doug
> {} \;
> /home/dougb/testglob/4785/file
> real 0m26.344s
> user 0m1.706s
> sys 0m22.771s
>
> Piping to xargs instead of using -exec is actually quite a bit faster,
> roughly 3.5 seconds wall clock time using the same setup (post caching).
Yes, I know this (and since I did not know about \+ instead of \;
in find -exec, I used xargs). Now I prefer \+.
>> I have about 1380 ports installed, and this number can grow. One
>> That's the reason for find. And it is 5-30 times faster then grep
>> -r
>
> I didn't say anything about grep -r, look carefully at what I wrote.
Oh, so 'the complicated thing' is not 'find | xargs' but the rest of
the command? Then you ask wrong person, I have nothing to do with it.
I do not know who wrote it and how it works.
I only wanted to get rid of grep -r in this small patch.
>> (depending on existence and size of /var/pkg/db/pkgdb.db).
>
> ... which is why the command I pasted above skips it altogether.
...the same way as any of the tricks with find.
I tried to paste your command into the place. It gives errors.
>> As to the rest of the command - I do not want to mess with it yet.
>> It works.
>
> I think you need to read des' rules on optimization, especially the
> bit about not doing optimization unless you're doing extensive
> benchmarking.
I believe I read it many years ago. If you give a link I will check it
out. (No, I do not want to google it.)
> Doug
>
Thanks!
Vladimir
--
V. Chukharev
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?op.ufc9w5ny0g54sc>