From owner-freebsd-ports@FreeBSD.ORG Mon Aug 4 06:20:09 2008 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01CC41065678 for ; Mon, 4 Aug 2008 06:20:09 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx23.fluidhosting.com [204.14.89.6]) by mx1.freebsd.org (Postfix) with ESMTP id A572B8FC15 for ; Mon, 4 Aug 2008 06:20:08 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: (qmail 22317 invoked by uid 399); 4 Aug 2008 06:20:07 -0000 Received: from localhost (HELO lap.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 4 Aug 2008 06:20:07 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <48969F94.2060100@FreeBSD.org> Date: Sun, 03 Aug 2008 23:20:04 -0700 From: Doug Barton Organization: http://www.FreeBSD.org/ User-Agent: Thunderbird 2.0.0.16 (X11/20080726) MIME-Version: 1.0 To: Vladimir Chukharev References: <08GlBzRclM@dmeyer.dinoex.sub.org> <488A0B2E.9070403@FreeBSD.org> <488F79F8.1030105@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 0.95.6 OpenPGP: id=D5B2F0FB Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: Dirk Meyer , freebsd-ports@freebsd.org Subject: Re: Mk/bsd.openssl.mk optimization X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2008 06:20:09 -0000 Vladimir Chukharev wrote: > On Tue, 29 Jul 2008 23:13:44 +0300, Doug Barton > wrote: > >> V.Chukharev wrote: >>> Another patch, just one line. It can be applied independently >>> from the patch for bsd.port.subdir.mk. >> I use: grep -l "@comment ORIGIN:${1}$" $pdb/*/+CONTENTS for >> similar purposes in portmaster with no complaints so far. Is >> there a reason you need to do the complicated thing? > > I am a bit worried by the size of * expansion. I'm starting to wonder if you've actually tested and/or benchmarked this stuff. I've run tests of the construction above for 5,000 directories which is way more ports than a user would ever have installed. Rerunning this contrived example: 101$ time grep -l doug /home/dougb/testglob/*/file /home/dougb/testglob/4785/file real 0m0.718s user 0m0.026s sys 0m0.690s time find /home/dougb/testglob/* -type f -name file -exec grep -l doug {} \; /home/dougb/testglob/4785/file real 0m26.344s user 0m1.706s sys 0m22.771s Piping to xargs instead of using -exec is actually quite a bit faster, roughly 3.5 seconds wall clock time using the same setup (post caching). Those are the "best case" scenarios with everything cached. Very first run of the grep test (nothing in the file cache): time grep -l doug /home/dougb/testglob/*/file /home/dougb/testglob/4785/file real 0m6.454s user 0m0.114s sys 0m2.992s > I have about 1380 ports installed, and this number can grow. One > example of this kind limitation: $ ls /usr/ports/*/* | wc bash: > /bin/ls: Argument list too long Sorry, that's a ridiculous example. We have over 18,000 ports, and you're talking about two layers of globals, not one. > That's the reason for find. And it is 5-30 times faster then grep > -r I didn't say anything about grep -r, look carefully at what I wrote. > (depending on existence and size of /var/pkg/db/pkgdb.db). ... which is why the command I pasted above skips it altogether. > As to the rest of the command - I do not want to mess with it yet. > It works. I think you need to read des' rules on optimization, especially the bit about not doing optimization unless you're doing extensive benchmarking. Doug -- This .signature sanitized for your protection