From owner-freebsd-ports@FreeBSD.ORG Mon May 14 15:34:01 2007 Return-Path: X-Original-To: freebsd-ports@freebsd.org Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A78F316A402; Mon, 14 May 2007 15:34:01 +0000 (UTC) (envelope-from stephen@math.missouri.edu) Received: from math.missouri.edu (math.missouri.edu [128.206.184.200]) by mx1.freebsd.org (Postfix) with ESMTP id 5B5BC13C45E; Mon, 14 May 2007 15:34:01 +0000 (UTC) (envelope-from stephen@math.missouri.edu) Received: from [128.206.184.213] (cauchy.math.missouri.edu [128.206.184.213]) by math.missouri.edu (8.13.1/8.13.1) with ESMTP id l4EFY06S081011; Mon, 14 May 2007 10:34:00 -0500 (CDT) (envelope-from stephen@math.missouri.edu) Message-ID: <46488168.3030300@math.missouri.edu> Date: Mon, 14 May 2007 10:34:00 -0500 From: Stephen Montgomery-Smith User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.2) Gecko/20070505 SeaMonkey/1.1.1 MIME-Version: 1.0 To: "[LoN]Kamikaze" References: <464597C6.3030406@gmx.de> <20070512174011.GA22526@xor.obsecurity.org> <4645FF71.60100@gmx.de> <20070512175824.GA23103@xor.obsecurity.org> <20070512133054.B5588@math.missouri.edu> <4646193E.5040503@gmx.de> <464665D5.1090509@math.missouri.edu> <464676CC.2000203@math.missouri.edu> <46469BF5.9030603@math.missouri.edu> <1179058696.13237.3.camel@ikaros.oook.cz> <20070513101332.L82202@math.missouri.edu> <46479215.1080401@math.missouri.edu> <4647C4FB.9050607@math.missouri.edu> <4647F182.6000402@gmx.de> <46487491.6020806@math.missouri.edu> In-Reply-To: <46487491.6020806@math.missouri.edu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88/3243/Mon May 14 05:49:49 2007 on math.missouri.edu X-Virus-Status: Clean Cc: Pav Lucistnik , freebsd-ports@freebsd.org, Kris Kennaway Subject: Re: Time to abandon recursive pulling of dependencies? X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2007 15:34:01 -0000 Stephen Montgomery-Smith wrote: > [LoN]Kamikaze wrote: >> Stephen Montgomery-Smith wrote: >>> Basically I think we are stuck on making "make package-depends" go any >>> faster. >>> >>> However I do think that the modifications I made to pkg_create go a >>> very >>> significant way to solving the problem of registration taking so >>> very long. >>> >>> Stephen >>> >> >> You are right about invoking make being the problem. However, I was >> able to >> increase the performance of package-depends by 20 to 30 percent, by >> avoiding >> recursive make calls and moving the recursion into the stack of the >> shell >> invocation. >> >> With my changes there's less than 10% overhead over the sum of make >> calls. I.e. >> for 56 seconds of make calls (x11/xorg on my system) there are only >> 4 more >> seconds of processing, instead of 15 to 20 seconds. >> >> The original make package-depends does some things I don't >> understand. Once >> I've figured them out I will post my little patch here. >> >> > > > Someone pointed out that what I was proposing in +DEPENDENCIES is > already to be found in +CONTENTS. So here is a proof of concept patch > to /usr/ports/Mk/bsd.port.mk (proof of concept because no error > checking, and things like that). > > For me it makes registration about 3 times faster. And also, the only reason it goes slow is because it has to do (cd $$dir; make -V PKGNAME) for every dir in _LIB_RUN_DEPENDS. But if instead we kept a file in /var/db/pkg called something like +PACKAGE_NAMES, where as each port is created we add in a one line entry with this mapping of origin to package name, then registration would take less than a second. It also has the advantage that the created +REQUIRED_BY really would be an accurate reflection of the true state of affairs rather than what it should be (meaning that if you cvsup ports and then build on top of old dependencies, +REQUIRED_BY would still be accurate). Stephen