From owner-freebsd-ports@FreeBSD.ORG Sat Jan 28 16:28:45 2012 Return-Path: Delivered-To: ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DDAB106564A for ; Sat, 28 Jan 2012 16:28:45 +0000 (UTC) (envelope-from conrads@cox.net) Received: from eastrmfepo203.cox.net (eastrmfepo203.cox.net [68.230.241.218]) by mx1.freebsd.org (Postfix) with ESMTP id 98D628FC12 for ; Sat, 28 Jan 2012 16:28:44 +0000 (UTC) Received: from eastrmimpo305.cox.net ([68.230.241.237]) by eastrmfepo203.cox.net (InterMail vM.8.01.04.00 201-2260-137-20101110) with ESMTP id <20120128162838.MYWW25070.eastrmfepo203.cox.net@eastrmimpo305.cox.net>; Sat, 28 Jan 2012 11:28:38 -0500 Received: from serene.no-ip.org ([98.164.86.55]) by eastrmimpo305.cox.net with bizsmtp id T4Ud1i00W1BeFqy024Ue5S; Sat, 28 Jan 2012 11:28:38 -0500 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A02020A.4F242236.0094,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=1.1 cv=7YxdblQhh6tVU6ZVMrnfXyGefIif2LYgDHDYNRNiwDM= c=1 sm=1 a=ojH9cbcSZWsA:10 a=G8Uczd0VNMoA:10 a=kj9zAlcOel0A:10 a=fdHYxQQoAueMHNSmXppgDg==:17 a=mK_AVkanAAAA:8 a=kviXuzpPAAAA:8 a=z0iWqdLPz6cCUAxPLH4A:9 a=CjuIK1q_8ugA:10 a=9xyTavCNlvEA:10 a=4vB-4DCPJfMA:10 a=fdHYxQQoAueMHNSmXppgDg==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Received: from cox.net (localhost [127.0.0.1]) by serene.no-ip.org (8.14.5/8.14.5) with ESMTP id q0SGSaB3001596; Sat, 28 Jan 2012 10:28:37 -0600 (CST) (envelope-from conrads@cox.net) Date: Sat, 28 Jan 2012 10:28:31 -0600 From: "Conrad J. Sabatier" To: RW Message-ID: <20120128102831.51cf0f63@cox.net> In-Reply-To: <20120128143734.6166f5fc@gumby.homeunix.com> References: <20111203173149.224a64e6@cox.net> <20111214004838.GK1593@dormouse.experts-exchange.com> <20120112212905.GA78819@dormouse.experts-exchange.com> <20120127200325.66f36090@cox.net> <20120128143734.6166f5fc@gumby.homeunix.com> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: ports@freebsd.org Subject: Re: A new and better way to do "make readmes"? X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jan 2012 16:28:45 -0000 On Sat, 28 Jan 2012 14:37:34 +0000 RW wrote: > On Fri, 27 Jan 2012 20:03:25 -0600 > Conrad J. Sabatier wrote: > > > I've been thinking for a long time that we need a better way to do > > "make readmes", one that would be properly integrated into our > > ports Mk infrastructure, to take advantage of make's ability to > > recognize which files are up-to-date and which really do need > > rebuilding. > > This wont help and I think there's a better way that will make it up > to 700 times faster. > > When a make readmes is done at the top-level, the top-level and > category READMEs are created by make targets and the per port READMEs > are created by a perl script in one go from the INDEX- file. > > I once timed this and the 64 category READMEs took 2 hours, but the > ~20,000 port READMEs only took about 9 seconds. Am I understanding you correctly? Are you saying you built 20,000+ port READMEs in only 9 seconds?! How is that possible? Or do you mean 9 seconds for each one? > Selective updating isn't going to help because 99.9% of the time is > spent in the categories and it only takes a single port update to > make a category file obsolete. This is the part I find troubling. It would seem that it should be more work to create an individual port README, with its plucking the appropriate line out of the INDEX-* file and then parsing it into its respective pieces and filling in a template, than to simply string together a list of references to a bunch of already built port READMEs into a category README. What am I not getting here? > I think the way to speed this up is to have the script generate the > category files too. There's no point in bringing in the top-level > README since that's already fast. So what's making the category READMEs so slow then? > I've been toying with the idea of doing this, but have never got > around to it. If anyone wants to have a go I think it would be > sensible to write it in awk, since perl is no longer in the base > system and the existing perl script isn't really complex enough to be > worth hanging-on to. Oooo, awk! Been a while since I wrote any sizeable bit of code in it, but I do remember it was rather fun to work with. :-) I'm still not sure I read that paragraph above correctly, though (re: the times). :-) -- Conrad J. Sabatier conrads@cox.net