Date: Thu, 27 Sep 2001 01:55:48 -0300 From: "Mario Sergio Fujikawa Ferreira" <lioux@uol.com.br> To: Maxim Sobolev <sobomax@FreeBSD.org> Cc: Will Andrews <will@physics.purdue.edu>, FreeBSD-ports@FreeBSD.org, portmgr@FreeBSD.org, pat@databits.net, fenner@FreeBSD.org, kris@FreeBSD.org, green@FreeBSD.org, knu@FreeBSD.org, julian@FreeBSD.org, petef@FreeBSD.org, cwasser@v-wave.com, sjh-cl@horan.net.au, john_m_cooper@yahoo.com, matt@ipperformance.com, branson@windborne.net Subject: Re: review plz MASTER_SITES_n (ala OpenBSD) patch for bsd.port.mk Message-ID: <20010927015548.A21765@exxodus.fedaykin.here> In-Reply-To: <3BB05FD4.3E235748@FreeBSD.org>; from sobomax@FreeBSD.org on Tue, Sep 25, 2001 at 01:43:10PM %2B0300 References: <20010923055224.A93855@exxodus.fedaykin.here> <E15l7Nf-00072f-00@mx10.port.ru> <20010923133030.K69123@curie.physics.purdue.edu> <3BB05FD4.3E235748@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Sep 23, 2001 at 01:30:08PM -0500, Will Andrews wrote: > On Sun, Sep 23, 2001 at 03:25:05PM +0400, Maxim Sobolev (sobomax@mail.ru) wrote: [snip] > like to test a final version of this patch plus patches to the > various ports in my bento tree (mainly KDE and its dependneices) > and see how it goes. Any objections to this method of testing? I am ALL for it. :) What better way to test if it works than trying it against the whole ports tree. :PPP Though, I stand by the patch, I am worried since I am fully aware that no piece of software has not been fully tested untill it has been, will this trial time disrupt anything? If not, no problem. Just playing safe. :) Also, I apologize for the delay on replying but I wanted to both gather as much feedback as possible and be thurel on my analysis. > > Why not in addition to postfixes in DISTFILES introduce the same > > postfixes in MASTER_SITES and MASTER_SITE_SUBDIR? This would allow > > to avoid introducing MASTER_SITES_foo, turning your example below > > into: > > I like this: > > > MASTER_SITES= http://master7/:n \ > > http://master1/:t http://master2/:t http://master3/%SUBDIR%/:t \ > > http://master4/:y http://master5/:y http://master6/%SUBDIR%/:y \ > > http://master8/:p > > DISTFILES= file1:t file2:y file3:n file4:o:p > > MASTER_SITE_SUBDIR= subdir:t subdir_test:y I find the idea appealing, it looks small and clean..... BUT (there is always a but on long sentence:) I will compare the previous proposal with our later proposal exploring possible usage conditions: previous proposal (items prefixed with P) -> several MASTER_SITES_n later proposal (items prefixed with L) -> single MASTER_SITES, all sites postfixed 1) how do we deal with the sites inside bsd.sites.mk? I will use MASTER_SITE_SOURCEFORGE as an example. I have 3 files: 1a) fileA can be obtained only in MASTER_SITE_SOURCEFORGE 1b) fileB can be obtained in MASTER_SITE_SOURCEFORGE and in another_place 1c) fileC can only be obtained in another_site Since the idea of using bsd.sites.mk is that we can do MASTER_SITES= ${MASTER_SITE_SOURCEFORGE} * According to later proposal: L1.1) how do we deal with (1a) which seems to be both a common and simple case? L1.1a) we could leave this as the default for the files that no postfix :n; i.e., use the variable as it is now, no postfixes L1.1b) or, we could postfix all elements of MASTER_SITE_SOURCEFORGE inside bsd.sites.mk with :SOURCEFORGE; then, we just postfix the files with it L1.2) how do we deal with (1b) which is a common variation of (1a)? L1.2a) we need to postfix fileB if we did (L1.1a), but how we postfix if the original variable is not? No idea.... Can be accomplished by some per-call crazy regexp inside the calling Makefile but defeats the simplification purpose of this proposal. L1.2b) if we did (L1.1b), we will need the , (comma) operator suggested in the proposal which allows a file to belong to more than one group at the same time. Therefore, we could do DISTFILES= fileA:SOURCEFORGE fileB:SOURCEFORGE,OTHERSITE L1.3) (1c) seems a case pretty simple enough; as seen before, solution to previous problems has led to (L1.2b) which leads us to L1.3a) we can use the default case for fileC DISTFILES= fileA:SOURCEFORGE fileB:SOURCEFORGE,OTHERSITE \ fileC L1.3b) or, we can add an additional group DISTFILES= fileA:SOURCEFORGE fileB:SOURCEFORGE,OTHERSITE \ fileC:SOMESITE *** Result: we need to postfix the sites inside bsd.sites.mk *** Problem: we will need to fix ALL ports using these sites, since they do not belong anymore to default which is the group currently used by ALL ports * According to previous proposal: P1.1) how do we deal with (1a) which seems to be both a common and simple case? P1.1a) use the default group for this port (no postfix) MASTER_SITES= ${MASTER_SITE_SOURCEFORGE} DISTFILES= fileA MASTER_SITE_SUBDIR= something something_else P1.1b) or, create a separate group MASTER_SITES_GROUPA= ${MASTER_SITE_SOURCEFORGE} DISTFILES= fileA:GROUPA MASTER_SITE_SUBDIR_GROUPA= something something_else P1.1c) or, have the patch automagically use bsd.sites.mk DISTFILES= fileA:SOURCEFORGE MASTER_SITE_SUBDIR_SOURCEFORGE= something something_else P1.2) how do we deal with (1b) which is a common variation of (1a)? P1.2a) if we did (P1.1a), we have 2 options: P1.2a.1) create an additional group which adds the default group MASTER_SITES= ${MASTER_SITE_SOURCEFORGE} MASTER_SITES_GROUPB= ${MASTER_SITES} siteB DISTFILES= fileA fileB:GROUPB MASTER_SITE_SUBDIR= something something_else MASTER_SITE_SUBDIR_GROUPB= other yetanother p1.2a.2) use the comma operator with an empty argument and an additional group DISTFILES= fileA fileB:,GROUPB I don't think this is a good idea, it is not semanticaly clean not to mention visually confusing. We could create a keyword _DEFAULT which always refers to the default group DISTFILES= fileA fileB:_DEFAULT,GROUPB P1.2b) if we did (P1.1b), we have 3 options: P1.2b.1) add the additional group to the default group MASTER_SITES= ${MASTER_SITES_GROUPA} siteB DISTFILES= fileA:GROUPA fileB P1.2b.2) create an additional group which adds the separate group MASTER_SITES_GROUPB= ${MASTER_SITES_GROUPA} siteB DISTFILES= fileA:GROUPA fileB:GROUPB P1.2b.3) create an additional group but use the comma separator to include the separate group MASTER_SITES_GROUPB= siteB DISTFILES= fileA:GROUPA fileB:GROUPA,GROUPB P1.2c) if we did (P1.1c), we have 3 options: P1.2b.1) create an additional group and use the comma operator MASTER_SITES_GROUPB= siteB DISTFILES= fileA:SOURCEFORGE fileB:SOURCEFORGE,GROUPB P1.3) (1c) seems a case pretty simple enough P1.3a) we can either use the default case MASTER_SITES= siteC DISTIFILES= fileC P1.3b) or, create an additional group MASTER_SITES_GROUPC= siteC DISTIFILES= fileC:GROUPC *** Result: all permutations are possible (no intrinsic design "limitations") and we won't need to "fix" already working ports 2) how do we parse {MASTER,PATCH}_SITES{,_n}, {DIST,PATCH}FILES{,_n}? P2.1) previous implementation already solved this issue with a sample implementation *** Result: does not require "working" ports to be fixed L2.1) later implementation has this already solved by *sobomax* and *will* > But, there's a small problem here. I believe ':' is a valid URI > character, and I believe _some_ places have actually used this > character. But I am not sure. Would it be possible to use a > different character for the delimiter? As pointed out later in the thread, this shouldn't be a problem as long as we do a ports sweep. > > It would not a big problem, because in MASTER_SITES presence of > > postfix means absence of trailing `/' (hence it is easy to sort > > it out), while possibility to find `:' in the MASTER_SITE_SUBDIR > > is close to zero. > > Good point. How about we change the rules for MASTER_SITE_SUBDIR > so that a / is required on the end? *** Result: does not require ports to be fixed 3) Pollution of port Makefiles P3.1) Let's examine an example MASTER_SITES= http://master7/ MASTER_SITES_today= http://master1/ http://master2/ http://master3/%SUBDIR%/ MASTER_SITES_yesterday= http://master4/ http://master5/ http://master6/%SUBDIR%/ MASTER_SITES_p= http://master8/ DISTFILES= file1:today file2:yesterday file3:n file4:o:p MASTER_SITE_SUBDIR_today= subdir MASTER_SITE_SUBDIR_yesterday= subdir_test This one looks polluted. TOO much noise inside the Makefile. However, it is easy to spot the groups. Also, it scales without a problem with several dozen group sites. L3.1) Let's examine the same example of (P3.1) MASTER_SITES= http://master7/ http://master1/:today http://master2/:today \ http://master3/%SUBDIR%/:today http://master4/:yesterday \ http://master5/:yesterday http://master6/%SUBDIR%/:yesterday \ http://master8/:p DISTFILES= file1:today file2:yesterday file3:n file4:o:p MASTER_SITE_SUBDIR= subdir:today subdir_test:yesterday This one, obviously, looks cleaner in the eye. Less noise inside the Makefile at first glance. However, as the number of sites increase, too many postfixes need to be added. Scales with overhead (1 postfix per site added per group) 4) How to share sites amongst groups P4.1) add one group to the other MASTER_SITES_SHARED= siteA MASTER_SITES_ONE= ${MASTER_SITES_SHARED} P4.2) use the , (comma) operator MASTER_SITES_SHARED= siteA MASTER_SITES_ONE= ${MASTER_SITES_SHARED} DISTFILES= fileA:SHARED,ONE L4.1) replicate sites with different groups MASTER_SITES= siteA:GROUPA siteA:GROUPB siteB:GROUPB DISTFILES= fileA:GROUPB L4.2) use the , (comma) operator L4.2a) normal form MASTER_SITES= siteA:GROUPA siteB:GROUPB DISTFILES= fileA:GROUPA,GROUPB L4.2b) advanced (an idea for development) This proposes that we could use the comma operator in ${MASTER,PATCH}_SITES and {MASTER,PATCH}_SITE_SUBDIR MASTER_SITES= siteA:GROUPA,GROUPB siteB:GROUPB DISTFILES= fileA:GROUPB ----- This evaluation took quite some time. What is my conclusion? Well, the previous implementation is less intrusive in the ports system, requiring no modification to existing ports but has a pollution side which is undesirable but scales without a problem. (P3.1) The later proposal is intrusive requiring a ports sweep mostly due to the handling of bsd.sites.mk ((L1) conclusion). Also, it "does not scale" well (at least to my eyes :) as seen in (L3.1), but this shouldn't be a problem since most ports have only a few master sites. We can accept some level of pollution in a few ports. This analysis should get more ideas rolling and even more feedback coming. Please, if you think I missed something or you find a solution for something I said was a problem: LET US KNOW, contribute. :-) ----- Ideas which came from this analysis: i1) , (comma) operator on all variables (L4.2b) i2) (based on an original idea of *green*) if we pursue the previous proposal, we could: i2.1) ditch the PATCH_SITES_n; therefore, we would use {MASTER,PATCHES}_SITES which are the defaults for respectively {MASTER,PATCH}FILES. Whenever we want to use a group, we only use MASTER_SITES_n, so postfixed :n files inside {PATCH,MASTER}FILES all look for MASTER_SITES_n. This simplifies the previous patch a bit. i2.2) also, following (i2.1) cue, we could collapse {MASTER,PATCH}_SITE_SUBDIR_n using the ideas from later proposal (L2.1) {MASTER,PATCH}_SITE_SUBDIR= a/:a b/:b i3) a keyword _DEFAULT (or, whatever) to identify the default {MASTER,PATCHES}_SITES if we need to use them with the comma operator (P1.2a) i4) if you get any, please let me know. We are in need of good ideas. Keep them coming. ----- Regards, -- Mario S F Ferreira - UnB - Brazil - "I guess this is a signature." lioux at ( freebsd dot org | linf dot unb dot br ) flames to beloved devnull@someotherworldbeloworabove.org feature, n: a documented bug | bug, n: an undocumented feature To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010927015548.A21765>