Date: Sat, 5 May 2001 10:48:15 -0300 From: "Mario Sergio Fujikawa Ferreira" <lioux@uol.com.br> To: freebsd-ports@FreeBSD.ORG Subject: Re: RFC: New port target fetch-check (bsd.port.mk patch) Message-ID: <20010505104814.A21474@Fedaykin.here> In-Reply-To: <20010428040507.A8145@Fedaykin.here>; from lioux@uol.com.br on Sat, Apr 28, 2001 at 04:04:45AM -0300 References: <20010428040507.A8145@Fedaykin.here>
next in thread | previous in thread | raw e-mail | index | archive | help
--liOOAslEiF7prFVr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, An updated version of the last bsd.port.mk fetch-check target. On Sat, Apr 28, 2001 at 04:04:45AM -0300, Mario Sergio Fujikawa Ferreira wrote: [snip] > I would like some comments on this new target. > One the most common problems we have when either creating > or updating ports is verifying if the DISTFILES exist under the > MASTER_SITES. Some of us have automated tools to do that but the > ports system could use an automated target. Are my port DISTFILES > available under my MASTER_SITES? > I wrote a simple fetch-check target that does exactly that. > It is pretty naive in which it currently depends on the existence > of fetch in the target system but we can improve it in the future. [snip] Now, there are 2 targets: fetch-check-per-file and fetch-check-per-site. To suit all tastes. Sample code and output are better than any explanation: :) Regards, $ cd /usr/ports/mail/mutt $ make fetch-check-per-file >>> Checking distfile mutt-1.2.5i.tar.gz > FOUND under ftp://ftp.guug.de/pub/mutt/ with 1973923 bytes > FOUND under ftp://ftp.uib.no/pub/mutt/ with 1973923 bytes > NOT FOUND under ftp://pgp.rasip.fer.hr/pub/mutt/international/ > NOT FOUND under ftp://ftp.gbnet.net/pub/mutt-international/ > NOT FOUND under ftp://riemann.iam.uni-bonn.de/pub/mutt/ > NOT FOUND under ftp://ftp.fu-berlin.de/pub/unix/mail/mutt/mutt-international/ > FOUND under ftp://ftp.gwdg.de/pub/unix/mail/mutt/international/ with 1973923 bytes > NOT FOUND under ftp://ftp.iks-jena.de/pub/mitarb/lutz/crypt/software/pgp/mutt/ > NOT FOUND under ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/mutt/ >>> Checking distfile charmaps-0.0.tar.gz > FOUND under ftp://ftp.guug.de/pub/mutt/ with 220133 bytes > FOUND under ftp://ftp.uib.no/pub/mutt/ with 220133 bytes > NOT FOUND under ftp://pgp.rasip.fer.hr/pub/mutt/international/ > NOT FOUND under ftp://ftp.gbnet.net/pub/mutt-international/ > NOT FOUND under ftp://riemann.iam.uni-bonn.de/pub/mutt/ > NOT FOUND under ftp://ftp.fu-berlin.de/pub/unix/mail/mutt/mutt-international/ > FOUND under ftp://ftp.gwdg.de/pub/unix/mail/mutt/international/ with 220133 bytes > NOT FOUND under ftp://ftp.iks-jena.de/pub/mitarb/lutz/crypt/software/pgp/mutt/ > NOT FOUND under ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/mutt/ >>> Checking patchfile patch-1.2.5.rr.compressed.1.gz > FOUND under http://www.spinnaker.de/mutt/compressed/ with 12938 bytes > NOT FOUND under http://www.frmug.org/mutt/mutt/mutt-1.2.5/ > NOT FOUND under http://www.math.fu-berlin.de/~leitner/mutt/mutt-1.2.5/ > NOT FOUND under http://www.efrei.fr/~parmelan/mutt/mutt-1.2.5/ > NOT FOUND under ftp://www.frmug.org/pub/mutt/mutt/mutt-1.2.5/ > NOT FOUND under ftp://riemann.iam.uni-bonn.de/pub/mutt/ > NOT FOUND under ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/mutt/ $ cd /usr/ports/mail/mutt $ make fetch-check-per-site >>> Checking MASTER SITE ftp://ftp.guug.de/pub/mutt/ > FOUND mutt-1.2.5i.tar.gz with 1973923 bytes > FOUND charmaps-0.0.tar.gz with 220133 bytes >>> Checking MASTER SITE ftp://ftp.uib.no/pub/mutt/ > FOUND mutt-1.2.5i.tar.gz with 1973923 bytes > FOUND charmaps-0.0.tar.gz with 220133 bytes >>> Checking MASTER SITE ftp://pgp.rasip.fer.hr/pub/mutt/international/ > NOT FOUND mutt-1.2.5i.tar.gz > NOT FOUND charmaps-0.0.tar.gz >>> Checking MASTER SITE ftp://ftp.gbnet.net/pub/mutt-international/ > NOT FOUND mutt-1.2.5i.tar.gz > NOT FOUND charmaps-0.0.tar.gz >>> Checking MASTER SITE ftp://riemann.iam.uni-bonn.de/pub/mutt/ > NOT FOUND mutt-1.2.5i.tar.gz > NOT FOUND charmaps-0.0.tar.gz >>> Checking MASTER SITE ftp://ftp.fu-berlin.de/pub/unix/mail/mutt/mutt-international/ > NOT FOUND mutt-1.2.5i.tar.gz > NOT FOUND charmaps-0.0.tar.gz >>> Checking MASTER SITE ftp://ftp.gwdg.de/pub/unix/mail/mutt/international/ > FOUND mutt-1.2.5i.tar.gz with -4629714551674241025 bytes > FOUND charmaps-0.0.tar.gz with -4629714551674241025 bytes >>> Checking MASTER SITE ftp://ftp.iks-jena.de/pub/mitarb/lutz/crypt/software/pgp/mutt/ > NOT FOUND mutt-1.2.5i.tar.gz > NOT FOUND charmaps-0.0.tar.gz >>> Checking MASTER SITE ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/mutt/ > NOT FOUND mutt-1.2.5i.tar.gz > NOT FOUND charmaps-0.0.tar.gz >>> Checking PATCH SITE http://www.spinnaker.de/mutt/compressed/ > FOUND patch-1.2.5.rr.compressed.1.gz with 12938 bytes >>> Checking PATCH SITE http://www.frmug.org/mutt/mutt/mutt-1.2.5/ > NOT FOUND patch-1.2.5.rr.compressed.1.gz >>> Checking PATCH SITE http://www.math.fu-berlin.de/~leitner/mutt/mutt-1.2.5/ > NOT FOUND patch-1.2.5.rr.compressed.1.gz >>> Checking PATCH SITE http://www.efrei.fr/~parmelan/mutt/mutt-1.2.5/ > NOT FOUND patch-1.2.5.rr.compressed.1.gz >>> Checking PATCH SITE ftp://www.frmug.org/pub/mutt/mutt/mutt-1.2.5/ > NOT FOUND patch-1.2.5.rr.compressed.1.gz >>> Checking PATCH SITE ftp://riemann.iam.uni-bonn.de/pub/mutt/ > NOT FOUND patch-1.2.5.rr.compressed.1.gz >>> Checking PATCH SITE ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/mutt/ > NOT FOUND patch-1.2.5.rr.compressed.1.gz -- Mario S F Ferreira - UnB - Brazil - "I guess this is a signature." lioux at ( freebsd dot org | linf dot unb dot br ) flames to beloved devnull@someotherworldbeloworabove.org feature, n: a documented bug | bug, n: an undocumented feature --liOOAslEiF7prFVr Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="patch-bsd.port.mk" --- bsd.port.mk.orig Sun Apr 22 13:47:33 2001 +++ bsd.port.mk Sat May 5 10:00:03 2001 @@ -286,6 +286,12 @@ # # fetch - Retrieves ${DISTFILES} (and ${PATCHFILES} if defined) # into ${DISTDIR} as necessary. +# fetch-check - Gives a simple (has to be analyzed by a human operator :) check +# of whether the ${DISTFILES} (and ${PATCHFILES if defined) +# exist on the defined ${MASTER_SITES} (and ${PATCHSITES}). +# Useful for port writing and updating. (default: fetch-check-per-file) +# fetch-check-per-site - Does a fetch-check testing all ${DISTFILES} grouped per site basis +# fetch-check-per-file - Does a fetch-check testing all ${MASTER_SITES} grouped per file basis # fetch-list - Show list of files that would be retrieved by fetch. # fetch-recursive - Retrieves ${DISTFILES} (and ${PATCHFILES} if defined), # for port and dependencies into ${DISTDIR} as necessary. @@ -946,15 +952,19 @@ .endif .if exists(/usr/bin/fetch) +FETCH= /usr/bin/fetch # avoid -A for 2.2 -- it's not ported to that branch +FETCH_CHECK_CMD?= ${FETCH} +FETCH_CHECK_BEFORE_ARGS?= -s .if ${OSVERSION} < 300000 -FETCH_CMD?= /usr/bin/fetch +FETCH_CMD?= ${FETCH} .else -FETCH_CMD?= /usr/bin/fetch -A +FETCH_CMD?= ${FETCH} -A .endif #FETCH_BEFORE_ARGS+= $${CKSIZE:+-S $$CKSIZE} .else -FETCH_CMD?= /usr/bin/ftp +FTP= /usr/bin/ftp +FETCH_CMD?= ${FTP} .endif TOUCH?= /usr/bin/touch @@ -2343,6 +2353,96 @@ .endif .endif +# Checks if MASTER_SITES contain the DISTFILES + +.if !target(fetch-check-per-file) +fetch-check-per-file: +.if defined(FETCH_CHECK_CMD) + @(for file in ${DISTFILES}; do \ + ${ECHO_MSG} ">>> Checking distfile $${file}" ; \ + for site in `${SORTED_MASTER_SITES_CMD}`; do \ + DIR=${DIST_SUBDIR}; \ + CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ + case $${file} in \ + */*) args="-o $${file} $${site}$${file}";; \ + *) args=$${site}$${file};; \ + esac; \ + FETCH_CHECK_FILE_SIZE=`if ${SETENV} ${FETCH_ENV} ${FETCH_CHECK_CMD} ${FETCH_CHECK_BEFORE_ARGS} $${args} 2>/dev/null; then continue 2; fi` ; \ + if [ -n "$${FETCH_CHECK_FILE_SIZE}" ] ; then \ + ${ECHO_MSG} "> FOUND under $${site} with $${FETCH_CHECK_FILE_SIZE} bytes" ; \ + else \ + ${ECHO_MSG} "> NOT FOUND under $${site}" ; \ + fi \ + done; \ + done) +.endif +.if defined(PATCHFILES) + @(for file in ${PATCHFILES}; do \ + ${ECHO_MSG} ">>> Checking patchfile $${file}" ; \ + for site in `${SORTED_PATCH_SITES_CMD}`; do \ + DIR=${DIST_SUBDIR}; \ + CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ + case $${file} in \ + */*) args="-o $${file} $${site}$${file}";; \ + *) args=$${site}$${file};; \ + esac; \ + FETCH_CHECK_FILE_SIZE=`if ${SETENV} ${FETCH_ENV} ${FETCH_CHECK_CMD} ${FETCH_CHECK_BEFORE_ARGS} $${args} 2>/dev/null; then continue 2; fi` ; \ + if [ -n "$${FETCH_CHECK_FILE_SIZE}" ] ; then \ + ${ECHO_MSG} "> FOUND under $${site} with $${FETCH_CHECK_FILE_SIZE} bytes" ; \ + else \ + ${ECHO_MSG} "> NOT FOUND under $${site}" ; \ + fi \ + done; \ + done) +.endif +.endif + +.if !target(fetch-check-per-site) +fetch-check-per-site: +.if defined(FETCH_CHECK_CMD) + @(for site in `${SORTED_MASTER_SITES_CMD}`; do \ + ${ECHO_MSG} ">>> Checking MASTER SITE $${site}" ; \ + for file in ${DISTFILES}; do \ + DIR=${DIST_SUBDIR}; \ + CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ + case $${file} in \ + */*) args="-o $${file} $${site}$${file}";; \ + *) args=$${site}$${file};; \ + esac; \ + FETCH_CHECK_FILE_SIZE=`if ${SETENV} ${FETCH_ENV} ${FETCH_CHECK_CMD} ${FETCH_CHECK_BEFORE_ARGS} $${args} 2>/dev/null; then continue 2; fi` ; \ + if [ -n "$${FETCH_CHECK_FILE_SIZE}" ] ; then \ + ${ECHO_MSG} "> FOUND $${file} with $${FETCH_CHECK_FILE_SIZE} bytes" ; \ + else \ + ${ECHO_MSG} "> NOT FOUND $${file}" ; \ + fi \ + done; \ + done) +.endif +.if defined(PATCHFILES) + @(for site in `${SORTED_PATCH_SITES_CMD}`; do \ + ${ECHO_MSG} ">>> Checking PATCH SITE $${site}" ; \ + for file in ${PATCHFILES}; do \ + DIR=${DIST_SUBDIR}; \ + CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ + case $${file} in \ + */*) args="-o $${file} $${site}$${file}";; \ + *) args=$${site}$${file};; \ + esac; \ + FETCH_CHECK_FILE_SIZE=`if ${SETENV} ${FETCH_ENV} ${FETCH_CHECK_CMD} ${FETCH_CHECK_BEFORE_ARGS} $${args} 2>/dev/null; then continue 2; fi` ; \ + if [ -n "$${FETCH_CHECK_FILE_SIZE}" ] ; then \ + ${ECHO_MSG} "> FOUND $${file} with $${FETCH_CHECK_FILE_SIZE} bytes" ; \ + else \ + ${ECHO_MSG} "> NOT FOUND $${file}" ; \ + fi \ + done; \ + done) +.endif +.endif + +.if !target(fetch-check) +fetch-check: fetch-check-per-file +.endif + # Checksumming utilities .if !target(makesum) @@ -2645,6 +2745,14 @@ fetch-recursive-list: @for dir in ${.CURDIR} $$(${ALL-DEPENDS-LIST}); do \ (cd $$dir; ${MAKE} fetch-list); \ + done +.endif + +.if !target(fetch-check-recursive) +fetch-check-recursive: + @${ECHO_MSG} "===> Fetch checking all distfiles for ${PKGNAME} and dependencies" + @for dir in ${.CURDIR} $$(${ALL-DEPENDS-LIST}); do \ + (cd $$dir; ${MAKE} fetch-check); \ done .endif --liOOAslEiF7prFVr-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010505104814.A21474>