Date: Tue, 11 Mar 2008 12:05:19 GMT From: Anton Statutov <astatutov@gmail.com> To: freebsd-gnats-submit@FreeBSD.org Subject: ports/121596: Revision of early sended new port 'getdelta' Message-ID: <200803111205.m2BC5J49038029@www.freebsd.org> Resent-Message-ID: <200803111210.m2BCA4q9017170@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 121596 >Category: ports >Synopsis: Revision of early sended new port 'getdelta' >Confidential: no >Severity: non-critical >Priority: medium >Responsible: freebsd-ports-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: maintainer-update >Submitter-Id: current-users >Arrival-Date: Tue Mar 11 12:10:04 UTC 2008 >Closed-Date: >Last-Modified: >Originator: Anton Statutov >Release: FreeBSD 6.3-RELEASE-p1 >Organization: BeStyle Ltd >Environment: >Description: Changes by Anrew R. Muhametsin: - added refetch feature - added 'tgz' mask - added check of FETCH_CMD parameters - distfiles path fixed - fixed 'date' calls - some work on CHECK_OLD_FILE - fixed litte ouput bug >How-To-Repeat: >Fix: Patch attached with submission follows: diff -ruN getdelta.bak/Makefile getdelta/Makefile --- getdelta.bak/Makefile Tue Mar 11 14:56:23 2008 +++ getdelta/Makefile Tue Mar 11 14:17:41 2008 @@ -7,7 +7,7 @@ PORTNAME= getdelta PORTVERSION= 0.7.8 -#PORTREVISION= 0 +PORTREVISION= 2 CATEGORIES= sysutils MASTER_SITES= http://132.207.4.160/distfiles/ diff -ruN getdelta.bak/files/patch-getdelta.sh getdelta/files/patch-getdelta.sh --- getdelta.bak/files/patch-getdelta.sh Tue Mar 11 14:56:23 2008 +++ getdelta/files/patch-getdelta.sh Tue Mar 11 14:50:44 2008 @@ -1,22 +1,28 @@ --- getdelta.sh.orig Fri Jul 13 20:30:26 2007 -+++ getdelta.sh Sat Feb 9 14:02:17 2008 -@@ -1,7 +1,7 @@ ++++ getdelta.sh Tue Mar 11 14:49:37 2008 +@@ -1,4 +1,4 @@ -#!/bin/sh --# getdelta.sh --# A download-wrapper script for gentoo that tries to get dtu files --# created by deltup instead of downloading complete source-files +#!/usr/local/bin/bash -+# getdelta.sh -+# A download-wrapper script for gentoo that tries to get dtu files -+# created by deltup instead of downloading complete source-files - # to save bandwidth. - # - # (C) 2004-2007 Nicolai Lissner <nlissne@linux01.gwdg.de> -@@ -25,12 +25,17 @@ + # getdelta.sh + # A download-wrapper script for gentoo that tries to get dtu files + # created by deltup instead of downloading complete source-files +@@ -25,6 +25,23 @@ VERSION="0.7" # Changelog -+# version 0.7.8-freebsd 2009/02/09 by Anton Statutov stant@bestyle.ru ++# version 0.7.8-freebsd_2 2008/03/11 ++# Changes by Anrew R. Muhametsin: ++# - added refetch feature ++# - added 'tgz' mask ++# - added check of FETCH_CMD parameters ++# - distfiles path fixed ++# - fixed 'date' calls ++# - some work on CHECK_OLD_FILE ++# ++# - fixed litte ouput bug ++# version 0.7.8-freebsd_1 2008/02/12 ++# - handled situation when required file already exists in distfiles ++# version 0.7.8-freebsd 2008/02/09 by Anton Statutov stant@bestyle.ru +# - replaced "stat -c %s" to "stat -f "%z" +# - replaced "/etc/deltup" to "/usr/local/etc/deltup" +# - replaced $1 to $2 because $1 is the size of file on freebsd @@ -24,235 +30,35 @@ # version 0.7.8 2007/07/13 # - added option -O to call of emerge when finding mirrors # - replaced some calls of coreutils with bash-internals - # - removed some unnecessary code - # version 0.7.7 2007/01/30 --# - added support for more than one local mirror in getdelta.rc -+# - added support for more than one local mirror in getdelta.rc - # thanks to Alexander Polozov, who sent me the patch for this - # version 0.7.6 2006/10/08 - # - fixed a typo - thanks to Andrey, who reported this problem in gentoo bug #150426 -@@ -39,7 +44,7 @@ - # if configured it will reduce the waiting timeout to the expected download-time (based on size of old version) - # version 0.7.4 2006/09/06 - # - do not remove log file but reset only to make this work with FEATURE userfetch --# version 0.7.3 -+# version 0.7.3 - # 2006/09/01 - # - fixed a bug in detection of original url (sometimes no url was found) - # version 0.7.2 -@@ -48,25 +53,25 @@ - # version 0.7.1 - # 2006/08/08 - # - fixed a problem in counting differences in filenames (aka bug #105011) --# - -+# - - # version 0.7 - # 2005/05/09 - # - servers create dtu files based on bdelta instead of xdelta - # this happens for two reasons: smaller dtu-files and amd64-compatibilty - # (yes, it's true - welcome to all new amd64 users of the servers) --# this is the reason for major update - and update forcing - you really -+# this is the reason for major update - and update forcing - you really - # NEED bdelta now to use the servers, while you do *not* need xdelta anymore :) - # - integrity change of old candidate is optional now and *disabled* by default - # if you want this time consuming "safe way" re-enable it via the config-file --# - added some files to DO_NOT_REMOVE file -+# - added some files to DO_NOT_REMOVE file - # - added "&time=<timestamp>" to prevent ANY proxy from returning results from cache +@@ -63,7 +80,7 @@ # instead of asking the server - the server does not use this parameter - it just # exists to create unique request-URLS (as proposed by bodo "bothie" thiesen) # version 0.6.9 2005/03/11 --# ? exit with exitcode of wget to signal to portage if -+# ? exit with exitcode of wget to signal to portage if +-# exit with exitcode of wget to signal to portage if ++# � exit with exitcode of wget to signal to portage if # download was successful # - handle metadata.dtd as exception (repoman uses FETCHCOMMAND) --# - you can disable fetching of dtu-files now by setting -+# - you can disable fetching of dtu-files now by setting - # the environment variable GETDELTA=0 - # version 0.6.8 2005/01/09 - # - init frontmatch and backmatch with 0 (thanks, Torsten Veller) -@@ -85,7 +90,7 @@ - # - inserted "break" to the TSTAMP>=QTMOUNT condition, too - # (reported by Torsten Veller) - # version 0.6.3 2004/11/08 --# - added "^bash" and "^gtk-engines" to the default -+# - added "^bash" and "^gtk-engines" to the default - # do_not_remove file - # - inserted a "break" to prevent infinite looping - # when the server returns a queueposition higher than -@@ -96,16 +101,16 @@ - # version 0.6.1 2004/10/18 - # - give better original URL to the server - # - enhanced detection of former version (thanks to Jimmy Wennlund) --# - the COLOR variable didn't work since 0.5.3, because -+# - the COLOR variable didn't work since 0.5.3, because - # the config file was not read before evaluating the variable - # - check, if the user have set RESUMECOMMAND to getdelta.sh - # and if so complain about this - # - # version 0.6 2004/10/12 - # - dropped the client-side mirror-detection --# - Jimmy Wennlund <jimmy@jw.dyndns.org> sent me patch to --# make getdelta.sh work in an own tempdir and to --# remove any temporary files even when user pressed -+# - Jimmy Wennlund <jimmy@jw.dyndns.org> sent me patch to -+# make getdelta.sh work in an own tempdir and to -+# remove any temporary files even when user pressed - # ctrl-c -- I really like that. Thanks, Jimmy. - # version 0.5.4 2004/10/11 - # - fixed a security leak (possible symlink attack) -@@ -118,14 +123,14 @@ - # - fixed wrong path-detection with thirdpartymirrors - # version 0.5.3.3 2004/09/12 - # - changed the way the script finds the mirror-group to use --# - setting GENTOO_MIRRORS="" is *not* necessary anymore -+# - setting GENTOO_MIRRORS="" is *not* necessary anymore - # removed check and warning about that - # version 0.5.3.2 2004/09/12 - # - fixed a bug in the ouput of remove() (thx to wiebel) - # version 0.5.3.1 2004/09/12 --# - fixed a typo (FILESDIR) -+# - fixed a typo (FILESDIR) - # - fixed a missing "]" --# version 0.5.3 2004/09/12 -+# version 0.5.3 2004/09/12 - # - some code cleanups - # - use a separated config file now - # - old file in DISTDIR is tested on corruption before trying to download a dtu -@@ -137,7 +142,7 @@ - # - added MAXIMUM_ACCEPTABLE_QUEUEPOS - # version 0.5.2.3 2004/09/06 - # - new variable QUERY_RETRY --# - dont remove file added -+# - dont remove file added - # version 0.5.2.2 2004/08/30 - # - fixed a typo - # version 0.5.2.1 2004/08/29 -@@ -147,14 +152,14 @@ - # - server sends a queued-message including queue-position now - # show this. - # version 0.5.1 2003/08/24 --# - for some reason a "broken pipe" message appears when --# this script is called by portage/python, caused by -+# - for some reason a "broken pipe" message appears when -+# this script is called by portage/python, caused by - # "ls -c|head -n1" - Ok, that *IS* a broken pipe, "head" - # would not read anything more than 1 line, but I do not - # really understand, why it does not happen when the script - # is called manually -- ANY use of "head" in a pipe-construction --# would result in a "broken-pipe", but bash itself never --# complains about that. a cosmetic change to make the -+# would result in a "broken-pipe", but bash itself never -+# complains about that. a cosmetic change to make the - # output clean and the script-code ugly :-/ - # version 0.5.0 2003/08/21 - # - the exception handling for kde changed to the server -@@ -165,7 +170,7 @@ - # the deltup-server queues requests now - # and sends back a document "deltup-queued" - # the client then waits 10 seconds and tries --# again until it either gets the dtu or a file -+# again until it either gets the dtu or a file - # named *.failed - # version 0.3.3 2003/05/06 - # transmit version to server -@@ -208,7 +213,9 @@ + # - you can disable fetching of dtu-files now by setting +@@ -208,7 +225,9 @@ # This file: # #################################################### -GETDELTA_CONFIGFILE=/etc/deltup/getdelta.rc +PREFIX=/usr/local -+DISTDIR=/usr/ports/distfiles ++DISTDIR=$(pwd) +GETDELTA_CONFIGFILE="${PREFIX}/etc/deltup/getdelta.rc" # splitversion(){ -@@ -251,14 +258,14 @@ - ;; - [[:alpha:]]) - calced=0 -- if [ "${x[$i]:0:3}" = "pre" ] -+ if [ "${x[$i]:0:3}" = "pre" ] - then - vs=${x[$i]:3} - let vser=0x${vser}00-40+${vs:-0} - vser=$(printf "%02x" $vser) - calced=1 - fi -- if [ "${x[$i]:0:2}" = "rc" ] -+ if [ "${x[$i]:0:2}" = "rc" ] - then - vs=${x[$i]:2} - let vser=0x${vser}00-40+${vs:-0} -@@ -271,7 +278,7 @@ - let vser=0x${vser}00-80+${vs:-0} - vser=$(printf "%02x" $vser) - calced=1 -- fi -+ fi - if [ "${x[$i]:0:4}" = "beta" ] - then - vs=${x[$i]:4} -@@ -279,7 +286,7 @@ - vser=$(printf "%02x" $vser) - calced=1 - fi -- if [ "$calced" = "0" ] -+ if [ "$calced" = "0" ] - then - vs=$(echo -n ${x[$i]} | od -t x1 | head -n1| cut -d" " -f2-| tr -d " ") - vser="${vser}${vs}" -@@ -308,21 +315,21 @@ - then - echo -e "\n# ${3}\n${1}=\"${2}\"" >>$GETDELTA_CONFIGFILE - output "${CYAN}Added new variable ${YELLOW}$1${CYAN} to config file ${GETDELTA_CONFIGFILE}\n" -- output "please check if it fits your needs\n" -+ output "please check if it fits your needs\n" - fi - } - - # this checks for an entry in our do_not_remove-file and adds it if does not exist - # $1 is the name (as grep regexp) of the file not to be removed - add_to_donotremove() { -- -+ - if ! grep -q "^${1}" $DO_NOT_REMOVE - then - echo "${1}" >>$DO_NOT_REMOVE - output "${CYAN}Added new grep-regex \"${1}\" to config file ${DO_NOT_REMOVE}\n" - fi - } -- -+ - - remove() { - output "${GREEN}You have chosen to remove ${CYAN}$1\n" -@@ -351,20 +358,20 @@ - -e "s/src6/srcFFF/g" \ - -e "s/src7/srcGGG/g" <<< $MASK_FILENAME) - fi -- -+ - # ignore PR for src-files of firefox - if [ "${MASK_FILENAME:0:7}" = "firefox" ] - then - MASK_FILENAME="${MASK_FILENAME//PR}" - fi -- -+ - # ignore "try" with new mplayer - if [ "${MASK_FILENAME:0:7}" = "MPlayer" ] - then - MASK_FILENAME="${MASK_FILENAME//try}" - fi -- -- -+ -+ +@@ -368,6 +387,7 @@ # ignore some strings in any filename echo $(sed -e "s/\.bz2$//g" \ -e "s/\.gz$//g" \ -@@ -383,13 +390,13 @@ ++ -e "s/\.tgz$//g" \ + -e "s/[0-9]//g" \ + -e "s/pre//g" \ + -e "s/preview//g" \ +@@ -383,13 +403,13 @@ add_to_configfile KDE_MIRROR "ftp://ftp.kde.org/pub/kde/stable" "we de not get kde-deltas from a delta-up-server, since kde provides own xdelta-files" add_to_configfile LOCAL_MIRROR "" "set this to one or more (space separated) URI ending with '/' if you want to check one or more local mirror(s) first\n# most people just leave it empty." add_to_configfile DELTUP_SERVER "http://linux01.gwdg.de/~nlissne/deltup.php" "deltup-server to use" @@ -268,361 +74,192 @@ add_to_configfile REMOVE_INCOMPLETE_OLD_FILES "false" "set this to \"true\" if you want getdelta.sh to delete old versions that seems to be corrupt,\n# or to \"false\" if you want to delete them manually\n# note: getdelta.sh will not use these files anyway" add_to_configfile VERBOSITY true "set to \"true\", if you want verbose outputs (later to be set to a level [0-3])" add_to_configfile COLOR true "set to \"true\", if you want colorful messages, \"false\" if not." -@@ -402,12 +409,12 @@ - source $GETDELTA_CONFIGFILE - - # create or update DO_NOT_REMOVE file --# these files have "old" versions that are needed to build the new versions -+# these files have "old" versions that are needed to build the new versions - # so they should never removed by the REMOVE_OLD feature - DO_NOT_REMOVE_DIR=$(dirname $DO_NOT_REMOVE) --if [ ! -e $DO_NOT_REMOVE ] --then -- mkdir -p $DO_NOT_REMOVE_DIR -+if [ ! -e $DO_NOT_REMOVE ] -+then -+ mkdir -p $DO_NOT_REMOVE_DIR - echo "# This file contains regexp in 'grep-style' for files that should not be removed" >$DO_NOT_REMOVE - echo "# if REMOVE_OLD is set to 'true'" >>$DO_NOT_REMOVE - echo "# Some examples (actually these files are known to result" >>$DO_NOT_REMOVE -@@ -424,7 +431,7 @@ +@@ -424,23 +444,9 @@ add_to_donotremove "^rp-pppoe" -if [ -z $1 ] -+if [ -z "$1" ] - then - COLOR=true - echo -e "${YELLOW}getdelta.sh version ${VERSION}" -@@ -435,13 +442,13 @@ - echo -e "\n${YELLOW}There is a config-file ${CYAN}${GETDELTA_CONFIGFILE}${YELLOW} with some variables to control the behaviour of this script." - echo -e "Edit it to your needs.${NORMAL}" - exit 1 +-then +- COLOR=true +- echo -e "${YELLOW}getdelta.sh version ${VERSION}" +- echo "This script has to be called like this:" +- echo -e "${CYAN}$0 <URI>" +- echo -e "\n${YELLOW}To use it, you should just put the following line into your /etc/make.conf" +- echo -e "${GREEN}FETCHCOMMAND=\"$0 \\\${URI}\"" +- echo -e "\n${YELLOW}There is a config-file ${CYAN}${GETDELTA_CONFIGFILE}${YELLOW} with some variables to control the behaviour of this script." +- echo -e "Edit it to your needs.${NORMAL}" +- exit 1 -fi -# include variables from gentoo make.globals and make.conf -source /etc/make.globals -+fi -+# include variables from freebsd make.globals and make.conf -+#source /etc/make.globals ++# include variables from freebsd make.conf source /etc/make.conf - --if ${COLOR} -+if ${COLOR} +- + if ${COLOR} then RED="\033[01;31m" - GREEN="\033[01;32m" -@@ -459,14 +466,15 @@ - CYAN="" - NORMAL="" - fi --grep -q "getdelta.sh" <<< "${RESUMECOMMAND}" && -- output "${RED}do NOT set RESUMECOMMAND to use getdelta.sh" && -+grep -q "getdelta.sh" <<< "${RESUMECOMMAND}" && -+ output "${RED}do NOT set RESUMECOMMAND to use getdelta.sh" && +@@ -464,9 +470,25 @@ output "use getdelta.sh for your FETCHCOMMAND, only." && sleep 5 && exit 1 - pushd $DISTDIR >/dev/null 2>/dev/null +-pushd $DISTDIR >/dev/null 2>/dev/null -ORIG_URI=$1 --NEW_FILE=$(basename $ORIG_URI) -+ORIG_URI=$2 ++if [[ "${1%%-S *}" ]] ; then ORIG_URI=$1 ; else ORIG_URI=$2 ; fi +output "${GREEN} Original URI: ${YELLOW}${ORIG_URI}" -+NEW_FILE=$(basename "$ORIG_URI") + NEW_FILE=$(basename $ORIG_URI) ++TMP_DWN_FILE="${DISTDIR}/.download_tmp-$NEW_FILE" ++ ++if [ -z "$1" ] ++then ++ COLOR=true ++ echo -e "${YELLOW}getdelta.sh version ${VERSION}" ++ echo "This script has to be called like this:" ++ echo -e "${CYAN}$0 <URI>" ++ echo -e "\n${YELLOW}To use it, you should just put the following line into your /etc/make.conf" ++ echo -e "${GREEN}FETCHCOMMAND=\"$0 \\\${URI}\"" ++ echo -e "\n${YELLOW}There is a config-file ${CYAN}${GETDELTA_CONFIGFILE}${YELLOW} with some variables to control the behaviour of this script." ++ echo -e "Edit it to your needs.${NORMAL}" ++ exit 1 ++fi ++ ++pushd $DISTDIR >/dev/null 2>/dev/null # repoman downloads metadata.dtd with FETCHCOMMAND # this should not be done with getdelta - so just fetch the file and exit -@@ -495,8 +503,8 @@ - for localn in $LOCAL_MIRROR - do - output "${YELLOW}Trying to get ${CYAN}${NEW_FILE}${YELLOW} from local mirror ${CYAN}${localn}\n" -- if $FETCH "${localn}${NEW_FILE}" -- then -+ if $FETCH "${localn}${NEW_FILE}" -+ then - output "${GREEN}success.\n" - exit 0 - else -@@ -508,11 +516,11 @@ - # find an old file in $DISTDIR that matches the new one. This is tricky, - # and probably it will fail sometimes. - # --# we just ignore any occurence of -+# we just ignore any occurence of - # "pre","rc","[0-9]","_","-","." in the filenames and test - # if they are the same (or VERY similar (differ only in 1 char)). --# to reduce the files to check, we only check files --# with the same beginning -+# to reduce the files to check, we only check files -+# with the same beginning - # - output "${GREEN}Searching for a previously downloaded file in ${YELLOW}${DISTDIR}\n" - -@@ -522,7 +530,7 @@ - - # if filename is lib* use first 4 letters to increase performance - [ "${NEW_FILE:0:3}" = "lib" ] && [ $length_first_chars -lt 5 ] && first_chars="${NEW_FILE:0:4}" -- -+ - mask=$(mask_name "${NEW_FILE}") - let len1=${#mask} - filelist="" -@@ -532,7 +540,7 @@ - mask2=$(mask_name "${name}") - # add any file, that results in the same mask or differ not more than two letters - let len2=${#mask2} -- if [ $len1 -gt $len2 ] -+ if [ $len1 -gt $len2 ] - then - max=${len1} - let min=${len2} -@@ -540,10 +548,10 @@ - let min=${len1} - max=${len2} - fi -- let df=${max}-${min} -- -+ let df=${max}-${min} -+ - # if masks differ in length more than 1 they cannot match -- if [ $df -le 1 ] -+ if [ $df -le 1 ] - then - let frontmatch=0 - let backmatch=0 -@@ -554,7 +562,7 @@ - else break - fi +@@ -578,7 +600,13 @@ + if ! [ -z "$filelist" ] + then + # we have got a list of candidates in $filelist now. find the best match . +- output "${GREEN}We have the following candidates to choose from \n${YELLOW}`sed -e \"s/\ /\\n/g\" <<< $filelist` \n" ++ output "${GREEN}We have the following candidates to choose from:" ++ for name in $filelist ++ do ++ output "${YELLOW}$name" ++ done ++ ++ output "" + + # find matching part of filename - first: frontmatch + x=0; +@@ -588,7 +616,7 @@ + do + for ((i=0;i<${#a[@]};i++)) + do +- [ ${a[0]:${x}:1} != ${a[$i]:${x}:1} ] && match=$x ++ [[ ${a[0]:${x}:1} != ${a[$i]:${x}:1} ]] && match=$x done -- -+ - # now backwards - mask=$(rev <<< ${mask}) - mask2=$(rev <<< ${mask2}) -@@ -565,18 +573,18 @@ - else break - fi + ((x++)) + done +@@ -601,7 +629,7 @@ + do + for ((i=0;i<${#a[@]};i++)) + do +- [ ${a[0]:${#a[0]}-${x}:1} != ${a[$i]:${#a[$i]}-${x}:1} ] && match=$x ++ [[ ${a[0]:${#a[0]}-${x}:1} != ${a[$i]:${#a[$i]}-${x}:1} ]] && match=$x done -- -+ - # forwards for mask again (need this for the next run of the loop) - mask=$(rev <<< ${mask}) -- -+ - let matchall=${frontmatch}+${backmatch} - let minmatch=${min}-1 - [ ${matchall} -ge ${minmatch} ] && filelist="${filelist} $name" - fi - done - --if ! [ -z "$filelist" ] --then -+if ! [ -z "$filelist" ] -+then - # we have got a list of candidates in $filelist now. find the best match . - output "${GREEN}We have the following candidates to choose from \n${YELLOW}`sed -e \"s/\ /\\n/g\" <<< $filelist` \n" - -@@ -607,7 +615,7 @@ + ((x++)) done - ((match--)) - backmatch=${a[0]:${#a[0]}-${match}} -- -+ - # isolate version from filename (foobar-1.2.3.tar.gz -> 1.2.3) - new_version=${NEW_FILE#${frontmatch}} - new_version=${new_version%${backmatch}} -@@ -642,7 +650,7 @@ - else - let serial_diff=0x${old_serial}-0x${new_serial} - fi -- if [ $serial_diff -le ${minimal_diff:-${serial_diff}} ] -+ if [ $serial_diff -le ${minimal_diff:-${serial_diff}} ] - then - best_candidate="$name" - minimal_diff=${serial_diff} -@@ -651,18 +659,18 @@ - - output "${GREEN}The best of all is ... ${CYAN}${best_candidate}\n" - output "${YELLOW}Checking if this file is OK.\n" -- -+ - # this part is based on Pkunk's code posted on http://bugs.gentoo.org/show_bug.cgi?id=63525 - # but with some changes +@@ -657,21 +685,11 @@ FILE_IS_CORRUPT=false -- if $CHECK_OLD_FILE -+ if $CHECK_OLD_FILE + if $CHECK_OLD_FILE then - file_digest=$(grep -h ${best_candidate} ${FILESDIR}/digest-* | sed -n 1p) - if [ "$file_digest" ] - then +- file_digest=$(grep -h ${best_candidate} ${FILESDIR}/digest-* | sed -n 1p) +- if [ "$file_digest" ] +- then - file_md5=$(cut -d ' ' -f2 <<< "$file_digest") -+ file_md5=$(cut -d ' ' -f2 <<< "$file_digest") - file_origsize=$(cut -d ' ' -f4 <<< "$file_digest") +- file_origsize=$(cut -d ' ' -f4 <<< "$file_digest") - file_currentsize=$(stat -c %s ${best_candidate}) -+ file_currentsize=$(stat -f "%z" ${best_candidate}) - if [ $file_currentsize -ne $file_origsize ] +- if [ $file_currentsize -ne $file_origsize ] ++ _sfx1="$(rev <<< ${best_candidate} | cut -d. -f1 | rev)" ++ _sfx2="$(rev <<< ${best_candidate} | cut -d. -f2 | rev)" ++ if [ ${_sfx2} = "tar" -o ${_sfx1} = "tgz" ] then - output "${RED}Found ${best_candidate}, but filesize ${CYAN}${file_currentsize} ${RED} does not match ${CYAN}${file_origsize} (found in digest-file)\n" -@@ -677,7 +685,7 @@ +- output "${RED}Found ${best_candidate}, but filesize ${CYAN}${file_currentsize} ${RED} does not match ${CYAN}${file_origsize} (found in digest-file)\n" +- FILE_IS_CORRUPT=true +- fi +- else +- if [ $(rev <<< ${best_candidate} | cut -d. -f2 | rev) = "tar" ] +- then +- output "${YELLOW}Could not find a digest-file for ${CYAN}${best_candidate}${YELLOW}. Testing file integrity with tar.\n" ++ output "Testing file integrity with tar.\n" + if ! tar -tf ${best_candidate} >/dev/null + then output "${RED}reported an error while testing ${CYAN}${best_candidate}${RED} - so this file is unusable.\n" - FILE_IS_CORRUPT=true +@@ -685,26 +703,26 @@ fi -- -+ - if $FILE_IS_CORRUPT && $REMOVE_INCOMPLETE_OLD_FILES - then - output "${YELLOW}You have chosen to automatically delete such broken files from your distfiles-directory, so here we go...\n" -@@ -689,9 +697,9 @@ + fi + fi +- fi # end of file-corruption check for $best_candidate found in distfiles if ! $FILE_IS_CORRUPT then - - QUERY_URL=$(GENTOO_MIRRORS="" emerge -fOp =${CATEGORY}/${PF} 2>&1 | - sed -e "s/ /\\n/g" | egrep "(http|ftp)://" | -+ -+ QUERY_URL=$(GENTOO_MIRRORS="" emerge -fOp =${CATEGORY}/${PF} 2>&1 | -+ sed -e "s/ /\\n/g" | egrep "(http|ftp)://" | - grep "${NEW_FILE}" | tail -n 1) +- grep "${NEW_FILE}" | tail -n 1) ++ if [ "${best_candidate}" = "${NEW_FILE}" ] ++ then ++ output "${GREEN}Required file already exists. Nothing to download.\n" ++ else ++ QUERY_URL=${ORIG_URI} query="?have=${best_candidate}&want=${NEW_FILE}&url=${QUERY_URL}&version=${VERSION}&time=$(date +%s)" output "${GREEN}Trying to download ${YELLOW}${best_candidate}-${NEW_FILE}.dtu\n" -@@ -704,22 +712,22 @@ + + # Remember where we are, and go to a new dir there we can work +- tmp_dwn_dest="${DISTDIR}/.getdelta-`date +%N`-tmp" ++ tmp_dwn_dest="${DISTDIR}/.getdelta-`date +%s`_${RANDOM}-tmp" + mkdir ${tmp_dwn_dest} + # If user abort Ctrl+C (signal 2), remove tmp-dir; enabable trap again and send it again to stop wget + trap "rm -r ${tmp_dwn_dest}; trap 2; kill -2 $$" 2 pushd ${tmp_dwn_dest} >/dev/null 2>&1 # thanks to MATSUI Fe2+ Tetsushi for idea and patch - FILESIZE=$(stat -c %s "${DISTDIR}/${best_candidate}") -+ FILESIZE=$(stat -f "%z" "${DISTDIR}/${best_candidate}") ++ FILESIZE=$(stat -f "%z" "${DISTDIR}/${best_candidate}") let TIMELIMIT=${FILESIZE}/${BANDWIDTH} [[ $TIMELIMIT -lt $QUEUETIMEOUT ]] && QUEUETIMEOUT=$TIMELIMIT -- -+ - if $FETCH "${DELTUP_SERVER}${query}" - then - # thanks to deelkar for this much more elegant solution to the "broken pipe" problem with "head -n1" -- GOTFILE=$(ls -c | sed -n 1p) -+ GOTFILE=$(ls -c | sed -n 1p) - output "${YELLOW}GOT ${CYAN}${GOTFILE}\n" -- -+ - # There are some possibilities what the deltup-server - # may have sento to us. -- -+ - # first: the request have been queued - if [ "${GOTFILE}" = "deltup-queued" ] -- then -+ then - let QTMOUT=$(date +%s)+QUEUETIMEOUT - while [ -f deltup-queued ] - do -@@ -732,11 +740,11 @@ - then - for ((sec=QUEUERETRY;sec>0;sec--)) - do -- if ((sec>1)) -+ if ((sec>1)) - then -- ${VERBOSITY} && echo -n -e "${YELLOW} I will try again in ${sec} seconds. \r" -+ ${VERBOSITY} && echo -n -e "${YELLOW} I will try again in ${sec} seconds. \r" - else -- ${VERBOSITY} && echo -n -e "${YELLOW} I will try again in ${sec} second. \r" -+ ${VERBOSITY} && echo -n -e "${YELLOW} I will try again in ${sec} second. \r" - fi - sleep 1 - done -@@ -745,7 +753,7 @@ - GOTFILE=$(ls -c | sed -n 1p) - else - if ((TSTAMP>=QTMOUT)) -- then -+ then - GOTFILE="timeout" - output "\n${RED}TIMEOUT exceeded.\n" - break -@@ -756,31 +764,31 @@ - output "\n${RED}You have configured getdelta.sh not to accept this queue-position.\n" - output "${YELLOW}We are going to download the ${RED}full archive${YELLOW} instead.\n" - break -- fi -+ fi - fi - done - fi -- -+ - if [ -f ${best_candidate}-${NEW_FILE}.failed ] - then -- output "\n${RED}The server could not build the dtu-file for ${NEW_FILE}\n" -- output "${YELLOW}reason:\n${RED}$(cat ${best_candidate}-${NEW_FILE}.failed)\n" -+ output "\n${RED}The server could not build the dtu-file for ${NEW_FILE}\n" -+ output "${YELLOW}reason:\n${RED}$(cat ${best_candidate}-${NEW_FILE}.failed)\n" - rm -rf ${best_candidate}-${NEW_FILE}.failed - fi - + +@@ -771,10 +789,10 @@ if [ -f ${best_candidate}-${NEW_FILE}.dtu ] then -- output "${GREEN}Successfully fetched the dtu-file - let's build ${NEW_FILE}...\n" + output "${GREEN}Successfully fetched the dtu-file - let's build ${NEW_FILE}...\n" - downloadsize=$(stat -c %s ${best_candidate}-${NEW_FILE}.dtu) -- if deltup -p -v -D ${DISTDIR} ${best_candidate}-${NEW_FILE}.dtu -- then ++ downloadsize=$(stat -f "%z" ${best_candidate}-${NEW_FILE}.dtu) + if deltup -p -v -D ${DISTDIR} ${best_candidate}-${NEW_FILE}.dtu + then - newsize=$(stat -c %s ${NEW_FILE}) -+ output "${GREEN}Successfully fetched the dtu-file - let's build ${NEW_FILE}...\n" -+ downloadsize=$(stat -f "%z" ${best_candidate}-${NEW_FILE}.dtu) -+ if deltup -p -v -D ${DISTDIR} ${best_candidate}-${NEW_FILE}.dtu -+ then -+ newsize=$(stat -f "%z" ${NEW_FILE}) ++ newsize=$(stat -f "%z" ${NEW_FILE}) let savedsize=${newsize}-${downloadsize} let percent=${savedsize}*100/${newsize} unit="bytes" - [ $savedsize -gt 1024 ] && let savedsize=$savedsize/1024 && unit="kB" - [ $savedsize -gt 1024 ] && let savedsize=$savedsize/1024 && unit="MB" -- -+ - case $unit in - bytes) UCOLOR=${RED} - ;; -@@ -796,13 +804,13 @@ - fi - - fi # if $FETCH "${DELTUP_SERVER}${query}" -- -+ - # Clean up. - # We might got an important message - if [ -f ${tmp_dwn_dest}/deltup-server.msg ] - then -- echo -e "${RED}IMPORTANT MESSAGE FROM DELTUP-SERVER${YELLOW}$(cat ${tmp_dwn_dest}/deltup-server.msg)\n" -- for ((i=1;i<=5;i++)) -+ echo -e "${RED}IMPORTANT MESSAGE FROM DELTUP-SERVER${YELLOW}$(cat ${tmp_dwn_dest}/deltup-server.msg)\n" -+ for ((i=1;i<=5;i++)) - do - echo -n -e "\a" - sleep 1 -@@ -816,15 +824,15 @@ +@@ -815,6 +833,7 @@ + rm -rf ${tmp_dwn_dest} #stop respond to trap2 trap 2 ++ fi # if [ "${best_candidate}" = "${NEW_FILE}" ] fi # if ! FILE_IS_CORRUPT --else # if ! [ -z "$filelist" ] -+else # if ! [ -z "$filelist" ] + else # if ! [ -z "$filelist" ] # No filelist - probably we do not have an old version of the file -- output "${RED}No old version of the requested file found.\n" -+ output "${RED}No old version of the requested file found.\n" - fi - -- -+ - # Ok, once we are here, we should have got the delta (and used it) - # or we still have to download the full file --if ! [ -f ${DISTDIR}/${NEW_FILE} ] -+if ! [ -f ${DISTDIR}/${NEW_FILE} ] +@@ -827,11 +846,19 @@ + if ! [ -f ${DISTDIR}/${NEW_FILE} ] then output "${RED}The dtu could not be fetched,${YELLOW} downloading full file from original URL\n" - $FETCH $ORIG_URI -@@ -833,7 +841,7 @@ - fi - +- $FETCH $ORIG_URI ++ ++ $FETCH -O "${TMP_DWN_FILE}" -c "$ORIG_URI" ++ + # remember we had a fallback to use correct exitcode for portage + FALLBACK=$? +-fi + ++ if [ $FALLBACK = 0 ] ; then ++ output "${YELOW}Download complete\n" ++ mv "${TMP_DWN_FILE}" "${DISTDIR}/${NEW_FILE}" ++ fi ++else ++ [ -f "${TMP_DWN_FILE}" ] && rm "${TMP_DWN_FILE}" ++fi --if $SEPARATED_WINDOW -+if $SEPARATED_WINDOW + if $SEPARATED_WINDOW then - sleep 3 - kill $termpid diff -ruN getdelta.bak/pkg-message getdelta/pkg-message --- getdelta.bak/pkg-message Tue Mar 11 14:56:23 2008 +++ getdelta/pkg-message Tue Mar 11 14:18:21 2008 @@ -1,8 +1,8 @@ ========================================================================================= ATTENTION! -To use Getdelta for upgrading ports you need to add following string into /etc/make.conf: +To use Getdelta for upgrading ports you need to add the following string into /etc/make.conf: FETCH_CMD=/usr/local/bin/getdelta.sh -To do this you can simple run following command: +To do this you can simply run the following command: echo "FETCH_CMD=/usr/local/bin/getdelta.sh" >> /etc/make.conf ========================================================================================= >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200803111205.m2BC5J49038029>