From owner-freebsd-ports Sun Feb 9 18:21: 5 2003 Delivered-To: freebsd-ports@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8245437B401; Sun, 9 Feb 2003 18:21:02 -0800 (PST) Received: from perrin.int.nxad.com (internal.ext.nxad.com [66.250.180.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id A557843F93; Sun, 9 Feb 2003 18:21:00 -0800 (PST) (envelope-from sean@perrin.int.nxad.com) Received: by perrin.int.nxad.com (Postfix, from userid 1001) id D873B21065; Sun, 9 Feb 2003 18:19:53 -0800 (PST) Date: Sun, 9 Feb 2003 18:19:53 -0800 From: Sean Chittenden To: ports@FreeBSD.org Subject: Randomizing master sites for distfile downloads... Message-ID: <20030210021953.GT15936@perrin.int.nxad.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="I2DZnI03i1DS0lOz" Content-Disposition: inline User-Agent: Mutt/1.4i X-PGP-Key: finger seanc@FreeBSD.org X-PGP-Fingerprint: 6CEB 1B06 BFD3 70F6 95BE 7E4D 8E85 2E0A 5F5B 3ECB X-Web-Homepage: http://sean.chittenden.org/ Sender: owner-freebsd-ports@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --I2DZnI03i1DS0lOz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Pending a few more days of review of the updated version of random(6) on -audit, I'd like to see about introducing the ability to randomize the downloads of distfiles from the various master sites. I think mirror operators who are listed in the first three or four sites would love the repreive as well, nevermind improving the resiliancy of the system. Mark Murry posted a bit about how to randomize the sites without using the updated random(6), so pending any objections, I'll likely incorporate his suggestion for OS versions less than 500100 or 400800. I've got a few ideas for randomizing load for a given geographic region (Europe, Asia, Affrica, and the US being the 4 biggies: bandwidth isn't free, especially when crossing continents/oceans) and then allowing for a fallback to a randomized set of global mirrors as well, but those'll take some time to hash out. Thoughts/comments? -sc -- Sean Chittenden --I2DZnI03i1DS0lOz Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=patch Index: bsd.port.mk =================================================================== RCS file: /home/ncvs/ports/Mk/bsd.port.mk,v retrieving revision 1.437 diff -u -r1.437 bsd.port.mk --- bsd.port.mk 5 Dec 2002 22:43:22 -0000 1.437 +++ bsd.port.mk 10 Feb 2003 02:05:37 -0000 @@ -1308,6 +1307,23 @@ FETCH_CMD?= /usr/bin/ftp .endif +.if defined(RANDOMIZE_MASTER_SITES) +.if exists(/usr/games/random) +RANDOM_CMD?= /usr/games/random +RANDOM_ARGS?= "-w -f -" +.if ${OSVERSION} > 500100 +_RANDOMIZE_SITES= " |${RANDOM_CMD} ${RANDOM_ARGS}" +.else +_RANDOMIZE_SITES= '' +.endif +.if ${OSVERSION} > 400800 +_RANDOMIZE_SITES= " |${RANDOM_CMD} ${RANDOM_ARGS}" +.else +_RANDOMIZE_SITES= '' +.endif +.endif +.endif + TOUCH?= /usr/bin/touch TOUCH_FLAGS?= -f @@ -2368,7 +2384,7 @@ else \ SORTED_MASTER_SITES_CMD_TMP="${SORTED_MASTER_SITES_DEFAULT_CMD}" ; \ fi ; \ - for site in `eval $$SORTED_MASTER_SITES_CMD_TMP`; do \ + for site in `eval $$SORTED_MASTER_SITES_CMD_TMP ${_RANDOMIZE_SITES}`; do \ ${ECHO_MSG} ">> Attempting to fetch from $${site}."; \ DIR=${DIST_SUBDIR}; \ CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ @@ -3112,7 +3128,7 @@ else \ SORTED_MASTER_SITES_CMD_TMP="${SORTED_MASTER_SITES_DEFAULT_CMD}" ; \ fi ; \ - for site in `eval $$SORTED_MASTER_SITES_CMD_TMP`; do \ + for site in `eval $$SORTED_MASTER_SITES_CMD_TMP ${_RANDOMIZE_SITES}`; do \ DIR=${DIST_SUBDIR}; \ CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ case $${file} in \ @@ -3144,7 +3160,7 @@ else \ SORTED_PATCH_SITES_CMD_TMP="${SORTED_PATCH_SITES_DEFAULT_CMD}" ; \ fi ; \ - for site in `eval $$SORTED_PATCH_SITES_CMD_TMP`; do \ + for site in `eval $$SORTED_PATCH_SITES_CMD_TMP ${_RANDOMIZE_SITES}`; do \ DIR=${DIST_SUBDIR}; \ CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \ case $${file} in \ --I2DZnI03i1DS0lOz-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ports" in the body of the message