Date: Sun, 9 Feb 2003 18:19:53 -0800 From: Sean Chittenden <seanc@FreeBSD.org> To: ports@FreeBSD.org Subject: Randomizing master sites for distfile downloads... Message-ID: <20030210021953.GT15936@perrin.int.nxad.com>
next in thread | raw e-mail | index | archive | help
--I2DZnI03i1DS0lOz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Pending a few more days of review of the updated version of random(6)
on -audit, I'd like to see about introducing the ability to randomize
the downloads of distfiles from the various master sites. I think
mirror operators who are listed in the first three or four sites would
love the repreive as well, nevermind improving the resiliancy of the
system. Mark Murry posted a bit about how to randomize the sites
without using the updated random(6), so pending any objections, I'll
likely incorporate his suggestion for OS versions less than 500100 or
400800.
I've got a few ideas for randomizing load for a given geographic
region (Europe, Asia, Affrica, and the US being the 4 biggies:
bandwidth isn't free, especially when crossing continents/oceans) and
then allowing for a fallback to a randomized set of global mirrors as
well, but those'll take some time to hash out.
Thoughts/comments? -sc
--
Sean Chittenden
--I2DZnI03i1DS0lOz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=patch
Index: bsd.port.mk
===================================================================
RCS file: /home/ncvs/ports/Mk/bsd.port.mk,v
retrieving revision 1.437
diff -u -r1.437 bsd.port.mk
--- bsd.port.mk 5 Dec 2002 22:43:22 -0000 1.437
+++ bsd.port.mk 10 Feb 2003 02:05:37 -0000
@@ -1308,6 +1307,23 @@
FETCH_CMD?= /usr/bin/ftp
.endif
+.if defined(RANDOMIZE_MASTER_SITES)
+.if exists(/usr/games/random)
+RANDOM_CMD?= /usr/games/random
+RANDOM_ARGS?= "-w -f -"
+.if ${OSVERSION} > 500100
+_RANDOMIZE_SITES= " |${RANDOM_CMD} ${RANDOM_ARGS}"
+.else
+_RANDOMIZE_SITES= ''
+.endif
+.if ${OSVERSION} > 400800
+_RANDOMIZE_SITES= " |${RANDOM_CMD} ${RANDOM_ARGS}"
+.else
+_RANDOMIZE_SITES= ''
+.endif
+.endif
+.endif
+
TOUCH?= /usr/bin/touch
TOUCH_FLAGS?= -f
@@ -2368,7 +2384,7 @@
else \
SORTED_MASTER_SITES_CMD_TMP="${SORTED_MASTER_SITES_DEFAULT_CMD}" ; \
fi ; \
- for site in `eval $$SORTED_MASTER_SITES_CMD_TMP`; do \
+ for site in `eval $$SORTED_MASTER_SITES_CMD_TMP ${_RANDOMIZE_SITES}`; do \
${ECHO_MSG} ">> Attempting to fetch from $${site}."; \
DIR=${DIST_SUBDIR}; \
CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \
@@ -3112,7 +3128,7 @@
else \
SORTED_MASTER_SITES_CMD_TMP="${SORTED_MASTER_SITES_DEFAULT_CMD}" ; \
fi ; \
- for site in `eval $$SORTED_MASTER_SITES_CMD_TMP`; do \
+ for site in `eval $$SORTED_MASTER_SITES_CMD_TMP ${_RANDOMIZE_SITES}`; do \
DIR=${DIST_SUBDIR}; \
CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \
case $${file} in \
@@ -3144,7 +3160,7 @@
else \
SORTED_PATCH_SITES_CMD_TMP="${SORTED_PATCH_SITES_DEFAULT_CMD}" ; \
fi ; \
- for site in `eval $$SORTED_PATCH_SITES_CMD_TMP`; do \
+ for site in `eval $$SORTED_PATCH_SITES_CMD_TMP ${_RANDOMIZE_SITES}`; do \
DIR=${DIST_SUBDIR}; \
CKSIZE=`${GREP} "^SIZE ($${DIR:+$$DIR/}$$file)" ${MD5_FILE} | ${AWK} '{print $$4}'`; \
case $${file} in \
--I2DZnI03i1DS0lOz--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030210021953.GT15936>