From owner-freebsd-hubs@FreeBSD.ORG Tue Apr 8 08:49:58 2008 Return-Path: Delivered-To: hubs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD8CA1065676 for ; Tue, 8 Apr 2008 08:49:58 +0000 (UTC) (envelope-from pav@FreeBSD.org) Received: from nat-application.b1.lan.prg.vol.cz (nat-application.b1.lan.prg.vol.cz [195.122.204.152]) by mx1.freebsd.org (Postfix) with ESMTP id 2C34A8FC12 for ; Tue, 8 Apr 2008 08:49:57 +0000 (UTC) (envelope-from pav@FreeBSD.org) Received: from pav.hide.vol.cz (localhost [127.0.0.1]) by nat-application.b1.lan.prg.vol.cz (8.14.2/8.14.2) with ESMTP id m388FmSq084461; Tue, 8 Apr 2008 10:15:48 +0200 (CEST) (envelope-from pav@FreeBSD.org) Received: (from pav@localhost) by pav.hide.vol.cz (8.14.2/8.14.2/Submit) id m388FmUE084460; Tue, 8 Apr 2008 10:15:48 +0200 (CEST) (envelope-from pav@FreeBSD.org) X-Authentication-Warning: pav.hide.vol.cz: pav set sender to pav@FreeBSD.org using -f From: Pav Lucistnik To: Eberhard Moenkeberg In-Reply-To: References: <1207605059.1031.38.camel@ikaros.oook.cz> <20080408002005.W15502@fledge.watson.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-qF1+FDhiuoVWGcf7/PI/" Date: Tue, 08 Apr 2008 10:15:48 +0200 Message-Id: <1207642548.84150.10.camel@pav.hide.vol.cz> Mime-Version: 1.0 X-Mailer: Evolution 2.12.3 FreeBSD GNOME Team Port Cc: hubs@FreeBSD.org Subject: Re: package distribution crisis - CDN needed X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pav@FreeBSD.org List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2008 08:49:58 -0000 --=-qF1+FDhiuoVWGcf7/PI/ Content-Type: text/plain; charset=ISO8859-2 Content-Transfer-Encoding: quoted-printable Eberhard Moenkeberg p=ED=B9e v =FAt 08. 04. 2008 v 01:54 +0200: > > > I do propose a creation of a CDN (Content Delivery Network), having t= hese > > > features: > >=20 > > The success of a CDN is premised on the working set of active files bei= ng > > [significantly] smaller than the set of available files. While I don't= doubt > > that only a few packages are of interest to most users, for a CDN to pr= ovide a > > real benefit it needs to be the case that no users require most files. = The > > reason being that, if you have to transfer all or even most files every= time, > > a CDN may well be a significantly less efficient way to do it than rsyn= c. Has > > a working set analysis for our main mirrors been done that demonstrates= that > > the actual working set is sufficiently smaller than the total set that = a CDN > > would be more, rather than less, efficient? I don't have these statistics, but I'd like to see them very much. Can I ask some of the mirror admins to compile me statistics on downloads of files from /pub/FreeBSD/ports/*packages*, say for March 2008 ? I'd also like to have these numbers from ftp.FreeBSD.org so I have a comparision of volume. My suspicion is that 99% of users get their packages from ftp.FreeBSD.org, because that's what pkg_add -r uses without PACKAGEROOT specification. > Maybe it is interesting for you to have a look at=20 >=20 > http://www.poeml.de/~poeml/talks/redirector/ >=20 > The SUSE team has a redirection mechanism at work on single file basis,=20 > and they also have experienced that it would be "enough" for the users if= =20 > there are lots of mirrors with the "most requested" files plus some=20 > mirrors with all files. Now this looks so much better and easier than my CDN idea! We could use it practically as-is. We will need to patch pkg_install to use http://redirector.FreeBSD.org/ instead of ftp://ftp.FreeBSD.org/ --=20 Pav Lucistnik Logitech is infamous for providing 30 MB compressed driver software packages that install a bunch of crap applications on the system and round up the number of icons in the tray to the nearest power of two. -- Michael Stromberg --=-qF1+FDhiuoVWGcf7/PI/ Content-Type: application/pgp-signature; name=signature.asc Content-Description: Toto je =?UTF-8?Q?digit=C3=A1ln=C4=9B?= =?ISO-8859-1?Q?_podepsan=E1?= =?UTF-8?Q?_=C4=8D=C3=A1st?= =?ISO-8859-1?Q?_zpr=E1vy?= -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (FreeBSD) iEYEABECAAYFAkf7Ka4ACgkQntdYP8FOsoLkmACeNDmkKboZZemXwZN3Fb+gFICm Gi4AoKl7Dk8XM36bJAbRU1jnCl/eA2Y8 =Y101 -----END PGP SIGNATURE----- --=-qF1+FDhiuoVWGcf7/PI/--