From owner-freebsd-hubs@FreeBSD.ORG Wed Sep 5 19:23:31 2012 Return-Path: Delivered-To: hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EFA3106566B; Wed, 5 Sep 2012 19:23:31 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id BC2E78FC08; Wed, 5 Sep 2012 19:23:30 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id BE40D25D3887; Wed, 5 Sep 2012 19:23:29 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id CD3F8BE84BF; Wed, 5 Sep 2012 19:23:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id rEruHOjwu3yf; Wed, 5 Sep 2012 19:23:26 +0000 (UTC) Received: from nv.sbone.de (nv.sbone.de [IPv6:fde9:577b:c1a9:31::2013:138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id A02E3BE84BD; Wed, 5 Sep 2012 19:23:26 +0000 (UTC) Date: Wed, 5 Sep 2012 19:23:25 +0000 (UTC) From: "Bjoern A. Zeeb" To: Tim Bishop In-Reply-To: <20120905190901.GK66364@carrick-users.bishnet.net> Message-ID: References: <20120905112013.2d44783c@laptop> <09CB99A0-75BC-426C-BD44-9ACC7CD741D1@exonetric.com> <20551.35538.119912.329917@hergotha.csail.mit.edu> <20120905190901.GK66364@carrick-users.bishnet.net> X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: hubs@freebsd.org, Cluster Administrators Subject: Re: pkgng mirrors X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 19:23:31 -0000 On Wed, 5 Sep 2012, Tim Bishop wrote: > On Wed, Sep 05, 2012 at 06:35:48PM +0000, Bjoern A. Zeeb wrote: >> ... we need it, especially if run well (though the majority of mirrors >> sadly aren't run well); ... > > I run mirrorservice.org so I've interacted with quite a few open source And I noticed that service about two weeks ago and have a reminder to go back out of curiosity to see;-) > projects that have their stuff mirrored by third parties. The FreeBSD > system seems fairly primitive compared to others; whilst you have > various things to check the status of the mirrors there's nothing (as > far as I know) automatically happening with it. Other projects have > systems to automatically remove mirrors from listings if they're not > updating properly or are missing content. > > If FreeBSD was to do something similar you'd be able to retain a well > established mirror network whilst effectively weeding out broken sites. Part of the historic problem here is that we have little control over *.cc.FreeBSD.org currently though we gain more as time passes. So it's not that easy. We can remove them from website and release lists and I am not sure to which extend this happening but well worth investigating. I seem to remember that Release Engineering in the past had at least made sure that the mirrors mentioned in the release announcement had the bits at that time. The other problems would however equally remain - no statistics, maintainance, unable to "atomically" deploy things, and a couple of other stuff ... The fact that the world has moved by a decade but we did not may make these changes look more radical than they are. > That said, 10TB worth of packages? That's a lot and I can imagine most > mirrors couldn't handle that quantity of data. 7TB to start within the first year, yes, we are upping the game a little to give a lot better support to users we hope. It won't be there from day 1 but that's the direction thigns are heading. A lot of the mirrors can't cope with the 1G or 2G of ftp, and most of the old stuff is not on that anymore but on ftp-archive, so yeah... -- Bjoern A. Zeeb You have to have visions! Stop bit received. Insert coin for new address family.