Date: Mon, 10 Mar 2003 01:58:57 -0500 From: "Chris Demers" <admin@govital.net> To: jason andrade <jason@rtfmconsult.com>, Kris Kennaway <kris@obsecurity.org> Cc: "David O'Brien" <obrien@FreeBSD.ORG>, "" <hubs@FreeBSD.ORG> Subject: Re: Poor state of some top-level FTP mirrors Message-ID: <20030310064332.M84136@govital.net> In-Reply-To: <Pine.GSO.4.50.0303101139360.41-100000@luna.rtfmconsult.com> References: <20030309215448.GB30033@dragon.nuxi.com> <Pine.GSO.4.50.0303101116080.41-100000@luna.rtfmconsult.com> <20030310013355.GA70336@rot13.obsecurity.org> <Pine.GSO.4.50.0303101139360.41-100000@luna.rtfmconsult.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Greetings, On Mon, 10 Mar 2003 11:44:08 +1000 (EST), jason andrade wrote > On Sun, 9 Mar 2003, Kris Kennaway wrote: > > > I suspect you're wrong. Can you provide supporting evidence? > > not really - i should try and go back to some logs we kept, but > most of the evidence was anecdotal by asking on this list how > package trees are maintained. > > in any case, the up to dateness of mirrors is quite an important > issue and i guess worth pursuing. a lot of the other projects > either have systems in place or are also looking at putting > systems in place to provide verification and feedback on > mirrors that fall out of date. and (i am speaking out loud) > with the efforts in organizing a US master mirror, an EU > master mirror and possible one other, it is starting to again > look good for organizing tiers of mirrors to spread out the > load of updating and hopefully making it more reliable/faster. > > we stopped updating the packages tree daily at one point where it > reached the stage that the first update was not completing before > the next one started to delete files already - this was some > months ago though. > > regards, > > -jason > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hubs" in the body of the message I don't normally say too much around here but i thought i would throw in my two cents. :-) I took a quick look around to find out just how this was being done and i couldn't find it. So i'm going to do some assuming and make some suggestions. Assuming a full package build is normally just pushed out for distribution, this would cause much unneeded bandwidth usage among all of the mirror sites. Wouldn't it be just better to maybe keep a record of the fingerprint of a port and if it matched from the last build not send it out to the mirrors so that only things that have been updated get pushed out? The only problem with this method is that it would slow down the build system a bit while it checks the files. Or maybe instead of just blindly building the entire collection all the time, keep a database of the versions that have been built and only build the ones that change. There would have to be some extra checking in there too to make sure that all dependacys affected by a version change of any port also get rebuilt, but that method would probably work better as the build system wouldn't have to work as hard all the time to churn out packages for ports that rarely change. And then just do a full build from time to time to double check everything. Just some ideas... -- Chris Demers admin@govital.net www.govital.net www.govitalhosting.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hubs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030310064332.M84136>