Date: Tue, 6 Jan 2009 13:08:02 +0100 From: Daniel Bond <db@danielbond.org> To: Christopher Arnold <chris@arnold.se> Cc: freebsd-stable@freebsd.org Subject: Re: freebsd-update painfully slow - slower than source code build of world and kernel Message-ID: <08B216B4-79AB-45AB-AB4D-C8CD62196B87@danielbond.org> In-Reply-To: <alpine.BSF.2.00.0901061242120.1996@localhost> References: <DA7E7739-0631-4B00-8CA8-D8C9E22B2126@danielbond.org> <alpine.BSF.2.00.0901061121530.1996@localhost> <F6385D88-BFAF-47A0-B598-78C971FCBD7C@danielbond.org> <alpine.BSF.2.00.0901061242120.1996@localhost>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi again Christopher, reading your answer, you are obviously confusing what I am saying about freebsd-update with the portsnap program. Also, I also wrote in my first post that HTTP_PROXY / Caching proxy server does not help me much. This is because I download a lot of "initial tarball snapshots".. I would rarely see "Cache hits" in my proxy log. I guess I could set something up to fetch nightly via proxy, to keep the data in house, for when I need it. I don't want to use a PROXY server, I feel this is attacking the problem at the wrong end. I agree, I am interested to hear the views of the wise ones. Personally I'm going back to CVSup until freebsd-update and portsnap mirrors are in a more distributed or usable state. Cheers. On Jan 6, 2009, at 12:59 PM, Christopher Arnold wrote: > > > On Tue, 6 Jan 2009, Daniel Bond wrote: > >> Regarding portsnap in my previous post, I think you misunderstood >> me. This is not a new "one time" problem regarding a specific case, >> portsnap is allways slow. This is observed from heavy usage of it, >> over a long period of time. >> > This is not my experience, but shure i realise that mileages can vary. > >> Great to see that there will be an update2.freebsd.org - >> unfortunately, that a new release generates more traffic than >> update-server handles is not acceptable (imho). People should be >> able to upgrade to a new release, once it is out. Sadly, I don't >> think one more mirror will cut it. Especially if it is going to be >> of the same quality as the other portsnap mirrors. Also, sadly CP >> isn't looking for more mirrors, while a large chunk of users trying >> to upgrade *are* looking for mirrors. >> > portsnap is extremly lightweight, so it might be just fine. > > But then i am not arguing against you, more and better > infrastructure is always good. Lets wait untill the us has woken up > (And maybe add some extra time for the right person to look into the > current problems) and see what kind of feedback we get from people > who have more insight into this issue. > >> Look at CVSUP mirrors, they have always worked fine, even directly >> after a new release. We even have a few of them here in Norway, and >> they are fast as hell. Look how many there are of them, spread >> around the world.. This works out great! >> > My experience from when i was based in Sweden is the opposit. > Shortly after a major release cvsup always had problems syncing due > to the load on the servers. > >> However, freebsd-update is closed. I've searched the web for how >> the protocol works, how the server-part of it works, with metadata, >> checksums and all. How the mirroring of it works, basicly. There >> are no public available documents on this. Do we have to reverse- >> engineer it, or what? >> > If we start off with portsnap it is http-based and the in the manual > you can find: > "If you wish to use portsnap to keep a large number of machines up > to date, you may wish to set up a caching HTTP proxy. Since > portsnap uses fetch(1) to download updates, setting the HTTP_PROXY > environment variable will direct it to fetch updates from the given > proxy. This is much more efficient than mirroring the files on the > portsnap server, since the vast majority of files are not needed by > any particular client." > > So it's straight forward to speed up portsnap. (But then if the > central servers break like today this dosn't help.) > > Im not shure about freebsd-update, but since they are both written > by Colin and the fact that they seem simmilar in config etc. i would > guess that the same applies to freebsd-update. > > So lets wait for some input from Colin or someone else who know the > ins and outs of freebsd-update. > > /Chris > > -- > http://www.arnold.se/chris/ >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?08B216B4-79AB-45AB-AB4D-C8CD62196B87>