Date: Tue, 26 Feb 2013 17:34:37 +1100 From: Aristedes Maniatis <ari@ish.com.au> To: Alexander Yerenkow <yerenkow@gmail.com>, Dewayne Geraghty <dewayne.geraghty@heuristicsystems.com.au> Cc: freebsd-ports@freebsd.org Subject: Re: Share /var/cache/pkg/ between machines Message-ID: <512C577D.6060704@ish.com.au> In-Reply-To: <CAPJF9wkeHqVzTZ7_Mzo7eYbPqcqs1khcap2SHw2a=sV5kwSJ2A@mail.gmail.com> References: <512C249B.4090401@ish.com.au> <4749ABEC867549DAB07179A7E71CDB4F@white> <CAPJF9wkeHqVzTZ7_Mzo7eYbPqcqs1khcap2SHw2a=sV5kwSJ2A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 26/02/13 5:22pm, Alexander Yerenkow wrote: > I don't understand how do you imagine "magical appearing" 80 Mb JDK from any one place to other without pulling data from internet? :) > In any case, with any kind of sharing you probably will have almost same traffic, maybe even greater than if you choose simple `pkg install` traffic. Perhaps I oversimplified our setup. Here is what we have: - pkg.office.example.com (poudriere): builds all packages (pkgng). Runs apache to serve repository to other servers. -- server1.office.example.com pulls files from build server (PACKAGESITE points to pkg.office.example.com) -- server2.office.example.com, etc.. as above office <------ VPN tunnel ------> colo (data centre) - pkg.colo.example.com also pulls files from build server (PACKAGESITE points to pkg.office.example.com) -- server[2-10].colo.example.com... rather than pull packages over the slow VPN tunnel, I want to pull them from pkg.colo.example.com Now, one approach is to rsync the poudriere output folder on pkg.office.example.com to pkg.colo.example.com. And given the complexity of other options, that's probably what I'll do. It means more network traffic than optimal, because poudriere will bulk many packages which aren't actually needed in the colo (and more often than the colo machines are actually updated as well since poudriere in the office might do a run twice a week, but the colo is updated once every 3 months or when there is a security patch we care about). So I was hoping to rely on "pkg install" to pull the package over the VPN network, and then replicate the data out to the other servers (NFS/rsync/etc) in the colo. Sorry for confusing you with my abbreviated problem description. Hopefully this is clearer. Ari > 1 - pointless, you can set up simple nginx, pointing to poudriere build packages directory and set up packagesite in /u/l/e/pkg.conf, without any proxy machine, which must download these cached packages in any case to be served by apache. AFAIK that machine will download something only while installing, right? So you overcomplicate your setup. > > 2. rsync will produce packagesize traffic + constant little more, due to checks of alteration. You'll lose in that case in long shot. > > My advice - put nginx to serve poudriere's built packages dir, and trust to xz compression level = easy to setup, not big traffic really. > This is of course based on sentence "... onto 10 machines across the internet" - if you have some clusters with PCs in some nets, then you could think about rsync with primary poudriere, to serve locally packages for few other PCs. > > Hope this help :) > > -- > Regards, > Alexander Yerenkow -- --------------------------> Aristedes Maniatis ish http://www.ish.com.au Level 1, 30 Wilson Street Newtown 2042 Australia phone +61 2 9550 5001 fax +61 2 9550 4001 GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?512C577D.6060704>