Date: Sun, 21 Jul 2013 15:08:36 -0700 From: Peter Wemm <peter@wemm.org> To: hubs@freebsd.org Subject: FYI: Upcoming cvsup changes Message-ID: <CAGE5yCqbtGfFN_BGFjncD6%2BAhZ9knA8fL6cgorxmk=O-oopwaA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Right now, we have a rather fragile cvsup setup that's a bit painful to deal with. cvsup-master depends on a scan file for efficiency. So we have to double-cvsup internally to generate the scan files. It looks like this: www -> via cvsup(rsync mode) -> cvsup-in -> scan file -> cvsup-master -> mirrors www -> via rsync -> internal rsync mirror -> (not published) cvs -> via cvsup(cvs repo mode) -> cvsup-in -> scan file -> cvsup-master -> mirrors mail-> via cvsup(rsync mode) -> cvsup-in -> scan file -> cvsup-master -> mirrors mail -> via rsync -> internal rsync mirror -> (not published) gnats -> not in cvsup anymore, but it was like www/mail. gnats -> via rsync -> internal rsync -> semi-published on bit0 portsindex -> via rsync -> internal rsync -> semi-published on bit0 This means we have cvsupd on many more systems than we'd like. I'm planning to change it like this: www -> via rsync -> internal rsync mirror -> published mail -> via rsync -> internal rsync mirror -> published gnats -> via rsync -> internal rsync -> published portsindex -> via rsync -> internal rsync -> published and on a backend worker in the cluster: www -> rsync -> stage area mail -> rsync -> stage area gnats -> rsync -> stage area cvs(local file system) -> stage area then: stage area -> cvsupd (loopback) -> create scan file -> publish as cvsup-master. This should be mostly transparent for cvsup mirrors. As of today there is still some data that can only be obtained via cvsup and this change will finally fix it. After the change, cvsup will be 100% second-hand and/or converted data. cvsup has had a very good run, but this will likely be the final step in completing the deprecation of the system. Note that I did NOT say "turning off" - there is a fair bit of effort going into keeping it alive. This work is for sustaining it so it can wind down gracefully. But this is probably the last major chunk of work being spent on it at our end. (FYI, remember CTM? It's still going due to a madman^H^H^H^H^H^Hvolunteer. Said in the nicest possible way of course :) cvsup-master will likely be changing IP addresses but we'll reduce the dns TTL's to keep this as transparent as possible. Naming for the rsync infrastructure is internal right now but that'll be sorted out as part of this and documented. cvs tree status: 6.x (and earlier) hasn't been exported to cvs for quite a while. 7.x hasn't been exported to cvs for a few months. 8.x is still being exported once an hour. (including security support branches) 9.x is still being exported once an hour. (including security support branches) 10.x is not exported from svn to cvs. ports has not been exported since February. doc/www was never exported from svn to cvs. BTW; Some mirrors have been rapidly updating multiple times per hour. Please don't. Once an hour is more than enough - there will never be more than one cvsup-master update per hour. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV UTF-8: So you can \342\200\231 .. for when a ' just won't do <brueffer> ZFS must be the bacon of file systems. "everything's better with ZFS"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGE5yCqbtGfFN_BGFjncD6%2BAhZ9knA8fL6cgorxmk=O-oopwaA>