Date: Fri, 13 Jun 2003 16:24:09 -0400 From: Garance A Drosihn <drosih@rpi.edu> To: Ken Smith <kensmith@cse.Buffalo.EDU>, freebsd-hubs@freebsd.org Subject: Re: RFC - extra sync functionality... Message-ID: <p05210628bb0fdac75d02@[128.113.24.47]> In-Reply-To: <20030613141421.GD13868@electra.cse.Buffalo.EDU> References: <20030613141421.GD13868@electra.cse.Buffalo.EDU>
next in thread | previous in thread | raw e-mail | index | archive | help
At 10:14 AM -0400 6/13/03, Ken Smith wrote: > >Taking that view of things Jun and others have already started >to carve the FTP site into the equivalent of zones. The trick >though is coming up with the equivalent of what triggers a Zone >Transfer, which is a different serial number in the SOA for DNS. Well, let me mention another partially-formed idea that I have been thinking about... Another issue is how to get a consistent collection of what you are downloading, once you do start the download. I have been hit by cases where I cvsup the source tree, and I get *part* of some commit. Ie, when I go to do the buildworld, I get errors because I managed to get *some* files from some commit, but not *all* of the files that were changed by that single commit. From this comes the common advice "Just CVSUP again and the problem should go away". Let's say my machine does decide to start downloading. At the very time I'm downloading from some server, that server might be downloading from *its* source. Neither download can be an atomic operation, so the end user can end up with an inconsistent collection of files. It occurred to me that we could solve this by using snapshots on the mirrors (by this I mean the UFS-snapshots which are related to softupdates, and available in freebsd-5.x). When I connect to a mirror, I would not be downloading from "the real disk", but the mirror would instead connect me to a snapshot of the real disk. The mirror would only create a new snapshot when it was finished updating from it's source, so when any given snapshot is made the repository should be in a completely consistent state. Assuming that all works, the question was what event would the super-master source site use for creating it's snapshot. I was thinking it could create a snapshot, and then start the buildworld-testing processes. If the buildworld succeeds, then it would make *that* snapshot the one used for any subsequent connections to the server. (I am still a bit undecided about how all that should work, however...) I did start playing with some experiments along this line last year, but was tripped up by some bugs that were still in the UFS-snapshot code at that time. The relevance of this to your comments is that you would end up with a single "serial number" for an entire snapshot, and that would be *in* the snapshot. So, you don't need serial numbers for every file, you can have a single number for the entire repository. (or maybe break the single repository into subsets, and then have a serial number per subset). I realize this is another idea which would need a bit more fleshing-out before it would be practical to use, but I also think that it's a promising idea. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?p05210628bb0fdac75d02>