From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 22 00:57:56 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75E5F106566C for ; Sat, 22 Mar 2008 00:57:56 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id 73F0B8FC1E for ; Sat, 22 Mar 2008 00:57:55 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [202.108.54.204]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id 64E082845A for ; Sat, 22 Mar 2008 08:57:53 +0800 (CST) Received: from localhost (tarsier.geekcn.org [202.108.54.204]) by tarsier.geekcn.org (Postfix) with ESMTP id D2DE6EBBFFB; Sat, 22 Mar 2008 08:57:52 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([202.108.54.204]) by localhost (mail.geekcn.org [202.108.54.204]) (amavisd-new, port 10024) with ESMTP id tXf5pNGoWGbo; Sat, 22 Mar 2008 08:57:47 +0800 (CST) Received: from charlie.delphij.net (71.5.7.139.ptr.us.xo.net [71.5.7.139]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id DC7AAEB572B; Sat, 22 Mar 2008 08:57:45 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=rwS4LLpswS+fl7ZLk3wEj5irmdxghRXT3xz6KcERD5+LkduKSDMJOS4IN0y3ii4Hk 6mEJ03U5fFvoi4rWw6kZw== Message-ID: <47E45984.3080907@delphij.net> Date: Fri, 21 Mar 2008 17:57:40 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.12 (X11/20080312) MIME-Version: 1.0 To: Giorgos Keramidas References: <47E4405E.2080609@delphij.net> <874pazpqjq.fsf@kobe.laptop> In-Reply-To: <874pazpqjq.fsf@kobe.laptop> X-Enigmail-Version: 0.95.6 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, d@delphij.net Subject: Re: Some versioned storage program? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2008 00:57:56 -0000 Giorgos Keramidas wrote: > On Fri, 21 Mar 2008 16:10:22 -0700, Xin LI wrote: >> Hi, folks, >> >> I'm looking for some versioned storage program that can fulfill the >> following requirements: >> >> - Open source/Free Software that can run on FreeBSD, or not far >> (i.e. on other POSIX OS) >> - Support of atomic commit/rollback. >> - Fast checkin time (At least, when added/changed files are explicitly >> specified). >> - Fast update time (i.e. something like 'cvsup -s' that makes it >> possible to trust bookkeeping file rather than stat'ing every files) >> - Scalable for a large number of files, directories and revisions. Say, >> it is not acceptable for it to store a zillion of revisions as >> individual files within one directory. >> - Ideally it can support some sort of "hook" functions upon commit so >> that changes can be notified in some way such as e-mail. >> - Ideally it can support fast export of a snapshot for HEAD and >> "nearby" revision like HEAD - 1, etc. >> >> I think what I need is some SCM software like subversion or hg, but I do >> not know if there is some superior stuff that matches these requirements >> better. Any other suggestions? > > Before you start using Hg, Git or Subversion it may be worth > experimenting a bit with them. My apologies if you _have_ already and > the previous sentence sounds patronising. All I'm saying is that they > all have a fair share of good, not so good, or even bad aspects. So it > would be nice to have tried them all a bit and pick the one that seems > like the best fit for the job at hand :) Ah... Ok I think I should have mentioned the purpose of the system. It is supposed to be used in a CMS system, and the storage program will be used as one auxiliary backend where rendered pages are being stored. > To provide a few starting points: > > - Subversion, Git and Hg, all run on FreeBSD > - They support 'changesets' as the basic model of storing commits > - Commit speed varies a bit. For locally stored 'workspaces', Git and > Hg seem to be more or less equally fast, with Subversion being a close > second > - Update times tend to vary a bit too. Hg and Git will blow Subversion > away on locally stored repositories, but they might suck a bit on NFS > workspaces > - Storing individual revisions as 'a zillion directory entries in a > single tree' seem to point at Subversion. Have you already tried it, > and found that it doesn't scale for your sort of work? It is used by > many large-ish projects, so it would be surprising but not unrealistic > to have scalability issues after a few million commits > - Hooks _are_ supported by Subversion, Git and Hg (others too) > - Checkout speed (and `export' speed) is pretty fast in Git and Hg. > Subversion is a bit slower, but still usable. Changeset support is a > nice feature, because it doesn't matter if your `export' run takes 1.5 > minutes instead of 20 seconds. When a given changeset is exported in > any of svn/git/hg you _never_ get a mix of file revisions from > changesets ${FOO} ... ${FOO+j} for some arbitrarily random value of > 'j', because 'j+k' commits happened in the mean time. > > Before you _do_ embark on the journey of using a VCS for storing a bunch > of files, it would be nice to stop for a moment and consider if you need > one. If you do, there _are_ options, and they are definitely not > limited to the three systems mentioned so far. Thanks for all these information. I have tried svn and hg but neither is "just fit" and both have good stuff and drawbacks. I have even tried to use cvsup/cvs in a small system when I was in university and it served them well for many years, however I think it would not work well for larger systems. For now I am more inclined to use hg (if not using some home grown system as this is not exactly source code version management and a lot of complexity can be thus just skipped) but I think I need to find out how well it would work with 'pull+update' on large repositories, the largest hg repository I am aware of right now, is less than 1GB and the potential size of the repository I would use will be larger than that and grow from time to time. I think it's a good point that speed would vary when the repository is being stored locally (git and hg) and remotely (svn), also speed of synchronization between several hosts would be important as well. Cheers, -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve!