From owner-freebsd-fs@FreeBSD.ORG Tue Feb 28 07:32:46 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D3641065675; Tue, 28 Feb 2012 07:32:46 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-2.mit.edu (DMZ-MAILSEC-SCANNER-2.MIT.EDU [18.9.25.13]) by mx1.freebsd.org (Postfix) with ESMTP id 24D918FC0C; Tue, 28 Feb 2012 07:32:41 +0000 (UTC) X-AuditID: 1209190d-b7fbf6d0000008ba-40-4f4c7f93a315 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) by dmz-mailsec-scanner-2.mit.edu (Symantec Messaging Gateway) with SMTP id BE.91.02234.39F7C4F4; Tue, 28 Feb 2012 02:17:39 -0500 (EST) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id q1S7HdM2020938; Tue, 28 Feb 2012 02:17:39 -0500 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q1S7HbKf022360 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 28 Feb 2012 02:17:38 -0500 (EST) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id q1S7HaU1015734; Tue, 28 Feb 2012 02:17:36 -0500 (EST) Date: Tue, 28 Feb 2012 02:17:36 -0500 (EST) From: Benjamin Kaduk To: Lawrence Stewart In-Reply-To: <4F4C5CCB.3050806@freebsd.org> Message-ID: References: <3CEE2DA4348D944399A67E308B78D38A1A57CABA@janus.anserinae.net> <4F4C5CCB.3050806@freebsd.org> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrLIsWRmVeSWpSXmKPExsUixG6nrju53sffoPmtjsWxxz/ZLOYf2M5o sflWF7MDs8ebH1/YPGZ8ms8SwBTFZZOSmpNZllqkb5fAlXHt7km2ghd8FY97/7A0ML7h7mLk 5JAQMJF4fekbO4QtJnHh3nq2LkYuDiGBfYwSh1+/hHI2MEpsfnaJGcI5wCSxY/0pJgingVFi Re9ZRpB+FgFtiev/v7KA2GwCKhIz32xkA7FFgOLbzn5m7WLk4GAWyJdYtbYUxBQW8JM40xgL UsEJVNF87zYTiM0r4Cjx4+ZEZhBbSCBTomnCQ7DpogI6Eqv3T2GBqBGUODnzCZjNLGApce7P dbYJjIKzkKRmIUktYGRaxSibklulm5uYmVOcmqxbnJyYl5dapGukl5tZopeaUrqJERSynJK8 OxjfHVQ6xCjAwajEw1s8y9tfiDWxrLgy9xCjJAeTkiivT62PvxBfUn5KZUZicUZ8UWlOavEh RgkOZiURXv1soHLelMTKqtSifJiUNAeLkjivqtY7PyGB9MSS1OzU1ILUIpisDAeHkgTvvjqg oYJFqempFWmZOSUIaSYOTpDhPEDDNUBqeIsLEnOLM9Mh8qcYdTkWLN98kVGIJS8/L1VKnPcU SJEASFFGaR7cHFiqecUoDvSWMO9KkCoeYJqCm/QKaAkT0JIATpAPiksSEVJSDYw+3IfvGIXM Yw2cOd8z80n97P0rKtZJH60tLCtp+LHzKUfp2dwuxbK+1uWrfVj5ttnZcVWFLln8aeO0/x9P my3x8X/IeVXmBkPkV9kV21iXrolz7tOcf8WxtMwg4i7Lol92EVLSpjF6ngHBfqIdXZZdN2z5 W7SkUyszznPKhdqLLdftL2yQVWIpzkg01GIuKk4EAKhWBHkQAwAA Cc: "freebsd-fs@freebsd.org" , Kamil Choudhury Subject: Re: Distributed, snapshotting, checksumming filesystems for FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Feb 2012 07:32:46 -0000 On Tue, 28 Feb 2012, Lawrence Stewart wrote: > On 02/25/12 15:37, Kamil Choudhury wrote: >> The dream: a file system spread out over a variable, >> ever increasing number of hosts, presenting a single >> unified file system to any client host mounting the >> file system. >> >>> From the client's point of view, it is possible to >> snapshot the directory view that is presented. The >> client also has confidence that data written to the >> file system will be returned exactly as it went in. >> >> Now that I think about it, what I seem to be looking >> for is a network aware ZFS that uses hosts as vdevs. >> >> Is there such a thing out there? > > AFS perhaps? > > http://en.wikipedia.org/wiki/OpenAFS > http://en.wikipedia.org/wiki/Andrew_File_System > > /usr/ports/net/openafs I almost made that suggestion, but decided not to, given that the present implementation only allows administrators to make snapshots, and I don't think any checksumming would be quite what was described. (One could certainly use a ZFS partition for the fileserver backing store, which would give some checksum functionality, but it would not be exposed to users in a particularly useful fashion.) But, it does win at having a single unified file system presented to any client mounting it, which is distributed over multiple servers (and data can be shuffled between servers seamlessly). If the OP can compromise in the right directions it might be worth looking at, though, I suppose. I'll also add that the OpenAFS client is not production-ready on FreeBSD; there are some memory leaks and issues with cache eviction for large files that are not yet resolved. Unfortunately there have been a couple of regressions on the git master branch that I need to track down before I can get back to working on the FreeBSD-specific issues. -Ben Kaduk