Date: Sat, 18 Jun 2016 18:50:52 -0700 From: Jordan Hubbard <jkh@ixsystems.com> To: Chris Watson <bsdunix44@gmail.com> Cc: Rick Macklem <rmacklem@uoguelph.ca>, freebsd-fs <freebsd-fs@freebsd.org>, Alexander Motin <mav@freebsd.org> Subject: Re: pNFS server Plan B Message-ID: <B2907C1F-D32A-48FB-8E58-209E6AF1E86D@ixsystems.com> In-Reply-To: <7E27FA25-E18F-41D3-8974-EAE1EACABF38@gmail.com> References: <1524639039.147096032.1465856925174.JavaMail.zimbra@uoguelph.ca> <D20C793E-A2FD-49F3-AD88-7C2FED5E7715@ixsystems.com> <7E27FA25-E18F-41D3-8974-EAE1EACABF38@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Jun 18, 2016, at 6:14 PM, Chris Watson <bsdunix44@gmail.com> wrote:
>=20
> Since Jordan brought up clustering, I would be interested to hear =
Justin Gibbs thoughts here. I know about a year ago he was asked on an =
"after hours" video chat hosted by Matt Aherns about a feature he would =
really like to see and he mentioned he would really like, in a universe =
filled with time and money I'm sure, to work on a native clustering =
solution for FreeBSD. I don't know if he is subscribed to the list, and =
I'm certainly not throwing him under the bus by bringing his name up, =
but I know he has at least been thinking about this for some time and =
probably has some value to add here.=20
I think we should also be careful to define our terms in such a =
discussion. Specifically:
1. Are we talking about block-level clustering underneath ZFS (e.g. HAST =
or ${somethingElse}) or otherwise incorporated into ZFS itself at some =
low level? If you Google for =E2=80=9CHigh-availability ZFS=E2=80=9D =
you will encounter things like RSF-1 or the somewhat more mysterious =
Zetavault (http://www.zeta.systems/zetavault/high-availability/) but =
it=E2=80=99s not entirely clear how these technologies work, they simply =
claim to =E2=80=9Cscale-out ZFS=E2=80=9D or =E2=80=9Ccluster ZFS=E2=80=9D =
(which can be done within ZFS or one level above and still probably pass =
the Marketing Test for what people are willing to put on a web page).
2. Are we talking about clustering at a slightly higher level, in a =
filesystem-agnostic fashion which still preserves filesystem semantics?
3. Are we talking about clustering for data objects, in a fashion which =
does not necessarily provide filesystem semantics (a sharding database =
which can store arbitrary BLOBs would qualify)?
For all of the above: Are we seeking to be compatible with any other =
mechanisms, or are we talking about a FreeBSD-only solution?
This is why I brought up glusterfs / ceph / RiakCS in my previous =
comments - when talking to the $users that Rick wants to involve in the =
discussion, they rarely come to the table asking for =E2=80=9Csome or =
any sort of clustering, don=E2=80=99t care which or how it works=E2=80=9D =
- they ask if I can offer an S3 compatible object store with horizontal =
scaling, or if they can use NFS in some clustered fashion where =
there=E2=80=99s a single namespace offering petabytes of storage with =
configurable redundancy such that no portion of that namespace is ever =
unavailable.
I=E2=80=99d be interested in what Justin had in mind when he asked Matt =
about this. Being able to =E2=80=9Cattach ZFS pools to one another=E2=80=9D=
in such a fashion that all clients just see One Big Pool and ZFS=E2=80=99=
s own redundancy / snapshotting characteristics magically apply to the =
=C3=BCberpool would be Pretty Cool, obviously, and would allow one to do =
round-robin DNS for NFS such that any node could serve the same =
contents, but that also sounds pretty ambitious, depending on how it=E2=80=
=99s implemented.
- Jordan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B2907C1F-D32A-48FB-8E58-209E6AF1E86D>