Date: Tue, 21 Jun 2016 17:54:44 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Jordan Hubbard <jkh@ixsystems.com> Cc: Doug Rabson <dfr@rabson.org>, freebsd-fs <freebsd-fs@freebsd.org>, Alexander Motin <mav@freebsd.org> Subject: Re: pNFS server Plan B Message-ID: <901639916.165261477.1466546084028.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <74CD7EB1-1656-4511-8B63-5C4401D1BB8D@ixsystems.com> References: <1524639039.147096032.1465856925174.JavaMail.zimbra@uoguelph.ca> <D20C793E-A2FD-49F3-AD88-7C2FED5E7715@ixsystems.com> <CACA0VUibM1giAkJdNNkn1_m8QqqLzdNC86hFhRxMmY7gMb1nvg@mail.gmail.com> <74CD7EB1-1656-4511-8B63-5C4401D1BB8D@ixsystems.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Jordan Hubbard wrote:
> OK, wow. This appears to have turned into something of a referendum on N=
FS
> and, just based on Rick and Doug=E2=80=99s defense of pNFS, I also think =
my
> commentary on that may have been misconstrued somewhat.
>=20
Actually, I thought it had become a referendum on LDAP;-)
As for defending pNFS, all I was trying to say was that "although it is har=
d
to believe, it has taken 10years for pNFS to hit the streets". As such, it
is anyone's guess w.r.t. whether or not it will become widely adopted?
If it came across as more than that, I am the one that should be apologizin=
g
and am in no way discouraged by any of the comments.
> So, let me just set the record straight by saying that I=E2=80=99m all in=
favor of
> pNFS. It addresses a very definite need in the Enterprise marketplace an=
d
> gives FreeBSD yet another arrow in its quiver when it comes to being =E2=
=80=9Ca
> player=E2=80=9D in that (ever-growing) arena. The only point I was tryin=
g to make
> before was that if we could ALSO address clustering in a more general way=
as
> part of providing a pNFS solution, that would be great.
When I did a fairly superficial evaluation of the open source clustering sy=
stems
out there (looking at online doc and not actually their code), it seemed th=
at
GlusterFS was the best bet for "one size fits all".
It had:
- a distributed file system (replication, etc) with a POSIX/FUSE interface.
- SwiftOnFile that put the Swift/Openstack on top of this.
- It had decentralized metadata handling.
For pNFS:
- It had a NFSv3 server built into it.
- Was ported to FreeBSD.
The others were:
- Object store only with no POSIX file system support
or
- Single centralized metadata store (MooseFS, for example)
- No FreeBSD port and rumoured to be hard to port (Ceph, Lustre are two exa=
mples).
Now that I've worked with GlusterFS a little bit, I am skeptical that it ca=
n
deliver adequate performance for pNFS using the nfsd. I am still hoping I w=
ill
be proven wrong on this, but???
A GlusterFS/Ganesha-NFS user space solution may be feasible. This is what t=
he
GlusterFS folk are planning. However, for FreeBSD...
- Ganesha-NFS apparently was ported to FreeBSD, but the port was removed fr=
om
their source tree and it is said it now uses Linux-specific thread primit=
ives.
--> As such, I have no idea what effort is involved in getting this porte=
d and
working well on FreeBSD is.
- I would also wait until this is working in Linux and would want to do an
evaluation of that, to make sure it actually works/performs well, before
considering this.
*** For me personally, I am probably not interested in working on this. I
know the FreeBSD nfsd kernel code well and can easily work with that,
but Ganesha-NFS would be an entirely different beast.
Bottom line, at this point I am skeptical that a generic clustering system
will work for pNFS.
> I am not, however,
> the one writing the code and if my comments were in any way discouraging =
to
> the folks that are, I apologize and want to express my enthusiasm for it.
> If iXsystems engineering resources can contribute in any way to moving th=
is
> ball forward, let me know and we=E2=80=99ll start doing so.
>=20
Well, although they may not be useful for building a pNFS server, but some =
sort
of evaluation of the open source clustering systems might be useful.
Sooner or later, the Enterprise marketplace may want one or more of these a=
nd
it seems to me that having one of these layered on top of ZFS may be an att=
ractive
solution.
- Some will never be ported to FreeBSD, but the ones that are could probabl=
y be
evaluated fairly easily, if you have the resources.
Since almost all the code I've written gets reused if I do a PlanB, I will
probably pursue that, leaving the GlusterFS interface bits in place in case
they are useful.
Thanks for all the interesting comments, rick
> On the more general point of =E2=80=9CNFS is hard, let=E2=80=99s go shopp=
ing=E2=80=9D let me also say
> that it=E2=80=99s kind of important not to conflate end-user targeted sol=
utions with
> enterprise solutions. Setting up a Kerberized NFSv4, for example, is not
> really designed to be trivial to set up and if anyone is waiting for that=
to
> happen, they may be waiting a very long time (like, forever). NFS and SM=
B
> are both fairly simple technologies to use if you restrict yourself to
> using, say, just 20% of their overall feature-sets. Once you add ACLs,
> Directory Services, user/group and permissions mappings, and any of the
> other more enterprise-centric features of these filesharing technologies,
> however, things rapidly get more complicated and the DevOps people who
> routinely play in these kinds of environments are quite happy to have all
> those options available because they=E2=80=99re not consumers operating i=
n consumer
> environments.
>=20
> Sun didn=E2=80=99t design NFS to be particularly consumer-centric, for th=
at matter,
> and if you think SMB is =E2=80=9Csimple=E2=80=9D because you clicked Netw=
ork on Windows
> Explorer one day and stuff just automagically appeared, you should try
> operating it in a serious Windows Enterprise environment (just flip throu=
gh
> some of the SMB bugs in the FreeNAS bug tracker -
> https://bugs.freenas.org/projects/freenas/issues?utf8=3D=E2=9C=93&set_fil=
ter=3D1&f%5B%5D=3Dstatus_id&op%5Bstatus_id%5D=3D*&f%5B%5D=3Dcategory_id&op%=
5Bcategory_id%5D=3D%3D&v%5Bcategory_id%5D%5B%5D=3D57&f%5B%5D=3D&c%5B%5D=3Dt=
racker&c%5B%5D=3Dstatus&c%5B%5D=3Dpriority&c%5B%5D=3Dsubject&c%5B%5D=3Dassi=
gned_to&c%5B%5D=3Dupdated_on&c%5B%5D=3Dfixed_version&group_by=3D
> - if you want to see the kinds of problems users wrestle with all the tim=
e).
>=20
> Anyway, I=E2=80=99ll get off the soapbox now, I just wanted to dispute th=
e premise
> that =E2=80=9Csimple file sharing=E2=80=9D that is also =E2=80=9Csecure f=
ile sharing=E2=80=9D and =E2=80=9Cflexible
> file sharing=E2=80=9D doesn=E2=80=99t really exist. The simplest end-use=
r oriented file
> sharing system I=E2=80=99ve used to date is probably AFP, and Apple has b=
een trying
> to kill it for years, probably because it doesn=E2=80=99t have all those =
extra knobs
> and Kerberos / Directory Services integration business users have been
> asking for (it=E2=80=99s also not particularly industry standard).
>=20
> - Jordan
>=20
>=20
>=20
> _______________________________________________
> freebsd-fs@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?901639916.165261477.1466546084028.JavaMail.zimbra>