Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 20 Dec 2018 11:03:59 +0000
From:      Bob Bishop <rb@gid.co.uk>
To:        Matthew Macy <mmacy@freebsd.org>
Cc:        Steven Hartland <killing@multiplay.co.uk>, freebsd-current <freebsd-current@freebsd.org>, freebsd-fs <freebsd-fs@freebsd.org>
Subject:   Re: The future of ZFS in FreeBSD
Message-ID:  <B527B90E-9C5E-4DC5-8DE5-35A85221BA64@gid.co.uk>
In-Reply-To: <CAPrugNpU2nnyuxetwvgeHXVNKRiDMk3uO-0OSBZ5nK55PKpY8A@mail.gmail.com>
References:  <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com> <CAHEMsqbhdcy1GKznVhxKAqCabYHvm_G9oxQR6eQrJ372215E0w@mail.gmail.com> <CAPrugNpr1YCo%2Br%2BCCts-uEfdU-nOF6-g15eqODJa_nKia_PwWw@mail.gmail.com> <CAHEMsqaJERWM_HpD4ru8Jj6QhVEb_CoOotzH1oPuYa95AoqWeA@mail.gmail.com> <CAPrugNpU2nnyuxetwvgeHXVNKRiDMk3uO-0OSBZ5nK55PKpY8A@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
Hi,

> On 19 Dec 2018, at 23:16, Matthew Macy <mmacy@freebsd.org> wrote:
>=20
> On Wed, Dec 19, 2018 at 15:11 Steven Hartland =
<killing@multiplay.co.uk>
> wrote:
>=20
>> Sorry been off for a few weeks so must have missed that, please do =
prod me
>> on again if you don=E2=80=99t see any response to anything not just =
this. Like many
>> others I get so may emails across so many lists it=E2=80=99s more =
than likely I
>> just missed it.
>>=20
>> That said would you say that with the right support we can make =
progress
>> on the this prior to the port? I have to ask as the alternative =
version has
>> been on the cusp for many years now so it=E2=80=99s feels more like a =
distant
>> memory than something that may happen, no disrespect to anyone =
involved, as
>> I know all too well how hard it can be to get something like this =
over the
>> line, especially when people have competing priorities.
>>=20
>=20
> I am hoping that it's sufficiently important to FreeBSD ZFS developers =
that
> they'll give the PR the attention it needs so that it can be merged =
before
> summer. My understanding is that it's mostly suffered from neglect. =
TRIM is
> most important to FreeBSD and it already had its own implementation.
>=20
> https://github.com/zfsonlinux/zfs/pull/5925

Please correct me if I=E2=80=99m wrong but this looks a lot less mature =
than FreeBSD=E2=80=99s existing TRIM support for ZFS which we=E2=80=99ve =
had in production for six years.

What is the rationale here? I=E2=80=99m concerned that it looks like an =
opportunity for mighty regressions.

> I forwarded you the private communication again as well.
>=20
> -M
>=20
>=20
>>=20
>> On Wed, 19 Dec 2018 at 22:52, Matthew Macy <mmacy@freebsd.org> wrote:
>>=20
>>>=20
>>>=20
>>> On Wed, Dec 19, 2018 at 14:47 Steven Hartland =
<killing@multiplay.co.uk>
>>> wrote:
>>>=20
>>>> Thanks for the write up most appreciated. One of the more meaty
>>>> differences is that FreeBSD ZFS still has the only merged and =
production
>>>> ready TRIM support so my question would be are their any plans to =
address
>>>> this before creating the new port as going back to a world without =
TRIM
>>>> support wouldn=E2=80=99t be something I=E2=80=99d look forward to.
>>>>=20
>>>=20
>>> Well, then please follow up on the request I CC'd you on a week ago
>>> asking that you engage on the deadlist based TRIM  PR. That's a =
better
>>> forum for discussing these details than lamenting in public lists.
>>>=20
>>> Thanks.
>>>=20
>>> -M
>>>=20
>>>=20
>>>=20
>>>=20
>>>> On Wed, 19 Dec 2018 at 06:51, Matthew Macy <mmacy@freebsd.org> =
wrote:
>>>>=20
>>>>> The sources for FreeBSD's ZFS support are currently taken directly
>>>>> from Illumos with local ifdefs to support the peculiarities of =
FreeBSD
>>>>> where the Solaris Portability Layer (SPL) shims fall short. =
FreeBSD
>>>>> has regularly pulled changes from Illumos and tried to push back =
any
>>>>> bug fixes and new features done in the context of FreeBSD. In the =
past
>>>>> few years the vast majority of new development in ZFS has taken =
place
>>>>> in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix =
announced
>>>>> that they will be moving to ZoL
>>>>> https://www.delphix.com/blog/kickoff-future-eko-2018 This shift =
means
>>>>> that there will be little to no net new development of Illumos. =
While
>>>>> working through the git history of ZoL I have also discovered that
>>>>> many races and locking bugs have been fixed in ZoL and never made =
it
>>>>> back to Illumos and thus FreeBSD. This state of affairs has led to =
a
>>>>> general agreement among the stakeholders that I have spoken to =
that it
>>>>> makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf
>>>>> has graciously encouraged me to add FreeBSD support directly to =
ZoL
>>>>> https://github.com/zfsonfreebsd/ZoF so that we might all have a =
single
>>>>> shared code base.
>>>>>=20
>>>>> A port for ZoF can be found at =
https://github.com/miwi-fbsd/zof-port
>>>>> Before it can be committed some additional functionality needs to =
be
>>>>> added to the FreeBSD opencrypto framework. These can be found at
>>>>> https://reviews.freebsd.org/D18520
>>>>>=20
>>>>> This port will provide FreeBSD users with multi modifier =
protection,
>>>>> project quotas, encrypted datasets, allocation classes, vectorized
>>>>> raidz, vectorized checksums, and various command line =
improvements.
>>>>>=20
>>>>> Before ZoF can be merged back in to ZoL several steps need to be =
taken:
>>>>> - Integrate FreeBSD support into ZoL CI
>>>>> - Have most of the ZFS test suite passing
>>>>> - Complete additional QA testing at iX
>>>>>=20
>>>>> We at iX Systems need to port ZoL's EC2 CI scripts to work with
>>>>> FreeBSD and make sure that most of the ZFS Test Suite (ZTS) =
passes.
>>>>> Being integrated in to their CI will mean that, among other =
things,
>>>>> most integration issues will be caught before a PR is merged =
upstream
>>>>> rather than many months later when it is MFVed into FreeBSD. I=E2=80=
=99m
>>>>> hoping to submit the PR to ZoL some time in January.
>>>>>=20
>>>>> This port will make it easy for end users on a range of releases =
to
>>>>> run the latest version of ZFS. Nonetheless, transitioning away =
from an
>>>>> Illumos based ZFS is not likely to be entirely seamless. The
>>>>> stakeholders I=E2=80=99ve spoken to all agree that this is the =
best path
>>>>> forward but some degree of effort needs to be made to accommodate
>>>>> downstream consumers. The current plan is to import ZoF and unhook =
the
>>>>> older Illumos based sources from the build on April 15th or two =
months
>>>>> after iX systems QA deems ZoF stable - which ever comes later. The
>>>>> Illumos based sources will be removed some time later - but well
>>>>> before 13. This will give users a 3 month period during which both =
the
>>>>> port and legacy Illumos based ZFS will be available to users. =
Pools
>>>>> should interoperate between ZoF and legacy provided the user does =
not
>>>>> enable any features available only in ZoF. We will try to =
accommodate
>>>>> any downstream consumers in the event that they need that date =
pushed
>>>>> back. We ask that any downstream consumers who are particularly
>>>>> sensitive to changes start testing the port when it is formally
>>>>> announced and report back any issues they have. I will do my best =
to
>>>>> ensure that this message is communicated to all those who it may
>>>>> concern. However, I can=E2=80=99t ensure that everyone reads these =
lists. That
>>>>> is the responsibility of -CURRENT users.
>>>>>=20
>>>>> -M
>>>>>=20
>>>> _______________________________________________
>>>>> freebsd-fs@freebsd.org mailing list
>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>>>>> To unsubscribe, send any mail to =
"freebsd-fs-unsubscribe@freebsd.org"
>>>>>=20
>>>>=20
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to =
"freebsd-current-unsubscribe@freebsd.org"



--
Bob Bishop
rb@gid.co.uk







Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B527B90E-9C5E-4DC5-8DE5-35A85221BA64>