Date: Tue, 18 Dec 2018 23:49:13 -0800 From: Enji Cooper <yaneurabeya@gmail.com> To: Matthew Macy <mmacy@freebsd.org> Cc: freebsd-fs <freebsd-fs@freebsd.org>, freebsd-current <freebsd-current@freebsd.org> Subject: Re: The future of ZFS in FreeBSD Message-ID: <9C23F0C0-DEF7-45A4-ADEF-58A00F9714D8@gmail.com> In-Reply-To: <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com> References: <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_5B9C960B-5159-4A51-ACDD-81A425A795AA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hello Matthew, I appreciate the long write up, as someone who still uses FreeBSD ZFS on = my NAS box and knowing some of the history with ZFS on *Solaris, etc. = Something like this was bound to happen with post the Oracle buyout. > On Dec 18, 2018, at 10:49 PM, Matthew Macy <mmacy@freebsd.org> wrote: >=20 > The sources for FreeBSD's ZFS support are currently taken directly > from Illumos with local ifdefs to support the peculiarities of FreeBSD > where the Solaris Portability Layer (SPL) shims fall short. FreeBSD > has regularly pulled changes from Illumos and tried to push back any > bug fixes and new features done in the context of FreeBSD. In the past > few years the vast majority of new development in ZFS has taken place > in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced > that they will be moving to ZoL > https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means > that there will be little to no net new development of Illumos. While > working through the git history of ZoL I have also discovered that > many races and locking bugs have been fixed in ZoL and never made it > back to Illumos and thus FreeBSD. This state of affairs has led to a > general agreement among the stakeholders that I have spoken to that it > makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf > has graciously encouraged me to add FreeBSD support directly to ZoL > https://github.com/zfsonfreebsd/ZoF so that we might all have a single > shared code base. >=20 > A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port > Before it can be committed some additional functionality needs to be > added to the FreeBSD opencrypto framework. These can be found at > https://reviews.freebsd.org/D18520 >=20 > This port will provide FreeBSD users with multi modifier protection, > project quotas, encrypted datasets, allocation classes, vectorized > raidz, vectorized checksums, and various command line improvements. >=20 > Before ZoF can be merged back in to ZoL several steps need to be = taken: > - Integrate FreeBSD support into ZoL CI > - Have most of the ZFS test suite passing > - Complete additional QA testing at iX Can you please describe the testing process that will be employed to = verify the sanity of the ZoL on FreeBSD port? Should other large = companies who use ZFS on FreeBSD (Panzura?) chime in and the ZFS on = FreeBSD community (as a whole) collaborate to better suss out issues = with the ZoL port? > We at iX Systems need to port ZoL's EC2 CI scripts to work with > FreeBSD and make sure that most of the ZFS Test Suite (ZTS) passes. > Being integrated in to their CI will mean that, among other things, > most integration issues will be caught before a PR is merged upstream > rather than many months later when it is MFVed into FreeBSD. I=E2=80=99m= > hoping to submit the PR to ZoL some time in January. Could you please better describe this, in particular provide pointers to = any scripts that might be executed with the ZoL port? > This port will make it easy for end users on a range of releases to > run the latest version of ZFS. Nonetheless, transitioning away from an > Illumos based ZFS is not likely to be entirely seamless. The > stakeholders I=E2=80=99ve spoken to all agree that this is the best = path > forward but some degree of effort needs to be made to accommodate > downstream consumers. The current plan is to import ZoF and unhook the > older Illumos based sources from the build on April 15th or two months > after iX systems QA deems ZoF stable - which ever comes later. The > Illumos based sources will be removed some time later - but well > before 13. This will give users a 3 month period during which both the > port and legacy Illumos based ZFS will be available to users. Pools > should interoperate between ZoF and legacy provided the user does not > enable any features available only in ZoF. We will try to accommodate > any downstream consumers in the event that they need that date pushed > back. We ask that any downstream consumers who are particularly > sensitive to changes start testing the port when it is formally > announced and report back any issues they have. I will do my best to > ensure that this message is communicated to all those who it may > concern. However, I can=E2=80=99t ensure that everyone reads these = lists. That > is the responsibility of -CURRENT users. As a thought: is there a way for the ZoL port to run in parallel with = the non-ZoL port, kind of like in an audit/read-only/verify-only mode? Thanks so much, -Enji --Apple-Mail=_5B9C960B-5159-4A51-ACDD-81A425A795AA Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE5bk3FaGcY5rvqmb79YOpJmkwhhUFAlwZ9/kACgkQ9YOpJmkw hhXeBA//aK5RivUGqkKcORtDQ+P+l/0jXGs0lE42C7Yo89QO3QXnfM2po7PL/ZFV GaZGc8J+LIsiM2T48lx1u9S8qVbFRhdSbOnQKRFgm4ouQVJJUKiSCFLdugP2ADG4 q/EmRaO6tPo5Dc1XeyONahLymH/izPv0E97ua+HHTG2LSpe3uG0Wq4HMVGvqFklS b5Zy1HfNmBD8MxklHe6WDaCAtyns5l/zrNrIH8J3p9votYJ8zjJy3FMcXpu0BqSy 6eohgGykDtG/rL3BjgKaIi1JtSjZ/LBOJtkwM+owagBpnFvBoUHiem7dWxYuzosc 9NoFppsPflnAS3OC+2xwLg9ZtPcsOlDg88y9iChRBZYG/Lit6lBYxbjYC7E8GgFO ceqstQTgjkRTAe3+C4tGD0thI9DcjtUX5a3zoeu0glH8KcNDLAPy+ovqY8UaY4CK k18rVpk/j2pI7lmfPBZssuJop0idcJ6dUW+bicN1XsJZgdrzAIQQpqdrwjON35Ho bmseHWTQbhlit0+I+/HYchM1j1oAwdizuNF2MmMJGWrb/fiDqWp6n55kA1zmnTHV F7/v0qS4zqeyxq0zJrk177wb/ZhA/GcFpi+wAJ8+a4vEKFzkXgeYnmik9BucOplB ecjdqN1CsFR5E+PiVTUSk9dtoP7pUBVqvJjLC+EuYn212AO9syQ= =ypW9 -----END PGP SIGNATURE----- --Apple-Mail=_5B9C960B-5159-4A51-ACDD-81A425A795AA--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9C23F0C0-DEF7-45A4-ADEF-58A00F9714D8>