From owner-freebsd-jail@freebsd.org Tue Jan 22 16:23:43 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D306D14A7309 for ; Tue, 22 Jan 2019 16:23:42 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 34CD392FEA for ; Tue, 22 Jan 2019 16:23:42 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: by mailman.ysv.freebsd.org (Postfix) id ECD9614A7308; Tue, 22 Jan 2019 16:23:41 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1B8C14A7307 for ; Tue, 22 Jan 2019 16:23:41 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mail.michaelwlucas.com (mail.michaelwlucas.com [104.236.197.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5852B92FE9 for ; Tue, 22 Jan 2019 16:23:41 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mail.michaelwlucas.com (localhost [127.0.0.1]) by mail.michaelwlucas.com (8.15.2/8.15.2) with ESMTP id x0MGNYVY000738; Tue, 22 Jan 2019 11:23:35 -0500 (EST) (envelope-from mwlucas@mail.michaelwlucas.com) Received: (from mwlucas@localhost) by mail.michaelwlucas.com (8.15.2/8.15.2/Submit) id x0MGNYbH000737; Tue, 22 Jan 2019 11:23:34 -0500 (EST) (envelope-from mwlucas) Date: Tue, 22 Jan 2019 11:23:34 -0500 From: "Michael W. Lucas" To: Willem Jan Withagen Cc: jail@freebsd.org Subject: Re: delegating ZFS of jail's root directory Message-ID: <20190122162334.GA668@mail.michaelwlucas.com> References: <20190121164242.GB91955@mail.michaelwlucas.com> <946528bf-f9a9-724f-b4c0-1a734800d16d@digiware.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <946528bf-f9a9-724f-b4c0-1a734800d16d@digiware.nl> User-Agent: Mutt/1.10.1 (2018-07-13) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.2 (mail.michaelwlucas.com [127.0.0.1]); Tue, 22 Jan 2019 11:23:37 -0500 (EST) X-Rspamd-Queue-Id: 5852B92FE9 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.992,0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 16:23:43 -0000 On Tue, Jan 22, 2019 at 11:39:57AM +0100, Willem Jan Withagen wrote: > On 21-1-2019 17:42, Michael W. Lucas wrote: > > Hi, > > > > Two more book research questions, sorry. If the answer is "it doesn't > > work that way," cool, I'll document and move on. It looks like ZFS > > delegation isn't widely used. > > > > 1) It seems I can successfully delegate managing ZFS datasets to a jail, > > sort of. A restart removes my ability to destroy and rename datasets I > > created, though. > > > > 2) I can't delegate the jail's root to the jail. Obvious question: CAN > > you delegate a jail's root dataset, or am I chasing an impossibility > > here? > > > > Details: > > > > Real hardware, running yesterday's -current: > > > > FreeBSD storm 13.0-CURRENT FreeBSD 13.0-CURRENT r343219 GENERIC amd64 > > > > > > Here's my jail.conf. > > > > exec.start="sh /etc/rc"; > > exec.stop="sh /etc/rc.shutdown"; > > > > filedump { > > host.hostname="filedump.mwl.io"; > > ip4.addr="203.0.113.224"; > > path="/jail/filedump/zroot"; > > persist=true; > > mount.devfs=true; > > allow.mount=true; > > allow.mount.zfs=true; > > enforce_statfs=1; > > exec.poststart="/sbin/zfs jail filedump jail/filedump/zroot"; > > exec.poststop="/sbin/zfs unjail filedump jail/filedump/zroot"; > > } > > > > /jail/filedump/zroot contains FreeBSD 12.0 base.tgz extract. > > > > # ls /jail/filedump/zroot/ > > .cshrc dev media root var > > .profile etc mnt sbin > > COPYRIGHT jail net sys > > bin lib proc tmp > > boot libexec rescue usr > > > > Initial ZFS "jailed" parameter: > > > > # zfs get -r jailed jail/filedump > > NAME PROPERTY VALUE SOURCE > > jail/filedump jailed off default > > jail/filedump/zroot jailed off default > > jail/filedump/zroot/cdr jailed on local > > jail/filedump/zroot/home jailed on local > > jail/filedump/zroot/home/mwl jailed on inherited from jail/filedump/zroot/home > > > > > > Running "service jail start filedump" gives me a working jail. I can > > create and destroy datasets. > > > > root@filedump:~ # zfs create jail/filedump/zroot/home/abc > > root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc > > > > Gonna recreate that dataset for testing purposes: > > > > root@filedump:~ # zfs create jail/filedump/zroot/home/abc > > > > Now back to the host, restart the jail, and: > > > > root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc > > cannot unmount '/jail/filedump/zroot/home/abc': Operation not permitted > > > > I created this dataset within the jail, and can manage it only so long > > as it's the same jail instance. A restart wrecks my ability to manage > > the dataset. > > > > > > > > Second problem: > > > > I would also like to delegate management of the jail's root fileset, > > so on the host I run: > > > > # zfs set jailed=on jail/filedump/zroot > > # service jail start filedump > > Starting jails: cannot start jail "filedump": > > jail: filedump: mount.devfs: /jail/filedump/zroot/dev: No such file or directory > > . > > > > Which--of course, the root dir isn't mounted, so /dev can't be mounted. > > > > > > I'm vaguely confident I've heard of people delegating management of > > the root dataset to the jail, though I can't find it. Am I > > misremembering? > > Hi Michael, > > I think I asked that question a some time ago, to be able to run a > ceph-setup script in a jail.... > > The basic answer was that the jail needs to have access to /dev/zfs in the > jail to be effectively controlling zfs. But then I think you delegate the > whole set of zfs capabilities to the jail. > > Which in my case was not a problem. But if you want to use a jail as > separation of control, then this will be way too liberal. > > There is a set of configs for devfs in /etc. See `man -k devfs` > But I've not used this in the end. That fixes the first problem, thank you. I still can't delegate the jail's root directory to the jail, though. Once I set jailed=on to the jail's zroot, it's unmounted and jail(8) can't find the jail's /dev to mount it. There seems to be a chicken-and-egg problem here that I have no idea how to resolve. Any suggestions? ==ml -- Michael W. Lucas https://mwl.io/ author of: Absolute OpenBSD, SSH Mastery, git commit murder, Immortal Clay, PGP & GPG, Absolute FreeBSD, etc, etc, etc...