From owner-freebsd-jail@freebsd.org Mon Jan 21 16:42:54 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A0B114AF079 for ; Mon, 21 Jan 2019 16:42:54 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B41E06ACC7 for ; Mon, 21 Jan 2019 16:42:53 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: by mailman.ysv.freebsd.org (Postfix) id 77B5D14AF071; Mon, 21 Jan 2019 16:42:53 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5550D14AF070 for ; Mon, 21 Jan 2019 16:42:53 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mail.michaelwlucas.com (mail.michaelwlucas.com [104.236.197.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DBB546ACC2 for ; Mon, 21 Jan 2019 16:42:52 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mail.michaelwlucas.com (localhost [127.0.0.1]) by mail.michaelwlucas.com (8.15.2/8.15.2) with ESMTP id x0LGgg01092109 for ; Mon, 21 Jan 2019 11:42:42 -0500 (EST) (envelope-from mwlucas@mail.michaelwlucas.com) Received: (from mwlucas@localhost) by mail.michaelwlucas.com (8.15.2/8.15.2/Submit) id x0LGggVe092108 for jail@freebsd.org; Mon, 21 Jan 2019 11:42:42 -0500 (EST) (envelope-from mwlucas) Date: Mon, 21 Jan 2019 11:42:42 -0500 From: "Michael W. Lucas" To: jail@freebsd.org Subject: delegating ZFS of jail's root directory Message-ID: <20190121164242.GB91955@mail.michaelwlucas.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.2 (mail.michaelwlucas.com [127.0.0.1]); Mon, 21 Jan 2019 11:42:44 -0500 (EST) X-Rspamd-Queue-Id: DBB546ACC2 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [4.62 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.94)[0.938,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[jail@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: mail.michaelwlucas.com]; NEURAL_SPAM_LONG(1.00)[0.999,0]; DMARC_NA(0.00)[michaelwlucas.com]; NEURAL_SPAM_MEDIUM(0.96)[0.958,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[mwlucas@michaelwlucas.com,mwlucas@mail.michaelwlucas.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14061, ipnet:104.236.192.0/18, country:US]; FROM_NEQ_ENVFROM(0.00)[mwlucas@michaelwlucas.com,mwlucas@mail.michaelwlucas.com]; IP_SCORE(0.54)[asn: 14061(2.76), country: US(-0.08)] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2019 16:42:54 -0000 Hi, Two more book research questions, sorry. If the answer is "it doesn't work that way," cool, I'll document and move on. It looks like ZFS delegation isn't widely used. 1) It seems I can successfully delegate managing ZFS datasets to a jail, sort of. A restart removes my ability to destroy and rename datasets I created, though. 2) I can't delegate the jail's root to the jail. Obvious question: CAN you delegate a jail's root dataset, or am I chasing an impossibility here? Details: Real hardware, running yesterday's -current: FreeBSD storm 13.0-CURRENT FreeBSD 13.0-CURRENT r343219 GENERIC amd64 Here's my jail.conf. exec.start="sh /etc/rc"; exec.stop="sh /etc/rc.shutdown"; filedump { host.hostname="filedump.mwl.io"; ip4.addr="203.0.113.224"; path="/jail/filedump/zroot"; persist=true; mount.devfs=true; allow.mount=true; allow.mount.zfs=true; enforce_statfs=1; exec.poststart="/sbin/zfs jail filedump jail/filedump/zroot"; exec.poststop="/sbin/zfs unjail filedump jail/filedump/zroot"; } /jail/filedump/zroot contains FreeBSD 12.0 base.tgz extract. # ls /jail/filedump/zroot/ .cshrc dev media root var .profile etc mnt sbin COPYRIGHT jail net sys bin lib proc tmp boot libexec rescue usr Initial ZFS "jailed" parameter: # zfs get -r jailed jail/filedump NAME PROPERTY VALUE SOURCE jail/filedump jailed off default jail/filedump/zroot jailed off default jail/filedump/zroot/cdr jailed on local jail/filedump/zroot/home jailed on local jail/filedump/zroot/home/mwl jailed on inherited from jail/filedump/zroot/home Running "service jail start filedump" gives me a working jail. I can create and destroy datasets. root@filedump:~ # zfs create jail/filedump/zroot/home/abc root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc Gonna recreate that dataset for testing purposes: root@filedump:~ # zfs create jail/filedump/zroot/home/abc Now back to the host, restart the jail, and: root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc cannot unmount '/jail/filedump/zroot/home/abc': Operation not permitted I created this dataset within the jail, and can manage it only so long as it's the same jail instance. A restart wrecks my ability to manage the dataset. Second problem: I would also like to delegate management of the jail's root fileset, so on the host I run: # zfs set jailed=on jail/filedump/zroot # service jail start filedump Starting jails: cannot start jail "filedump": jail: filedump: mount.devfs: /jail/filedump/zroot/dev: No such file or directory . Which--of course, the root dir isn't mounted, so /dev can't be mounted. I'm vaguely confident I've heard of people delegating management of the root dataset to the jail, though I can't find it. Am I misremembering? Thanks, ==ml -- Michael W. Lucas https://mwl.io/ author of: Absolute OpenBSD, SSH Mastery, git commit murder, Immortal Clay, PGP & GPG, Absolute FreeBSD, etc, etc, etc... From owner-freebsd-jail@freebsd.org Mon Jan 21 22:04:11 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F11B81481822 for ; Mon, 21 Jan 2019 22:04:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8C10584D35 for ; Mon, 21 Jan 2019 22:04:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4FEDA148181D; Mon, 21 Jan 2019 22:04:10 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D192148181C for ; Mon, 21 Jan 2019 22:04:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CBE8884D31 for ; Mon, 21 Jan 2019 22:04:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0675FCEF0 for ; Mon, 21 Jan 2019 22:04:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x0LM48iI045130 for ; Mon, 21 Jan 2019 22:04:08 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x0LM48ph045129 for jail@FreeBSD.org; Mon, 21 Jan 2019 22:04:08 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: jail@FreeBSD.org Subject: [Bug 228066] Jail hook missing between launch and exec.start Date: Mon, 21 Jan 2019 22:04:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: gonzo@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: jail@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2019 22:04:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D228066 Oleksandr Tymoshenko changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed CC| |gonzo@FreeBSD.org Resolution|--- |FIXED --- Comment #3 from Oleksandr Tymoshenko --- There is a commit referencing this PR, but it's still not closed and has be= en inactive for some time. Closing the PR as fixed but feel free to re-open it= if the issue hasn't been completely resolved. Thanks --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-jail@freebsd.org Tue Jan 22 10:40:04 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE5D114B7B2D for ; Tue, 22 Jan 2019 10:40:04 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2E23D85869 for ; Tue, 22 Jan 2019 10:40:04 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: by mailman.ysv.freebsd.org (Postfix) id DCC3114B7B2C; Tue, 22 Jan 2019 10:40:03 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B897214B7B2B for ; Tue, 22 Jan 2019 10:40:03 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0A8EC85866 for ; Tue, 22 Jan 2019 10:40:02 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 32A1BA91E0; Tue, 22 Jan 2019 11:39:59 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eDJokTtCUgcK; Tue, 22 Jan 2019 11:39:58 +0100 (CET) Received: from [192.168.10.67] (opteron [192.168.10.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 510F7A91DF; Tue, 22 Jan 2019 11:39:58 +0100 (CET) Subject: Re: delegating ZFS of jail's root directory To: "Michael W. Lucas" , jail@freebsd.org References: <20190121164242.GB91955@mail.michaelwlucas.com> From: Willem Jan Withagen Message-ID: <946528bf-f9a9-724f-b4c0-1a734800d16d@digiware.nl> Date: Tue, 22 Jan 2019 11:39:57 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <20190121164242.GB91955@mail.michaelwlucas.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 10:40:04 -0000 On 21-1-2019 17:42, Michael W. Lucas wrote: > Hi, > > Two more book research questions, sorry. If the answer is "it doesn't > work that way," cool, I'll document and move on. It looks like ZFS > delegation isn't widely used. > > 1) It seems I can successfully delegate managing ZFS datasets to a jail, > sort of. A restart removes my ability to destroy and rename datasets I > created, though. > > 2) I can't delegate the jail's root to the jail. Obvious question: CAN > you delegate a jail's root dataset, or am I chasing an impossibility > here? > > Details: > > Real hardware, running yesterday's -current: > > FreeBSD storm 13.0-CURRENT FreeBSD 13.0-CURRENT r343219 GENERIC amd64 > > > Here's my jail.conf. > > exec.start="sh /etc/rc"; > exec.stop="sh /etc/rc.shutdown"; > > filedump { > host.hostname="filedump.mwl.io"; > ip4.addr="203.0.113.224"; > path="/jail/filedump/zroot"; > persist=true; > mount.devfs=true; > allow.mount=true; > allow.mount.zfs=true; > enforce_statfs=1; > exec.poststart="/sbin/zfs jail filedump jail/filedump/zroot"; > exec.poststop="/sbin/zfs unjail filedump jail/filedump/zroot"; > } > > /jail/filedump/zroot contains FreeBSD 12.0 base.tgz extract. > > # ls /jail/filedump/zroot/ > .cshrc dev media root var > .profile etc mnt sbin > COPYRIGHT jail net sys > bin lib proc tmp > boot libexec rescue usr > > Initial ZFS "jailed" parameter: > > # zfs get -r jailed jail/filedump > NAME PROPERTY VALUE SOURCE > jail/filedump jailed off default > jail/filedump/zroot jailed off default > jail/filedump/zroot/cdr jailed on local > jail/filedump/zroot/home jailed on local > jail/filedump/zroot/home/mwl jailed on inherited from jail/filedump/zroot/home > > > Running "service jail start filedump" gives me a working jail. I can > create and destroy datasets. > > root@filedump:~ # zfs create jail/filedump/zroot/home/abc > root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc > > Gonna recreate that dataset for testing purposes: > > root@filedump:~ # zfs create jail/filedump/zroot/home/abc > > Now back to the host, restart the jail, and: > > root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc > cannot unmount '/jail/filedump/zroot/home/abc': Operation not permitted > > I created this dataset within the jail, and can manage it only so long > as it's the same jail instance. A restart wrecks my ability to manage > the dataset. > > > > Second problem: > > I would also like to delegate management of the jail's root fileset, > so on the host I run: > > # zfs set jailed=on jail/filedump/zroot > # service jail start filedump > Starting jails: cannot start jail "filedump": > jail: filedump: mount.devfs: /jail/filedump/zroot/dev: No such file or directory > . > > Which--of course, the root dir isn't mounted, so /dev can't be mounted. > > > I'm vaguely confident I've heard of people delegating management of > the root dataset to the jail, though I can't find it. Am I > misremembering? Hi Michael, I think I asked that question a some time ago, to be able to run a ceph-setup script in a jail.... The basic answer was that the jail needs to have access to /dev/zfs in the jail to be effectively controlling zfs. But then I think you delegate the whole set of zfs capabilities to the jail. Which in my case was not a problem. But if you want to use a jail as separation of control, then this will be way too liberal. There is a set of configs for devfs in /etc. See `man -k devfs` But I've not used this in the end. --WjW From owner-freebsd-jail@freebsd.org Tue Jan 22 16:23:43 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D306D14A7309 for ; Tue, 22 Jan 2019 16:23:42 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 34CD392FEA for ; Tue, 22 Jan 2019 16:23:42 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: by mailman.ysv.freebsd.org (Postfix) id ECD9614A7308; Tue, 22 Jan 2019 16:23:41 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1B8C14A7307 for ; Tue, 22 Jan 2019 16:23:41 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mail.michaelwlucas.com (mail.michaelwlucas.com [104.236.197.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5852B92FE9 for ; Tue, 22 Jan 2019 16:23:41 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mail.michaelwlucas.com (localhost [127.0.0.1]) by mail.michaelwlucas.com (8.15.2/8.15.2) with ESMTP id x0MGNYVY000738; Tue, 22 Jan 2019 11:23:35 -0500 (EST) (envelope-from mwlucas@mail.michaelwlucas.com) Received: (from mwlucas@localhost) by mail.michaelwlucas.com (8.15.2/8.15.2/Submit) id x0MGNYbH000737; Tue, 22 Jan 2019 11:23:34 -0500 (EST) (envelope-from mwlucas) Date: Tue, 22 Jan 2019 11:23:34 -0500 From: "Michael W. Lucas" To: Willem Jan Withagen Cc: jail@freebsd.org Subject: Re: delegating ZFS of jail's root directory Message-ID: <20190122162334.GA668@mail.michaelwlucas.com> References: <20190121164242.GB91955@mail.michaelwlucas.com> <946528bf-f9a9-724f-b4c0-1a734800d16d@digiware.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <946528bf-f9a9-724f-b4c0-1a734800d16d@digiware.nl> User-Agent: Mutt/1.10.1 (2018-07-13) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.2 (mail.michaelwlucas.com [127.0.0.1]); Tue, 22 Jan 2019 11:23:37 -0500 (EST) X-Rspamd-Queue-Id: 5852B92FE9 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.992,0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 16:23:43 -0000 On Tue, Jan 22, 2019 at 11:39:57AM +0100, Willem Jan Withagen wrote: > On 21-1-2019 17:42, Michael W. Lucas wrote: > > Hi, > > > > Two more book research questions, sorry. If the answer is "it doesn't > > work that way," cool, I'll document and move on. It looks like ZFS > > delegation isn't widely used. > > > > 1) It seems I can successfully delegate managing ZFS datasets to a jail, > > sort of. A restart removes my ability to destroy and rename datasets I > > created, though. > > > > 2) I can't delegate the jail's root to the jail. Obvious question: CAN > > you delegate a jail's root dataset, or am I chasing an impossibility > > here? > > > > Details: > > > > Real hardware, running yesterday's -current: > > > > FreeBSD storm 13.0-CURRENT FreeBSD 13.0-CURRENT r343219 GENERIC amd64 > > > > > > Here's my jail.conf. > > > > exec.start="sh /etc/rc"; > > exec.stop="sh /etc/rc.shutdown"; > > > > filedump { > > host.hostname="filedump.mwl.io"; > > ip4.addr="203.0.113.224"; > > path="/jail/filedump/zroot"; > > persist=true; > > mount.devfs=true; > > allow.mount=true; > > allow.mount.zfs=true; > > enforce_statfs=1; > > exec.poststart="/sbin/zfs jail filedump jail/filedump/zroot"; > > exec.poststop="/sbin/zfs unjail filedump jail/filedump/zroot"; > > } > > > > /jail/filedump/zroot contains FreeBSD 12.0 base.tgz extract. > > > > # ls /jail/filedump/zroot/ > > .cshrc dev media root var > > .profile etc mnt sbin > > COPYRIGHT jail net sys > > bin lib proc tmp > > boot libexec rescue usr > > > > Initial ZFS "jailed" parameter: > > > > # zfs get -r jailed jail/filedump > > NAME PROPERTY VALUE SOURCE > > jail/filedump jailed off default > > jail/filedump/zroot jailed off default > > jail/filedump/zroot/cdr jailed on local > > jail/filedump/zroot/home jailed on local > > jail/filedump/zroot/home/mwl jailed on inherited from jail/filedump/zroot/home > > > > > > Running "service jail start filedump" gives me a working jail. I can > > create and destroy datasets. > > > > root@filedump:~ # zfs create jail/filedump/zroot/home/abc > > root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc > > > > Gonna recreate that dataset for testing purposes: > > > > root@filedump:~ # zfs create jail/filedump/zroot/home/abc > > > > Now back to the host, restart the jail, and: > > > > root@filedump:~ # zfs destroy jail/filedump/zroot/home/abc > > cannot unmount '/jail/filedump/zroot/home/abc': Operation not permitted > > > > I created this dataset within the jail, and can manage it only so long > > as it's the same jail instance. A restart wrecks my ability to manage > > the dataset. > > > > > > > > Second problem: > > > > I would also like to delegate management of the jail's root fileset, > > so on the host I run: > > > > # zfs set jailed=on jail/filedump/zroot > > # service jail start filedump > > Starting jails: cannot start jail "filedump": > > jail: filedump: mount.devfs: /jail/filedump/zroot/dev: No such file or directory > > . > > > > Which--of course, the root dir isn't mounted, so /dev can't be mounted. > > > > > > I'm vaguely confident I've heard of people delegating management of > > the root dataset to the jail, though I can't find it. Am I > > misremembering? > > Hi Michael, > > I think I asked that question a some time ago, to be able to run a > ceph-setup script in a jail.... > > The basic answer was that the jail needs to have access to /dev/zfs in the > jail to be effectively controlling zfs. But then I think you delegate the > whole set of zfs capabilities to the jail. > > Which in my case was not a problem. But if you want to use a jail as > separation of control, then this will be way too liberal. > > There is a set of configs for devfs in /etc. See `man -k devfs` > But I've not used this in the end. That fixes the first problem, thank you. I still can't delegate the jail's root directory to the jail, though. Once I set jailed=on to the jail's zroot, it's unmounted and jail(8) can't find the jail's /dev to mount it. There seems to be a chicken-and-egg problem here that I have no idea how to resolve. Any suggestions? ==ml -- Michael W. Lucas https://mwl.io/ author of: Absolute OpenBSD, SSH Mastery, git commit murder, Immortal Clay, PGP & GPG, Absolute FreeBSD, etc, etc, etc... From owner-freebsd-jail@freebsd.org Tue Jan 22 17:50:31 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C201A14A9A84 for ; Tue, 22 Jan 2019 17:50:31 +0000 (UTC) (envelope-from SRS0=bgq/=P6=quip.cz=000.fbsd@elsa.codelab.cz) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4894D95F5B for ; Tue, 22 Jan 2019 17:50:31 +0000 (UTC) (envelope-from SRS0=bgq/=P6=quip.cz=000.fbsd@elsa.codelab.cz) Received: by mailman.ysv.freebsd.org (Postfix) id 0B97314A9A83; Tue, 22 Jan 2019 17:50:31 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECCFB14A9A82 for ; Tue, 22 Jan 2019 17:50:30 +0000 (UTC) (envelope-from SRS0=bgq/=P6=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 814AC95F59 for ; Tue, 22 Jan 2019 17:50:30 +0000 (UTC) (envelope-from SRS0=bgq/=P6=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 1161828422; Tue, 22 Jan 2019 18:50:22 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 0F85028417; Tue, 22 Jan 2019 18:50:16 +0100 (CET) Subject: Re: delegating ZFS of jail's root directory To: "Michael W. Lucas" , Willem Jan Withagen Cc: jail@freebsd.org References: <20190121164242.GB91955@mail.michaelwlucas.com> <946528bf-f9a9-724f-b4c0-1a734800d16d@digiware.nl> <20190122162334.GA668@mail.michaelwlucas.com> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <2ac2db6a-9fb8-36d7-2481-b6983da5bc04@quip.cz> Date: Tue, 22 Jan 2019 18:50:15 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: <20190122162334.GA668@mail.michaelwlucas.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 814AC95F59 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.982,0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 17:50:31 -0000 Michael W. Lucas wrote on 2019/01/22 17:23: > On Tue, Jan 22, 2019 at 11:39:57AM +0100, Willem Jan Withagen wrote: >> On 21-1-2019 17:42, Michael W. Lucas wrote: >> Hi Michael, >> >> I think I asked that question a some time ago, to be able to run a >> ceph-setup script in a jail.... >> >> The basic answer was that the jail needs to have access to /dev/zfs in the >> jail to be effectively controlling zfs. But then I think you delegate the >> whole set of zfs capabilities to the jail. >> >> Which in my case was not a problem. But if you want to use a jail as >> separation of control, then this will be way too liberal. >> >> There is a set of configs for devfs in /etc. See `man -k devfs` >> But I've not used this in the end. > > That fixes the first problem, thank you. > > I still can't delegate the jail's root directory to the jail, > though. Once I set jailed=on to the jail's zroot, it's unmounted and > jail(8) can't find the jail's /dev to mount it. > > There seems to be a chicken-and-egg problem here that I have no idea > how to resolve. Any suggestions? What about to mount it with exec.prestart before the jail is created? (I didn't tried it) Miroslav Lachman