From owner-freebsd-jail@freebsd.org Mon Feb 11 15:48:24 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5962C14DA83E for ; Mon, 11 Feb 2019 15:48:24 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CE3057332E for ; Mon, 11 Feb 2019 15:48:23 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: by mailman.ysv.freebsd.org (Postfix) id 8E6E114DA83D; Mon, 11 Feb 2019 15:48:23 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A37B14DA83C for ; Mon, 11 Feb 2019 15:48:23 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mail.michaelwlucas.com (mail.michaelwlucas.com [104.236.197.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2C1F7332D for ; Mon, 11 Feb 2019 15:48:22 +0000 (UTC) (envelope-from mwlucas@mail.michaelwlucas.com) Received: from mail.michaelwlucas.com (localhost [127.0.0.1]) by mail.michaelwlucas.com (8.15.2/8.15.2) with ESMTP id x1BFmJJk010239 for ; Mon, 11 Feb 2019 10:48:20 -0500 (EST) (envelope-from mwlucas@mail.michaelwlucas.com) Received: (from mwlucas@localhost) by mail.michaelwlucas.com (8.15.2/8.15.2/Submit) id x1BFmJ0b010238 for jail@freebsd.org; Mon, 11 Feb 2019 10:48:19 -0500 (EST) (envelope-from mwlucas) Date: Mon, 11 Feb 2019 10:48:19 -0500 From: "Michael W. Lucas" To: jail@freebsd.org Subject: init in a jail Message-ID: <20190211154819.GB10183@mail.michaelwlucas.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.2 (mail.michaelwlucas.com [127.0.0.1]); Mon, 11 Feb 2019 10:48:22 -0500 (EST) X-Rspamd-Queue-Id: A2C1F7332D X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [4.58 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.88)[0.884,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[jail@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: mail.michaelwlucas.com]; NEURAL_SPAM_LONG(0.99)[0.990,0]; DMARC_NA(0.00)[michaelwlucas.com]; NEURAL_SPAM_MEDIUM(0.98)[0.984,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[mwlucas@michaelwlucas.com,mwlucas@mail.michaelwlucas.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14061, ipnet:104.236.192.0/18, country:US]; FROM_NEQ_ENVFROM(0.00)[mwlucas@michaelwlucas.com,mwlucas@mail.michaelwlucas.com]; IP_SCORE(0.53)[asn: 14061(2.72), country: US(-0.07)] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2019 15:48:24 -0000 Hi, Sadly, my google-fu has turned up thousands of man pages but no real discussion on this. According to init(8), you can run init inside a jail. If init is run in a jail, the security level of the "host system" will not be affected. Part of the information set up in the kernel to support a jail is a per-jail security level. This allows running a higher security level inside of a jail than that of the host system. See jail(8) for more information about jails. If you actually try, though, the jail dies: storm~;jail -vc loghost loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 netmask 255.255.255.255 alias loghost: run command: /sbin/mount -t devfs -oruleset=4 . /jail/loghost/dev loghost: run command: logger trying to start jail loghost... loghost: jail_set(JAIL_CREATE) persist name=loghost path=/jail/loghost host.hostname=loghost.mwl.io ip4.addr=19 8.51.100.225 loghost: created loghost: run command in jail: /sbin/init jail: loghost: /sbin/init: failed loghost: removed loghost: run command: /sbin/umount /jail/loghost/dev loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 netmask 255.255.255.255 -alias Is that init(8) text left over from an earlier jail incarnation? Or is there some other way to run init in a jail? And WHY would you run init in a jail? Thanks, ==ml -- Michael W. Lucas https://mwl.io/ author of: Absolute OpenBSD, SSH Mastery, git commit murder, Immortal Clay, PGP & GPG, Absolute FreeBSD, etc, etc, etc... From owner-freebsd-jail@freebsd.org Mon Feb 11 17:24:02 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B806614DE256 for ; Mon, 11 Feb 2019 17:24:02 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3E92B80B25 for ; Mon, 11 Feb 2019 17:24:02 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id EC31E14DE253; Mon, 11 Feb 2019 17:24:01 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D9B6C14DE252 for ; Mon, 11 Feb 2019 17:24:01 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [199.192.165.131]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6F99D80B23 for ; Mon, 11 Feb 2019 17:24:01 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org ([127.0.0.131]) by gritton.org (8.15.2/8.15.2) with ESMTP id x1BHNsG7092737; Mon, 11 Feb 2019 10:23:54 -0700 (MST) (envelope-from jamie@freebsd.org) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 11 Feb 2019 10:23:54 -0700 From: James Gritton To: jail@freebsd.org Subject: Re: init in a jail In-Reply-To: <20190211154819.GB10183@mail.michaelwlucas.com> References: <20190211154819.GB10183@mail.michaelwlucas.com> Message-ID: X-Sender: jamie@freebsd.org User-Agent: Roundcube Webmail/1.3.8 X-Greylist: inspected by milter-greylist-4.6.2 (gritton.org [127.0.0.131]); Mon, 11 Feb 2019 10:23:54 -0700 (MST) for IP:'127.0.0.131' DOMAIN:'[127.0.0.131]' HELO:'gritton.org' FROM:'jamie@freebsd.org' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (gritton.org [127.0.0.131]); Mon, 11 Feb 2019 10:23:54 -0700 (MST) X-Rspamd-Queue-Id: 6F99D80B23 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.979,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2019 17:24:03 -0000 On 2019-02-11 08:48, Michael W. Lucas wrote: > Sadly, my google-fu has turned up thousands of man pages but no real > discussion on this. > > According to init(8), you can run init inside a jail. > > If init is run in a jail, the security level of the "host system" > will > not be affected. Part of the information set up in the kernel to > support > a jail is a per-jail security level. This allows running a higher > security level inside of a jail than that of the host system. See > jail(8) for more information about jails. > > > If you actually try, though, the jail dies: > > storm~;jail -vc loghost > loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 > netmask > 255.255.255.255 alias > loghost: run command: /sbin/mount -t devfs -oruleset=4 . > /jail/loghost/dev > loghost: run command: logger trying to start jail loghost... > loghost: jail_set(JAIL_CREATE) persist name=loghost path=/jail/loghost > host.hostname=loghost.mwl.io ip4.addr=19 8.51.100.225 > loghost: created > loghost: run command in jail: /sbin/init > jail: loghost: /sbin/init: failed > loghost: removed > loghost: run command: /sbin/umount /jail/loghost/dev > loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 > netmask > 255.255.255.255 -alias > > Is that init(8) text left over from an earlier jail incarnation? Or is > there some other way to run init in a jail? > > And WHY would you run init in a jail? Interesting - I wonder how long it's been since init worked inside jails. From the look of your error messages, probably not since devfs started being used. I wasn't even aware the init(8) had anything to say on the matter, but it's clearly erroneous. AS to why it would be good to have a per-jail init, there would be a few advantages. Orphaned processes could then reparent to the jail's init instead of the real init, and the jail root could easily reboot jails. Doing it right would require presenting jailed init as pid 1, but that's not really very hard. - Jamie From owner-freebsd-jail@freebsd.org Mon Feb 11 18:15:44 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F110C14DFA2D for ; Mon, 11 Feb 2019 18:15:43 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 83A7183130 for ; Mon, 11 Feb 2019 18:15:43 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4144D14DFA2C; Mon, 11 Feb 2019 18:15:43 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0241614DFA2B for ; Mon, 11 Feb 2019 18:15:42 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C2258312E; Mon, 11 Feb 2019 18:15:42 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:13b:39f::9f:25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) (Authenticated sender: bz/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 175969C31; Mon, 11 Feb 2019 18:15:42 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id B72C78D4A13C; Mon, 11 Feb 2019 18:15:39 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id B78AED22F83; Mon, 11 Feb 2019 18:15:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id jx67TVmW2aNR; Mon, 11 Feb 2019 18:15:36 +0000 (UTC) Received: from [192.168.2.110] (unknown [IPv6:fde9:577b:c1a9:31:2ef0:eeff:fe03:ee34]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 83E32D22F7A; Mon, 11 Feb 2019 18:15:36 +0000 (UTC) From: "Bjoern A. Zeeb" To: "James Gritton" Cc: jail@freebsd.org Subject: Re: init in a jail Date: Mon, 11 Feb 2019 18:15:36 +0000 X-Mailer: MailMate (2.0BETAr6135) Message-ID: <902F2BFD-5D33-4C16-8DA5-C16E2F2CD373@FreeBSD.org> In-Reply-To: References: <20190211154819.GB10183@mail.michaelwlucas.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 8C2258312E X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2019 18:15:44 -0000 On 11 Feb 2019, at 17:23, James Gritton wrote: > On 2019-02-11 08:48, Michael W. Lucas wrote: >> Sadly, my google-fu has turned up thousands of man pages but no real >> discussion on this. >> >> According to init(8), you can run init inside a jail. >> >> If init is run in a jail, the security level of the "host >> system" will >> not be affected. Part of the information set up in the kernel >> to support >> a jail is a per-jail security level. This allows running a >> higher >> security level inside of a jail than that of the host system. >> See >> jail(8) for more information about jails. >> >> >> If you actually try, though, the jail dies: >> >> storm~;jail -vc loghost >> loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 >> netmask >> 255.255.255.255 alias >> loghost: run command: /sbin/mount -t devfs -oruleset=4 . >> /jail/loghost/dev >> loghost: run command: logger trying to start jail loghost... >> loghost: jail_set(JAIL_CREATE) persist name=loghost >> path=/jail/loghost >> host.hostname=loghost.mwl.io ip4.addr=19 8.51.100.225 >> loghost: created >> loghost: run command in jail: /sbin/init >> jail: loghost: /sbin/init: failed >> loghost: removed >> loghost: run command: /sbin/umount /jail/loghost/dev >> loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 >> netmask >> 255.255.255.255 -alias >> >> Is that init(8) text left over from an earlier jail incarnation? Or >> is >> there some other way to run init in a jail? >> >> And WHY would you run init in a jail? > > Interesting - I wonder how long it's been since init worked inside > jails. From the look of your error messages, probably not since devfs > started being used. I wasn't even aware the init(8) had anything to > say on the matter, but it's clearly erroneous. Ken Smith added that message to init(8) 15 years ago and from the sounds of it, I think it was more related to securelevels. > AS to why it would be good to have a per-jail init, there would be a > few advantages. Orphaned processes could then reparent to the jail's > init instead of the real init, and the jail root could easily reboot > jails. Doing it right would require presenting jailed init as pid 1, > but that's not really very hard. It’s not just PID 1 but yeah; I have open reviews (which I should update) from the vps branch to do a virtualised pid space, real init to jails along with it, console, and then init would also manage ttys, .. I need to work on the management bits from the host side to make it a real thing (ps, kill, etc. to work with a (jid, pid) combination as jexec won’t work anymore (possible collisions etc). But that’s unrelated to this thread. /bz From owner-freebsd-jail@freebsd.org Mon Feb 11 18:26:59 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E007A14DFFE6 for ; Mon, 11 Feb 2019 18:26:58 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EB90B837ED for ; Mon, 11 Feb 2019 18:26:57 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id ABA5614DFFE5; Mon, 11 Feb 2019 18:26:57 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CEE814DFFE4 for ; Mon, 11 Feb 2019 18:26:57 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [199.192.165.131]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 09597837EB; Mon, 11 Feb 2019 18:26:56 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org ([127.0.0.131]) by gritton.org (8.15.2/8.15.2) with ESMTP id x1BIQs9X096099; Mon, 11 Feb 2019 11:26:54 -0700 (MST) (envelope-from jamie@freebsd.org) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Mon, 11 Feb 2019 11:26:54 -0700 From: James Gritton To: jail@freebsd.org Cc: "Bjoern A. Zeeb" Subject: Re: init in a jail In-Reply-To: <902F2BFD-5D33-4C16-8DA5-C16E2F2CD373@FreeBSD.org> References: <20190211154819.GB10183@mail.michaelwlucas.com> <902F2BFD-5D33-4C16-8DA5-C16E2F2CD373@FreeBSD.org> Message-ID: X-Sender: jamie@freebsd.org User-Agent: Roundcube Webmail/1.3.8 X-Greylist: inspected by milter-greylist-4.6.2 (gritton.org [127.0.0.131]); Mon, 11 Feb 2019 11:26:55 -0700 (MST) for IP:'127.0.0.131' DOMAIN:'[127.0.0.131]' HELO:'gritton.org' FROM:'jamie@freebsd.org' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (gritton.org [127.0.0.131]); Mon, 11 Feb 2019 11:26:55 -0700 (MST) X-Rspamd-Queue-Id: 09597837EB X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.970,0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2019 18:26:59 -0000 On 2019-02-11 11:15, Bjoern A. Zeeb wrote: > On 11 Feb 2019, at 17:23, James Gritton wrote: > >> On 2019-02-11 08:48, Michael W. Lucas wrote: >>> Sadly, my google-fu has turned up thousands of man pages but no real >>> discussion on this. >>> >>> According to init(8), you can run init inside a jail. >>> >>> If init is run in a jail, the security level of the "host >>> system" will >>> not be affected. Part of the information set up in the kernel >>> to support >>> a jail is a per-jail security level. This allows running a >>> higher >>> security level inside of a jail than that of the host system. >>> See >>> jail(8) for more information about jails. >>> >>> >>> If you actually try, though, the jail dies: >>> >>> storm~;jail -vc loghost >>> loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 >>> netmask >>> 255.255.255.255 alias >>> loghost: run command: /sbin/mount -t devfs -oruleset=4 . >>> /jail/loghost/dev >>> loghost: run command: logger trying to start jail loghost... >>> loghost: jail_set(JAIL_CREATE) persist name=loghost >>> path=/jail/loghost >>> host.hostname=loghost.mwl.io ip4.addr=19 8.51.100.225 >>> loghost: created >>> loghost: run command in jail: /sbin/init >>> jail: loghost: /sbin/init: failed >>> loghost: removed >>> loghost: run command: /sbin/umount /jail/loghost/dev >>> loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 >>> netmask >>> 255.255.255.255 -alias >>> >>> Is that init(8) text left over from an earlier jail incarnation? Or >>> is >>> there some other way to run init in a jail? >>> >>> And WHY would you run init in a jail? >> >> Interesting - I wonder how long it's been since init worked inside >> jails. From the look of your error messages, probably not since devfs >> started being used. I wasn't even aware the init(8) had anything to >> say on the matter, but it's clearly erroneous. > > Ken Smith added that message to init(8) 15 years ago and from the > sounds of it, I think it was more related to securelevels. > > > >> AS to why it would be good to have a per-jail init, there would be a >> few advantages. Orphaned processes could then reparent to the jail's >> init instead of the real init, and the jail root could easily reboot >> jails. Doing it right would require presenting jailed init as pid 1, >> but that's not really very hard. > > It’s not just PID 1 but yeah; I have open reviews (which I should > update) from the vps branch to do a virtualised pid space, real init > to jails along with it, console, and then init would also manage ttys, > .. I need to work on the management bits from the host side to make > it a real thing (ps, kill, etc. to work with a (jid, pid) combination > as jexec won’t work anymore (possible collisions etc). But that’s > unrelated to this thread. I was just talking the easy part! Your project has some phenomenal ramifications, but it's also possible to just slide in virtual init on its own without the rest - I used to do that back when I was running jail-ish containers of my own. I'd been tempted every now and then to add it to jails, but now that you've got something so much bigger, it makes sense just to sit back and watch :-). - Jamie From owner-freebsd-jail@freebsd.org Tue Feb 12 07:07:58 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E46214D94FB for ; Tue, 12 Feb 2019 07:07:58 +0000 (UTC) (envelope-from crapsh@monkeybrains.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0CEC87552C for ; Tue, 12 Feb 2019 07:07:58 +0000 (UTC) (envelope-from crapsh@monkeybrains.net) Received: by mailman.ysv.freebsd.org (Postfix) id C3E1814D94FA; Tue, 12 Feb 2019 07:07:57 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B16ED14D94F9 for ; Tue, 12 Feb 2019 07:07:57 +0000 (UTC) (envelope-from crapsh@monkeybrains.net) Received: from sfo.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.monkeybrains.net", Issuer "AlphaSSL CA - SHA256 - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0ACD75526 for ; Tue, 12 Feb 2019 07:07:46 +0000 (UTC) (envelope-from crapsh@monkeybrains.net) Received: from mail.monkeybrains.net (mail.monkeybrains.net [208.69.40.19]) by sfo.monkeybrains.net (8.15.2/8.15.2) with ESMTP id x1C6b7x2071448 for ; Mon, 11 Feb 2019 22:37:07 -0800 (PST) (envelope-from crapsh@monkeybrains.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=monkeybrains.net; s=dkim; t=1549953427; bh=ObBE6WFxfBWr47gtlJuBaxtVNrkWTjt46dziD+FVDao=; h=Date:Subject:From:To; b=OnicB1cTIVlRCWbrSKfdEfr8pSPPHxCurR992NSAJLhJKT0Iyszhqhr2EHToai+5l pHdOg25fd3RWCoIExykD8WilzGuLVr+ox5feh/d8XKEjV4EtbPyfM9oIroxoN5D1We 6CF76XHex/rd77fmGHfofhLr9dstouITEEtm4tKE= Received: from 148.64.99.20 (SquirrelMail authenticated user rudy-ftp@monkeybrains.net) by mail.monkeybrains.net with HTTP; Mon, 11 Feb 2019 22:37:07 -0800 Message-ID: <2331cedc410f2123b2a0e142f81bf92e.squirrel@mail.monkeybrains.net> Date: Mon, 11 Feb 2019 22:37:07 -0800 Subject: "ipfw log" messages from jail show in host syslog From: "Rudy (bulk address)" To: jail@freebsd.org User-Agent: SquirrelMail/1.4.22 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Virus-Scanned: clamav-milter 0.100.2 at mail.monkeybrains.net X-Virus-Status: Clean X-Rspamd-Queue-Id: E0ACD75526 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=monkeybrains.net header.s=dkim header.b=OnicB1cT; dmarc=pass (policy=none) header.from=monkeybrains.net; spf=pass (mx1.freebsd.org: domain of crapsh@monkeybrains.net designates 208.69.40.19 as permitted sender) smtp.mailfrom=crapsh@monkeybrains.net X-Spamd-Result: default: False [-2.94 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.980,0]; R_DKIM_ALLOW(-0.20)[monkeybrains.net:s=dkim]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[jail@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-0.01)[country: US(-0.07)]; NEURAL_SPAM_SHORT(0.07)[0.067,0]; DKIM_TRACE(0.00)[monkeybrains.net:+]; DMARC_POLICY_ALLOW(-0.50)[monkeybrains.net,none]; HAS_X_PRIO_THREE(0.00)[3]; MX_GOOD(-0.01)[mx1.harsh.monkeybrains.net]; RCVD_NO_TLS_LAST(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[19.40.69.208.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:32329, ipnet:208.69.40.0/22, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[monkeybrains.net.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 07:07:58 -0000 I've switched to VNET (love it) in jails. Neat, you an have ipfw running in your jail! I added some log lines to test it out and was a bit confused when /var/log/security wasn't showing the log lines. Turns out, the kernel is grabbing them and logging in the host and not the chrooted environment. Bug? Feature? :) Rudy From owner-freebsd-jail@freebsd.org Tue Feb 12 08:35:11 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8E4C14DC528 for ; Tue, 12 Feb 2019 08:35:11 +0000 (UTC) (envelope-from srs0=cpp7=qt=vega.codepro.be=kp@codepro.be) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4C584802FF for ; Tue, 12 Feb 2019 08:35:11 +0000 (UTC) (envelope-from srs0=cpp7=qt=vega.codepro.be=kp@codepro.be) Received: by mailman.ysv.freebsd.org (Postfix) id 0AF9514DC525; Tue, 12 Feb 2019 08:35:11 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAE2A14DC524 for ; Tue, 12 Feb 2019 08:35:10 +0000 (UTC) (envelope-from srs0=cpp7=qt=vega.codepro.be=kp@codepro.be) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 865DE802FC for ; Tue, 12 Feb 2019 08:35:10 +0000 (UTC) (envelope-from srs0=cpp7=qt=vega.codepro.be=kp@codepro.be) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id 7C1C9F926; Tue, 12 Feb 2019 09:35:08 +0100 (CET) Received: by vega.codepro.be (Postfix, from userid 1001) id 76F8B2B82C; Tue, 12 Feb 2019 09:35:08 +0100 (CET) Date: Tue, 12 Feb 2019 09:35:08 +0100 From: Kristof Provost To: "Rudy (bulk address)" Cc: jail@freebsd.org Subject: Re: "ipfw log" messages from jail show in host syslog Message-ID: <20190212083508.GG8450@vega.codepro.be> References: <2331cedc410f2123b2a0e142f81bf92e.squirrel@mail.monkeybrains.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <2331cedc410f2123b2a0e142f81bf92e.squirrel@mail.monkeybrains.net> X-Checked-By-NSA: Probably User-Agent: Mutt/1.11.2 (2019-01-07) X-Rspamd-Queue-Id: 865DE802FC X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 08:35:12 -0000 On 2019-02-11 22:37:07 (-0800), Rudy (bulk address) wrote: > I've switched to VNET (love it) in jails. Neat, you an have ipfw running > in your jail! > > I added some log lines to test it out and was a bit confused when > /var/log/security wasn't showing the log lines. Turns out, the kernel is > grabbing them and logging in the host and not the chrooted environment. > > Bug? Feature? :) > "Known limitation", I think[*]. >From a quick look at the ipfw log code it appears to simply write the logging information to the kernel log, which is not a per-jail things. I don't expect this to be easy to change either. Regards, Kristof [*] Not an ipfw maintainer. Warranty void where prohibited. Do not feed after midnight. From owner-freebsd-jail@freebsd.org Tue Feb 12 14:37:59 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44BBC14E8680 for ; Tue, 12 Feb 2019 14:37:59 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4B18D8EA68 for ; Tue, 12 Feb 2019 14:37:58 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id EF1D714E867F; Tue, 12 Feb 2019 14:37:57 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7CE214E867E for ; Tue, 12 Feb 2019 14:37:57 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mail-it1-x12f.google.com (mail-it1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 238458EA67 for ; Tue, 12 Feb 2019 14:37:57 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mail-it1-x12f.google.com with SMTP id c9so8044539itj.1 for ; Tue, 12 Feb 2019 06:37:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-transfer-encoding; bh=YL9B1znesViNc//Rj2nDVCBTbo73aKU7xztS6sluwHI=; b=BAcCq7yxhYir7/g7gAGHWRN7shMFw4WeTtsJTQMZgadpcsYHZiZaj8IeAYsHy+ZkhA WnJ/49mQX1liKcHdZOwgkSuq8Q/e5dzfmUoim4m64nt1oHwi+Uq4lGOwJEOjAC2YNgpv oSY4xfDfStiCNNIOcdjNv3Q6MFMT0qruOEP+/5RjW4lv/NoqAjo2l71Pp7MThdGpULcU Mm5/PQn5wyhh+Ojtfl5sSLqBg5E83hkbpWNawpAIogPRA7qDmysmDNifgVVi7fCxXzO1 vCM4xVQtgdse2yeVrssp7XwHyw8KfLhrXIHn1hhckPeIEhqioe1maZ06dEA0taeBHWwv D2lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-transfer-encoding; bh=YL9B1znesViNc//Rj2nDVCBTbo73aKU7xztS6sluwHI=; b=DsXerMP4MJidkAD3CDUQRueRnfIyvPWptiN7ee5MmltogFJDpb8WIvpR0+BPLnTL8F JwmrlNTiu1IUd2IpUoamM50Dz9N2kj2gelHu0qB99d4+ZJAqi5600q+7gUPAQmeiN6IV t7MsH541uP+w995H78diYMXWDg1qH5h5pp7vab/HCsyG9ZrvqAsvITb0Bx3XZ51HddoH agOP6kxXri/HGzM9QzZoohhmBQnp/OfxagUbbtevZLN5e/JixCG9QN9BhbzYVCMfzyYB 5a/XPcojOOq2Bz6mk1HTWsZlOKoFi0iPh7M6wnnq9wTtJa8M4K2fY0U5sbHhfI+scErf U0VA== X-Gm-Message-State: AHQUAuYAPZSYTvM1Ld/goN0PyfR4OhX0pTPlG2jLg8fwXEYVVYhSY9eG Ml1zaolBOQWTkE/Kq4WN7EFcXDlq X-Google-Smtp-Source: AHgI3Ibrk5cny4w56RqskC9+D4wt/XE+QQKsJ+/IiLuQZyxz8rv7pnQLpQXBzzzsGNEvPskj2VKISw== X-Received: by 2002:a6b:bc83:: with SMTP id m125mr2045234iof.83.1549982276349; Tue, 12 Feb 2019 06:37:56 -0800 (PST) Received: from [10.0.10.8] (cpe-65-25-53-210.neo.res.rr.com. [65.25.53.210]) by smtp.googlemail.com with ESMTPSA id o18sm1449623ioa.40.2019.02.12.06.37.54 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 12 Feb 2019 06:37:55 -0800 (PST) Message-ID: <5C62DA43.8050202@gmail.com> Date: Tue, 12 Feb 2019 09:37:55 -0500 From: Ernie Luzar User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: "Rudy (bulk address)" CC: jail@freebsd.org Subject: Re: "ipfw log" messages from jail show in host syslog References: <2331cedc410f2123b2a0e142f81bf92e.squirrel@mail.monkeybrains.net> In-Reply-To: <2331cedc410f2123b2a0e142f81bf92e.squirrel@mail.monkeybrains.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 238458EA67 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 14:37:59 -0000 Rudy (bulk address) wrote: > I've switched to VNET (love it) in jails. Neat, you an have ipfw running > in your jail! > > I added some log lines to test it out and was a bit confused when > /var/log/security wasn't showing the log lines. Turns out, the kernel is > grabbing them and logging in the host and not the chrooted environment. > > Bug? Feature? :) > > Rudy > This is a known bug problem. There is a PR about this filed a few years ago. Now here is the good news. There is a simple solution. IPFW has the option to use an un-documented log file named ipfw0. When this log file is used in a vnet jail, IPFW does log to it at /var/log/security in the vnet jail. Add this to the rc.conf file of the vnet jail and restart the vnet jail to activate. firewall_logging ="NO" firewall_logif="YES" nohup tcpdump -lnti ipfw0 | logger -t jailname -p security.info & I am having network problems configuring my vnet jail on 12.0, using bridge/epair with ipfw/nated. I sure would appreciate your help in figuring out what is incorrect with my setup. If your agreeable, contact me off list. Thanks From owner-freebsd-jail@freebsd.org Tue Feb 12 14:47:39 2019 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8505114E8E2E for ; Tue, 12 Feb 2019 14:47:39 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 277758F182 for ; Tue, 12 Feb 2019 14:47:39 +0000 (UTC) (envelope-from pi@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DAE4C14E8E2D; Tue, 12 Feb 2019 14:47:38 +0000 (UTC) Delivered-To: jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8C0F14E8E2C for ; Tue, 12 Feb 2019 14:47:38 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 602798F180 for ; Tue, 12 Feb 2019 14:47:38 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from pi by home.opsec.eu with local (Exim 4.91 (FreeBSD)) (envelope-from ) id 1gtZLT-000Pme-Jz; Tue, 12 Feb 2019 15:47:35 +0100 Date: Tue, 12 Feb 2019 15:47:35 +0100 From: Kurt Jaeger To: Ernie Luzar Cc: "Rudy (bulk address)" , jail@freebsd.org Subject: Re: "ipfw log" messages from jail show in host syslog Message-ID: <20190212144735.GL2748@home.opsec.eu> References: <2331cedc410f2123b2a0e142f81bf92e.squirrel@mail.monkeybrains.net> <5C62DA43.8050202@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5C62DA43.8050202@gmail.com> X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2019 14:47:39 -0000 Hi! > Rudy (bulk address) wrote: > > I've switched to VNET (love it) in jails. Neat, you an have ipfw running > > in your jail! > > > > I added some log lines to test it out and was a bit confused when > > /var/log/security wasn't showing the log lines. Turns out, the kernel is > > grabbing them and logging in the host and not the chrooted environment. > This is a known bug problem. There is a PR about this filed a few years ago. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=178482 -- pi@opsec.eu +49 171 3101372 One year to go !