From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 20 07:39:37 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4BE3E2EB for ; Wed, 20 Feb 2013 07:39:37 +0000 (UTC) (envelope-from freebsd@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id 97C3F826 for ; Wed, 20 Feb 2013 07:39:36 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) by mx1.psconsult.nl (8.14.5/8.14.4) with ESMTP id r1K7dUXs059834 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 20 Feb 2013 08:39:35 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.5/8.14.4/Submit) id r1K7dUkN059833; Wed, 20 Feb 2013 08:39:30 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Wed, 20 Feb 2013 08:39:30 +0100 From: Paul Schenkeveld To: Alexander Yerenkow Subject: Re: Chicken and egg, encrypted root FS on remote server Message-ID: <20130220073930.GA59518@psconsult.nl> References: <20130220065810.GA25027@psconsult.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Feb 2013 07:39:37 -0000 On Wed, Feb 20, 2013 at 09:14:22AM +0200, Alexander Yerenkow wrote: > 2013/2/20 Paul Schenkeveld > > > Hi, > > > > I've been trying to find a solution for this chicken and egg problem, > > how to have an encrypted root filesystem on a remote server. > > > > Geli can ask for a root password at the console to unlock the root fs > > but that of course won't work for a remote server. > > > > Ideally I'd like the server to start, do minimal network config, run > > a minimal ssh client (dropbear?) and wait for someone to log in, > > provide the passphrase to unlock the root filesystem and then mount > > the root filesystem and do a normal startup. > > > > I read about a pivotroot call in other OS-es, that would allow for a > > very small unencrypted root filesystem to be mounted temporarily until > > the passphrase has been entered and then swap that for a real, encrypted > > root filesystem. But AFAIK we don't have pivotroot. > > > > The problem could also be solved if the real root fs could be union > > mounted over the small unencrypted one but unionfs won't mount over /. > > > > I found out that a ZFS pool where a specific dataset has the > > mountpoint=/ option set can be used to 'buri' the unencrypted root under > > the real root but that would render the unencrypted one unchangable > > after the real one is mounted (prohibiting sysadmin to change the ssh > > credentials or network config there). It would also make maintenance > > a bit more difficult because an import of the zpool would automatically > > remount /, even when running from a cd-rom or USB stick. And of course > > this approach would not work in non-zfs environments (like very small > > systems). > > > > Looking at sys/kern/init_main.c and sys/kern/vfs_mount.c I could > > imagine having a kind of "pre root environment", an unencrypted root > > that gets mounted first (along with a devfs) and a /sbin/init that > > sets up minimal networking and runs sshd. Aftre that one dies the > > unencrypted root and devfs would be unmounted, the real root mounted > > and the real /sbin/init started. But this may be a considered a dirty > > approach. > > > > Did I miss the obvious and easy solution? Any ideas? > > > > I'd like to propose you to see my similar setup - it's used for VMs. > Idea is to have base OS in little partition, and use it only in RO. > All data, and configs goes in different partitions, which could be > connected manually or automatically. So far, I've done something similar using the nanobsd(8) framework. Disadvantage is that usually /etc is either part of the root filesystem or a md(4) filesystem (like the diskless framework and nanobsd use). Another disadvantage is that upgrades to the root filesystem (installworld, installkernel) do not work out of the box. I was hoping to get closer to a 'normal' installation for certain servers I maintain. My ramblings about using nanobsd for servers can be found here: http://www.psconsult.nl/talks/AsiaBSDcon2010-Servers/ > What you need is to specify in loader.conf > init_script="/etc/find-rwfs.sh" Hey, this one is new to me. Very interesting, won't solve this problem but would allow things like having /etc on a separate filesystem and things like that. > My example: > https://github.com/yerenkow/freebsd-vm-image/blob/master/freebsd-firmware/find-rwfs.sh Interesting. > In this way, you can get RO booted OS, just waited for you to login and > mount all required data. > > Also, you could contact Andriy Gapon, he managed to do other trick - boot > from RO media such as CD, and run all OS as chroot, all transparently. Thought about the chroot trick too but prefer to stay away from chroot if possible. > Hope this helps! With kind regards, Paul Schenkeveld