From owner-freebsd-jail@FreeBSD.ORG Fri Sep 10 15:14:26 2010 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1211B106564A for ; Fri, 10 Sep 2010 15:14:26 +0000 (UTC) (envelope-from gnrp@gnrp.in-berlin.de) Received: from einhorn.in-berlin.de (einhorn.in-berlin.de [192.109.42.8]) by mx1.freebsd.org (Postfix) with ESMTP id 9065A8FC0A for ; Fri, 10 Sep 2010 15:14:25 +0000 (UTC) X-Envelope-From: gnrp@gnrp.in-berlin.de X-Envelope-To: Received: from adolfputzen (c-68-42-215-192.hsd1.mi.comcast.net [68.42.215.192]) (authenticated bits=0) by einhorn.in-berlin.de (8.13.6/8.13.6/Debian-1) with ESMTP id o8AEgR7P020360 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Fri, 10 Sep 2010 16:42:29 +0200 Date: Fri, 10 Sep 2010 16:42:22 +0200 From: Julian Fagir To: freebsd-jail@freebsd.org Message-ID: <20100910164222.31ae35d8@adolfputzen> In-Reply-To: <4C89B3DF.7050004@snap.net.nz> References: <4C89B3DF.7050004@snap.net.nz> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.14.7; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/kjJ.tJZvPz=tWh/XGF0rPBP"; protocol="application/pgp-signature"; micalg=PGP-SHA1 X-Scanned-By: MIMEDefang_at_IN-Berlin_e.V. on 192.109.42.8 Subject: Re: Jail hot migration X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-jail@freebsd.org List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Sep 2010 15:14:26 -0000 --Sig_/kjJ.tJZvPz=tWh/XGF0rPBP Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi, correct me if I'm wrong, I never actually worked with VMware (only a bit with Xen) but read a bit about the theory behind it. I don't think this can be as easilly achieved as it is with VMware (or Xen). > I was lately thinking around jail hot-migration feature where one jail > could be moved from one host to another without > shutting it down, something like vmotion in VMware world. With paravirtualisation, you have the machine in a 'container', the host system does not know except for the interfaces what the guest system is doi= ng with the ram it has reserved. So moving a machine is 'not more' than copying a bunch of memory to another machine and transferring the interfaces. Jails are no virtualisation, in the end, what makes up a jail, are only a f= ew data structures. Moving a jail would mean to transfer all these single fragments of data reasonably to another machine. I.e. you would have to transfer all the kernel-things a process does to another machine. How sould locks or access to system ressources be managed when transferring to another machine? I don't even know if the kernel does know all the stuff about a process on a central place to do this transfer. > The storage layer should be easy with zfs send and receive or some form > of shared storage. The tricky part would > be a memory copy from one node to another and also the CPU handling. Afaik you have containers for the machines with VMware/Xen, as the host system does not have to know the file system underneath. Again, transferring/sharing those is simple as it is only one file to the host system. With jails, you only have mount points. What happens to locks and other fs-specific stuff, ACLs etc (ok, zfs perhaps handles that)? But you can't r= ely on the other system having the same filesystem underneath. Also, sharing fi= le systems is not as easy as sharing a single file (without caring for permissions). You see the problem when you just look at all the different ways for locking over the network... > Anyone has an idea how this could be achieved? I guess it would require > a kernel module which could take care of memory > reservations and a daemon to copy and incrementally sync the jails > memory across. In case you could build a jail-transportation, a framework for transferring= a single process to another machine would be more appropriate imho as more generic and simpler. Then just adding the possibility of jailing such a process is not much work. A single example why the whole target cannot be achieved easily: Imagine a program which writes its pid into a lockfile. As jails and host share the pids, this pid could be already in use by the other host you want to transf= er the jail to. So what to do know? Just change the pid? Then the process would be confused and possibly die or malfunction. Not changing the pid would not be possible as then there would be two processes with the same pid. Sure, remapping pids would be possible but not within the current jails and the complications could be great (if the process sees another pid than the host-system, what will rc-files do which check lockfiles?). And there are other cases where processes do things the kernel cannot predict and so cannot transfer that easily. Also, the nic-transfer would make problems. When you usually have a bridge = or so with Xen/VMware, you have real interfaces or aliases on a BSD-machine, h= ow should they be taken out of service in time and back to service without problems (I often have cases when just taking up an alias does not work)? Jails are no virtualisation. They are an 'extended chroot'-environment and thus moving a jail is like moving a working machine except for the system drivers. I hope this was not all too wrong. Regards, Julian --Sig_/kjJ.tJZvPz=tWh/XGF0rPBP Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkyKQ84ACgkQc7h7cu1Hpp5blwCcC0mn5684xDjIW2prE/Ex8JwA 5GQAnA2ivRnmVyEsL5Ccd7v26ZDsXlqo =ugw9 -----END PGP SIGNATURE----- --Sig_/kjJ.tJZvPz=tWh/XGF0rPBP--