Date: Thu, 25 Feb 2016 11:50:37 +1100 From: Aristedes Maniatis <ari@ish.com.au> To: Mark Felder <feld@freebsd.org>, freebsd-jail <freebsd-jail@freebsd.org> Subject: Re: Jail management Message-ID: <ed308c0d-16e1-1dda-5e6a-0f8ecdfebb53@ish.com.au> In-Reply-To: <1456347178.2985454.530948890.160132B1@webmail.messagingengine.com> References: <ff8307f6-1264-30ec-1ef8-ed3b0a18dd84@ish.com.au> <1456347178.2985454.530948890.160132B1@webmail.messagingengine.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Vx58mOf2efqg0dl5cDTduxfIfolEQIx5v Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 25/02/2016 7:52am, Mark Felder wrote: >=20 >=20 > On Sun, Feb 21, 2016, at 19:13, Aristedes Maniatis wrote: >> * It is hard to reproduce the environment exactly, matching the >> application to the same version of Java that was available at the time= of >> deployment. Again I'm fighting against the pkg system which always wan= ts >> the latest version. >=20 > The package system *could* handle this, but it doesn't fit our design. > We aren't like RedHat/Debian where we "freeze" packages at a certain > version at the OS release and then backport only changes. With that > method different versions of packages will just work with everything > else in the system. With FreeBSD's ports system it's really a rolling > release as the entire ports tree moves together. Mixing packages build > from different checkouts of the ports tree is dangerous and not > guaranteed to work. Hi Mark Yes, that makes sense and I've frequently struggled in Linux systems to g= et all the bits to match each other properly. So the FreeBSD solution her= e works. But for me, where I use poudriere and roll my own custom packages, versio= ning becomes complicated. I'd need to either create a new poudriere jail = for every release of my software, or go down the path of snap-shotting th= e entire jail (I'm choosing the second). > I don't use ezjail. It doesn't upgrade well, and changes to the base > jail require you stop all your jails. FreeBSD fat jails are so small > (300MB?) it's not worth it in my opinion. I simply wrote a shell script= > to create fat jails and another script to handle updating them all. > They're all treated like full servers/VMs, and configs/roles are manage= d > with Ansible/Salt/etc. That's a good point. And after all the excellent advice here, this is pro= bably what I'll do: 1. Discard salt-minions inside every jail. That's become more trouble to = look after as the number of jails grows. That's also really tricky to han= dle in salt when I want to fail over jails to another host. Since then ev= ery jail is running in two places and that confuses salt. 2. Create a single master/template jail. That might include the basejail = or perhaps I'll keep the nullfs thing which works fine. 3. With every release of my software, upgrade that jail using 'pkg upgrad= e', test and 'zfs snapshot pool/template@v8.10'. I'll keep accumulating s= napshots for every release, bundling up the changes to my software plus a= ll the dependency packages and config. 4. When I want to upgrade a customer jail, I stop the jail and: # zfs destroy pool/customerJail # zfs clone pool/template@v8.10 pool/customerJail I'll have salt help with automating this of course. By using the zfs clon= e command, even the 300Mb of FreeBSD userland takes zero bytes to clone. = These commands should really take less than a second to execute, so the u= pgrade speed is great. 5. Then salt will write out the appropriate customer specific configurati= on into the newly cloned jail (for us that's just 2-3 files) 6. Start jail Where having no basejail falls down is when the host system goes from Fre= eBSD 10 to 11, then I'll need to upgrade every jail but I'll have no way = to go backwards to an older version once this is done since the old snaps= hot will include the old userland. That's probably not a problem for some= thing that is rare. Also, all the above works only because we use logstash and therefore don'= t care about losing all the logs inside each jail with every upgrade. I g= uess syslog would do the same thing for other people. I hope this little summary helps other people facing similar challenges. Ari --=20 --------------------------> Aristedes Maniatis CEO, ish https://www.ish.com.au GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A --Vx58mOf2efqg0dl5cDTduxfIfolEQIx5v Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAlbOT94ACgkQ72p9Lj5JECpUfQCfSlzdHFgqdFugV4UsPeaYQjQG 2EMAnjD99kAmItp4EeIh/FOo7MP4/ppA =7ker -----END PGP SIGNATURE----- --Vx58mOf2efqg0dl5cDTduxfIfolEQIx5v--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ed308c0d-16e1-1dda-5e6a-0f8ecdfebb53>