Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 14 Oct 2010 13:45:25 -0700
From:      Devin Teske <dteske@vicor.com>
To:        Christian Degen <bubulein@freenas.org>
Cc:        freebsd-jail@freebsd.org
Subject:   Re: jail_build(8) -- a dialog(1)-based utility for building jails from binary distributions
Message-ID:  <1287089125.15487.135.camel@localhost.localdomain>
In-Reply-To: <AANLkTi=y7ZL%2Bxp81wxucYi0cwxSQL84wn-cjxXYMs8Hf@mail.gmail.com>
References:  <1287017429.26377.47.camel@localhost.localdomain> <AANLkTi=y7ZL%2Bxp81wxucYi0cwxSQL84wn-cjxXYMs8Hf@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 2010-10-14 at 14:16 +0200, Christian Degen wrote:
> Hello Devin,
> 
> you can take a look at http://www.chruetertee.ch/jailscript/ , this
> script creates a jail from src

Surely there may be _some_ similarities between your script and mine,
though admittedly, I believe that the similaries end at our usage of
dialog(1).

My script builds jails from binary [pre-compiled] distributions, not
from source. This difference alone means that our scripts will share
very little in-common because the process of assembling a jail from
source is completely different than assembling from binary.

The reason that I wrote my script is because provisioning a jail from a
binary distribution is several orders of magnitude _faster_ than
building from source.

When building a jail from source, one must:

First prep:
a. c[v]sup/portsnap the source code
b. `buildworld' in /usr/src
Then for each jail:
c. `installworld' to jail root

When building a jail from binary, one must:

First prep:
a. Download/copy the binary release from FTP or CD/DVD to /usr/repos
Then for each jail:
b. Unpack the binary release to the jail root

The high-level difference doesn't look very substantial, but it _is_
because the `buildworld' step in the source-setup routine:

1. Takes hours (your mileage may vary -- based on hardware performance)
2. Takes up disk space (`/usr/obj' will occupy approx. 600MB depending
what features you've enabled/disabled in your build)

It's also worth noting...

When you build from source, you need approx. 600MB of disk space for
`/usr/src' (this is in addition to the approx. 600MB needed for
`/usr/obj' -- the compiled source that you `installworld' from).

Meanwhile, when you build a jail from the binary distribution, you only
need approx. 300MB for the binary distribution release:

[dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ ls
ERRATA.HTM      README.TXT      cdrom.inf       info            proflibs
ERRATA.TXT      RELNOTES.HTM    dict            kernels         src
HARDWARE.HTM    RELNOTES.TXT    doc             manpages
HARDWARE.TXT    base            docbook.css     packages
README.HTM      catpages        games           ports
[dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ du -hs .
741M    .
[dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ du -hs packages
438M    packages
[dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ expr 741 - 438
303

Even less (241MB) if you consider that jails don't need kernels:

[dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ du -hs kernels
 62M    kernels
[dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ expr 303 - 62
241

NOTE: My script will omit the kernels dist if it doesn't exist when
building the jail.

Last, but not least...

$ time sr jail_build
[...]
Jail successfully created.

real    0m40.593s
user    0m8.383s
sys     0m6.551s

I wrote this script with speed in-mind, and I'd say that being able to
provision a jail from a binary release in 40 seconds is pretty-damned
efficient.




> , perhaps you find some good ideas. It
> is written with German dialogues, but code is code.

You're right, code is code ^_^

And I'm pleasantly surprised that I haven't forgotten _all_ of my German
^_^

First, a few comments...

=========================================

I've never fathomed putting a jail into an md(4) image.

What advantages to using an md(4) based disk-image have you encountered?
What disadvantages have you encountered?

=========================================

Nice usage of truncate(1) -- I would have used dd(1) with /dev/zero

But then again, I didn't know that truncate(1) went back so far (it's
available on my FreeBSD-4.11 system ... amazing).





Now, a few points where I noticed you could optimize your script...

=========================================

I notice that you're script immediately prompts the user for two
inputs... first the hostname of the new jail ("Name der Jail") and then
immediately thereafter the domain name of the jail ("Domain Name der
Jail").

Why not prompt the user for the fully-qualified host name (FQHN) and
then split the result?

For example:

foo=fully.qualified.host.com
name="${foo%%.*}" ## "fully"
domain="${foo#*.}" ## "qualified.host.com"

=========================================

I notice that you're sometimes trapping the return status of dialog(1),
later comparing it against either 1 or 255, and exiting for both 1 and
255.

In those cases where you exit for both 1 and 255, (in example:)

case ${retval} in
  1)
    exit;;
  255)
    exit;;
esac

why not simply instead test for non-success (non-zero):

# written in same if/then style preened from your source
if [ $retval -ne 0 ] ;
	then
		exit
fi


Which (FYI) can be written shorter as:

[ $retval -eq 0 ] || exit

or

[ $retval -ne 0 ] && exit

=========================================

The following is not required:

verz=`echo ${verz} | sed -e 's/\/\//\//g'`

The path: /usr/bar//baz///abc

Is understood to be the same as: /usr/bar/baz/abc

Yes, the former looks ugly, but it is still usable.

If you *really* want to clean up ugly paths, this is a better way:

verz=`eval realpath ${verz}`

NOTE: It may not be immediately obvious, but the "eval" portion is
required to support tilde-expansion (this is because tilde-expansion is
performed only-once _and before_ parameter-expansion; the `eval' causes
re-evaluation after initial parameter-expansion, therefore allowing
tilde-expansion to be performed on the expanded parameter).

NOTE: The above does not account for paths containing spaces (though it
should also be noted that the rest of your script also lacks support for
paths with spaces, so this should be of little concern -- for example,
the above echo to sed(1) for replacing '//' with '/', ... paths with
multiple spaces will be munged due to lacking surrounding double-
quotes... the echo will see a series of positional parameters after the
parameter-expansion of ${verz} is performed, so if the path contains a
series of whitespace, the whitespace will not be retained but rather
replaced with a single space, or whatever the first character of $IFS
happens to be).

=========================================

Your script makes use of jexec(8) and jail(8).

My script does not require any jail utilities to unpack the binary
distribution to the desired jail root directory.

The core dependencies of my script (jail_build(8)) are below:
   find(1)     sort(1)   awk(1)
   dialog(1)   cat(1)    rm(1)
   mkdir(1)    tar(1)    mtree(8)

All versions of FreeBSD should be able to provide the above.

And since jails first appeared in FreeBSD-4.0, that means that my script
is still usable to people running FreeBSD-4.0 that want a better way to
provision a jail.

For example... [God Forbid] should you want to be able to provision a
FreeBSD-1.0 jail within FreeBSD-4.8, -6.2, -7.1, -8.1, -whatever... my
script will help make it happen.

=========================================

The below can be optimized...

if [ "${filedisk}" = "true" ] ;
  then

Because `true' and `false' are both valid built-ins to the shell, you
can do this (your style retained):

if $filedisk ;
  then


Parameter expansion is performed and then the command is executed, so
what is executed is really:

if true ;
  then


Which is a valid statement.

It's also more efficient, because rather than having the shell: 

1. perform parameter expansion of filedisk, then...
2. `[' is executed with 3 arguments: "$true", "=", and "true"
3. `[' reads arguments
4. `[' performs string comparison between "true" and "true"
5. `[' returns zero (for success)
6. return status is evaluated for success
7. if-block is fired on success

The following is performed instead:

1. parameter expansion of filedisk is performed
2. `true' is executed with zero arguments
3. `true' returns zero (for success)
4. return status is evaluated for success
5. if-block is fired on success

=========================================

Last,...

It should be noted that your script goes above and beyond what my script
does. My script (aptly named "jail_build") only _builds_ the jail for
you. It does not (and will never) be used for configuring a jail (I view
the process of building a jail and setting up a jail to run as two
separate routines that should be handled by two different programs -- if
at all).

This is further compounded by the fact that jails in FreeBSD-4.0 to
pre-8.0 can be considered "jails 1.0", while FreeBSD-8.x can be
considered to be "jails 2.0" and HEAD is working on "jails
3.0" (introduction of VIMAGE, etc.).

Provisioning a jail has not changed in all that time, but meanwhile the
way you go about setting up a jail has changed for each iteration.

In "jails 1.0", you had to:

a. manage your own aliases manually in rc.conf(5)
b. Look at /proc/PID/status to determine which processes where running
in association with a given jail...

In example:

$ uname -spr
FreeBSD 4.11-STABLE i386
$ grep jail /usr/local/etc/rc.d/zzjail.sh
jail /usr/vm/jail1 jail1.localdomain 192.168.1.127 /bin/sh /etc/rc
jail /usr/vm/jail2 jail2.localdomain 192.168.1.128 /bin/sh /etc/rc
jail /usr/vm/jail3 jail3.localdomain 192.168.1.129 /bin/sh /etc/rc
$ /bin/sh -c 'grep -Ilrs jail2.localdomain /proc 2> /dev/null'
/proc/503/status
/proc/364/status
/proc/360/status
/proc/358/status
/proc/355/status
/proc/353/status
/proc/346/status
$ cat /proc/360/status
sshd 360 1 360 360 -1,-1 sldr 1266020515,536071 1,28410 0,0 select 0 0 0,0 jail2.localdomain

c. Roll your own killall like-so:

/bin/sh -c 'kill -9 $(grep -Ilrs jailhostname /proc 2> /dev/null | awk -F/ "{print \$3}")'

d. Manually deconfigure aliases via ifconfig(8) when bringing the jail
down.

Meanwhile, in "jails 2.0"...

a. rc.conf(5) now has a bevy of setup options for jails
b. aliases are automatically de/configured by /etc/rc.d/jail
c. ps(1) now has a `jid' keyword (e.g.): ps axo pid,jid,command
d. killall(1) now has a `-j JID' option
e. There is now the jls(8) utility for listing running jails
f. There is now the jexec(8) utility for executing commands within a
running jail
g. Many sysctl MIBs have been added
h. Certain filesystems can be marked as ``jail-friendly'' (currently
only nullfs [through simple patch] and zfs)
i. jails can now be configured with multiple IP addresses
j. jails can run within jails

And of course... "jails 3.0" is in the works and some of the focal
points could perhaps be:

a. "Productionized" VIMAGE support
b. ``jail-friendly'' NFS
c. ``jail-friendly'' umount(8)
c. additional utilities

...

Because the landscape for managing, configuring, and maintaining jails
has changed dramatically from each iterative revision of jails, I think
it would be best if the process of building a jail were kept seperate
from the process of configuring a jail.

That is to say...

That a script whose purpose is to build a jail can act universally for
all versions of FreeBSD.

Meanwhile, any script whose purpose is to configure a jail must perform
differently depending on the faculties of the base host (since the only
thing that FreeBSD-4.x shares in-common with FreeBSD-8.x and eventually
9.x is the jail(8) utility and nothing else).

Of course... you could just call me an "over pedantic nut who for some
unknown reason cares about legacy operating systems"... but then I'd
remind you that FreeBSD-4.x was not officially EOL'd until about 3 years
ago (keep in-mind that FreeBSD develops multiple releases with separate
branch-tags simultaneoulsy and usually has three major releases in
active-support at any given time: right now it's highest-7.x,
highest-8.x, and uncut-release 9/HEAD, but it wasn't but a few years ago
that it was 4.11, highest-5.x and highest-6.x ... pre-7.0).



> Christian
> 
> 
> 
> On Thu, Oct 14, 2010 at 2:50 AM, Devin Teske <dteske@vicor.com> wrote:
> 
> > I'd like to share a script that I wrote for provisioning jails NOT from
> > the standard `buildworld'/`installworld' mechanism, but rather from
> > binary distributions. For example, those made available here:
-- 
Cheers,
Devin Teske

-> CONTACT INFORMATION <-
Business Solutions Consultant II
FIS - fisglobal.com
510-735-5650 Mobile
510-621-2038 Office
510-621-2020 Office Fax
909-477-4578 Home/Fax
devin.teske@fisglobal.com

-> LEGAL DISCLAIMER <-
This message  contains confidential  and proprietary  information
of the sender,  and is intended only for the person(s) to whom it
is addressed. Any use, distribution, copying or disclosure by any
other person  is strictly prohibited.  If you have  received this
message in error,  please notify  the e-mail sender  immediately,
and delete the original message without making a copy.

-> FUN STUFF <-
-----BEGIN GEEK CODE BLOCK-----
Version 3.1
GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? o? K- w O
M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) G+>++ e>+ h
r>++ y+ 
------END GEEK CODE BLOCK------
http://www.geekcode.com/

-> END TRANSMISSION <-




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1287089125.15487.135.camel>