Date: Tue, 30 Apr 2019 13:24:04 +0200 From: "Martin Jakob" <mj-mailinglist@gmx.de> To: freebsd-pkgbase@freebsd.org, freebsd-stable@freebsd.org Subject: Creating jails from pkgbase packages Message-ID: <trinity-e90ed83d-dc34-437c-9243-c4447a1a9bf4-1556623444702@3c-app-gmx-bs07>
next in thread | raw e-mail | index | archive | help
My first attempt seems to be empty when its arrived at the maillinglist, so=
i try it again in pure text format (i hope)=2E Sorry, if there are unwante=
d duplicates=2E=20
Since a few days i was experimenting with jails build from pkgbase (i got =
the idea from Michael W=2E Lucas Freebsd Jails book)=2E Now there is some d=
iscussion about pkgbase again :)
So i want to share my experience, i hope this is not offtopic to this conv=
ersation and my writing style is not too offputting=2E=2E=2E
=C2=A0
This experiments were made on an resently updated 12-stable system=2E Sorr=
y, i dont have a system with head=C2=A0available right now=2E
=C2=A0
Anyway, this is what i did:
- I first set the variable "REPODIR=3D/usr/repo" in /etc/make=2Econf, to h=
ave a nonvolatile repository (is this the right place/variable to do this?)=
=2E
- then after the standard "make buildworld|installworld|buildkernel|instal=
lkernel" in /usr/src, i created the pkgbase packages with the command "make=
packages"=2E
=C2=A0 the files are placed in "/usr/repo/FreeBSD:12:amd64/latest" where l=
atest is a symlink to the, well latest build=2E
=C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 14 Apr=2E 14:59 12=2E=
0=2Es20190414123806/
=C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 21 Apr=2E 09:27 12=2E=
0=2Es20190419155807/
=C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 22 Apr=2E 13:53 12=2E=
0=2Es20190422094219/
=C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 27 Apr=2E 14:12 12=2E=
0=2Es20190427084356/
=C2=A0 lrwxr-xr-x =C2=A01 root =C2=A0wheel =C2=A0 20 27 Apr=2E 11:04 lates=
t@ -> 12=2E0=2Es20190427084356
=C2=A0=C2=A0
- this repo is published via the file "/usr/local/etc/pkg/repos/FreeBSD-ba=
se=2Econf"which has this content:
root@betablock:~ # cat "/usr/local/etc/pkg/repos/FreeBSD-base=2Econf"
# FreeBSD base system repository
FreeBSD-base: {
=C2=A0 url: "file:///usr/repo/${ABI}/latest",
=C2=A0 mirror_type: "none",
=C2=A0 enabled: yes
}
=C2=A0
now to the jail stuff:
- this command installs the FreeBSD-runtime and the 72 neccesary dependenc=
ies in the jails root directory /jails/test03:
root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru=
e' install -r FreeBSD-base FreeBSD-runtime
=C2=A0
- the newly created jail "test03" (an entry in /etc/jail=2Econf already ex=
ists) is started with the command:
root@betablock:~ # jail -vc test03
=C2=A0
sadly the jail wont start, it exits with this message:
=2E=2E=2E
test03: created
test03: run command in jail: /bin/sh /etc/rc
jail: test03: getpwnam: No such file or directory
jail: test03: /bin/sh /etc/rc: failed
test03: removed
=2E=2E=2E
=C2=A0
This happens, because there are no user-database files (like master=2Epass=
wd, pwd=2Edb, spwd=2Edb) and no group files in the jails /etc/ directory=2E=
hmmm=2E=2E=2E
I saw in the pkg install output of the FreeBSD-runtime =C2=A0pkg this mess=
age:
=2E=2E=2E
[2/72] Extracting FreeBSD-runtime-12=2E0=2Es20190414123806: 100%
pwd_mkdb: /jails/test03/etc/master=2Epasswd: No such file or directory
=2E=2E=2E
so pkg tries to run the pwd_mkdb as defined in the runtime=2Eucl file, or =
embedded in the pkg file, it is viewable with this command:
root@betablock:~ # pkg info --raw --file /usr/repo/FreeBSD:12:amd64/latest=
/FreeBSD-runtime-12=2E0=2Es20190414123806=2Etxz
=2E=2E=2E
scripts: {
=C2=A0 =C2=A0 post-install: "cap_mkdb -l ${PKG_ROOTDIR}/etc/login=2Econf\n=
\tpwd_mkdb -i -p -d =C2=A0${PKG_ROOTDIR}/etc ${PKG_ROOTDIR}/etc/master=2Epa=
sswd\n\tservices_mkdb -l -q -o ${PKG_ROOTDIR}/var/db/services=2Edb ${PKG_RO=
OTDIR}/etc/services\n\tchmod 1777 ${PKG_ROOTDIR}/tmp"
}
=2E=2E=2E
- so for a quick fix, i copied the "missing" files (master=2Epasswd, pwd=
=2Edb, spwd=2Edb, group) from a FreeBSD base=2Etxz package to my jails etc =
directory and tried to start the jail again=2E Read below for a "better" so=
lution=2E
Now the jail starts:
root@betablock:~ # jls
=C2=A0 =C2=A0 =C2=A0JID =C2=A0IP Address =C2=A0 =C2=A0 =C2=A0Hostname =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Pa=
th
=C2=A0 =C2=A0 =C2=A0=2E=2E=2E
=C2=A0 =C2=A0 =C2=A05 =C2=A0192=2E168=2E0=2E113 =C2=A0 test03=2Elocal =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/jails/test03
=C2=A0
i can "enter" it with this command:
root@betablock:~ # jexec -l test03
=C2=A0=C2=A0
now to some more experiments with the jail which results in some questions=
(also, see below)
=C2=A0
- trying to ping something:
root@test03:~ # ping google=2Ecom
ld-elf=2Eso=2E1: Shared object "libcap_dns=2Eso=2E1" not found, required b=
y "ping"
=C2=A0
the required library libcap_dns=2Eso=2E1 is in the file FreeBSD-libcasper-=
12=2E0=2Es20190414123806=2Etxz, so lets install it:
Outside of the jail:
root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru=
e' install -r FreeBSD-base FreeBSD-libcasper
=C2=A0
and inside the jail:
root@test03:~ # ping google=2Ecom
PING google=2Ecom (216=2E58=2E213=2E206): 56 data bytes
64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D0 ttl=3D52 time=3D23=2E398 =
ms
64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D1 ttl=3D52 time=3D23=2E975 =
ms
64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D2 ttl=3D52 time=3D23=2E252 =
ms
=2E=2E=2E
success!
=C2=A0
The size of this freshly installed jail:
du -h -d 1 /jails/
=2E=2E=2E
90M =C2=A0 =C2=A0/jails/test03
=C2=A0
- now lets install some packages from outside of the jail:
root@betablock:~ # pkg --rootdir /jails/test03 install nginx-full
Updating FreeBSD repository catalogue=2E=2E=2E
pkg: Repository FreeBSD load error: access repo file(/jails/test03/var/db/=
pkg/repo-FreeBSD=2Esqlite) failed: No such file or directory
Fetching meta=2Etxz: 100% =C2=A0 =C2=A0944 B =C2=A0 0=2E9kB/s =C2=A0 =C2=
=A000:01
Fetching packagesite=2Etxz: 100% =C2=A0 =C2=A06 MiB =C2=A0 2=2E2MB/s =C2=
=A0 =C2=A000:03
Processing entries: 100%
FreeBSD repository update completed=2E 31883 packages processed=2E
Updating Synth repository catalogue=2E=2E=2E
Synth repository is up to date=2E
Updating FreeBSD-base repository catalogue=2E=2E=2E
FreeBSD-base repository is up to date=2E
All repositories are up to date=2E
The following 146 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 nginx-full: 1=2E16=2E0_2,2 [FreeBSD]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 apache24: 2=2E4=2E39 [FreeBSD]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 libnghttp2: 1=2E38=2E0 [FreeBSD]
=2E=2E=2E
=C2=A0 =C2=A0 =C2=A0 =C2=A0 libdrizzle: 0=2E8_6 [FreeBSD]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 brotli: 1=2E0=2E7_1,1 [FreeBSD]
Number of packages to be installed: 146
The process will require 953 MiB more space=2E
232 MiB to be downloaded=2E
=2E=2E=2E
=C2=A0
Funfact -=C2=A0the jailsize after this installation:
1,0G =C2=A0 =C2=A0/jails/test03
=C2=A0
- lets enable the nginx service
root@betablock:~ # sysrc -R /jails/test03 nginx_enable=3DYES
nginx_enable: =C2=A0-> YES
=C2=A0
- and start it (after a jail restart, without it nginx does not find its l=
ibraries, installed above)
root@test03:~ # service nginx start
Performing sanity check on nginx configuration:
ld-elf=2Eso=2E1: Shared object "libexecinfo=2Eso=2E1" not found, required =
by "libprofiler=2Eso=2E0"
=C2=A0
- lets install this missing library with the command=C2=A0
root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru=
e' install -r FreeBSD-base FreeBSD-libexecinfo
=C2=A0
- and lets also install vi:
root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru=
e' install -r FreeBSD-base FreeBSD-vi
=C2=A0
- so lets try again (after creating the missing config files nginx=2Econf =
and mime=2Etypes):
root@betablock:~ # jexec -l test03 service nginx start
Starting nginx=2E
nginx: [emerg] gethostbyname() failed in /usr/local/etc/nginx/nginx=2Econf=
:122
=C2=A0
- and the check:
root@betablock:~ # jexec -l test03 service nginx status
nginx is running as pid 41359=2E
=C2=A0
(Browser access works too=2E=2E=2E)
=C2=A0
- Now Lets work with packages within the jail:
root@test03:~ # pkg update
The package management tool is not yet installed on your system=2E
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/lat=
est, please wait=2E=2E=2E
pkg: Error loading revoked certificates
=C2=A0
- Hm, this is because the directory "/usr/share/keys/pkg/revoked/" is miss=
ing, lets create it:
root@test03:~ # mkdir /usr/share/keys/pkg/revoked/
=C2=A0
- and now:
root@test03:~ # pkg update
The package management tool is not yet installed on your system=2E
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/lat=
est[http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/latest], please wait=2E=2E=
=2E
Verifying signature with trusted certificate pkg=2Efreebsd=2Eorg=2E2013102=
301=2E=2E=2E done
[test03=2Elocal] Installing pkg-1=2E10=2E5_5=2E=2E=2E
[test03=2Elocal] Extracting pkg-1=2E10=2E5_5: 100%
Updating FreeBSD repository catalogue=2E=2E=2E
FreeBSD repository is up to date=2E
All repositories are up to date=2E
root@test03:~ # pkg info
FreeBSD-casper-12=2E0=2Es20190414123806 casper package
FreeBSD-clibs-12=2E0=2Es20190414123806 Core C Libraries
FreeBSD-jail-12=2E0=2Es20190414123806 Jail Utilities
FreeBSD-lib-12=2E0=2Es20190414123806 lib package
FreeBSD-lib80211-12=2E0=2Es20190414123806 lib80211 package
=2E=2E=2E
x265-3=2E0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 H=2E265/High Efficiency Video Coding (HEVC) format
xorg-fonts-truetype-7=2E7_1 =C2=A0 =C2=A0 =C2=A0X=2EOrg TrueType fonts
xorgproto-2018=2E4 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 xorg p=
rotocol headers
xvid-1=2E3=2E5,1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 Opensource MPEG-4 codec, based on OpenDivx
yajl-2=2E1=2E0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 Portable JSON parsing and serialization library in ANSI C
Btw=2E The missing "revoked" directory problem shoud be fixed with https:/=
/reviews=2Efreebsd=2Eorg/D20056[https://reviews=2Efreebsd=2Eorg/D20056]
---
So this is a basic jail installation with pkgbase, now to some further exp=
eriments and resulting questions:
=C2=A0
- i looked into why the userdatabase was missing in the newly created jail=
=2E Well, it is, because the files are not included in the default Freebsd-=
runtime package=2E
=C2=A0 To check if it would work, i built my own Freebsd-runtime package w=
ith these files added=2E to do this, i copied the runtime=2Eplist file from=
the $WSTAGEDIR and modified it=2E
=C2=A0 - first i removed a lot of duplicated directory entries like
=C2=A0 @dir(root,wheel,0755,) /boot
=C2=A0 @dir(root,wheel,0755,) /boot
=C2=A0 @dir(root,wheel,0755,) /boot
=C2=A0 @dir(root,wheel,0755,) /boot
=C2=A0 =2E=2E=2E (and many more)
=C2=A0 - then i added the "missing" files and directory
=C2=A0 @config(root,wheel,0600,) /etc/master=2Epasswd
=C2=A0 @config(root,wheel,0644,) /etc/group
=C2=A0 @dir(root,wheel,0755,) /usr/share/keys/pkg/revoked
=C2=A0=C2=A0
=C2=A0 - after that i rebuild the Freebsd-runtime package with these chang=
es (found in /usr/src/Makefile=2Einc1):
=C2=A0 root@betablock:~ # pkg -o ABI_FILE=3D/usr/obj/usr/src/amd64=2Eamd64=
/worldstage/usr/bin/uname -o ALLOW_BASE_SHLIBS=3Dyes create -M /usr/obj/usr=
/src/amd64=2Eamd64/worldstage/runtime=2Eucl -p /root/pkgs/runtime=2Eplist -=
r /usr/obj/usr/src/amd64=2Eamd64/worldstage --output-dir /usr/repo/FreeBSD:=
12:amd64/latest
=C2=A0=C2=A0
=C2=A0 and rebuilding the repo metadata with:
=C2=A0 root@betablock:~ # pkg repo /usr/repo/FreeBSD:12:amd64/latest
=C2=A0 a jail created with this Freebsd-runtime starts directly, without t=
he steps mentioned above=2E
=C2=A0=C2=A0
=C2=A0 My Question: Is the ommission of these files intentional?=C2=A0
=C2=A0=C2=A0
=C2=A0 - one last experiment:=C2=A0i deleted the manpages, the examples, a=
nd the /boot directory (i guessed a jail has no need for it), the jail buil=
d from this file is 75M, the installation of packages like nginx=C2=A0works=
like in the original file=2E
=C2=A0=C2=A0
=C2=A0 - should the FreeBSD-libcasper package be installed as a dependency=
of the FreeBSD-casper package?
=C2=A0
=C2=A0 - i did not observe more missing libraries like libcasper above, bu=
t depending on the tasks that are=C2=A0executed or packages that are instal=
led, there might be more that are missing=2E
=C2=A0
So that was my pkgbase-jail experiment, it was fun and i learned some thin=
gs about pkgbase=2E Any comments, tips, tricks and (mild) critique are welc=
ome=2E
=C2=A0
Greetings
=C2=A0
Martin
=C2=A0
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?trinity-e90ed83d-dc34-437c-9243-c4447a1a9bf4-1556623444702>