Date: Tue, 11 Jun 2019 11:52:31 -0700 From: Johannes Lundberg <johalun@FreeBSD.org> To: jbwlists@hilltopgroup.com, Baptiste Daroussin <bapt@freebsd.org> Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: pkg: Cannot open /dev/null:No such file or directory Message-ID: <a97754ed-9e20-f21f-ec4f-67b9ef5b2294@FreeBSD.org> In-Reply-To: <935a44aa7587cdc07fadc2e33caed1f7@hilltopgroup.com> References: <20190604073209.4e42a0eb@freyja> <20190604054409.4anei2ljzimqc75m@ivaldir.net> <d4f93ec9-da6b-cfed-000e-2488e5619c1d@FreeBSD.org> <935a44aa7587cdc07fadc2e33caed1f7@hilltopgroup.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------F545AD8270A6D91ADDA344E0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Hi This is probably overkill but I've attached a diff that show my patches to image.sh. It's just a hack so far to make it do what I want and not meant as general purpose. Use the changes you need for your application. Changes (only meant to work for building usb images on amd64 and i386) - mount devfs so pkg will work properly - add "install packages from official repo" option so you won't have to build your own packages for each jail - downloaded packages from official repo is cached locally to avoid excessive downloads - hack for i386 so packages are installed properly - increase swap space to allow for kernel core dumps (for usb image) - execute post-install script "overlay.sh" in the jail if provided in the overlay folder Bapt: Maybe you'll find some of these changes useful? :) /Johannes On 6/11/19 8:52 AM, jbwlists@hilltopgroup.com wrote: > I'm having the same issue with poudriere image; could you please let > me know what you did to fix it? I'm assuming the image.sh you're > referring to is /usr/local/share/poudriere/image.sh, but I'm not sure > where the change would need to be made. > > Thanks in advance, > Joseph > > > On 2019-06-04 13:02, Johannes Lundberg wrote: >> On 6/3/19 10:44 PM, Baptiste Daroussin wrote: >>> On Tue, Jun 04, 2019 at 07:32:16AM +0200, O. Hartmann wrote: >>>> Hello List, >>>> >>>> lately I ran into a serious problem installing packages in a nanoBSD >>>> environment, in which the package repository server is "remotely" >>>> on site. The >>>> issue as documented below occurs on both 12-STABLE r348529 and >>>> CURRENT r348600 >>>> and must have been introduced shortly, since the last known good >>>> installation >>>> with the environment of ours was on 21st May 2019. >>>> >>>> As far as I know,, the package installation is performed via >>>> "chroot'ed" >>>> environment and somehow /dev/null is out of a sudden not accessible >>>> anymore >>>> while pkg tries to delegate some output to /dev/null. >>>> >>>> What happened here? >>>> >>>> Kind regards and thanks in advance, >>>> >>>> oh >>>> >>>> [...] >>>> All repositories are up to date. >>>> The following 10 package(s) will be affected (of 0 checked): >>>> >>>> New packages to be INSTALLED: >>>> python3: 3_3 [zeit4] >>>> sudo: 1.8.27_1 [zeit4] >>>> devcpu-data: 1.22 [zeit4] >>>> python36: 3.6.8_2 [zeit4] >>>> readline: 8.0.0 [zeit4] >>>> indexinfo: 0.3.1 [zeit4] >>>> libffi: 3.2.1_3 [zeit4] >>>> gettext-runtime: 0.19.8.1_2 [zeit4] >>>> openldap-sasl-client: 2.4.47 [zeit4] >>>> cyrus-sasl: 2.1.27 [zeit4] >>>> >>>> Number of packages to be installed: 10 >>>> >>> What is new is that pkg is using /dev/null as input when running >>> script? this is >>> new since pkg 1.11 . Somehow this does not seems to be avaalaible in >>> your >>> environement. >> >> Hi >> >> Same things applies to poudriere-image. I had to add a mount devfs >> command to the image.sh script. >> >>> >>> Best regards, >>> Bapt >> >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org" --------------F545AD8270A6D91ADDA344E0 Content-Type: text/x-patch; name="image.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="image.diff" --- image.sh.orig 2019-06-06 08:24:36.445264000 -0700 +++ image.sh 2019-06-08 17:44:57.852022000 -0700 @@ -40,6 +40,7 @@ -n imagename -- The name of the generated image -o outputdir -- Image destination directory -p portstree -- Ports tree + -P -- Install packages from official repo -s size -- Set the image size -t type -- Type of image can be one of (default iso+zmfs): -- iso, iso+mfs, iso+zmfs, usb, usb+mfs, usb+zmfs, @@ -112,7 +113,7 @@ . ${SCRIPTPREFIX}/common.sh HOSTNAME=poudriere-image -while getopts "c:f:h:j:m:n:o:p:s:t:X:z:" FLAG; do +while getopts "c:f:h:j:m:n:o:Pp:s:t:X:z:" FLAG; do case "${FLAG}" in c) [ -d "${OPTARG}" ] || err 1 "No such extract directory: ${OPTARG}" @@ -146,6 +147,9 @@ OPTARG="${SAVED_PWD}/${OPTARG}" OUTPUTDIR=${OPTARG} ;; + P) + PKGREPO=1 + ;; p) PTNAME=${OPTARG} ;; @@ -239,7 +243,6 @@ mkdir -p ${WRKDIR}/out [ -z "${EXCLUDELIST}" ] || cat ${EXCLUDELIST} > ${excludelist} cat >> ${excludelist} << EOF -usr/src var/db/freebsd-update var/db/etcupdate boot/kernel.old @@ -351,8 +354,11 @@ make -C ${mnt}/usr/src DESTDIR=${WRKDIR}/world BATCH_DELETE_OLD_FILES=yes SRCCONF=${WRKDIR}/src.conf delete-old delete-old-libs [ ! -d "${EXTRADIR}" ] || cp -fRPp ${EXTRADIR}/ ${WRKDIR}/world/ -mv ${WRKDIR}/world/etc/login.conf.orig ${WRKDIR}/world/etc/login.conf -cap_mkdb ${WRKDIR}/world/etc/login.conf +if [ -e "${WRKDIR}/world/etc/login.conf.orig" ]; then + # No login.conf.orig on 13.0-CURRENT-i386 + mv ${WRKDIR}/world/etc/login.conf.orig ${WRKDIR}/world/etc/login.conf + cap_mkdb ${WRKDIR}/world/etc/login.conf +fi # Set hostname if [ -n "${HOSTNAME}" ]; then @@ -367,12 +373,15 @@ local REPOS_DIR=$(mktemp -dt poudriere_repo) local ABI_FILE - # This pkg rquery is always ran in host so we need a host-centric - # repo.conf always. - cat > "${REPOS_DIR}/repo.conf" <<-EOF - FreeBSD: { enabled: false } - local: { url: file:///${WRKDIR}/world/tmp/packages } - EOF + if [ -n "${PKGREPO}" ]; then + else + # This pkg rquery is always ran in host so we need a host-centric + # repo.conf always. + cat > "${REPOS_DIR}/repo.conf" <<-EOF + FreeBSD: { enabled: false } + local: { url: file:///${WRKDIR}/world/tmp/packages } + EOF + fi export REPOS_DIR PKG_DBDIR # Always need this from host. @@ -387,36 +396,83 @@ # install packages if any is needed if [ -n "${PACKAGELIST}" ]; then mkdir -p ${WRKDIR}/world/tmp/packages - ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages + mount -t devfs devfs ${WRKDIR}/world/dev + chroot "${WRKDIR}/world" env /usr/sbin/pwd_mkdb -p /etc/master.passwd + [ -n "${RESOLV_CONF}" ] && cp -v "${RESOLV_CONF}" "${WRKDIR}/world/etc/" + mkdir -p ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} + mkdir -p ${WRKDIR}/world/var/cache/pkg if [ "${arch}" == "${host_arch}" ]; then - cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF - FreeBSD: { enabled: false } - local: { url: file:///tmp/packages } - EOF - convert_package_list "${PACKAGELIST}" | \ - xargs chroot "${WRKDIR}/world" env \ - REPOS_DIR=/tmp ASSUME_ALWAYS_YES=yes \ - pkg install + if [ -n "${PKGREPO}" ]; then + ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \ + ${WRKDIR}/world/var/cache/pkg + convert_package_list "${PACKAGELIST}" | \ + xargs chroot "${WRKDIR}/world" env \ + ASSUME_ALWAYS_YES=yes \ + pkg install + umount ${WRKDIR}/world/var/cache/pkg + else + ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages + cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF + FreeBSD: { enabled: false } + local: { url: file:///tmp/packages } + EOF + convert_package_list "${PACKAGELIST}" | \ + xargs chroot "${WRKDIR}/world" env \ + REPOS_DIR=/tmp ASSUME_ALWAYS_YES=yes \ + pkg install + umount ${WRKDIR}/world/tmp/packages + fi else - cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF - FreeBSD: { enabled: false } - local: { url: file:///${WRKDIR}/world/tmp/packages } - EOF - ( - export ASSUME_ALWAYS_YES=yes SYSLOG=no \ - REPOS_DIR="${WRKDIR}/world/tmp/" \ - ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o" - pkg -r "${WRKDIR}/world/" install pkg + if [ -n "${PKGREPO}" ]; then + # Try do same as for amd64 (we only care about i386) + ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \ + ${WRKDIR}/world/var/cache/pkg convert_package_list "${PACKAGELIST}" | \ - xargs pkg -r "${WRKDIR}/world/" install - ) + xargs chroot "${WRKDIR}/world" env \ + ASSUME_ALWAYS_YES=yes \ + pkg install + umount ${WRKDIR}/world/var/cache/pkg + # ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \ + # ${WRKDIR}/world/var/cache/pkg + # ( + # export ASSUME_ALWAYS_YES=yes SYSLOG=no \ + # ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o" + # pkg -r "${WRKDIR}/world/" install pkg + # convert_package_list "${PACKAGELIST}" | \ + # xargs pkg -r "${WRKDIR}/world/" install + # ) + # umount ${WRKDIR}/world/var/cache/pkg + else + ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages + cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF + FreeBSD: { enabled: false } + local: { url: file:///${WRKDIR}/world/tmp/packages } + EOF + ( + export ASSUME_ALWAYS_YES=yes SYSLOG=no \ + REPOS_DIR="${WRKDIR}/world/tmp/" \ + ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o" + pkg -r "${WRKDIR}/world/" install pkg + convert_package_list "${PACKAGELIST}" | \ + xargs pkg -r "${WRKDIR}/world/" install + ) + umount ${WRKDIR}/world/tmp/packages + fi fi - rm -rf ${WRKDIR}/world/var/cache/pkg - umount ${WRKDIR}/world/tmp/packages + umount ${WRKDIR}/world/dev rmdir ${WRKDIR}/world/tmp/packages + rm -rf ${WRKDIR}/world/var/cache/pkg rm ${WRKDIR}/world/var/db/pkg/repo-* 2>/dev/null || : fi + +if [ -e "${WRKDIR}/world/overlay.sh" ]; then + export MASTERNAME + mount -t devfs devfs ${WRKDIR}/world/dev + chroot "${WRKDIR}/world" env LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib /overlay.sh + umount ${WRKDIR}/world/dev +fi + case ${MEDIATYPE} in *mfs) cat >> ${WRKDIR}/world/etc/fstab <<-EOF @@ -574,8 +630,8 @@ mkimg -s gpt -b ${mnt}/boot/pmbr \ -p efi:=${mnt}/boot/boot1.efifat \ -p freebsd-boot:=${mnt}/boot/gptboot \ - -p freebsd-ufs:=${WRKDIR}/raw.img \ - -p freebsd-swap::1M \ + -p freebsd-ufs/gfx-root:=${WRKDIR}/raw.img \ + -p freebsd-swap/gfx-swap::1G \ -o ${OUTPUTDIR}/${FINALIMAGE} ;; tar) --------------F545AD8270A6D91ADDA344E0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a97754ed-9e20-f21f-ec4f-67b9ef5b2294>