Date: Tue, 20 Jan 2015 23:56:05 +0000 (UTC) From: Glen Barber <gjb@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r277458 - in head: release release/amd64 release/i386 release/scripts release/tools share/man/man7 Message-ID: <201501202356.t0KNu5ts016548@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: gjb Date: Tue Jan 20 23:56:04 2015 New Revision: 277458 URL: https://svnweb.freebsd.org/changeset/base/277458 Log: Merge the following revisions from ^/projects/release-vmimage: r273823-r273826, r273833, r273836, r273944, r274069-r274071, r274134, r274211, r274280-r274285, r274287-r274288, r274292, r274296-r274297, r274356, r274533, r274725, r274726, r274729, r274734, r274771, r274945-r274946, r277180, r277183-r277184, r277186-r277187, r277250-r277253, r277263-r277264, r277383-r277384, r277393-r277395, r277438-r277439, r277447, r277455: r273823: Move virtual machine / cloud provider targets and options from release/Makefile to their own Makefile. r273824: Add glue to allow enabling building cloud provider VM images by default. When WITH_CLOUDWARE is not empty, add CLOUDTARGETS to the release/Makefile 'release' target. r273825: Avoid hard-coding the Azure image file format. While here, avoid using OSRELEASE for the output file name. r273826: Remove a few vestiges of passing an exit code to panic(). r273833: Initial commit providing a mechanism to create openstack images as part of the release build. r273836: Fix output file name for openstack images. No further conversion is necessary for this VM file target, so there is no need to append the '.raw' suffix here. r273944: Uncomment the cloudinit rc.conf(5) line. r274069: Add line continuation so OPENSTACKCONF is actually included in the env(1). r274070: Add a 'vm-cloudware' target, used to drive all targets in CLOUDTARGETS. r274071: Add examples for WITH_CLOUDWARE to release.conf.sample. Add WITH_CLOUDWARE evaluation to RELEASE_RMAKEFLAGS. r274134: Initial rewrite to consolidate VM image build scripts into one. r274211: Add write_partition_layout() used to populate the final image. Fix duplicated mkimg(1) call in vm_create_disk(). Add primitive (untested) PowerPC/PowerPC64 VM image support. Note: As it is currently written, the /boot/pmbr and /boot/{gptboot,boot1.hfs} use the build host and not the target build. Fixing this is likely going to be a hack in itself. r274280: Return if vm_create_disk() is unsuccessful. r274281: Add CLEANFILES entry for VM targets r274282: Add vm_extra_pre_umount() prototype to vmimage.subr. r274283: Fix DESTDIR for installworld, and make sure it is created before use. r274284: Move usage() from vmimage.subr to mk-vmimage.sh, in case vmimage.subr has not been sourced. r274285: Spell 'OPTARG' correctly. Actually call vm_create_base(). r274287: Fix line continuation in write_partition_layout(). Remove variable test that is no longer needed. r274288: Fix scheme flag to mkimg(1). r274292: mount(8) and umount(8) devfs(5) as needed. r274296: Change path for mk-vmimage.sh from ${TARGET}/ to scripts/ now that it is consolidated into one file. Fix paths for the base image and output disk image files. r274297: Call cleanup() after everything is done. r274356: Remove a stray directory from CLEANFILES. r274533: Set the boot partition type to 'apple-boot' for powerpc. r274725: In vm_install_base(), copy the host resolv.conf into the build chroot before attempting to do anything that requires working DNS (i.e., pkg bootstrap). In vm_extra_pre_umount(), remove the resolv.conf before the disk image is unmounted from the backing md(4). r274726 (cperciva): Silence errors when umounting the chroot's /dev, since it probably doesn't exist when we're running this. Unmount filesystems before attempting to destroy the md which holds them. r274729 (cperciva): Unmount filesystem and destroy md before we read the vnode from disk and package it into a disk image. Otherwise we end up packaging an unclean filesystem. r274734 (cperciva): Merge duplicative vm-CLOUDTYPE targets before additional duplication gets added by the impending arrival of ec2 and gcloud. r274771 (cperciva): Add NOSWAP option which can be set by a vmimage.conf file to specify that no swap space should be created in the image. This will be used by EC2 builds, since FreeBSD/EC2 allocates swap space on "ephemeral" disks which are physically attached to the Xen host node. r274945: In vm_extra_install_packages(), only bootstrap pkg(8) if VM_EXTRA_PACKAGES is empty. In vm_extra_pre_umount(), cleanup downloaded packages if pkg(8) was bootstrapped earlier. r274946: Fix indentation nit. r277180: In vm_extra_install_base(), do not install waagent in the openstack image, because it is not used. This appears to be a copy mistake. Remove vm_extra_install_base() from the openstack.conf entirely, since it does not need to be overridden. r277183: Enable the textmode console by default for VM images, since there is no way to tell if the environment will be able to use the graphics-mode console. r277184: Enable password-less sudo for openstack images. r277186: Update the VM_EXTRA_PACKAGES list for the openstack images. The documentation suggests doing a "just fetch this and run it"-style bootstrap, from which the list of dependencies was obtained (in github, at: pellaeon/bsd-cloudinit-installer) There is one Python dependency unmet, oslo.config, which is not in the Ports Collection. r277187: Add a comment to note that setting hw.vga.textmode=1 is temporary. r277250: Remove vm_extra_install_base() for the Azure image, now that the waagent exists in the ports tree. Add sysutils/azure-agent to the VM_EXTRA_PACKAGES list. In vm_extra_pre_umount(), remove the explicit pkg(8) install list, as dependencies are resolved by sysutils/azure-agent. r277251: Add a 'list-cloudware' target to print the list of supported CLOUDWARE values and a description. Add the AZURE_DESC and OPENSTACK_DESC descriptions. r277252: Update release(7) r277253: Add 'list-vmtargets' target, which produces a list of all supported VM and cloud provider images. Add VHD_DESC, VMDK_DESC, QCOW2_DESC, RAW_DESC image descriptions. Format the output to make a bit more readable. Update release(7) to document the list-vmtargets target. r277263: Add initial support for the GCE (Google Compute Engine) cloud hosting provider image. r277264: Style and line length cleanup. r277383: Remove the console setting from rc.conf(5), which is not used there. While here, set console to include vidconsole in the loader.conf(5). r277384: Fix an indentation nit. No functional changes. r277393: Remove the pkg-clean(8) call from vm_extra_pre_umount() since the function is often overridden. Add vm_extra_pkg_rmcache() to call pkg-clean(8) to avoid duplicated code. r277394: Move resolv.conf(5) removal back to vm_extra_pre_umount() where it belongs. The GCE image needs resolv.conf(5) to exist (created as part of the image setup), so it cannot be removed. r277395: Comment the line that configures ttys(5) to 'off', which makes it impossible to test that the image boots. Add a note explaining why the line is commented, and not (yet) removed entirely. r277438: Move the 'install' bits that are specific to virtual machine images from the Makefile to Makefile.vm. Rename the 'install' target to 'release-install', and add a new 'vm-install' target. Add a new 'install' target that invokes the new targets. r277439: Add WITH_CLOUDWARE to the list of make(1) variables for the release build. r277447: Remove hw.vga.textmode=1 from the VM image loader.conf, which was included during test builds and not intended to be included when merging this project branch back to head. r277455: Remove mk-azure.sh, which is no longer needed. MFC after: 1 month X-MFC-To: stable/10 (requires mkimg(1)) Help from: cperciva, swills Relnotes: yes Sponsored by: The FreeBSD Foundation Added: head/release/Makefile.vm - copied, changed from r273823, projects/release-vmimage/release/Makefile.vm head/release/scripts/mk-vmimage.sh - copied, changed from r274134, projects/release-vmimage/release/scripts/mk-vmimage.sh head/release/tools/gce.conf - copied, changed from r277263, projects/release-vmimage/release/tools/gce.conf head/release/tools/openstack.conf - copied, changed from r273833, projects/release-vmimage/release/tools/openstack.conf head/release/tools/vmimage.subr (contents, props changed) - copied, changed from r274134, projects/release-vmimage/release/tools/vmimage.subr Deleted: head/release/amd64/mk-azure.sh head/release/amd64/mk-vmimage.sh head/release/i386/mk-azure.sh head/release/i386/mk-vmimage.sh Modified: head/release/Makefile head/release/release.conf.sample head/release/release.sh head/release/tools/azure.conf head/share/man/man7/release.7 Modified: head/release/Makefile ============================================================================== --- head/release/Makefile Tue Jan 20 23:39:08 2015 (r277457) +++ head/release/Makefile Tue Jan 20 23:56:04 2015 (r277458) @@ -9,7 +9,9 @@ # mini-memstick: Builds minimal memory stick image (mini-memstick.img) # ftp: Sets up FTP distribution area (ftp) # release: Build all media and FTP distribution area -# install: Copies all release media into ${DESTDIR} +# install: Invokes the release-install and vm-install targets +# release-install: Copies all release installation media into ${DESTDIR} +# vm-install: Copies all virtual machine images into ${DESTDIR} # # Variables affecting the build process: # WORLDDIR: location of src tree -- must have built world and default kernel @@ -26,6 +28,7 @@ # WITH_VMIMAGES: if set, build virtual machine images with the release # WITH_COMPRESSED_VMIMAGES: if set, compress virtual machine disk images # with xz(1) (extremely time consuming) +# WITH_CLOUDWARE: if set, build cloud hosting disk images with the release # TARGET/TARGET_ARCH: architecture of built release # @@ -103,12 +106,6 @@ IMAGES+= memstick.img IMAGES+= mini-memstick.img .endif -VMTARGETS= vm-base vm-image -VMFORMATS?= vhd vmdk qcow2 raw -VMSIZE?= 20G -VMBASE?= vm -AZURECONF?= ${.CURDIR}/tools/azure.conf - CLEANFILES= packagesystem *.txz MANIFEST system ${IMAGES} .if defined(WITH_COMPRESSED_IMAGES) && !empty(WITH_COMPRESSED_IMAGES) . for I in ${IMAGES} @@ -118,22 +115,7 @@ CLEANFILES+= ${I}.xz .if defined(WITH_DVD) && !empty(WITH_DVD) CLEANFILES+= pkg-stage .endif -.if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) -CLEANFILES+= ${VMBASE}.img -. for FORMAT in ${VMFORMATS} -CLEANFILES+= ${VMBASE}.${FORMAT} -. endfor -.endif CLEANDIRS= dist ftp release bootonly dvd -.if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) -CLEANDIRS+= ${VMTARGETS} -.endif -.if exists(${.CURDIR}/${TARGET}/mk-azure.sh) -CLEANFILES+= ${OSRELEASE}.vhd \ - ${OSRELEASE}.vhd.raw \ - azure.img -CLEANDIRS+= vm-azure -.endif beforeclean: chflags -R noschg . .include <bsd.obj.mk> @@ -299,8 +281,13 @@ release: .if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) ${MAKE} -C ${.CURDIR} ${.MAKEFLAGS} ${VMTARGETS} .endif +.if defined(WITH_CLOUDWARE) && !empty(WITH_CLOUDWARE) && !empty(CLOUDWARE) + ${MAKE} -C ${.CURDIR} ${.MAKEFLAGS} ${CLOUDTARGETS} +.endif + +install: release-install vm-install -install: +release-install: .if defined(DESTDIR) && !empty(DESTDIR) mkdir -p ${DESTDIR} .endif @@ -313,52 +300,5 @@ install: .endfor cd ${DESTDIR} && sha256 ${OSRELEASE}* > ${DESTDIR}/CHECKSUM.SHA256 cd ${DESTDIR} && md5 ${OSRELEASE}* > ${DESTDIR}/CHECKSUM.MD5 -.if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) - mkdir -p ${DESTDIR}/vmimages -. for FORMAT in ${VMFORMATS} - cp -p ${VMBASE}.${FORMAT} \ - ${DESTDIR}/vmimages/${OSRELEASE}.${FORMAT} -. endfor -. if defined(WITH_COMPRESSED_VMIMAGES) && !empty(WITH_COMPRESSED_VMIMAGES) -# This is very time consuming, so defer it after the images are moved to -# the DESTDIR. -. for FORMAT in ${VMFORMATS} - # Don't keep the originals. There is a copy in ${.OBJDIR} if needed. - ${XZCMD} ${DESTDIR}/vmimages/${OSRELEASE}.${FORMAT} -. endfor -. endif - cd ${DESTDIR}/vmimages && sha256 ${OSRELEASE}* > \ - ${DESTDIR}/vmimages/CHECKSUM.SHA256 - cd ${DESTDIR}/vmimages && md5 ${OSRELEASE}* > \ - ${DESTDIR}/vmimages/CHECKSUM.MD5 -.endif -vm-base: -.if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) -. if exists(${.CURDIR}/${TARGET}/mk-vmimage.sh) - env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} \ - ${.CURDIR}/${TARGET}/mk-vmimage.sh ${.TARGET} \ - ${VMBASE}.img ${WORLDDIR} ${.OBJDIR}/${.TARGET} ${VMSIZE} -. endif -.endif - touch ${.TARGET} - -vm-image: vm-base -.if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) -. if exists(${.CURDIR}/${TARGET}/mk-vmimage.sh) -. for FORMAT in ${VMFORMATS} - env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} \ - ${.CURDIR}/${TARGET}/mk-vmimage.sh ${.TARGET} \ - ${VMBASE}.img ${FORMAT} ${VMBASE}.${FORMAT} -. endfor -. endif -.endif - touch ${.TARGET} - -vm-azure: -.if exists(${.CURDIR}/${TARGET}/mk-azure.sh) - env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} AZURECONF=${AZURECONF} \ - ${.CURDIR}/${TARGET}/mk-azure.sh ${.TARGET} azure.img \ - ${WORLDDIR} ${.TARGET} ${VMSIZE} ${OSRELEASE}.vhd -.endif - touch ${.TARGET} +.include "${.CURDIR}/Makefile.vm" Copied and modified: head/release/Makefile.vm (from r273823, projects/release-vmimage/release/Makefile.vm) ============================================================================== --- projects/release-vmimage/release/Makefile.vm Wed Oct 29 14:57:30 2014 (r273823, copy source) +++ head/release/Makefile.vm Tue Jan 20 23:56:04 2015 (r277458) @@ -5,11 +5,48 @@ # Makefile for building virtual machine and cloud provider disk images. # -VMTARGETS= vm-base vm-image +VMTARGETS= vm-image VMFORMATS?= vhd vmdk qcow2 raw VMSIZE?= 20G VMBASE?= vm -AZURECONF?= ${.CURDIR}/tools/azure.conf + +VHD_DESC= Azure, VirtualPC, Hyper-V, Xen disk image +VMDK_DESC= VMWare, VirtualBox disk image +QCOW2_DESC= Qemu, KVM disk image +RAW_DESC= Unformatted raw disk image + +CLOUDWARE?= AZURE \ + GCE \ + OPENSTACK +AZURE_FORMAT= vhdf +AZURE_DESC= Microsoft Azure platform image +GCE_FORMAT= raw +GCE_DESC= Google Compute Engine image +OPENSTACK_FORMAT=qcow2 +OPENSTACK_DESC= OpenStack platform image + +.if defined(WITH_CLOUDWARE) && !empty(WITH_CLOUDWARE) && !empty(CLOUDWARE) +. for _CW in ${CLOUDWARE} +CLOUDTARGETS+= vm-${_CW:tl} +CLEANDIRS+= vm-${_CW:tl} +CLEANFILES+= ${_CW:tl}.img \ + ${_CW:tl}.${${_CW:tu}_FORMAT} \ + ${_CW:tl}.${${_CW:tu}_FORMAT}.raw +${_CW:tu}IMAGE= ${_CW:tl}.${${_CW:tu}_FORMAT} +. if exists(${.CURDIR}/tools/${_CW:tl}.conf) && !defined(${_CW:tu}CONF) +${_CW:tu}CONF?= ${.CURDIR}/tools/${_CW:tl}.conf +. endif + +vm-${_CW:tl}: + mkdir -p ${.OBJDIR}/${.TARGET} + env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} \ + ${.CURDIR}/scripts/mk-vmimage.sh \ + -C ${.CURDIR}/tools/vmimage.subr -d ${.OBJDIR}/${.TARGET} \ + -i ${.OBJDIR}/${_CW:tl}.img -s ${VMSIZE} -f ${${_CW}_FORMAT} \ + -S ${WORLDDIR} -o ${.OBJDIR}/${${_CW}IMAGE} -c ${${_CW}CONF} + touch ${.TARGET} +. endfor +.endif .if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) CLEANDIRS+= ${VMTARGETS} @@ -19,39 +56,56 @@ CLEANFILES+= ${VMBASE}.${FORMAT} . endfor .endif -.if exists(${.CURDIR}/${TARGET}/mk-azure.sh) -CLEANFILES+= ${OSRELEASE}.vhd \ - ${OSRELEASE}.vhd.raw \ - azure.img -CLEANDIRS+= vm-azure -.endif +vm-base: vm-image -vm-base: +vm-image: .if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) -. if exists(${.CURDIR}/${TARGET}/mk-vmimage.sh) +. for FORMAT in ${VMFORMATS} + mkdir -p ${.OBJDIR}/${.TARGET} env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} \ - ${.CURDIR}/${TARGET}/mk-vmimage.sh ${.TARGET} \ - ${VMBASE}.img ${WORLDDIR} ${.OBJDIR}/${.TARGET} ${VMSIZE} -. endif + ${.CURDIR}/scripts/mk-vmimage.sh \ + -C ${.CURDIR}/tools/vmimage.subr -d ${.OBJDIR}/${.TARGET} \ + -i ${.OBJDIR}/${VMBASE}.img -s ${VMSIZE} -f ${FORMAT} \ + -S ${WORLDDIR} -o ${.OBJDIR}/${VMBASE}.${FORMAT} +. endfor .endif touch ${.TARGET} -vm-image: vm-base +vm-cloudware: ${CLOUDTARGETS} + +list-vmtargets: list-cloudware + @${ECHO} + @${ECHO} "Supported virtual machine disk image formats:" +.for FORMAT in ${VMFORMATS:tu} + @${ECHO} " ${FORMAT:tl}: ${${FORMAT}_DESC}" +.endfor + +list-cloudware: +.if !empty(CLOUDWARE) + @${ECHO} + @${ECHO} "Supported cloud hosting provider images:" +. for _CW in ${CLOUDWARE} + @${ECHO} " ${_CW:tu}: ${${_CW:tu}_DESC}" +. endfor +.endif + +vm-install: .if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) -. if exists(${.CURDIR}/${TARGET}/mk-vmimage.sh) + mkdir -p ${DESTDIR}/vmimages +. for FORMAT in ${VMFORMATS} + cp -p ${VMBASE}.${FORMAT} \ + ${DESTDIR}/vmimages/${OSRELEASE}.${FORMAT} +. endfor +. if defined(WITH_COMPRESSED_VMIMAGES) && !empty(WITH_COMPRESSED_VMIMAGES) +# This is very time consuming, so defer it after the images are moved to +# the DESTDIR. . for FORMAT in ${VMFORMATS} - env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} \ - ${.CURDIR}/${TARGET}/mk-vmimage.sh ${.TARGET} \ - ${VMBASE}.img ${FORMAT} ${VMBASE}.${FORMAT} + # Don't keep the originals. There is a copy in ${.OBJDIR} if needed. + ${XZCMD} ${DESTDIR}/vmimages/${OSRELEASE}.${FORMAT} . endfor . endif + cd ${DESTDIR}/vmimages && sha256 ${OSRELEASE}* > \ + ${DESTDIR}/vmimages/CHECKSUM.SHA256 + cd ${DESTDIR}/vmimages && md5 ${OSRELEASE}* > \ + ${DESTDIR}/vmimages/CHECKSUM.MD5 .endif - touch ${.TARGET} - -vm-azure: -.if exists(${.CURDIR}/${TARGET}/mk-azure.sh) - env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} AZURECONF=${AZURECONF} \ - ${.CURDIR}/${TARGET}/mk-azure.sh ${.TARGET} azure.img \ - ${WORLDDIR} ${.TARGET} ${VMSIZE} ${OSRELEASE}.vhd -.endif - touch ${.TARGET} Modified: head/release/release.conf.sample ============================================================================== --- head/release/release.conf.sample Tue Jan 20 23:39:08 2015 (r277457) +++ head/release/release.conf.sample Tue Jan 20 23:56:04 2015 (r277458) @@ -98,3 +98,11 @@ PORTBRANCH="ports/head@rHEAD" ## image formats to create. Valid values are listed in the mkimg(1) ## manual page, as well as 'mkimg --formats' output. #VMFORMATS="vhdf vmdk qcow2 raw" + +## Set to a non-empty value to build virtual machine images for various +## cloud providers as part of the release build. +#WITH_CLOUDWARE= + +## If WITH_CLOUDWARE is set to a non-empty value, this is a list of providers +## to create disk images. +#CLOUDWARE="AZURE OPENSTACK" Modified: head/release/release.sh ============================================================================== --- head/release/release.sh Tue Jan 20 23:39:08 2015 (r277457) +++ head/release/release.sh Tue Jan 20 23:56:04 2015 (r277458) @@ -94,6 +94,10 @@ WITH_COMPRESSED_IMAGES= WITH_VMIMAGES= WITH_COMPRESSED_VMIMAGES= +# Set to non-empty value to build virtual machine images for various +# cloud providers as part of the release. +WITH_CLOUDWARE= + usage() { echo "Usage: $0 [-c release.conf]" exit 1 @@ -174,7 +178,8 @@ CHROOT_DMAKEFLAGS="${CONF_FILES}" RELEASE_WMAKEFLAGS="${MAKE_FLAGS} ${WORLD_FLAGS} ${ARCH_FLAGS} ${CONF_FILES}" RELEASE_KMAKEFLAGS="${MAKE_FLAGS} ${KERNEL_FLAGS} KERNCONF=\"${KERNEL}\" ${ARCH_FLAGS} ${CONF_FILES}" RELEASE_RMAKEFLAGS="${ARCH_FLAGS} KERNCONF=\"${KERNEL}\" ${CONF_FILES} \ - ${DOCPORTS} WITH_DVD=${WITH_DVD} WITH_VMIMAGES=${WITH_VMIMAGES}" + ${DOCPORTS} WITH_DVD=${WITH_DVD} WITH_VMIMAGES=${WITH_VMIMAGES} \ + WITH_CLOUDWARE=${WITH_CLOUDWARE}" # Force src checkout if configured FORCE_SRC_KEY= Copied and modified: head/release/scripts/mk-vmimage.sh (from r274134, projects/release-vmimage/release/scripts/mk-vmimage.sh) ============================================================================== --- projects/release-vmimage/release/scripts/mk-vmimage.sh Wed Nov 5 13:22:19 2014 (r274134, copy source) +++ head/release/scripts/mk-vmimage.sh Tue Jan 20 23:56:04 2015 (r277458) @@ -32,6 +32,12 @@ # $FreeBSD$ # +usage() { + echo "${0} usage:" + echo "${@}" + return 1 +} + main() { local arg while getopts "C:c:d:f:i:o:s:S:" arg; do @@ -49,7 +55,7 @@ main() { VMFORMAT="${OPTARG}" ;; i) - VMBASE="${VMBASE}" + VMBASE="${OPTARG}" ;; o) VMIMAGE="${OPTARG}" @@ -87,13 +93,16 @@ main() { . "${VMCONFIG}" fi + vm_create_base vm_install_base vm_extra_install_base vm_extra_install_packages vm_extra_install_ports vm_extra_enable_services vm_extra_pre_umount - vm_create_disk + vm_extra_pkg_rmcache + cleanup + vm_create_disk || return 0 vm_extra_create_disk return 0 Modified: head/release/tools/azure.conf ============================================================================== --- head/release/tools/azure.conf Tue Jan 20 23:39:08 2015 (r277457) +++ head/release/tools/azure.conf Tue Jan 20 23:56:04 2015 (r277458) @@ -6,9 +6,37 @@ # Set to a list of packages to install. # Example: #export VM_EXTRA_PACKAGES="www/apache24" -export VM_EXTRA_PACKAGES= +export VM_EXTRA_PACKAGES="sysutils/azure-agent" # Set to a list of third-party software to enable in rc.conf(5). # Example: #export VM_RC_LIST="apache24" export VM_RC_LIST= + +vm_extra_pre_umount() { + chroot ${DESTDIR} /usr/sbin/waagent -verbose -install + yes | chroot ${DESTDIR} /usr/sbin/waagent -deprovision + echo 'sshd_enable="YES"' >> ${DESTDIR}/etc/rc.conf + echo 'ifconfig_hn0="SYNCDHCP"' >> ${DESTDIR}/etc/rc.conf + echo 'waagent_enable="YES"' >> ${DESTDIR}/etc/rc.conf + echo 'console="comconsole vidconsole"' >> ${DESTDIR}/boot/loader.conf + echo 'comconsole_speed="115200"' >> ${DESTDIR}/boot/loader.conf + + rm -f ${DESTDIR}/etc/resolv.conf + + return 0 +} + +vm_extra_create_disk() { + if [ ! -x "/usr/local/bin/qemu-img" ]; then + env ASSUME_ALWAYS_YES=yes pkg install -y emulators/qemu-devel + fi + + mv ${VMIMAGE} ${VMIMAGE}.raw + size=$(qemu-img info -f raw --output json ${VMIMAGE}.raw | awk '/virtual-size/ {print $2}' | tr -d ',') + size=$(( ( ${size} / ( 1024 * 1024 ) + 1 ) * ( 1024 * 1024 ) )) + qemu-img resize ${VMIMAGE}.raw ${size} + qemu-img convert -f raw -o subformat=fixed -O vpc ${VMIMAGE}.raw ${VMIMAGE} + + return 0 +} Copied and modified: head/release/tools/gce.conf (from r277263, projects/release-vmimage/release/tools/gce.conf) ============================================================================== --- projects/release-vmimage/release/tools/gce.conf Fri Jan 16 19:27:19 2015 (r277263, copy source) +++ head/release/tools/gce.conf Tue Jan 20 23:56:04 2015 (r277458) @@ -4,7 +4,9 @@ # # Set to a list of packages to install. -export VM_EXTRA_PACKAGES="firstboot-freebsd-update firstboot-pkgs google-cloud-sdk google-daemon panicmail sudo firstboot-growfs google-startup-scripts" +export VM_EXTRA_PACKAGES="firstboot-freebsd-update firstboot-pkgs \ + google-cloud-sdk google-daemon panicmail sudo firstboot-growfs \ + google-startup-scripts" # Set to a list of third-party software to enable in rc.conf(5). export VM_RC_LIST="google_accounts_manager ntpd" @@ -16,8 +18,7 @@ vm_extra_install_base() { } vm_extra_pre_umount() { - cat << EOF >> ${DESTDIR}/etc/rc.conf -console="comconsole" + cat << EOF >> ${DESTDIR}/etc/rc.conf dumpdev="AUTO" ifconfig_vtnet0="SYNCDHCP mtu 1460" ntpd_sync_on_start="YES" @@ -34,12 +35,12 @@ firstboot_growfs_enable="YES" google_startup_enable="YES" EOF - cat << EOF >> ${DESTDIR}/boot/loader.conf + cat << EOF >> ${DESTDIR}/boot/loader.conf autoboot_delay="-1" beastie_disable="YES" loader_logo="none" hw.memtest.tests="0" -console="comconsole" +console="comconsole,vidconsole" hw.vtnet.mq_disable=1 kern.timecounter.hardware=ACPI-safe aesni_load="YES" @@ -50,7 +51,7 @@ EOF ${DESTDIR}/etc/hosts # overwrite ntp.conf - cat << EOF > ${DESTDIR}/etc/ntp.conf + cat << EOF > ${DESTDIR}/etc/ntp.conf server metadata.google.internal iburst restrict default kod nomodify notrap nopeer noquery @@ -61,11 +62,11 @@ restrict -6 ::1 restrict 127.127.1.0 EOF - cat << EOF >> ${DESTDIR}/etc/syslog.conf + cat << EOF >> ${DESTDIR}/etc/syslog.conf *.err;kern.warning;auth.notice;mail.crit /dev/console EOF - cat << EOF >> ${DESTDIR}/etc/ssh/sshd_config + cat << EOF >> ${DESTDIR}/etc/ssh/sshd_config ChallengeResponseAuthentication no X11Forwarding no AcceptEnv LANG @@ -74,11 +75,11 @@ AllowAgentForwarding no ClientAliveInterval 420 EOF - cat << EOF >> ${DESTDIR}/etc/crontab + cat << EOF >> ${DESTDIR}/etc/crontab 0 3 * * * root /usr/sbin/freebsd-update cron EOF - cat << EOF >> ${DESTDIR}/etc/sysctl.conf + cat << EOF >> ${DESTDIR}/etc/sysctl.conf net.inet.icmp.drop_redirect=1 net.inet.ip.redirect=0 net.inet.tcp.blackhole=2 @@ -88,9 +89,12 @@ debug.trace_on_panic=1 debug.debugger_on_panic=0 EOF - sed -E -i '' 's/^([^#].*[[:space:]])on/\1off/' ${DESTDIR}/etc/ttys + ## XXX: Verify this is needed. I do not see this requirement + ## in the docs, and it impairs the ability to boot-test a copy + ## of the image prior to packaging for upload to GCE. + #sed -E -i '' 's/^([^#].*[[:space:]])on/\1off/' ${DESTDIR}/etc/ttys - touch ${DESTDIR}/firstboot + touch ${DESTDIR}/firstboot return 0 } Copied and modified: head/release/tools/openstack.conf (from r273833, projects/release-vmimage/release/tools/openstack.conf) ============================================================================== --- projects/release-vmimage/release/tools/openstack.conf Wed Oct 29 17:04:09 2014 (r273833, copy source) +++ head/release/tools/openstack.conf Tue Jan 20 23:56:04 2015 (r277458) @@ -4,7 +4,22 @@ # # Set to a list of packages to install. -export VM_EXTRA_PACKAGES="net/cloud-init" +export VM_EXTRA_PACKAGES="net/cloud-init devel/py-pbr devel/py-iso8601 \ + net/py-eventlet net/py-netaddr comms/py-serial devel/py-six \ + devel/py-babel net/py-oauth net/py-netifaces" # Set to a list of third-party software to enable in rc.conf(5). -export VM_RC_LIST="#cloudinit" +export VM_RC_LIST="cloudinit" + +vm_extra_pre_umount() { + echo 'sshd_enable="YES"' >> ${DESTDIR}/etc/rc.conf + echo 'ifconfig_DEFAULT="SYNCDHCP"' >> ${DESTDIR}/etc/rc.conf + + # Openstack wants sudo(8) usable by default without a password. + echo 'ALL ALL=(ALL) NOPASSWD:ALL' >> \ + ${DESTDIR}/usr/local/etc/sudoers.d/cloud-init + + rm -f ${DESTDIR}/etc/resolv.conf + + return 0 +} Copied and modified: head/release/tools/vmimage.subr (from r274134, projects/release-vmimage/release/tools/vmimage.subr) ============================================================================== --- projects/release-vmimage/release/tools/vmimage.subr Wed Nov 5 13:22:19 2014 (r274134, copy source) +++ head/release/tools/vmimage.subr Tue Jan 20 23:56:04 2015 (r277458) @@ -9,15 +9,33 @@ export PATH="/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin" trap "cleanup" INT QUIT TRAP ABRT TERM -mkimg_bootcode="/boot/pmbr" -mkimg_partitions="-p freebsd-boot/bootfs:=/boot/gptboot" -mkimg_partitions="${mkimg_partitions} -p freebsd-swap/swapfs::1G" -mkimg_partitions="${mkimg_partitions} freebsd-ufs/rootfs:=${VMBASE}" - -usage() { - echo "${0} usage:" - echo "${@}" - return 1 +write_partition_layout() { + if [ -z "${NOSWAP}" ]; then + SWAPOPT="-p freebsd-swap/swapfs::1G" + fi + + case "${TARGET}:${TARGET_ARCH}" in + amd64:amd64 | i386:i386) + mkimg -s gpt -b /boot/pmbr \ + -p freebsd-boot/bootfs:=/boot/gptboot \ + ${SWAPOPT} \ + -p freebsd-ufs/rootfs:=${VMBASE} \ + -o ${VMIMAGE} + ;; + powerpc:powerpc*) + mkimg -s apm \ + -p apple-boot/bootfs:=/boot/boot1.hfs \ + ${SWAPOPT} \ + -p freebsd-ufs/rootfs:=${VMBASE} \ + -o ${VMIMAGE} + ;; + *) + # ENOTSUPP + return 1 + ;; + esac + + return 0 } err() { @@ -27,11 +45,11 @@ err() { } cleanup() { + umount ${DESTDIR}/dev 2>/dev/null + umount ${DESTDIR} if [ ! -z "${mddev}" ]; then mdconfig -d -u ${mddev} fi - umount ${DESTDIR}/dev - umount ${DESTDIR} return 0 } @@ -61,11 +79,18 @@ vm_install_base() { > ${DESTDIR}/etc/fstab echo '/dev/gpt/rootfs / ufs rw 1 1' \ >> ${DESTDIR}/etc/fstab - echo '/dev/gpt/swapfs none swap sw 0 0' \ - >> ${DESTDIR}/etc/fstab + if [ -z "${NOSWAP}" ]; then + echo '/dev/gpt/swapfs none swap sw 0 0' \ + >> ${DESTDIR}/etc/fstab + fi + mkdir -p ${DESTDIR}/dev + mount -t devfs devfs ${DESTDIR}/dev chroot ${DESTDIR} /usr/bin/newaliases chroot ${DESTDIR} /etc/rc.d/ldconfig forcestart + umount ${DESTDIR}/dev + + cp /etc/resolv.conf ${DESTDIR}/etc/resolv.conf return 0 } @@ -89,12 +114,16 @@ vm_extra_enable_services() { } vm_extra_install_packages() { + if [ -z "${VM_EXTRA_PACKAGES}" ]; then + return 0 + fi + mkdir -p ${DESTDIR}/dev + mount -t devfs devfs ${DESTDIR}/dev chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \ /usr/sbin/pkg bootstrap -y - if [ ! -z "${VM_EXTRA_PACKAGES}" ]; then - chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \ - /usr/sbin/pkg install -y ${VM_EXTRA_PACKAGES} - fi + chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \ + /usr/sbin/pkg install -y ${VM_EXTRA_PACKAGES} + umount ${DESTDIR}/dev return 0 } @@ -106,6 +135,23 @@ vm_extra_install_ports() { return 0 } +vm_extra_pre_umount() { + # Prototype. When overridden, installs additional ports within the + # virtual machine environment. + + rm -f ${DESTDIR}/etc/resolv.conf + return 0 +} + +vm_extra_pkg_rmcache() { + if [ -e ${DESTDIR}/usr/local/sbin/pkg ]; then + chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \ + /usr/local/sbin/pkg clean -y -a + fi + + return 0 +} + vm_umount_base() { i=0 sync @@ -124,21 +170,10 @@ vm_umount_base() { } vm_create_disk() { - if [ -z "${mkimg_paritions}" ]; then - err "No partition types specified. Skipping." - return 1 - fi echo "Creating image... Please wait." echo - mkimg -f ${mkimg_format} -s ${mkimg_scheme} \ - ${mkimg_bootcode} \ - ${mkimg_partitions} \ - ${mkimg_outfile} - - mkimg -b /boot/pmbr -p freebsd-boot/bootfs:=/boot/gptboot \ - -p freebsd-swap/swapfs::1G \ - -p freebsd-ufs/rootfs:=${VMBASE} \ - -o ${VMIMAGE}.raw + + write_partition_layout || return 1 return 0 } Modified: head/share/man/man7/release.7 ============================================================================== --- head/share/man/man7/release.7 Tue Jan 20 23:39:08 2015 (r277457) +++ head/share/man/man7/release.7 Tue Jan 20 23:56:04 2015 (r277458) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd October 2, 2014 +.Dd January 16, 2015 .Dt RELEASE 7 .Os .Sh NAME @@ -406,6 +406,57 @@ See for valid format values .Pq requires version 20140927 or later . .El +.Pp +For a list of supported +.Va VMFORMATS +values +.Pq including cloud hosting provider formats +along with a brief description, run: +.Bd -literal -offset indent +cd /usr/src +make -C release list-vmtargets +.Ed +.Sh CLOUD HOSTING MACHINE IMAGES +The +.Fx +release build tools support building virtual machine images for various +cloud hosting providers, each with their own specific configuration to +include support for each hosting provider by default. +.Pp +The following +.Xr make 1 +environment variables are supported: +.Pp +.Bl -tag -width Ev +.It Va CLOUDWARE +Set to a list of one or more cloud hosting providers, enclosed in quotes. +Requires +.Va WITH_CLOUDWARE +to also be set. +.It Va WITH_CLOUDWARE +Set to a non-empty value to enable building virtual machine images +for various cloud hosting providers. +Requires +.Va CLOUDWARE +to also be set. +.El +.Pp +Additionally, the +.Va CLOUDWARE +and +.Va WITH_CLOUDWARE +variables can be added to +.Pa release.conf , +and used in conjunction with +.Pa release.sh . +.Pp +For a list of supported +.Va CLOUDWARE +values, run: +.Bd -literal -offset indent +cd /usr/src +make -C release list-cloudware +.Ed .Sh MAKEFILE TARGETS The release makefile .Pq Pa src/release/Makefile @@ -470,6 +521,23 @@ target requires the .Va WITH_VMIMAGES .Xr make 1 envirionment variable to be set to a non-null value. +.It Cm vm-cloudware +Builds +.Fx +virtual machine images for various cloud hosting providers. +See +.Qq CLOUD HOSTING MACHINE IMAGES +for implementation details. +.It Cm list-cloudware +Displays the list of valid +.Va CLOUDWARE +values. +.It Cm list-vmtargets +Displays the list of valid +.Va VMFORMAT +and +.Va CLOUDWARE +values. .El .Pp Major subtargets called by targets above: @@ -576,8 +644,11 @@ Typically, one only needs to set .It Pa /usr/src/Makefile .It Pa /usr/src/Makefile.inc1 .It Pa /usr/src/release/Makefile +.It Pa /usr/src/release/Makefile.vm .It Pa /usr/src/release/release.sh .It Pa /usr/src/release/release.conf.sample +.It Pa /usr/src/release/tools/*.conf +.It Pa /usr/src/release/tools/vmimage.subr .El .Sh EXAMPLES The following sequence of commands can be used to build a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201501202356.t0KNu5ts016548>