From owner-freebsd-xen@FreeBSD.ORG Sat Feb 11 17:41:10 2012 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EDC21065675 for ; Sat, 11 Feb 2012 17:41:10 +0000 (UTC) (envelope-from trix@basement.net) Received: from anya.basement.net (vt42.crimsonhexagon.com [66.135.63.193]) by mx1.freebsd.org (Postfix) with ESMTP id 664C78FC15 for ; Sat, 11 Feb 2012 17:41:09 +0000 (UTC) Received: from willow.basement.net (home.basement.net [173.162.16.100]) by anya.basement.net (8.13.8/8.13.8) with ESMTP id q1BGvBqZ011509; Sat, 11 Feb 2012 11:01:11 -0600 Received: from willow.basement.net (localhost [127.0.0.1]) by willow.basement.net (8.14.5/8.14.5) with ESMTP id q1BGvBS7071335; Sat, 11 Feb 2012 10:57:11 -0600 (CST) (envelope-from trix@willow.basement.net) Received: (from trix@localhost) by willow.basement.net (8.14.5/8.14.5/Submit) id q1BGvBxp071334; Sat, 11 Feb 2012 10:57:11 -0600 (CST) (envelope-from trix) Date: Sat, 11 Feb 2012 10:57:11 -0600 From: Trix Farrar To: FreeBSD Xen , Xen-Users Message-ID: <20120211165711.GB71096@basement.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="O3RTKUHj+75w1tg5" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: Subject: 9.0-RELEASE PV from scratch on XCP v1.1.0 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2012 17:41:10 -0000 --O3RTKUHj+75w1tg5 Content-Type: multipart/mixed; boundary="WYTEVAkct0FjGQmd" Content-Disposition: inline --WYTEVAkct0FjGQmd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Apologies for the re-post. Forgot to sign original. -Trx Title: HOWTO-FreeBSD-on-XCP Author: John D. "Trix" Farrar Date: 2012-02-09 * The Challenge - ParaVirtualized FreeBSD 9.0-RELEASE under XCP 1.1.0 The idea here is to create a PV FreeBSD VM under Xen Cloud Platform (XCP) without starting with an HVM first. The documentation I''ve been able to find on-line is at least a year old and is mostly written for Xen (under Linux or NetBSD) rather than XCP. To make things more interesting, the hardware on which I have XCP installed doesn''t support Hardware Virtualization -- so I can''t just install an HVM and convert it over. A PV under XCP means using pygrub as a bootloader to load the FreeBSD kernel. It seems that pygrub doesn''t know enough about BSD disk slices and partitions to be able to find the kernel on its own, so it will need a patch This solution isn''t perfect, but I think it''s a good start. * The Equipment Yes, my hardware is old/cheap. * Xen Cloud Platform server - XCP v1.1.0 * Build Server - FreeBSD 9.0-RELEASE (32-bit) * NFS Server - anything w/ enough space. Can be the Build Server. (should NOT be the XCP Server) * The Process * Patching pygrub Adrian Chadd (http://wiki.freebsd.org/AdrianChadd/XenHackery) did a great job laying the groundwork for a Xen (not XCP) installation. I also got a lot of help from Adian''s patch to pygrub at http://people.freebsd.org/~adrian/xen/bsd_pygrub/. The problem is that Adrian''s patch "is against the pygrub shipped with the xen-3.0.3-80.el5_3.2 CentOS 5.3 package." Using Adrian''s patch, I wsa able to create a patch against the pygrub shipped with XCP v1.1.0: =3D=3DBEGIN=3D=3D Patch attached. =3D=3D=3DEND=3D=3D=3D =20 * Creating a disk image I mounted an exported filesystem from the NFS server so that I would have a large, sharable space in which to place the disk image. This space is also mounted on the XCP server. In both cases, the NFS filesystem is mounted under /mnt. The following steps are taken on the BUILD host where /usr/src has already been compiled with buiidworld and buildkernel targets. =3D=3DBEGIN=3D=3D export FSIMAGE=3D"FreeBSD-PV.img" truncate -s 10G /mnt/${FSIMAGE} # Create the file where the im= age will reside. mdconfig -a -t vnode -f /mnt/${FSIMAGE} -u 0 # Create the image itself fdisk -BI md0 # Write the DOS partition table # Create an inital BSD Label=20 # For testing, we'll assume one large partition bsdlabel -w -B md0s1 bsdlabel -e md0s1 # Change partition a from 'unused' to '4.2BSD' newfs -O1 /dev/md0s1a # UFS, because pygrub doesn't grok UFS2 # Mount up the image and do the installation mount /dev/md0s1a /media # Perform the installation to the disk image. cd /usr/src make -s DESTDIR=3D/media KERNCONF=3DXEN installworld make -s DESTDIR=3D/media KERNCONF=3DXEN installkernel make -s DESTDIR=3D/media KERNCONF=3DXEN distribution # Set up the ttys(5) file so the VM will use Xen's console. cat >>/media/etc/ttys <>/media/etc/fstab <>/media/etc/rc.conf <>xcp-optionb-bsdlabel.txt <>/media/etc/ttys <>/media/etc/fstab <>/media/etc/rc.conf <>sys.stderr, "get_freebsd_slice: file : %s" % file + print >>sys.stderr, "get_freebsd_slice: offset: %s" % offset + + fd =3D os.open(file, os.O_RDONLY) + os.lseek(fd, offset + (SECTOR_SIZE * BSD_LABEL_LABELSECTOR) + BSD_LABE= L_LABELOFFSET, 0) + buf =3D os.read(fd, 1024) + + # Check the magic - is it a valid disklabel? if not, barf + bsd_d_magic =3D struct.unpack(">sys.stderr, "bsd magic: %s" % bsd_d_magic + print >>sys.stderr, "needed magic: %s" % BSD_LABEL_DISKMAGIC + if bsd_d_magic <> BSD_LABEL_DISKMAGIC: + raise RuntimeError, "FreeBSD label diskmagic incorrect" + + # Get sectorsize - does it make sense? If not, barf + bsd_d_secsize =3D struct.unpack(">sys.stderr, "bsd sector size: %s" % bsd_d_secsize + + # Get the first partition - is it the right type? If not, barf + bsd_d_npartitions =3D struct.unpack(">sys.stderr, "bsd number of partitions: %s" % bsd_d_npartitions + + # Grab the filesystem offset in sectors + bsd_d_part_offset =3D get_freebsd_partition_offset(buf, 0) + bsd_d_part_size =3D get_freebsd_partition_size(buf, 0) + bsd_d_part_fstype =3D get_freebsd_partition_fstype(buf, 0) + print >>sys.stderr, "bsd partition 0 offset: %s" % bsd_d_part_offset + print >>sys.stderr, "bsd partition 0 size : %s" % bsd_d_part_size + print >>sys.stderr, "bsd partition 0 type : %s" % bsd_d_part_fstype + + if bsd_d_part_fstype <> BSD_FS_BSDFFS: + raise RuntimeError, "First BSD partition is not type BSDFFS (7)" + + # XXX one should now verify that the given partition size and offset f= all within the disk size.. + + # Calculate the filesystem offset, return that + return (bsd_d_part_offset * bsd_d_secsize) + + def get_fs_offset_gpt(file): fd =3D os.open(file, os.O_RDONLY) # assume the first partition is an EFI system partition. @@ -81,9 +185,11 @@ buf =3D os.read(fd, 512) return struct.unpack(">sys.stderr, "BAIL! I don't know what to do!" + sys.exit(1) + if type =3D=3D FDISK_PART_GPT: offset =3D get_fs_offset_gpt(file) =20 @@ -620,7 +733,7 @@ ["quiet", "interactive", "not-really",= =20 "help", "output=3D", "entry=3D", "kern= el=3D",=20 "ramdisk=3D", "args=3D", "default_args= =3D",=20 - "extra_args=3D", "vm=3D"]) + "extra_args=3D", "vm=3D", "debug"]) except getopt.GetoptError: usage() sys.exit(1) @@ -633,6 +746,7 @@ output =3D None entry =3D None interactive =3D True + debug =3D False not_really =3D False default_args =3D "" extra_args =3D "" @@ -654,6 +768,8 @@ sys.exit() elif o in ("-n", "--not-really"): not_really =3D True + elif o in ("--debug"): + debug =3D True elif o in ("--output",): output =3D a elif o in ("--kernel",): @@ -671,6 +787,9 @@ elif o in ("--extra_args",): extra_args =3D a =20 + if debug: + logging.basicConfig(level=3Dlogging.DEBUG) + if output is None or output =3D=3D "-": fd =3D sys.stdout.fileno() else: @@ -700,6 +819,11 @@ except: # IOErrors raised by fsimage.open # RuntimeErrors raised by run_grub if no menu.lst present + if debug: + traceback.print_exc()=20 + print >>sys.stderr, "Filename: %s" % file + print >>sys.stderr, "Offset : %s" % offset + print >>sys.stderr, "Options : %s" % bootfsoptions fs =3D None continue else: @@ -738,6 +862,8 @@ except: # IOErrors raised by fsimage.open # RuntimeErrors raised by run_grub if no menu.lst present + if debug: + traceback.print_exc()=20 fs =3D None continue =20 --WYTEVAkct0FjGQmd Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="pygrub.xcp-ufs-debug.patch.sig" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEABECAAYFAk80D8oACgkQOEu4UTzvwQpIfACbBF4P6E+3g1ESHv182EquFbyi Dn4An1MTbrOynH9ui85xCk31PukepmSS =XwGn -----END PGP SIGNATURE----- --WYTEVAkct0FjGQmd-- --O3RTKUHj+75w1tg5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk82necACgkQOEu4UTzvwQo1mQCghecWgvlYAYkwBanZD0yrXT/3 IZgAnj1l65FAEROUPL+REiBvLxLbVGD0 =UldF -----END PGP SIGNATURE----- --O3RTKUHj+75w1tg5--