Date: Sat, 5 Jan 2019 23:39:57 +1100 From: Ben Woods <woodsb02@gmail.com> To: Colin Percival <cperciva@tarsnap.com> Cc: freebsd-cloud@freebsd.org, "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: Any plans for EC2 AMIs with RootOnZFS? Message-ID: <CAOc73CBQBOYT6XTP35HuiP8VxXyoqXSxNkEt-TZ=Y-0u1aBT8A@mail.gmail.com> In-Reply-To: <01000167f27b4c13-41c15731-ae4c-4658-820d-80c4d4fbb01c-000000@email.amazonses.com> References: <CAOc73CBQ%2B282amu-d%2BcxUehLBuhPKSrWTnft1_eSvY5FNKynwg@mail.gmail.com> <01000167dd7b16cd-58ee733f-c662-42d5-9c75-2301fbc8377a-000000@email.amazonses.com> <01000167f27b4c13-41c15731-ae4c-4658-820d-80c4d4fbb01c-000000@email.amazonses.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 28 Dec 2018 at 12:43, Colin Percival <cperciva@tarsnap.com> wrote: > I've created an experimental FreeBSD 12.0 ZFS AMI in the us-east-1 region: > ami-0786f5b55d5aa573f > > Since I'm basically a ZFS newbie (I run it on my laptop, but only with the > default setup from the installer, and I've never used any of the fancy ZFS > features) I don't know if I've set everything up properly. So far I have > one piece of feedback, which is that I should have marked canmount=off for > zroot. > Hi Colin, Thanks a lot for doing this - much appreciated. I have just given it a try, and have the following feedback for you: 1. ZFS DATASETS AND PROPERTIES SHOULD MATCH INSTALLER DEFAULT To avoid any confusion or unexpected behaviour, I believe the list of ZFS datasets and their properties should exactly match the bsdinstall defaults (especially for releases). zpool create: https://svnweb.freebsd.org/base/head/usr.sbin/bsdinstall/scripts/zfsboot?revision=342283&view=markup#l1308 zfs datasets and properties: https://svnweb.freebsd.org/base/head/usr.sbin/bsdinstall/scripts/zfsboot?revision=342283&view=markup#l141 Looking at how you created the zpool and zfs datasets (using the "zpool history zroot" command), the following changes would achieve this outcome: --- zfs-commands.old +++ zfs-commands.new @@ -1,19 +1,15 @@ -zpool create -o altroot=/mnt zroot ada0p2 -zfs set compress=on zroot +zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f zroot ada0p2 zfs create -o mountpoint=none zroot/ROOT zfs create -o mountpoint=/ -o canmount=noauto zroot/ROOT/default zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp -zfs create -o canmount=off -o mountpoint=/usr zroot/usr +zfs create -o mountpoint=/usr -o canmount=off zroot/usr zfs create zroot/usr/home -zfs create -o exec=off -o setuid=off zroot/usr/src -zfs create zroot/usr/obj -zfs create -o mountpoint=/usr/ports -o setuid=off zroot/usr/ports -zfs create -o exec=off -o setuid=off zroot/usr/ports/distfiles -zfs create -o exec=off -o setuid=off zroot/usr/ports/packages -zfs create -o canmount=off -o mountpoint=/var zroot/var +zfs create -o setuid=off zroot/usr/ports +zfs create zroot/usr/src +zfs create -o mountpoint=/var -o canmount=off zroot/var zfs create -o exec=off -o setuid=off zroot/var/audit zfs create -o exec=off -o setuid=off zroot/var/crash zfs create -o exec=off -o setuid=off zroot/var/log -zfs create -o atime=on -o exec=off -o setuid=off zroot/var/mail -zfs create -o exec=on -o setuid=off zroot/var/tmp +zfs create -o atime=on zroot/var/mail +zfs create -o setuid=off zroot/var/tmp zpool set bootfs=zroot/ROOT/default zroot 2. LOADER.CONF SETTINGS The following settings should be included in /boot/loader.conf: kern.geom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" 3. SYSCTL.CONF SETTINGS The following settings should be included in /etc/sysctl.conf: vfs.zfs.min_auto_ashift=12 4. THE GPT TABLE IS INITIALLY CORRUPT I suspect this is because I used a 50Gb disk, and it hasn't been expanded correctly. The following error was in DMESG and I suspect is related: GEOM: ada0: the secondary GPT header is not in the last LBA. I was able to fix this with the following commands on firstboot: # gpart recover ada0 # gpart resize -i 2 ada0 # zpool online -e zroot ada0p2 The full background of my investigation and resolution is shown below: # gpart show => 40 20971440 ada0 GPT (50G) [CORRUPT] 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 20967424 2 freebsd-zfs (10G) 20969472 2008 - free - (1.0M) # gpart recover ada0 ada0 recovered # gpart show => 40 104857520 ada0 GPT (50G) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 20967424 2 freebsd-zfs (10G) 20969472 83888088 - free - (40G) # gpart resize -i 2 ada0 ada0p2 resized # gpart show => 40 104857520 ada0 GPT (50G) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 104855512 2 freebsd-zfs (50G) # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 9.50G 1.25G 8.25G - - 0% 13% 1.00x ONLINE - # zpool online -e zroot ada0p2 # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 49.5G 1.25G 48.3G - - 0% 2% 1.00x ONLINE - Thanks again for your efforts. Regards, Ben -- From: Benjamin Woods woodsb02@gmail.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOc73CBQBOYT6XTP35HuiP8VxXyoqXSxNkEt-TZ=Y-0u1aBT8A>