Date: Thu, 6 May 2021 15:56:18 -0700 From: Mark Millard <marklmi@yahoo.com> To: tech-lists <tech-lists@zyxst.net> Cc: freebsd-arm@freebsd.org Subject: Re: rpi4 zfs-on-root boot-to-usb3 [Example sequence that lead to booting zfs-on-root under GPT partitioning] Message-ID: <F399A252-76D1-42ED-93D6-35CD05DC03BB@yahoo.com> In-Reply-To: <EF4DFDA7-0388-4326-ACEC-4CAFDF296EEF@yahoo.com> References: <YJPPMBijtB5MDZIo@ceres.zyxst.net> <BE982BA1-0E6B-4805-B999-0A63B6C76BCF@yahoo.com> <E78287CB-2DCD-415E-B513-922D67E933AD@yahoo.com> <96F0BF83-D0FB-442F-B8A1-54194A724BD1@yahoo.com> <EF4DFDA7-0388-4326-ACEC-4CAFDF296EEF@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-May-6, at 13:42, Mark Millard <marklmi at yahoo.com> wrote: > On 2021-May-6, at 13:09, Mark Millard <marklmi at yahoo.com> wrote: >=20 >> On 2021-May-6, at 12:12, Mark Millard <marklmi at yahoo.com> wrote: >>=20 >>> On 2021-May-6, at 05:55, Mark Millard <marklmi at yahoo.com> wrote: >>>=20 >>>> On 2021-May-6, at 04:12, tech-lists <tech-lists at zyxst.net> = wrote: >>>>=20 >>>>> How can zfs-on-root boot-to-usb3 on rpi4 be accomplished? >>>>>=20 >>>>> I've tried bsdinstall from a mmcsd-booted rpi4 but there seems to = be >>>>> problems with it that I can't work around. What's really needed is = an >>>>> installer, but these aren't made for arm64.aarch64 rpi4 from what = I can >>>>> see (I'm no expert though, it's entirely feasible i've missed >>>>> something). >>>>>=20 >>>>> Maybe one way of doing it would be to have a usb key (as ufs2) for = the >>>>> system to boot on, then have /home /usr/obj and other larger dirs = on the >>>>> usb3-zfs disk. >>>>=20 >>>> I used bsdinstall from booting a releng/13.0's release/13.0.0.0 >>>> microsd card in a 8 GiBYte RPi4B to produce the: >>>> . . . >>>=20 >>> Various details shown will just be my specific >>> choices. (The RPi4B's that I have access to have >>> the 2021-Apr-29 default(/critical) EEPROM image.) >>>=20 >>> Taking notes as I go (and readjusting as I >>> progress and figure things out, eliminating >>> failing attempts as well) . . . >>>=20 >>> Booting based on a microsd card with releng/13.0 's >>> release/13.0.0 as its basis. The context has a >>> working network with internet access. >=20 > I'll note that setting up the microsd card context to > have the correct timezone and time is something I > presumed was already in place. But such is not the > case for an RPI image from the servers. >=20 > So I effectively presumed booting with /etc/rc.conf > having, say, >=20 > # > ntpd_enable=3D"YES" > ntpd_sync_on_start=3D"YES" > ntpd_user=3D"root" >=20 > already added (or a manual time set) > and having done a: >=20 > # tzsetup >=20 > sequence with appropriate selections. >=20 >>> # uname -apKU >>> FreeBSD generic 13.0-RELEASE FreeBSD 13.0-RELEASE #0 = releng/13.0-n244733-ea31abc261f: Fri Apr 9 06:06:55 UTC 2021 = root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC = arm64 aarch64 1300139 1300139 >>>=20 >>> Plug in USB3 SSD. Ends up as da0. >>>=20 >>> # /bin/sh # Just for my familiarity >>> # set -o vi >>>=20 >>> # mkdir -p /usr/freebsd-dist >>> # cd /usr/freebsd-dist >>> # fetch = http://ftp3.freebsd.org/pub/FreeBSD/releases/arm64/13.0-RELEASE/MANIFEST >>> MANIFEST 782 B 6147 = kBps 00s >>> # cd ~ >>>=20 >>> # bsdinstall >>>=20 >>> Continue with default keymap : Select >>>=20 >>> Enter hostname as ZFStest : OK >>>=20 >>> [*] base-dbg >>> [*] kernel-dbg >>> [ ] ports >>> [ ] src >>> [*] tests >>> then: OK >>>=20 >>> (Note: I use git for src and ports.) >>>=20 >>> Main Site : OK >>>=20 >>> Auto (ZFS) : OK >>>=20 >>> Pool Name : Select >>>=20 >>> Enter name for zpool ztstp : OK >>>=20 >>> Swap Size : Select >>>=20 >>> Enter swap size 24g : OK >>>=20 >>> Proceed with Installation : Select >>>=20 >>> Stripe - No Redundancy : OK >>>=20 >>> [*] da0 : OK >>>=20 >>> Last Chance for da0 : YES >>>=20 >>> Downloads . . . >>> Extracts . . . >>>=20 >>> New Password: . . . >>> Retype New Password: . . . >>>=20 >>> genet0 : OK >>>=20 >>> configure IPv4 : YES >>> configure DHCP : YES >>> configure IPv6 : YES >>> try SLAAC : YES >>> Resolver Configuration : OK >>>=20 >>> time is UTC? : YES >>>=20 >>> America : OK >>> United States of America : OK >>> Pacific : OK >>> Does PDT look reasonable? : Yes >>> May 2021 6 : Set Date >>> 11 07 00 : Set Time >>>=20 >>> [ ] local_unbound >>> [*] sshd >>> [ ] moused >>> [ ] ntpdate >>> [*] ntpd >>> [*] powerd >>> [*] dumpdev >>> Then : OK >>>=20 >>> No hardening options enabled : OK >>>=20 >>> Add uses? : Yes >>> . . . details omitted . . . >>> OK ? yes >>> Add another user? no >>>=20 >>> Handbook : OK >>> [*] en : OK >>>=20 >>> Apply configuration and exit installer : OK >>> open a shell : No >>=20 >> I suggested an inappropriate later stage if one >> wants to try the same vintage of rpi-firmware >> and u-boot as releng/13.0 itself uses. One >> can copy over materials before the shutdown >> by getting them from /boot/msdos/ . >>=20 >> Note that you might not want to copy over >> /boot/msdos/EFI/... as the install will >> already have such. >>=20 >> So, something like: >>=20 >> # mount -onoatime -tmsdosfs /dev/da0p1 /mnt >> # cp -aRx /boot/msdos/[LRa-z]* /mnt/ >> # umount /mnt >>=20 >> then do the shutdown and remove the microsd card. >>=20 >>> # shutdown -p now >>=20 >> The following is only if one did not copy from >> /boot/msdosfs/ or one needs more recent >> materials, such as U-Boot for *C0T revision >> processors. >>=20 >>> At this point it still can not boot an RPi4B >>> for lack of rpi firmware and U-Boot. >>>=20 >>> I have such available on other machine based >>> on the latest ports instead of quarterly. There >>> are RPi4B's in the world that need the more >>> modern U-Boot compared to the quarterly that >>> releng/13.0 is tied to by default. But you >>> likely could install rpi-firmware and >>> u-boot-rpi-arm64 on the microsd card and >>> then copy over materials from there. >>=20 >> The above is dumb suggestion unless newer >> material is needed. See before the shutdown >> above. >>=20 >> The below is me getting more recent materials >> (well, U-Boot) from another context. You might >> not do similarly. >>=20 >>> In my context . . . >>>=20 >>> # gpart show -p da1 >>> =3D> 40 468862048 da1 GPT (224G) >>> 40 532480 da1p1 efi (260M) >>> 532520 2008 - free - (1.0M) >>> 534528 50331648 da1p2 freebsd-swap (24G) >>> 50866176 417994752 da1p3 freebsd-zfs (199G) >>> 468860928 1160 - free - (580K) >>>=20 >>> # mount -onoatime -tmsdosfs /dev/da1p1 /mnt >>> # cp -aRx /usr/local/share/rpi-firmware/* /mnt/ >>> # cp -aRx /mnt/config_arm64.txt /mnt/config.txt=20 >>> # cp -aRx /usr/local/share/u-boot/u-boot-rpi-arm64/u-boot.bin /mnt/ >>> # umount /mnt >>=20 >> (Note: It is possible to be more selective in >> what to copy over. I did not complicate the >> sequence with such handling.) >>=20 >>=20 >> Back to the normal flow on the RPi4B given >> appropriate RPi* materials copied over to the >> msdos file system . . . >>=20 >>> Back to the RPi4B, no microsd card but plugging in the >>> USB3 SSD and booting and logging in: >>>=20 >>> Dec 31 16:00:48 ZFStest login[1351]: ROOT LOGIN (root) ON ttyu0 >>> FreeBSD 13.0-RELEASE (GENERIC) #0 releng/13.0-n244733-ea31abc261f: = Fri Apr 9 03:54:53 UTC 2021 >>>=20 >>> # gpart show -p >>> =3D> 40 468862048 da0 GPT (224G) >>> 40 532480 da0p1 efi (260M) >>> 532520 2008 - free - (1.0M) >>> 534528 50331648 da0p2 freebsd-swap (24G) >>> 50866176 417994752 da0p3 freebsd-zfs (199G) >>> 468860928 1160 - free - (580K) >>>=20 >>> # uname -apKU >>> FreeBSD ZFStest 13.0-RELEASE FreeBSD 13.0-RELEASE #0 = releng/13.0-n244733-ea31abc261f: Fri Apr 9 03:54:53 UTC 2021 = root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC = arm64 aarch64 1300139 1300139 >>>=20 >>> I end up adding to /etc/rc.conf: >>>=20 >>> # >>> ntpd_sync_on_start=3D"YES" >>> ntpd_user=3D"root" >>>=20 >>> The first boot's time will be messed up for >>> lack of the ntpd_sync_on_start=3D"YES" . >>>=20 >>> # shutdown -r now >>>=20 >>> After login: >>>=20 >>> # ls -Tld /etc/rc.conf >>> -rw-r--r-- 1 root wheel 279 Dec 31 16:12:37 1969 /etc/rc.conf >>> # touch /etc/rc.conf >>>=20 >>> There are other files around with such an odd timestamp. >>>=20 >>> # zpool list >>> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP = HEALTH ALTROOT >>> ztstp 199G 1.09G 198G - - 0% 0% 1.00x = ONLINE - >>>=20 >>> # zfs list >>> NAME USED AVAIL REFER MOUNTPOINT >>> ztstp 1.09G 192G 96K /ztstp >>> ztstp/ROOT 1.08G 192G 96K none >>> ztstp/ROOT/default 1.08G 192G 1.08G / >>> ztstp/tmp 96K 192G 96K /tmp >>> ztstp/usr 416K 192G 96K /usr >>> ztstp/usr/home 128K 192G 128K /usr/home >>> ztstp/usr/ports 96K 192G 96K /usr/ports >>> ztstp/usr/src 96K 192G 96K /usr/src >>> ztstp/var 680K 192G 96K /var >>> ztstp/var/audit 96K 192G 96K /var/audit >>> ztstp/var/crash 96K 192G 96K /var/crash >>> ztstp/var/log 200K 192G 200K /var/log >>> ztstp/var/mail 96K 192G 96K /var/mail >>> ztstp/var/tmp 96K 192G 96K /var/tmp >>>=20 >>> # more /etc/sysctl.conf=20 >>> # $FreeBSD$ >>> # >>> # This file is read when going to multi-user and its contents piped = thru >>> # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for = details. >>> # >>>=20 >>> # Uncomment this to prevent users from seeing information about = processes that >>> # are being run under another UID. >>> #security.bsd.see_other_uids=3D0 >>> vfs.zfs.min_auto_ashift=3D12 >>>=20 >>> I'll note that in: >>>=20 >>> # more /boot/efi/config.txt=20 >>> [all] >>> arm_64bit=3D1 >>> dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don >>> dtoverlay=3Dmmc >>> dtoverlay=3Ddisable-bt >>> device_tree_address=3D0x4000 >>> kernel=3Du-boot.bin >>>=20 >>> [pi4] >>> hdmi_safe=3D1 >>> armstub=3Darmstub8-gic.bin >>>=20 >>> The hdmi)safe=3D1 line restricts the HDMI display >>> resolution/scaling. Any of the following >>> replacements for that line will avoid that but >>> in some contexts one could end up in a "blind >>> display" context instead, which is why hdmi_safe >>> is enabled by default. >>>=20 >>> hdmi_safe=3D0 >>> or: >>> #hdmi_safe=3D1 >>> or just delete the line. >>=20 By warned that things are not set up with /etc/fstab being based on identification via unique labels: # more /etc/fstab # Device Mountpoint FStype Options Dump = Pass# /dev/da0p1 /boot/efi msdosfs rw 2 = 2 /dev/da0p2 none swap sw 0 0 This can be an issue when multiple USB devices are present: which will end up as /dev/da0 ? But default label names would also tend to not be unique as well. Setting and using unique labels for such can be appropriate, sort of like unique zpool naming on various media can be appropriate. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F399A252-76D1-42ED-93D6-35CD05DC03BB>