Date: Thu, 6 May 2021 13:42:50 -0700 From: Mark Millard <marklmi@yahoo.com> To: tech-lists <tech-lists@zyxst.net> Cc: freebsd-arm@freebsd.org Subject: Re: rpi4 zfs-on-root boot-to-usb3 [Example sequence that lead to booting zfs-on-root under GPT partitioning] Message-ID: <EF4DFDA7-0388-4326-ACEC-4CAFDF296EEF@yahoo.com> In-Reply-To: <96F0BF83-D0FB-442F-B8A1-54194A724BD1@yahoo.com> References: <YJPPMBijtB5MDZIo@ceres.zyxst.net> <BE982BA1-0E6B-4805-B999-0A63B6C76BCF@yahoo.com> <E78287CB-2DCD-415E-B513-922D67E933AD@yahoo.com> <96F0BF83-D0FB-442F-B8A1-54194A724BD1@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-May-6, at 13:09, Mark Millard <marklmi at yahoo.com> wrote: > On 2021-May-6, at 12:12, Mark Millard <marklmi at yahoo.com> wrote: >=20 >> On 2021-May-6, at 05:55, Mark Millard <marklmi at yahoo.com> wrote: >>=20 >>> On 2021-May-6, at 04:12, tech-lists <tech-lists at zyxst.net> wrote: >>>=20 >>>> How can zfs-on-root boot-to-usb3 on rpi4 be accomplished? >>>>=20 >>>> I've tried bsdinstall from a mmcsd-booted rpi4 but there seems to = be >>>> problems with it that I can't work around. What's really needed is = an >>>> installer, but these aren't made for arm64.aarch64 rpi4 from what I = can >>>> see (I'm no expert though, it's entirely feasible i've missed >>>> something). >>>>=20 >>>> Maybe one way of doing it would be to have a usb key (as ufs2) for = the >>>> system to boot on, then have /home /usr/obj and other larger dirs = on the >>>> usb3-zfs disk. >>>=20 >>> I used bsdinstall from booting a releng/13.0's release/13.0.0.0 >>> microsd card in a 8 GiBYte RPi4B to produce the: >>> . . . >>=20 >> Various details shown will just be my specific >> choices. (The RPi4B's that I have access to have >> the 2021-Apr-29 default(/critical) EEPROM image.) >>=20 >> Taking notes as I go (and readjusting as I >> progress and figure things out, eliminating >> failing attempts as well) . . . >>=20 >> Booting based on a microsd card with releng/13.0 's >> release/13.0.0 as its basis. The context has a >> working network with internet access. I'll note that setting up the microsd card context to have the correct timezone and time is something I presumed was already in place. But such is not the case for an RPI image from the servers. So I effectively presumed booting with /etc/rc.conf having, say, # ntpd_enable=3D"YES" ntpd_sync_on_start=3D"YES" ntpd_user=3D"root" already added (or a manual time set) and having done a: # tzsetup sequence with appropriate selections. >> # uname -apKU >> FreeBSD generic 13.0-RELEASE FreeBSD 13.0-RELEASE #0 = releng/13.0-n244733-ea31abc261f: Fri Apr 9 06:06:55 UTC 2021 = root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC = arm64 aarch64 1300139 1300139 >>=20 >> Plug in USB3 SSD. Ends up as da0. >>=20 >> # /bin/sh # Just for my familiarity >> # set -o vi >>=20 >> # mkdir -p /usr/freebsd-dist >> # cd /usr/freebsd-dist >> # fetch = http://ftp3.freebsd.org/pub/FreeBSD/releases/arm64/13.0-RELEASE/MANIFEST >> MANIFEST 782 B 6147 = kBps 00s >> # cd ~ >>=20 >> # bsdinstall >>=20 >> Continue with default keymap : Select >>=20 >> Enter hostname as ZFStest : OK >>=20 >> [*] base-dbg >> [*] kernel-dbg >> [ ] ports >> [ ] src >> [*] tests >> then: OK >>=20 >> (Note: I use git for src and ports.) >>=20 >> Main Site : OK >>=20 >> Auto (ZFS) : OK >>=20 >> Pool Name : Select >>=20 >> Enter name for zpool ztstp : OK >>=20 >> Swap Size : Select >>=20 >> Enter swap size 24g : OK >>=20 >> Proceed with Installation : Select >>=20 >> Stripe - No Redundancy : OK >>=20 >> [*] da0 : OK >>=20 >> Last Chance for da0 : YES >>=20 >> Downloads . . . >> Extracts . . . >>=20 >> New Password: . . . >> Retype New Password: . . . >>=20 >> genet0 : OK >>=20 >> configure IPv4 : YES >> configure DHCP : YES >> configure IPv6 : YES >> try SLAAC : YES >> Resolver Configuration : OK >>=20 >> time is UTC? : YES >>=20 >> America : OK >> United States of America : OK >> Pacific : OK >> Does PDT look reasonable? : Yes >> May 2021 6 : Set Date >> 11 07 00 : Set Time >>=20 >> [ ] local_unbound >> [*] sshd >> [ ] moused >> [ ] ntpdate >> [*] ntpd >> [*] powerd >> [*] dumpdev >> Then : OK >>=20 >> No hardening options enabled : OK >>=20 >> Add uses? : Yes >> . . . details omitted . . . >> OK ? yes >> Add another user? no >>=20 >> Handbook : OK >> [*] en : OK >>=20 >> Apply configuration and exit installer : OK >> open a shell : No >=20 > I suggested an inappropriate later stage if one > wants to try the same vintage of rpi-firmware > and u-boot as releng/13.0 itself uses. One > can copy over materials before the shutdown > by getting them from /boot/msdos/ . >=20 > Note that you might not want to copy over > /boot/msdos/EFI/... as the install will > already have such. >=20 > So, something like: >=20 > # mount -onoatime -tmsdosfs /dev/da0p1 /mnt > # cp -aRx /boot/msdos/[LRa-z]* /mnt/ > # umount /mnt >=20 > then do the shutdown and remove the microsd card. >=20 >> # shutdown -p now >=20 > The following is only if one did not copy from > /boot/msdosfs/ or one needs more recent > materials, such as U-Boot for *C0T revision > processors. >=20 >> At this point it still can not boot an RPi4B >> for lack of rpi firmware and U-Boot. >>=20 >> I have such available on other machine based >> on the latest ports instead of quarterly. There >> are RPi4B's in the world that need the more >> modern U-Boot compared to the quarterly that >> releng/13.0 is tied to by default. But you >> likely could install rpi-firmware and >> u-boot-rpi-arm64 on the microsd card and >> then copy over materials from there. >=20 > The above is dumb suggestion unless newer > material is needed. See before the shutdown > above. >=20 > The below is me getting more recent materials > (well, U-Boot) from another context. You might > not do similarly. >=20 >> In my context . . . >>=20 >> # gpart show -p da1 >> =3D> 40 468862048 da1 GPT (224G) >> 40 532480 da1p1 efi (260M) >> 532520 2008 - free - (1.0M) >> 534528 50331648 da1p2 freebsd-swap (24G) >> 50866176 417994752 da1p3 freebsd-zfs (199G) >> 468860928 1160 - free - (580K) >>=20 >> # mount -onoatime -tmsdosfs /dev/da1p1 /mnt >> # cp -aRx /usr/local/share/rpi-firmware/* /mnt/ >> # cp -aRx /mnt/config_arm64.txt /mnt/config.txt=20 >> # cp -aRx /usr/local/share/u-boot/u-boot-rpi-arm64/u-boot.bin /mnt/ >> # umount /mnt >=20 > (Note: It is possible to be more selective in > what to copy over. I did not complicate the > sequence with such handling.) >=20 >=20 > Back to the normal flow on the RPi4B given > appropriate RPi* materials copied over to the > msdos file system . . . >=20 >> Back to the RPi4B, no microsd card but plugging in the >> USB3 SSD and booting and logging in: >>=20 >> Dec 31 16:00:48 ZFStest login[1351]: ROOT LOGIN (root) ON ttyu0 >> FreeBSD 13.0-RELEASE (GENERIC) #0 releng/13.0-n244733-ea31abc261f: = Fri Apr 9 03:54:53 UTC 2021 >>=20 >> # gpart show -p >> =3D> 40 468862048 da0 GPT (224G) >> 40 532480 da0p1 efi (260M) >> 532520 2008 - free - (1.0M) >> 534528 50331648 da0p2 freebsd-swap (24G) >> 50866176 417994752 da0p3 freebsd-zfs (199G) >> 468860928 1160 - free - (580K) >>=20 >> # uname -apKU >> FreeBSD ZFStest 13.0-RELEASE FreeBSD 13.0-RELEASE #0 = releng/13.0-n244733-ea31abc261f: Fri Apr 9 03:54:53 UTC 2021 = root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC = arm64 aarch64 1300139 1300139 >>=20 >> I end up adding to /etc/rc.conf: >>=20 >> # >> ntpd_sync_on_start=3D"YES" >> ntpd_user=3D"root" >>=20 >> The first boot's time will be messed up for >> lack of the ntpd_sync_on_start=3D"YES" . >>=20 >> # shutdown -r now >>=20 >> After login: >>=20 >> # ls -Tld /etc/rc.conf >> -rw-r--r-- 1 root wheel 279 Dec 31 16:12:37 1969 /etc/rc.conf >> # touch /etc/rc.conf >>=20 >> There are other files around with such an odd timestamp. >>=20 >> # zpool list >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP = HEALTH ALTROOT >> ztstp 199G 1.09G 198G - - 0% 0% 1.00x = ONLINE - >>=20 >> # zfs list >> NAME USED AVAIL REFER MOUNTPOINT >> ztstp 1.09G 192G 96K /ztstp >> ztstp/ROOT 1.08G 192G 96K none >> ztstp/ROOT/default 1.08G 192G 1.08G / >> ztstp/tmp 96K 192G 96K /tmp >> ztstp/usr 416K 192G 96K /usr >> ztstp/usr/home 128K 192G 128K /usr/home >> ztstp/usr/ports 96K 192G 96K /usr/ports >> ztstp/usr/src 96K 192G 96K /usr/src >> ztstp/var 680K 192G 96K /var >> ztstp/var/audit 96K 192G 96K /var/audit >> ztstp/var/crash 96K 192G 96K /var/crash >> ztstp/var/log 200K 192G 200K /var/log >> ztstp/var/mail 96K 192G 96K /var/mail >> ztstp/var/tmp 96K 192G 96K /var/tmp >>=20 >> # more /etc/sysctl.conf=20 >> # $FreeBSD$ >> # >> # This file is read when going to multi-user and its contents piped = thru >> # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for = details. >> # >>=20 >> # Uncomment this to prevent users from seeing information about = processes that >> # are being run under another UID. >> #security.bsd.see_other_uids=3D0 >> vfs.zfs.min_auto_ashift=3D12 >>=20 >> I'll note that in: >>=20 >> # more /boot/efi/config.txt=20 >> [all] >> arm_64bit=3D1 >> dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don >> dtoverlay=3Dmmc >> dtoverlay=3Ddisable-bt >> device_tree_address=3D0x4000 >> kernel=3Du-boot.bin >>=20 >> [pi4] >> hdmi_safe=3D1 >> armstub=3Darmstub8-gic.bin >>=20 >> The hdmi)safe=3D1 line restricts the HDMI display >> resolution/scaling. Any of the following >> replacements for that line will avoid that but >> in some contexts one could end up in a "blind >> display" context instead, which is why hdmi_safe >> is enabled by default. >>=20 >> hdmi_safe=3D0 >> or: >> #hdmi_safe=3D1 >> or just delete the line. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?EF4DFDA7-0388-4326-ACEC-4CAFDF296EEF>