Date: Fri, 24 Apr 2020 15:09:35 -0700 From: Mark Millard <marklmi@yahoo.com> To: bob prohaska <fbsd@www.zefox.net> Cc: freebsd-arm <freebsd-arm@freebsd.org> Subject: Re: Booting from USB on RPI3 Message-ID: <5FA69E16-72DE-4175-A0FB-BEFA1A865633@yahoo.com> In-Reply-To: <20200424195953.GA6707@www.zefox.net> References: <20200423162124.GA3583@www.zefox.net> <CAJuc1zMHL6ke3LH67iV8%2BogKFO18xwW_v7bRAUsj8CbxD2EOTw@mail.gmail.com> <8D1F6A8D-4910-4C1E-8EB5-2F5F89E31120@yahoo.com> <CAJuc1zOTnoRmeaKXHMTH9po=6ipBYDxvxAoiVmow-sNgwUERVQ@mail.gmail.com> <20200423212614.GA3996@www.zefox.net> <328B7083-1119-43F1-A6E4-61F1FB9E922E@yahoo.com> <20200423233259.GD3996@www.zefox.net> <74DB7BC9-8F60-404B-BF7B-02B06D5A1011@yahoo.com> <20200424021808.GA4638@www.zefox.net> <F0C464CD-2E1A-487C-97D3-D79C7A197132@yahoo.com> <20200424195953.GA6707@www.zefox.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-Apr-24, at 12:59, bob prohaska <fbsd at www.zefox.net> wrote: > On Thu, Apr 23, 2020 at 09:26:53PM -0700, Mark Millard wrote: >> On 2020-Apr-23, at 19:18, bob prohaska <fbsd at www.zefox.net> wrote: >>=20 >>> On Thu, Apr 23, 2020 at 06:16:14PM -0700, Mark Millard wrote: >>>> On 2020-Apr-23, at 16:32, bob prohaska <fbsd at www.zefox.net> = wrote: >>>>=20 >>>>> On Thu, Apr 23, 2020 at 03:24:57PM -0700, Mark Millard wrote: >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> For USB-only or for microsd card also in use, >>>>>> have you tried the equivalent of: >>>>>>=20 >>>>>> OK lsdev >>>>>>=20 >>>>> Another surprise. First time I tried, after some time idle: >>>>> OK lsdev >>>>> unknown command >>>>> but a simple=20 >>>>> ls >>>>> listed the contents of the root directory. Apparently the >>>>> drive was asleep after an idle timeout. >>>>>=20 >>>>> A second attempt at lsdev produced >>>>> OK lsdev >>>>> disk devices: >>>>> disk0: 250085377 X 512 blocks (removable) >>>>> disk0s1: DOS/Windows >>>>> disk0s2: FreeBSD >>>>> disk0s2a: FreeBSD UFS >>>>> disk0s2b: FreeBSD swap >>>>> disk1: 78140161 X 512 blocks >>>>> disk1s1: DOS/Windows >>>>> disk1s2: FreeBSD >>>>> disk1s2a: FreeBSD UFS >>>>> disk1s2b: FreeBSD swap >>>>> http: (unknown) >>>>> net devices: >>>>> net0: >>>>=20 >>>> Where both the microsd card (disk0) and >>>> the USB drive (disk1) set up via installing >>>> FreeBSD distributed .iso's, at least for >>>> how the file systems were created (even >>>> if somewhat adjusted later)? >>>>=20 >>> (thinking you meant "were both...") >>=20 >> True. Dumb typo. >>=20 >>> The answer is yes, both were written with dd from a downloaded .img = file. >>>=20 >>>> If yes, it may be that you have two instances >>>> of each of: >>>>=20 >>>> /dev/msdosfs/MSDOSBOOT >>>> /dev/ufs/rootfs >>>>=20 >>>> one per disk* for each. >>>>=20 >>> Yes, but if I'm quick enough stopping u-boot wouldn't >>> only one be read per boot attempt?=20 >>=20 >> If you stop the boot at: >>=20 >> Hit any key to stop autoboot: 2 =08=08=08 1 =08=08=08 0=20 >>=20 >> then you are in u-boot/EFI before the FreeBSD >> loader is loaded/in-use. This is a very >> different context and does not involve or >> support the various types of labels or >> notations from FreeBSD. I assume you are >> not referring to this rather early stage >> of the boot sequence. >>=20 > There's a very early prompt from u-boot, which > must be invoked to stop booting the microSD. > That's the point where one gets the u-boot>=20 > prompt, and the place to run the usb reset and > run bootcmd_usb0 commands. The next chance to > intervene is at the pause before booting the > loaded kernel, which is where I was poking around. But the "run bootcmd_usb0" continues the boot and the FreeBSD loader still runs and there still is the finding of all the label information and the conflicting labeling resolution. The u-boot stage has no effect on this if the boot is continued. >> FreeBSD looks at disks before mounting them >> and it looks at partitions/slices before they >> are mounted. It creates the bindings for >> the labels of various types (msdosfs, ufs, >> gpt, label) before mounting is even possible. >> It does so across all the media present at >> the time. This is so all the labels are >> available to use to pick what to mount (in >> FreeBSD terms). But it means that the loader >> has to pick what to do about ambiguous labeling >> before you are involved. >>=20 >=20 > I suspect that had I not used a microSD with a > runnable FreeBSD installation the boot of USB > would have been trouble-free. =20 >=20 >> (I've no clue if it keeps just the first-find >> or last-find or what. But in the end a label >> identifies only one thing, not multiple things.) >>=20 >> There is meta-data on the media and in the >> partitions/slices/file-systems that indicates >> what labels to create. >>=20 > Are the labels created at each boot? I thought > they were defined at partitioning.=20 There is static information stored on the drive. There is run-time activity to use it to get the labeling behaviors. The run-time activity is certainly per-boot (and drive attachment/detachment while booted). There is code in FreeBSD's loader to create the behaviors. There is code in FreeBSD to create the behaviors. Only some types of labels are tied to partitioning. Some types of partitioning do not support labels. Some other types of labels are tied to file-systems, no matter if the partition type that the file system is in has partition/slice labeling or not. One type of label is tied to GEOM. It can be possible to have all 3 types of labels for the same content. Partitioning (when the partition type supports labels): gpart add -l gpart modify -l gpart show -l File Systems: newfs -L tunefs -L tunefs -p (will show the label assigned, if any) newfs_msdos -L GEOM (I'll only list the one command variant): glabel label >> (The kernel may well repeat doing some things >> that the loader had to do, for all I know. >> Refinding labels could be an example.) >>=20 >>>> These types of naming are expecting to have >>>=20 >>>> been set up with what is after the last / being >>>> a unique name relative to everything else >>>> potentially present at the same time. This >>>> means that two .iso installs would need >>>> to have been adjusted to make the names >>>> distinct before trying to use the media >>>> together. >>>>=20 >>>> If these names are duplicated, then use of >>>> the notations in /etc/fstab would be ambiguous >>>> for identifying the drives and I've no clue >>>> which instance wins --or if which wins is even >>>> stable. >>>>=20 >>>> For ufs, tunefs has an option for setting the >>>> file system label: >>>>=20 >>>> -L volname >>>> Add/modify an optional file system volume label. Legal >>>> characters are alphanumerics, dashes, and underscores. >>>>=20 >>>> Once the adjustment is seen (remount/reboot?), >>>> that should make: >>>>=20 >>>> /dev/ufs/... >>>>=20 >>>> change to the new name. >>>>=20 >>>> I'm only aware of file systsem creation-time setting >>>> for msdos via newfs_msdos : >>>>=20 >>>> -L label >>>> Volume label (up to 11 characters). The label should = consist of >>>> only those characters permitted in regular DOS (8+3) = filenames. >>>>=20 >>>> That should make >>>>=20 >>>> /dev/msdosfs/... >>>>=20 >>>> be the assigned name. But newfs_msdos would not >>>> preserve the contents of the file system that >>>> was present before (if any). >>>>=20 >>>> glabel can be used to assign labels that are >>>> independent of the file system and of the >>>> partition metadata, labels that would appear >>>> under /dev/label/ instead of under /dev/ufs/ >>>> or /dev/msdosfs/ or /dev/gpt/ . (gpt is not >>>> an option for MBR contexts.) >>>>=20 >>>>=20 >>>>>> and then used the naming convention shown in >>>>>> the output for any loader commands/settings >>>>>> instead of using FreeBSD-stage da0 based naming? >>>>>> (You might want to report what lsdev shows.) >>>>>>=20 >>>>>> Note: the disk0 and disk1 like prefixes are >>>>>> copied from the u-boot notations that the >>>>>> loader gets from u-boot, if I understand >>>>>> right. >>>>>>=20 >>>>>> Does your loader.conf do anything to control >>>>>> what is booted?=20 >>>>>=20 >>>>> Not in a way I recognize, but you might: >>>>> OK more /boot/loader.conf >>>>> *** FILE /boot/loader.conf BEGIN *** >>>>> # Configure USB OTG; see usb_template(4). >>>>> hw.usb.template=3D3 >>>>> umodem_load=3D"YES" >>>>> # Multiple console (serial+efi gop) enabled. >>>>> boot_multicons=3D"YES" >>>>> boot_serial=3D"YES" >>>>> # Disable the beastie menu and color >>>>> beastie_disable=3D"YES" >>>>> loader_color=3D"NO" >>>>> *** FILE /boot/loader.conf END *** >>>>=20 >>>> Nothing there to control what is booted. >>>>=20 >>>>> As Joanathan asked: what does >>>>>> your /etc/fstab have listed? (/etc/fstab should >>>>>> have FreeBSD notations, not loader device names.) >>>>>=20 >>>>> OK more /etc/fstab >>>>> *** FILE /etc/fstab BEGIN *** >>>>> # Custom /etc/fstab for FreeBSD embedded images >>>>> /dev/ufs/rootfs / ufs rw 1 1 >>>>> /dev/msdosfs/MSDOSBOOT /boot/msdos msdosfs rw,noatime 0 0 >>>>> tmpfs /tmp tmpfs rw,mode=3D1777,size=3D50m 0 0 >>>>> *** FILE /etc/fstab END *** >>>>> OK=20 >>>>=20 >>>> See earlier notes: each media may be defining >>>> what causes /dev/ufs/rootfs , leading to ambiguity. >>>> Similarly for /dev/msdosfs/MSDOSBOOT . >>>>=20 >>>> /etc/fstab should use notation that is a unique >>>> identification for each device. The media should >>>> have names assigned that are unique compared to >>>> anything else potentially connected at the same >>>> time if labeling style names are to be used. >>>>=20 >>>> Names like /dev/da* are not stable from boot to >>>> boot if multiple USB media are present at the >>>> same time: which media gets which da* name could >>>> vary from boot to boot. Using some form of unique >>>> label assignment is a way of avoiding this. >>>>=20 >>>> However, if you stick to only one USB drive >>>> being present, likely /dev/da0s2a would be >>>> a stable USB drive ufs file system reference >>>> for that context. Similarly for /dev/da0s1 >>>> being a stable reference to the USB msdosfs . >>>>=20 >>>> /dev/mmcsd0s1 would likely also be a stable >>>> microsd card msdosfs reference for your context. >>>> Similarly for /dev/mmcsd0s2a for being a stable >>>> reference to your microsd card ufs file system. >>>>=20 >>>> For now such notations may be better for your >>>> /etc/fstab file(s). >>>>=20 >>> That seems to be key. Using /dev/da... notation in /etc/fstab >>> on the USB device allows a successful boot. One very important >>> detail that I forgot was adding kern.cam.boot_delay=3D"20000" >>> to /boot/loader.conf. Otherwise boots are successful only if >>> something else (like me) delays the boot process. The erratic >>> results observed likely owe much to that oversight. >>=20 >> As I remember there is a RPi* way of increasing its >> 2 second wait for a device on a port to indicate it >> is present. If you end up with problems despite the >> FreeBSD stage materials indicating delays, you might >> have to have the RPi3 increase its delay. The RPi3 >> activity for this is before FreeBSD is involved, >> possibly even before u-boot is involved. >>=20 > There is, by adding an empty file named timeout to the > msdos partition holding bootcode.bin, but it only gives > six seconds. I think my hard drive needs about ten.=20 > That's why the first run of usb scan fails to see the > hard disk. Running usb reset at the u-boot prompt finds > the disk. >=20 >>>>>> Are there any other boot-contributing files that >>>>>> would be appropriate to show together in one >>>>>> message for review, including any on the >>>>>> msdos file system? >>>>>>=20 >>>>>=20 >>>>> I don't know how to answer that question. Since the machine >>>>> has gotten to loader and it's looking at the USB drive it's >>>>> very tempting to think the msdos part is done. I'll admit >>>>> that names like /dev/ufs/rootfs seem rather obscure. >>>>> Are they required for RAID? >>>>>=20 >>>>=20 >>>> An example: does either media have in its msdos file >>>> system: /EFI/FreeBSD/loader.env ? If yes, are the >>>> settings present in loader.env that contribute to what >>>> boots? >>>>=20 >>> Apparently not by default; nothing in microSD, but I added=20 >>> a /boot/loader.env to the USB device, holding rootdev=3Ddisk1p2: >>> It didn't seem to affect booting behavior, at least not on >>> the first try.=20 >>=20 >> There is no reason to have a /boot/loader.env as far >> as I know. It would be ignored: that is the wrong >> place for the file. >>=20 > Indeed, my mistake. And, in hindsight, probably unhelpful. >=20 >> EFI/FreeBSD/loader.env in the msdos file system is >> a way to supply input to EFI/BOOT/bootaa64.efi (the >> FreeBSD loader) before the loader tries to find the >> FreeBSD /boot it is going to use --and before you >> can type to the loader. That is why Jonathan was >> indicating to create the rootdev assignment there. >> (That rootdev is then used to find the /boot to use, >> as I understand things.) >>=20 >>>> (That is something from Jonathan's exchanges with >>>> you.) >>>>=20 >>>>=20 >>>> A separate question is: Other than bootcode.bin , >>>> are the contents in the two msdos files systems >>>> identical? >>>>=20 >>>>=20 >>> Clearly not. Those on the the USB device are about six months >>> newer and different in size. >>=20 >> You may want to make your msdosfs content matching and >> keep them matching as they progress, at least for >> anything that you do not have a specific reason to make >> a distinction for. (Avoids odd variations in behavior.) >>=20 >>> For now it looks as if the immediate problem is solved. Both >>> microSD and USB storage boot and run.=20 >>=20 >> Does that wording mean that you got USB-only to >> work too? >>=20 > Yes, bootstrapping _through_ microSD.=20 Now that using both the microsd card and USB drive as a pair to boot has been shown to work, have you made the USB drive match what is used on from the microsd card (such as the msdos file system) and checked what the USB drive does without involving the microsd card? (Although, I wonder if the "10 sec" issue ends up involved, possibly blocking this way of working.) >> Now that you can boot, you should be able to show >> the ufs labels (and more) with: >>=20 >> tunefs -p /dev/mmcsd0s2a >> tunefs -p /dev/da0s2a >>=20 >> The outputs will each have a line starting with: >>=20 >> tunefs: volume label: (-L) >>=20 >> and ending with the label (or empty for no label). >>=20 >> If the two volume labels match, you should >> change at least one such that they are >> distinct. After that future boots should >> end up with two separate /dev/ufs/* names >> that you can use to pick the ufs file system >> with. The -L is the command line option >> involved in specifying a volume/file-system >> label. >>=20 > Both microSD and USB root filesystems have the same > label. That seems to be the crux of the problem. > For now hard-coding device file names in /etc/fstab > is probably the most idiot-proof method.=20 For UFS tunefs -l can be used to change the label. It might have to be when the file system chanberd is not mounted/live. The msdosfs labeling is messier to deal with. >> Unfortunately, I'm not aware of anything >> directly in FreeBSD analogous to "tunefs -p" >> or to "tunefs -L NAME . . .". for the msdos >> file systems. There seems to be only >> newfs_msdos --which would wipe out the existing >> msdosfs content by creating a new file system. >> But it does have a -L option for indicating the >> volume/file-system label to put in place. >>=20 >> May be after doing such the content of the >> other msdos file system couple be copied into >> the new/empty file system. (Not via dd. dd'ing >> the slice would replace the label too, not just >> the content placed in the file system.) >=20 > Once, a long time ago, FreeBSD had a boot manager > that lived, I think, in the msdos partition of each > bootable hard disk. On startup, it presented a menu, > something like > 1 msdos > 2 freebsd > 3 next disk in the boot order > defaulting to whatever was last selected. I think > that recollection gave me false hope that dual-booting > a Pi would be much simpler than it turned out. Ahh. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5FA69E16-72DE-4175-A0FB-BEFA1A865633>