Date: Wed, 25 Jul 2018 11:46:07 +0300 From: Toomas Soome <tsoome@me.com> To: "O. Hartmann" <ohartmann@walstatt.org> Cc: freebsd-current <freebsd-current@freebsd.org>, Allan Jude <allanjude@freebsd.org> Subject: Re: [UEFI] Boot issues on some UEFI implementations Message-ID: <2FCF2BCC-05E5-466A-B1AE-90300990ED1A@me.com> In-Reply-To: <20180725095926.7d52a15a@freyja.zeit4.iv.bundesimmobilien.de> References: <20180713130001.219a0a84@freyja.zeit4.iv.bundesimmobilien.de> <68505F98-E840-4148-9E48-BDB350F7431A@me.com> <20180713164447.42430301@thor.intern.walstatt.dynvpn.de> <680FBB42-75BF-427F-AA3B-6D864E83ED1F@me.com> <20180723092735.12a5d2a8@freyja.zeit4.iv.bundesimmobilien.de> <80523BE6-C035-482D-B134-23812183DE83@me.com> <20180724071514.6cb9d111@freyja.zeit4.iv.bundesimmobilien.de> <16439773-3E9A-40FF-99A1-32E97CCEE2C3@me.com> <20180725095926.7d52a15a@freyja.zeit4.iv.bundesimmobilien.de>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 25 Jul 2018, at 10:59, O. Hartmann <ohartmann@walstatt.org> wrote:
>=20
> On Tue, 24 Jul 2018 08:53:36 +0300
> Toomas Soome <tsoome@me.com> wrote:
>=20
>=20
> Hello Toomas Soome,
>=20
> I CC Allan Jude since I discovered something weird today regarding =
the UEFI
> boot capabilities of USB flash devices and SSDs. See below.
>=20
>>> On 24 Jul 2018, at 08:16, O. Hartmann <ohartmann@walstatt.org> =
wrote:
>>>=20
>>> On Mon, 23 Jul 2018 10:56:04 +0300
>>> Toomas Soome <tsoome@me.com> wrote:
>>>=20
>>>>> On 23 Jul 2018, at 10:27, O. Hartmann <ohartmann@walstatt.org> =
wrote:
>>>>>=20
>>>>> On Fri, 13 Jul 2018 18:44:23 +0300
>>>>> Toomas Soome <tsoome@me.com <mailto:tsoome@me.com>> wrote:
>>>>>=20
>>>>>>> On 13 Jul 2018, at 17:44, O. Hartmann <o.hartmann@walstatt.org
>>>>>>> <mailto:o.hartmann@walstatt.org>> wrote:
>>>>>>>=20
>>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>>> Hash: SHA512
>>>>>>>=20
>>>>>>> Am Fri, 13 Jul 2018 14:26:51 +0300
>>>>>>> Toomas Soome <tsoome@me.com <mailto:tsoome@me.com> =
<mailto:tsoome@me.com
>>>>>>> <mailto:tsoome@me.com>>> schrieb: =20
>>>>>>>>> On 13 Jul 2018, at 14:00, O. Hartmann <ohartmann@walstatt.org> =
wrote:
>>>>>>>>>=20
>>>>>>>>> The problem is some kind of weird. I face UEFI boot problems =
on GPT
>>>>>>>>> drives where the first partition begins at block 40 of the =
hdd/ssd.
>>>>>>>>>=20
>>>>>>>>> I have two host in private use based on an
>>>>>>>>> outdated ASRock Z77-Pro4-M and Z77-Pro4 mainboard (IvyBridge, =
Socket
>>>>>>>>> LGA1155). Both boards are equipted with the lates official =
available
>>>>>>>>> AMI firmware revision, dating to 2013. This is for the =
Z77-Pro4-M
>>>>>>>>> revision 2.0 (2013/7/23) and for the Z77 Pro4 revision 1.8
>>>>>>>>> (2013/7/17). For both boards a BETA revision for the =
Spectre/Meltdown
>>>>>>>>> mitigation is available, but I didn't test that. But please =
read.
>>>>>>>>>=20
>>>>>>>>> The third box I realised this problem is a brand new Fujitsu =
Esprimo
>>>>>>>>> Q956, also AMI firmware, at V5.0.0.11 R 1.26.0 for 3413-A1x, =
date
>>>>>>>>> 05/25/2018 (or 20180525).
>>>>>>>>>=20
>>>>>>>>> Installing on any kind of HDD or SSD manually or via =
bsdinstall the OS
>>>>>>>>> using UEFI-only boot method on a GPT partitioned device fails. =
The
>>>>>>>>> ASRock boards jump immediately into the firmware, the Fujitsu =
offers
>>>>>>>>> some kind of CPU/Memory/HDD test facility.
>>>>>>>>>=20
>>>>>>>>> If on both type of vendor/boards CSM is disabled and UEFI boot =
only is
>>>>>>>>> implied, the MBR partitioned FreeBSD installation USB flash =
device
>>>>>>>>> does boot in UEFI! I guess I can assume this when the well =
known
>>>>>>>>> clumsy 80x25 char console suddenly gets bright and shiny with =
a much
>>>>>>>>> higher resoltion as long the GPU supports EFI GOP. Looking =
with gpart
>>>>>>>>> at the USB flash drives reveals that the EFI partition starts =
at
>>>>>>>>> block 1 of the device and the device has a MBR layout. I =
haven't
>>>>>>>>> found a way to force the GPT scheme, when initialised via =
gpart, to
>>>>>>>>> let the partitions start at block 1. This might be a naiv =
thinking,
>>>>>>>>> so please be patient with me.
>>>>>>>>>=20
>>>>>>>>> I do not know whether this is a well-known issue. On the =
ASRock
>>>>>>>>> boards, I tried years ago some LinuxRed Hat and Suse with UEFI =
and
>>>>>>>>> that worked - FreeBSD not. I gave up on that that time. Now, =
having
>>>>>>>>> the very same issues with a new Fujitsu system, leaves me with =
the
>>>>>>>>> impression that FreeBSD's UEFI implementation might have =
problems I'm
>>>>>>>>> not aware of.
>>>>>>>>>=20
>>>>>>>>> Can someone shed some light onto this?=20
>>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> The first thing to check is if the secure boot is disabled. We =
do not
>>>>>>>> support secure boot at all at this time. =20
>>>>>>>=20
>>>>>>> Secure boot is in every scenario disabled!
>>>>>>>=20
>>>>>>>>=20
>>>>>>>> If you have efi or bios version running - you can check from =
either
>>>>>>>> console variable value (it can have efi or vidconsole or =
comconsole) or
>>>>>>>> better yet, see if efi-version is set (show efi-version) - if
>>>>>>>> efi-version is not set, it is BIOS loader running. Another =
indirect
>>>>>>>> way is to see lsdev -v, with device paths present, it is uefi:) =
=20
>>>>>>>=20
>>>>>>> What are you talking about?
>>>>>>> What is the point of entry - running system, loader?
>>>>>>>=20
>>>>>>> sysct machdep.bootmethod: BIOS
>>>>>>>=20
>>>>>>> This makes me quite sure that the system has booted via BIOS - =
as I'm
>>>>>>> sure since I've checked that many times. UEFI doesn't work on =
those
>>>>>>> systems with FreeBSD. I'm not sure antmore, but I tried also =
Windows 7
>>>>>>> on those mainboards booting via UEFI - and I might recall that =
they
>>>>>>> failed also. I also recall that there were issues with earlier =
UEFI
>>>>>>> versions regarding booting only Windows 8/8.1 - and nothing =
else, but
>>>>>>> the fact that Linux worked confuses me a bit.
>>>>>>>=20
>>>>>>> If this ASRock crap (never ever again this brand!) doesn't work =
at all -
>>>>>>> who cares, I intend to purchase new server grade hardware. But =
the more
>>>>>>> puzzling issue is with the Fujitsu, which I consider serious and =
from
>>>>>>> the behaviour the Fujitsu failure looks exactly like the ASRock =
-
>>>>>>> Windows 7 works, RedHat 7.5 works (I assume I can trust the =
Firmware
>>>>>>> settings when I disable CSM support, that the Firmware will only
>>>>>>> EFI/UEFI capable loader? Or is there a ghosty override somwhere =
to be
>>>>>>> expected?). Also on ASRock disabling CSM should ensure not =
booting a
>>>>>>> dual-bootstrap-capable system. This said, on the recent Fujitsu, =
it
>>>>>>> seems to boil down to a FreeBSD UEFI-firmware interaction =
problem,
>>>>>>> while the ASRock is still under suspicion to be broken by =
design. =20
>>>>>>>>=20
>>>>>>>> GPT partitions can never start from disk absolute sector 1; =
this is
>>>>>>>> because at sector 0 there is MBR (for compatibility), sector 1 =
is GPT
>>>>>>>> table and then sectors 2-33 have GPT partition table entries, =
so the
>>>>>>>> first possible data sector is 34 (absolute 34). Thats assuming =
512B
>>>>>>>> sectors. For details see UEFI 2.7 Chapter 5.3.1 page 131. =20=
>>>>>>>=20
>>>>>>> Thanks for the explanation. That implies the installer did =
right, gpart
>>>>>>> did also right and therefore there must be an issue with the =
stuff
>>>>>>> located within the EFI partition? =20
>>>>>>=20
>>>>>> Ok, so, it is not about UEFI bootcode but BIOS, and if we reach =
BIOS
>>>>>> loader at all or not - that is, if the BIOS bootstrap is actually =
caring
>>>>>> to read the MBR code and start it, since once the MBR code is =
started,
>>>>>> it is all about our code. =20
>>>>>=20
>>>>> I'm getting confused a bit here. Do you mean by "BIOS" the CSM? or =
do you
>>>>> mean that specific portion of the UEFI firmware, which looks for =
the
>>>>> proper UEFI partition?
>>>>>=20
>>>>=20
>>>> BIOS as either native or CSM. Note that from boot code point of =
view the
>>>> CSM boot *is* BIOS boot, we have no access to UEFI features.
>>>>=20
>>>>>=20
>>>>> The boxes in question, most notably the more recent Fujitsu =
Esprimo Q956,
>>>>> refuse booting UEFI, even if properly setup (in terms of what =
FreeBSD
>>>>> provides on recent CURRENT) is applied and CSM is switched off in =
the
>>>>> firmware. Again: GPT partition scheme.
>>>>>=20
>>>>> The system boots properly if a second partition of type =
"freebsd-boot" is
>>>>> applied and bootcode is properly applied via "gpart bootcode -b =
/boot/pmbr
>>>>> -p /boot/gptboot -i 2 ada0" (ada0 is the device). =20
>>>>>>=20
>>>>>> btw, you can try to validate the installed boot blocks by using =
recent
>>>>>> enough loader (usb or iso) and then you can use from OK prompt: =
=20
>>>>>=20
>>>>> lsdev provides me with the follwoing informations (CSM enabled):
>>>>>=20
>>>>> OK lsdev
>>>>> disk devices:
>>>>> disk0: BIOS DRIVE C ...
>>>>>=20
>>>>> disk0p1: EFI
>>>>> disk0p2: FreeBSD BOOT
>>>>> disk0p3: FreeBSD SWAP
>>>>> disk0p4: FreeBSD ZFS
>>>>> zfs devices:
>>>>> zfs:zroot
>>>>>=20
>>>>> OK chain disk0
>>>>> open failed (so for disk0p{1-4}.
>>>>>=20
>>>>> OK chain zroot
>>>>> failed to read disk (just for completeness) =20
>>>>=20
>>>>=20
>>>> chain command does use only device name (such as disk0: or disk0p2: =
), but
>>>> not zfs pool as device. I just found I haven=E2=80=99t ported the =
code to read the
>>>> file. =20
>>>=20
>>> ??
>>>=20
>>>>=20
>>>> the point for chain command test is to see if the normal read and =
execute
>>>> would work, so in your case please try:
>>>>=20
>>>> chain disk0: =20
>>>=20
>>> As stated above, I did so, and the result is also mentioned above, I =
always
>>> get "open failed".
>>> This is the same for=20
>>>=20
>>> chain disk0
>>> chain disk0p1
>>> chain disk0p2
>>> chain disk0p3
>>> chain disk0p4
>>>=20
>>> as already said. CSM is enabled in this case. =20
>>=20
>> sigh=E2=80=A6 chain command does take device as argument, device must =
always end with
>> colon=E2=80=A6. in this case, the devil is in details:) as I wrote =
above, the command
>> should be:
>>=20
>> chain disk0:
>>=20
>> The disk0p1: etc will only work when partition boot code was =
installed (which
>> you most likely do not have - the only possible candidate could be =
FreeBSD
>> ZFS partition).
>=20
> The command "chain disk0:" works as expected (CSM enabled, GPT =
partition
> scheme, but with PMBR bootblock installed and freebsd-boot partition =
conatining
> gptzfsboot installed.
>=20
>=20
>>=20
>>>=20
>>>>=20
>>>> to read pmbr (512B) and execute it. The expected outcome would be =
that pmbr
>>>> boot code would browse the GPT, read stage1 from disk0p2: and =
execute it;
>>>> stage1 would detect FreeBSD ZFS from disk0p4: and load and
>>>> execute /boot/loader. If that will happen, it means the boot code =
in our
>>>> stages is just fine, but the bios (CSM) does not load pmbr=E2=80=A6. =
if thats
>>>> true, it would mean that you either need to use UEFI boot or need =
to have
>>>> some hack to fool the BIOS or just not use GPT on that machine with =
CSM. =20
>>>=20
>>> To make it clear here: The only way to boot this box is using CSM =
(as it is
>>> the same with the ASRock boards mentioned earlier). But my intention =
is to
>>> disable CSM and use a GPT/UEFI environment only! And GPT/UEFI =
doesn't work
>>> with FreeBSD, neither with 12-CURRENT, nor 11.2-RELENG.
>>>=20
>>> It would be nice if this could be fixed. I'm more interested in the =
fix on
>>> the recent Fujitsu device than the outdated ASRock crap, but if the =
fix for
>>> the Fujitsu Firmware could fix older issues as a byproduct, I'd =
appreciate
>>> that.
>>>=20
>>> Kind regards,
>>>=20
>>=20
>> ok, somehow I have lost that part of the discussion. Well, you wrote =
that the
>> UEFI boot fails when the first partition starts from sector 40 - does =
it mean
>> you have boot when the partition will start from some other sector? I =
think,
>> there is something else going on.
>=20
> Well, I simply try to describe what I "see" to make things =
disambiguous. I'm
> not familiar with the deeper insights of disk layouts on a binary =
level. So,
> you explained to me the reason, why ESP (EGI partition) starts at =
block 40. I
> compared that to the FreeBSD USB flash image FreeBSD provides, but =
this is
> another story since the image uses MBR scheme as I figured out.
>=20
>=20
>>=20
>> What you can do is to see if that firmware will offer you EFI shell =
option,
>> from there you can try to start the bootx64.efi manually and see what =
error
>> you will get. However, the number 1 cause for failing to start the =
bootloader
>> in UEFI is secure boot - we do not support it and secure boot must be
>> switched off.=20
>>=20
>> However, they seem to claim "The Secure Boot option is available in =
the
>> UEFI/BIOS of most if not all ASRock boards. It is disabled by =
default.=E2=80=9D=20
>>=20
>> Still suggest to double check if thats really the case. Also, if the
>> bootx64.efi start will fail and no messages are appearing on screen, =
then
>> either there is something in firmware logs or you could get them from =
trying
>> to start bootx64.efi from UEFI shell.
>=20
> Since I'm with this problem since 2014 and try from time to time, be =
ausred
> that I tried every possible permutationof all reasonable options, even =
those
> nonsense, to get rid of that problem.
>=20
> I never had any problems with any other UEFI capable =
server/workstation
> firmware so far booting FreeBSD off in UEFI-native (GPT partition =
scheme, CSM
> disabled) so far - until now, when I ran into this Fujitsu ESPRIMO =
Q956 with
> the most recent firmware (as of lat week, week 29 of 2018) having the =
very same
> problems.=20
>=20
>=20
>=20
> I figured out something strange on the Fujitsu - and that is the same =
with the
> ASRock boards.
>=20
> We/I prepare some USB flash drives to boot a NanoBSD for a very small
> appliance, but nevertheless, the USB flash device is booted on Fujitsu =
servers
> with UEFI-only configurations. I assume at this point that disabling =
on the
> most recent Fujitsu firmwares on reasonable "new" hardware (not older =
than
> three years) will disable any(!) legacy BIOS capabilities. The same is =
assumed
> for the Fujitus ESPRIMO Q956. I can not speak for the ASRock A77 =
Pro4/m boards
> mentioned above/earlier, they are from 2012/2013 and "quite old".
>=20
> The NanoBSD image of ours doesn't have a "freebsd-boot" partition. The
> partition scheme of the flash device is GPT. The layout looks like =
this:
>=20
> gpart show -l da4
> =3D> 40 15425456 da4 GPT (7.4G)
> 40 2000 1 efiboot0 (1.0M)
> 2040 1453584 3 disk1a (710M)
> 1455624 4096 5 disk3 (2.0M)
> 1459720 13965776 - free - (6.7G)
>=20
> I created the flash with md, gpart and dd straightforward, efiboot0 is =
the ESP
> partition and its format/content is created via dd =
if=3D/boot/boot1.efifat
> of=3D/dev/da4p1 - I presume this is very simple.
>=20
> This USB flash device boots(!) successfully (UEFI!) on both the ASRock =
boards
> and the Esprimo Q956!
>=20
> But any SSD prepared the same way doesn't. Why?=20
>=20
> On the ASRock, I recall having fiddled around with HDD also for a =
while
> conatining Windows 7/SP1 and FreeBSD. Windows7 booted, FreeBSD - I =
can't
> remember.=20
>=20
> In the lack of proper hardware I'm unable to check whether =
USB-attached HDD or
> SSD will boot or HDD will boot (just in case the local SATA has =
problems
> booting UEFI and USB not).
>=20
> Kind regards,
>=20
> Oliver=20
>=20
Am. well. I think the suggestion to test out FAT32 is still good one to =
test. This is because it is known that some vendors do not support =
booting FAT12/FAT16 from HDD (the likely reason is that UEFI =
specification does not tell which FAT must be supported, and only hint =
about FAT12/FAT16 in context of removable devices).
There are other possible causes too, for example: =
https://ubuntuforums.org/showthread.php?t=3D2147295=20
Also about the ESP sizes: https://www.ctrl.blog/entry/esp-size-guide
"The UEFI System Partition should be at least 260 MiB (273 MB) to ensure =
its properly formatted with FAT32 so that you avoid UEFI implementation =
compatibility issues. (If you do have incompatible hardware that =
requires FAT16-formatting, then I suggest you move aside the files on =
the UEFI System Partition, convert the partition to FAT16, and copy the =
files back over to it.)=E2=80=9D
So, as you see, even just telling =E2=80=9Cuse FAT32=E2=80=9D is not =
universal medicine, but I suspect it is still more universal than using =
FAT12/FAT16:)
Just to be clear, there is *no* standard size rule for ESP, there are =
only suggestions from vendors=E2=80=A6=20
Yes, this all means that if the solution from default installer does not =
work, the manual work is needed to identify why the default is not =
working and the findings should be reported, so the installer (and =
possibly other parts of the system) could be adjusted. Since this all is =
vendor specific, it has to be handled case by case.
rgds,
toomas
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2FCF2BCC-05E5-466A-B1AE-90300990ED1A>