Date: Sun, 7 Apr 2019 18:31:46 +0100 From: Jonathan Rowley <jonathans.email@icloud.com> To: freebsd-virtualization@freebsd.org Subject: unsubscribe Message-ID: <C268F727-A2E2-4503-B35E-030920FF417C@icloud.com> In-Reply-To: <mailman.1.1554638400.16553.freebsd-virtualization@freebsd.org> References: <mailman.1.1554638400.16553.freebsd-virtualization@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 7 Apr 2019, at 13:00, freebsd-virtualization-request@freebsd.org = wrote: >=20 > Send freebsd-virtualization mailing list submissions to > freebsd-virtualization@freebsd.org >=20 > To subscribe or unsubscribe via the World Wide Web, visit > = https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > or, via email, send a message with subject or body 'help' to > freebsd-virtualization-request@freebsd.org >=20 > You can reach the person managing the list at > freebsd-virtualization-owner@freebsd.org >=20 > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-virtualization digest..." >=20 >=20 > Today's Topics: >=20 > 1. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 2. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 3. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes) > 4. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes) > 5. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 6. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) >=20 >=20 > ---------------------------------------------------------------------- >=20 > Message: 1 > Date: Sun, 7 Apr 2019 09:37:43 +0700 > From: Victor Sudakov <vas@mpeks.tomsk.su> > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407023743.GB99339@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>=20 >>>> You can usually use the host by doing mdconfig -f = <path+to+diskimage> >>>=20 >>> Unfortunately mdconfig does not work with zvols: >>>=20 >>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>=20 >> If its a zvol cant you just do >> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>=20 >> and=20 >> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >=20 > No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >=20 > This is the default for a reason: it's no good exposing scores of = always > coming and going guest geoms to the host system. I think you can even > get a conflict of labels or something like that one day. >=20 >>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>> prompt and with a few commands: >>>>=20 >>>> Yes, the timeout is very long, and I do not know that we >>>> document anyplace that if you wait long enough at a failed >>>> boot you do get a EFI shell prompt eventually. >>>=20 >>> Can I press some key to escape to the EFI shell? >> Not that I am aware of. >=20 > It's a major problem! There must be a well-known way to break the boot > sequence any time and enter the EFI shell. >=20 >>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), = for >>>>> example? >>>>=20 >>>> I suspect that what ever guest you installed installed something >>>> else someplace, either within the eft partition, or possibly in >>>> the MBR? >>>=20 >>> Do you mean to say, the guest installing something else someplace = can >>> influence the boot sequence of bhyve efi? >>=20 >> The guest created all of the bits on that zvol, >> it can influence many things. There is probably a tiny initial >> stub that efi loads that has this bath to grubx64.efi codded in >> it and that is what is causing this issue. >=20 > It is very important to find and debug it because Oracle VirtualBox in > UEFI mode installs and runs this guest just fine. So it must be some > issue in bhyve itself. >=20 > Here is the complete archive of everything the guest created in the = EFI > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > can you find those confusing bits? >=20 > The standard procedure should be as follows: >=20 > Automated detection relies on standardized file paths to the OS > loader, with the path varying depending on the computer architecture. > The format of the file path is defined as > <EFI_SYSTEM_PARTITION>/EFI/BOOT/BOOT<MACHINE_TYPE_SHORT_NAME>.EFI; for > example, the file path to the OS loader on an x86-64 system is > /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture.=20= >=20 > Nothing about grub*.efi. But only bhyve is confused, VirtualBox is = not. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/201= 90407/aca49b08/attachment-0001.sig> >=20 > ------------------------------ >=20 > Message: 2 > Date: Sun, 7 Apr 2019 10:12:37 +0700 > From: Victor Sudakov <vas@mpeks.tomsk.su> > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407031237.GA7489@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Victor Sudakov wrote: >>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>> example? >>>>>=20 >>>>> I suspect that what ever guest you installed installed something >>>>> else someplace, either within the eft partition, or possibly in >>>>> the MBR? >>>>=20 >>>> Do you mean to say, the guest installing something else someplace = can >>>> influence the boot sequence of bhyve efi? >>>=20 >>> The guest created all of the bits on that zvol, >>> it can influence many things. There is probably a tiny initial >>> stub that efi loads that has this bath to grubx64.efi codded in >>> it and that is what is causing this issue. >>=20 >> It is very important to find and debug it because Oracle VirtualBox = in >> UEFI mode installs and runs this guest just fine. So it must be some >> issue in bhyve itself. >>=20 >> Here is the complete archive of everything the guest created in the = EFI >> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >> can you find those confusing bits? >=20 > I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but > BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some > kind of chain loader. >=20 > Watch the interactive session below. It does not however mean that = there is > nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and = runs this > guest just fine. >=20 > FS0:\> cd EFI > FS0:\EFI\> ls > Directory of: FS0:\EFI\ > 04/04/2019 15:53 <DIR> 2,048 . > 04/04/2019 15:53 <DIR> 0 .. > 04/04/2019 16:26 <DIR> 2,048 centos > 04/06/2019 04:19 <DIR> 2,048 BOOT > 0 File(s) 0 bytes > 4 Dir(s) > FS0:\EFI\> cd BOOT > FS0:\EFI\BOOT\> ls > Directory of: FS0:\EFI\BOOT\ > 04/04/2019 16:18 <DIR> 2,048 . > 04/04/2019 16:18 <DIR> 2,048 .. > 08/31/2017 21:30 1,296,176 BOOTX64.EFI > 08/31/2017 21:30 79,048 fbx64.efi > 2 File(s) 1,375,224 bytes > 2 Dir(s) > FS0:\EFI\BOOT\> BOOTX64.EFI > Failed to set MokListRT: Invalid Parameter > Failed to open \EFI\BOOT\grubx64.efi - Not Found > Failed to load image \EFI\BOOT\grubx64.efi: Not Found > start_image() returned Not Found > FS0:\EFI\BOOT\>=20 >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/201= 90407/92ff99aa/attachment-0001.sig> >=20 > ------------------------------ >=20 > Message: 3 > Date: Sat, 6 Apr 2019 21:19:37 -0700 (PDT) > From: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net> > To: Victor Sudakov <vas@mpeks.tomsk.su> > Cc: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <201904070419.x374JbIp048373@gndrsh.dnsmgr.net> > Content-Type: text/plain; charset=3DUS-ASCII >=20 >> Rodney W. Grimes wrote: >>>>>=20 >>>>> You can usually use the host by doing mdconfig -f = <path+to+diskimage> >>>>=20 >>>> Unfortunately mdconfig does not work with zvols: >>>>=20 >>>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>>=20 >>> If its a zvol cant you just do >>> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>>=20 >>> and=20 >>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >>=20 >> No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >>=20 >> This is the default for a reason: it's no good exposing scores of = always >> coming and going guest geoms to the host system. I think you can even >> get a conflict of labels or something like that one day. >=20 > So it may take a few more commands but it should be > possible to do this from the host side using host > side tools without having to boot a guest to make > these corrections. >=20 >>>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>>> prompt and with a few commands: >>>>>=20 >>>>> Yes, the timeout is very long, and I do not know that we >>>>> document anyplace that if you wait long enough at a failed >>>>> boot you do get a EFI shell prompt eventually. >>>>=20 >>>> Can I press some key to escape to the EFI shell? >>> Not that I am aware of. >>=20 >> It's a major problem! There must be a well-known way to break the = boot >> sequence any time and enter the EFI shell. >=20 > Agreed, hopefully those working on edk2 take note and either > chime in with what that way is, or create a bug and track > so that someone may fix this issue. >=20 >>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>> example? >>>>>=20 >>>>> I suspect that what ever guest you installed installed something >>>>> else someplace, either within the eft partition, or possibly in >>>>> the MBR? >>>>=20 >>>> Do you mean to say, the guest installing something else someplace = can >>>> influence the boot sequence of bhyve efi? >>>=20 >>> The guest created all of the bits on that zvol, >>> it can influence many things. There is probably a tiny initial >>> stub that efi loads that has this bath to grubx64.efi codded in >>> it and that is what is causing this issue. >>=20 >> It is very important to find and debug it because Oracle VirtualBox = in >> UEFI mode installs and runs this guest just fine. So it must be some >> issue in bhyve itself. >=20 > As I stated earlier bhyve is missing percistant efi variables, > and that is most likely the reason that VirtualBox just works > and bhyve does not. >=20 >> Here is the complete archive of everything the guest created in the = EFI >> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >> can you find those confusing bits? >=20 > Probably you well find in your VirtualBox directory a > file that is used to store efivars, that is where the > difference occurs. >=20 >> The standard procedure should be as follows: >>=20 >> Automated detection relies on standardized file paths to the OS >> loader, with the path varying depending on the computer architecture. >> The format of the file path is defined as >> <EFI_SYSTEM_PARTITION>/EFI/BOOT/BOOT<MACHINE_TYPE_SHORT_NAME>.EFI; = for >> example, the file path to the OS loader on an x86-64 system is >> /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 = architecture.=20 >>=20 >> Nothing about grub*.efi. But only bhyve is confused, VirtualBox is = not. >=20 > bhyve is not really confused, it is simply missing a feature > that this guest depends on for its boot procedures. This is > a well known miss-feature, but I do not know of anyone actively > working on fixing it. >=20 >> Victor Sudakov, VAS4-RIPE, VAS47-RIPN >> 2:5005/49@fidonet http://vas.tomsk.ru/ > --=20 > Rod Grimes = rgrimes@freebsd.org >=20 >=20 > ------------------------------ >=20 > Message: 4 > Date: Sat, 6 Apr 2019 21:26:06 -0700 (PDT) > From: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net> > To: Victor Sudakov <vas@mpeks.tomsk.su> > Cc: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <201904070426.x374Q641048406@gndrsh.dnsmgr.net> > Content-Type: text/plain; charset=3DUS-ASCII >=20 >> Victor Sudakov wrote: >>>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>>> example? >>>>>>=20 >>>>>> I suspect that what ever guest you installed installed something >>>>>> else someplace, either within the eft partition, or possibly in >>>>>> the MBR? >>>>>=20 >>>>> Do you mean to say, the guest installing something else someplace = can >>>>> influence the boot sequence of bhyve efi? >>>>=20 >>>> The guest created all of the bits on that zvol, >>>> it can influence many things. There is probably a tiny initial >>>> stub that efi loads that has this bath to grubx64.efi codded in >>>> it and that is what is causing this issue. >>>=20 >>> It is very important to find and debug it because Oracle VirtualBox = in >>> UEFI mode installs and runs this guest just fine. So it must be some >>> issue in bhyve itself. >>>=20 >>> Here is the complete archive of everything the guest created in the = EFI >>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >>> can you find those confusing bits? >>=20 >> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but >> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some >> kind of chain loader. >=20 > And it brobably tries to read a efivariable, and if that variable > is not set it defaults to grubx64.efi. This bootx64.efi is something > the guest installed into the EFI partition, hence my assertion that > the issue is with something the guest installed is some what valid. >=20 > There are some 3rd party EFI boot managers that might help > resolve this problem, or simply use the work around that > I provided earlier until we can get efivars working in > bhyve. >=20 >=20 >> Watch the interactive session below. It does not however mean that = there is >> nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and = runs this >> guest just fine. >>=20 >> FS0:\> cd EFI >> FS0:\EFI\> ls >> Directory of: FS0:\EFI\ >> 04/04/2019 15:53 <DIR> 2,048 . >> 04/04/2019 15:53 <DIR> 0 .. >> 04/04/2019 16:26 <DIR> 2,048 centos >> 04/06/2019 04:19 <DIR> 2,048 BOOT >> 0 File(s) 0 bytes >> 4 Dir(s) >> FS0:\EFI\> cd BOOT >> FS0:\EFI\BOOT\> ls >> Directory of: FS0:\EFI\BOOT\ >> 04/04/2019 16:18 <DIR> 2,048 . >> 04/04/2019 16:18 <DIR> 2,048 .. >> 08/31/2017 21:30 1,296,176 BOOTX64.EFI >> 08/31/2017 21:30 79,048 fbx64.efi >> 2 File(s) 1,375,224 bytes >> 2 Dir(s) >> FS0:\EFI\BOOT\> BOOTX64.EFI >> Failed to set MokListRT: Invalid Parameter >> Failed to open \EFI\BOOT\grubx64.efi - Not Found >> Failed to load image \EFI\BOOT\grubx64.efi: Not Found >> start_image() returned Not Found >> FS0:\EFI\BOOT\>=20 >>=20 >> --=20 >> Victor Sudakov, VAS4-RIPE, VAS47-RIPN >> 2:5005/49@fidonet http://vas.tomsk.ru/ >=20 > --=20 > Rod Grimes = rgrimes@freebsd.org >=20 >=20 > ------------------------------ >=20 > Message: 5 > Date: Sun, 7 Apr 2019 15:02:35 +0700 > From: Victor Sudakov <vas@mpeks.tomsk.su> > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407080235.GA40361@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>>>=20 >>>>>> You can usually use the host by doing mdconfig -f = <path+to+diskimage> >>>>>=20 >>>>> Unfortunately mdconfig does not work with zvols: >>>>>=20 >>>>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>>>=20 >>>> If its a zvol cant you just do >>>> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>>>=20 >>>> and=20 >>>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >>>=20 >>> No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >>>=20 >>> This is the default for a reason: it's no good exposing scores of = always >>> coming and going guest geoms to the host system. I think you can = even >>> get a conflict of labels or something like that one day. >>=20 >> So it may take a few more commands but it should be >> possible to do this from the host side using host >> side tools without having to boot a guest to make >> these corrections. >=20 > I'm not aware of such commands. If anyone knows them please share with = us. >=20 > Moreover, I already asked a similar question in February under the > topic "mounting/exporting/importing a zfs volume" and nobody gave a > recipe. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/201= 90407/17de57bb/attachment-0001.sig> >=20 > ------------------------------ >=20 > Message: 6 > Date: Sun, 7 Apr 2019 15:14:45 +0700 > From: Victor Sudakov <vas@mpeks.tomsk.su> > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407081445.GB40361@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>>>> "efi" but the rest is a mystery for me. Why is it trying to = find >>>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>>>> example? >>>>>>>=20 >>>>>>> I suspect that what ever guest you installed installed something >>>>>>> else someplace, either within the eft partition, or possibly in >>>>>>> the MBR? >>>>>>=20 >>>>>> Do you mean to say, the guest installing something else someplace = can >>>>>> influence the boot sequence of bhyve efi? >>>>>=20 >>>>> The guest created all of the bits on that zvol, >>>>> it can influence many things. There is probably a tiny initial >>>>> stub that efi loads that has this bath to grubx64.efi codded in >>>>> it and that is what is causing this issue. >>>>=20 >>>> It is very important to find and debug it because Oracle VirtualBox = in >>>> UEFI mode installs and runs this guest just fine. So it must be = some >>>> issue in bhyve itself. >>>>=20 >>>> Here is the complete archive of everything the guest created in the = EFI >>>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >>>> can you find those confusing bits? >>>=20 >>> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but >>> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some >>> kind of chain loader. >>=20 >> And it brobably tries to read a efivariable, and if that variable >> is not set it defaults to grubx64.efi. This bootx64.efi is something >> the guest installed into the EFI partition, hence my assertion that >> the issue is with something the guest installed is some what valid. >=20 > Do you think the guest OS installer set some efi variable during the > installation process, which bhyve did not save? That would explain a > lot. >=20 >>>>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>>>> prompt and with a few commands: >>>>>>=20 >>>>>> Yes, the timeout is very long, and I do not know that we >>>>>> document anyplace that if you wait long enough at a failed >>>>>> boot you do get a EFI shell prompt eventually. >>>>>=20 >>>>> Can I press some key to escape to the EFI shell? >>>> Not that I am aware of. >>>=20 >>> It's a major problem! There must be a well-known way to break the = boot >>> sequence any time and enter the EFI shell. >>=20 >> Agreed, hopefully those working on edk2 take note and either >> chime in with what that way is, or create a bug and track >> so that someone may fix this issue. >=20 > Would it be useful to create a PR in the FreeBSD bugtracker with a > feature request? >=20 >>>=20 >>> It is very important to find and debug it because Oracle VirtualBox = in >>> UEFI mode installs and runs this guest just fine. So it must be some >>> issue in bhyve itself. >>=20 >> As I stated earlier bhyve is missing percistant efi variables, >> and that is most likely the reason that VirtualBox just works >> and bhyve does not. >>=20 >> Probably you well find in your VirtualBox directory a >> file that is used to store efivars, that is where the >=20 > I'll look into the VirtualBox directory tomorrow and report here. I = was > under the impression that efivars are stored in a configuration file = in > the EFI partition but I was probably wrong, they are kept in NVRAM > somewhere, like BIOS settings, and not on a disk. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/201= 90407/c0afc2a8/attachment-0001.sig> >=20 > ------------------------------ >=20 > Subject: Digest Footer >=20 > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to = "freebsd-virtualization-unsubscribe@freebsd.org" >=20 >=20 > ------------------------------ >=20 > End of freebsd-virtualization Digest, Vol 436, Issue 7 > ******************************************************
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C268F727-A2E2-4503-B35E-030920FF417C>