From owner-freebsd-virtualization@freebsd.org Sun Apr 7 02:37:46 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2AD75157599A for ; Sun, 7 Apr 2019 02:37:46 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5127F76F0E for ; Sun, 7 Apr 2019 02:37:44 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=iVhvxqccICpHrzBLIbqfIZ4G+z/PU5Zo9yH+D3/WO94=; b=pihV+ASEkpsBAlPrjaCBteVszm uM9pCpqmhT6KdmsagmcSPxALLXWfDjL2jB32oVJh21K56lgNYXO6qmIulR488RrIVEITCfawlHlzI /tskibwSKytAxky4Np6OKirjpSlgrAOiy9q4zfNCB63JQJc0d6OiETeXpFtYeBfJmJ8Y=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hCxgl-0001i1-Sz for freebsd-virtualization@freebsd.org; Sun, 07 Apr 2019 09:37:43 +0700 Date: Sun, 7 Apr 2019 09:37:43 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: running FreePBX SNG7 Official Distro Message-ID: <20190407023743.GB99339@admin.sibptus.ru> References: <20190406085458.GA89832@admin.sibptus.ru> <201904061002.x36A2BZE044704@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="l76fUT7nc3MelDdI" Content-Disposition: inline In-Reply-To: <201904061002.x36A2BZE044704@gndrsh.dnsmgr.net> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 02:37:46 -0000 --l76fUT7nc3MelDdI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Rodney W. Grimes wrote: > > >=20 > > > You can usually use the host by doing mdconfig -f > >=20 > > Unfortunately mdconfig does not work with zvols: > >=20 > > root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 > > mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >=20 > If its a zvol cant you just do > gpart show /dev/zvol/d02/vm/freepbx/disk0 >=20 > and=20 > mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 No I can't if the zvol is in the "volmode=3Ddev" mode which is the default.= =20 This is the default for a reason: it's no good exposing scores of always coming and going guest geoms to the host system. I think you can even get a conflict of labels or something like that one day. > > > > Moreover, I waited (for a long time!) for the EFI interactive shell > > > > prompt and with a few commands: > > >=20 > > > Yes, the timeout is very long, and I do not know that we > > > document anyplace that if you wait long enough at a failed > > > boot you do get a EFI shell prompt eventually. > >=20 > > Can I press some key to escape to the EFI shell? > Not that I am aware of. It's a major problem! There must be a well-known way to break the boot sequence any time and enter the EFI shell. > > > > I can guess that it looks for a FAT16 partition in the GPT with the= type > > > > "efi" but the rest is a mystery for me. Why is it trying to find > > > > "grubx64.efi" and not the default "boot64.efi" (which is present), = for > > > > example? > > >=20 > > > I suspect that what ever guest you installed installed something > > > else someplace, either within the eft partition, or possibly in > > > the MBR? > >=20 > > Do you mean to say, the guest installing something else someplace can > > influence the boot sequence of bhyve efi? >=20 > The guest created all of the bits on that zvol, > it can influence many things. There is probably a tiny initial > stub that efi loads that has this bath to grubx64.efi codded in > it and that is what is causing this issue. It is very important to find and debug it because Oracle VirtualBox in UEFI mode installs and runs this guest just fine. So it must be some issue in bhyve itself. Here is the complete archive of everything the guest created in the EFI partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz can you find those confusing bits? The standard procedure should be as follows: Automated detection relies on standardized file paths to the OS loader, with the path varying depending on the computer architecture. The format of the file path is defined as /EFI/BOOT/BOOT.EFI; for example, the file path to the OS loader on an x86-64 system is /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture.=20 Nothing about grub*.efi. But only bhyve is confused, VirtualBox is not. --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --l76fUT7nc3MelDdI Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcqWJ3AAoJEA2k8lmbXsY0g3kH/2E54ixKEYsGZ+6mN+kETYAV F1OV5q9e17lDM+S0mGXX4RqEDuEn5YfFF4UN8Y8KVjroWPe2Gzi/Q/EAXi0aD2Pl Yq9ehits9OdVdqWbFR1O0EP1GfQ6+m2aC+ZegsTBQbcNe1uE5OSjV26DAVCsfnYZ wWikSrndrfHRmO1ZAjeKmLx6r2KTxORk84lAIljvAKJcocCxRP1UHV6IihpNOAce aLsILy8lg83/kcTH7itt3pyNWsWOOOK/Isl1/C0I8sCwRZ4GyxirLfKYjepWTLiD uPgJ9Lkp7T42jr+fN8gho/UzRJv2GQdint3ZiQw9Kvc4uDuP4ecb+wgf5BCBqHE= =nAUV -----END PGP SIGNATURE----- --l76fUT7nc3MelDdI-- From owner-freebsd-virtualization@freebsd.org Sun Apr 7 03:12:39 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 554D515766D5 for ; Sun, 7 Apr 2019 03:12:39 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 59807802E3 for ; Sun, 7 Apr 2019 03:12:38 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=gOJsbNGauuHZhGi2V7U8YGM1QbkJWk6J4OOT4aB/mx4=; b=jJHh43Srx5Ueyf9n/brnGk8VJo XzUTVyaaCBkaAiMC4kC6mAibIE1AoCfxu1yaxSpSl7E1fDSka3eApl8mwJb1vTVcMoqBGvN1tCeI8 SitDNCnBLT/T9K3ezVQ0eHwFitr/IXC+Yg/mifDeW3FwxJeBy4inoniALST9dl0hmrBY=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hCyEX-0002jL-Dl for freebsd-virtualization@freebsd.org; Sun, 07 Apr 2019 10:12:37 +0700 Date: Sun, 7 Apr 2019 10:12:37 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: running FreePBX SNG7 Official Distro Message-ID: <20190407031237.GA7489@admin.sibptus.ru> References: <20190406085458.GA89832@admin.sibptus.ru> <201904061002.x36A2BZE044704@gndrsh.dnsmgr.net> <20190407023743.GB99339@admin.sibptus.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sm4nu43k4a2Rpi4c" Content-Disposition: inline In-Reply-To: <20190407023743.GB99339@admin.sibptus.ru> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 03:12:39 -0000 --sm4nu43k4a2Rpi4c Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Victor Sudakov wrote: > > > > > I can guess that it looks for a FAT16 partition in the GPT with t= he type > > > > > "efi" but the rest is a mystery for me. Why is it trying to find > > > > > "grubx64.efi" and not the default "boot64.efi" (which is present)= , for > > > > > example? > > > >=20 > > > > I suspect that what ever guest you installed installed something > > > > else someplace, either within the eft partition, or possibly in > > > > the MBR? > > >=20 > > > Do you mean to say, the guest installing something else someplace can > > > influence the boot sequence of bhyve efi? > >=20 > > The guest created all of the bits on that zvol, > > it can influence many things. There is probably a tiny initial > > stub that efi loads that has this bath to grubx64.efi codded in > > it and that is what is causing this issue. >=20 > It is very important to find and debug it because Oracle VirtualBox in > UEFI mode installs and runs this guest just fine. So it must be some > issue in bhyve itself. >=20 > Here is the complete archive of everything the guest created in the EFI > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > can you find those confusing bits? I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, but BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be some kind of chain loader. Watch the interactive session below. It does not however mean that there is nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and runs = this guest just fine. FS0:\> cd EFI FS0:\EFI\> ls Directory of: FS0:\EFI\ 04/04/2019 15:53 2,048 . 04/04/2019 15:53 0 .. 04/04/2019 16:26 2,048 centos 04/06/2019 04:19 2,048 BOOT 0 File(s) 0 bytes 4 Dir(s) FS0:\EFI\> cd BOOT FS0:\EFI\BOOT\> ls Directory of: FS0:\EFI\BOOT\ 04/04/2019 16:18 2,048 . 04/04/2019 16:18 2,048 .. 08/31/2017 21:30 1,296,176 BOOTX64.EFI 08/31/2017 21:30 79,048 fbx64.efi 2 File(s) 1,375,224 bytes 2 Dir(s) FS0:\EFI\BOOT\> BOOTX64.EFI Failed to set MokListRT: Invalid Parameter Failed to open \EFI\BOOT\grubx64.efi - Not Found Failed to load image \EFI\BOOT\grubx64.efi: Not Found start_image() returned Not Found FS0:\EFI\BOOT\>=20 --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --sm4nu43k4a2Rpi4c Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcqWqlAAoJEA2k8lmbXsY0IisIAJ7a8pxf4UaIB1LcNq6MSbu+ tx4N+cQPfb3f2b4njruMvKdTp4S7RYRpwHs2gU0A6EKjm0DIHUqIfoJyT7Z7AqY6 AXMWAkVTkO04wP0lYdWPrqMhOmvzKewfgFD4DO3w7lvMYnaLO9j5WosSsKixZSYB hhuMelWk5Mu69CYp+ALa0S7YmuvV2AP8TZwZQek4vMVkFHzznXT+9I04NqbdDqky pk0vm3kE77BtlNu5lVK8x8dcGxAx3eILKuGwoHeL1wgvWd3RugEDRfqJGWlqq+5f zLxzouxZ2jRDyvLOHc5AYo8SgTzTMcvM09trl8lBTRytWa+ean88WqpeLahDCCw= =hzVw -----END PGP SIGNATURE----- --sm4nu43k4a2Rpi4c-- From owner-freebsd-virtualization@freebsd.org Sun Apr 7 04:19:40 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F4A91577490 for ; Sun, 7 Apr 2019 04:19:40 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 908798168B for ; Sun, 7 Apr 2019 04:19:39 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x374JbdZ048374; Sat, 6 Apr 2019 21:19:37 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x374JbIp048373; Sat, 6 Apr 2019 21:19:37 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201904070419.x374JbIp048373@gndrsh.dnsmgr.net> Subject: Re: running FreePBX SNG7 Official Distro In-Reply-To: <20190407023743.GB99339@admin.sibptus.ru> To: Victor Sudakov Date: Sat, 6 Apr 2019 21:19:37 -0700 (PDT) CC: freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 908798168B X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 04:19:40 -0000 > Rodney W. Grimes wrote: > > > > > > > > You can usually use the host by doing mdconfig -f > > > > > > Unfortunately mdconfig does not work with zvols: > > > > > > root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0 > > > mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file > > > > If its a zvol cant you just do > > gpart show /dev/zvol/d02/vm/freepbx/disk0 > > > > and > > mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 > > No I can't if the zvol is in the "volmode=dev" mode which is the default. > > This is the default for a reason: it's no good exposing scores of always > coming and going guest geoms to the host system. I think you can even > get a conflict of labels or something like that one day. So it may take a few more commands but it should be possible to do this from the host side using host side tools without having to boot a guest to make these corrections. > > > > > Moreover, I waited (for a long time!) for the EFI interactive shell > > > > > prompt and with a few commands: > > > > > > > > Yes, the timeout is very long, and I do not know that we > > > > document anyplace that if you wait long enough at a failed > > > > boot you do get a EFI shell prompt eventually. > > > > > > Can I press some key to escape to the EFI shell? > > Not that I am aware of. > > It's a major problem! There must be a well-known way to break the boot > sequence any time and enter the EFI shell. Agreed, hopefully those working on edk2 take note and either chime in with what that way is, or create a bug and track so that someone may fix this issue. > > > > > I can guess that it looks for a FAT16 partition in the GPT with the type > > > > > "efi" but the rest is a mystery for me. Why is it trying to find > > > > > "grubx64.efi" and not the default "boot64.efi" (which is present), for > > > > > example? > > > > > > > > I suspect that what ever guest you installed installed something > > > > else someplace, either within the eft partition, or possibly in > > > > the MBR? > > > > > > Do you mean to say, the guest installing something else someplace can > > > influence the boot sequence of bhyve efi? > > > > The guest created all of the bits on that zvol, > > it can influence many things. There is probably a tiny initial > > stub that efi loads that has this bath to grubx64.efi codded in > > it and that is what is causing this issue. > > It is very important to find and debug it because Oracle VirtualBox in > UEFI mode installs and runs this guest just fine. So it must be some > issue in bhyve itself. As I stated earlier bhyve is missing percistant efi variables, and that is most likely the reason that VirtualBox just works and bhyve does not. > Here is the complete archive of everything the guest created in the EFI > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > can you find those confusing bits? Probably you well find in your VirtualBox directory a file that is used to store efivars, that is where the difference occurs. > The standard procedure should be as follows: > > Automated detection relies on standardized file paths to the OS > loader, with the path varying depending on the computer architecture. > The format of the file path is defined as > /EFI/BOOT/BOOT.EFI; for > example, the file path to the OS loader on an x86-64 system is > /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture. > > Nothing about grub*.efi. But only bhyve is confused, VirtualBox is not. bhyve is not really confused, it is simply missing a feature that this guest depends on for its boot procedures. This is a well known miss-feature, but I do not know of anyone actively working on fixing it. > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Sun Apr 7 04:26:11 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D443515776ED for ; Sun, 7 Apr 2019 04:26:10 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 44CB8819EA for ; Sun, 7 Apr 2019 04:26:10 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x374Q6sS048407; Sat, 6 Apr 2019 21:26:06 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x374Q641048406; Sat, 6 Apr 2019 21:26:06 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201904070426.x374Q641048406@gndrsh.dnsmgr.net> Subject: Re: running FreePBX SNG7 Official Distro In-Reply-To: <20190407031237.GA7489@admin.sibptus.ru> To: Victor Sudakov Date: Sat, 6 Apr 2019 21:26:06 -0700 (PDT) CC: freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 44CB8819EA X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.973,0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 04:26:11 -0000 > Victor Sudakov wrote: > > > > > > I can guess that it looks for a FAT16 partition in the GPT with the type > > > > > > "efi" but the rest is a mystery for me. Why is it trying to find > > > > > > "grubx64.efi" and not the default "boot64.efi" (which is present), for > > > > > > example? > > > > > > > > > > I suspect that what ever guest you installed installed something > > > > > else someplace, either within the eft partition, or possibly in > > > > > the MBR? > > > > > > > > Do you mean to say, the guest installing something else someplace can > > > > influence the boot sequence of bhyve efi? > > > > > > The guest created all of the bits on that zvol, > > > it can influence many things. There is probably a tiny initial > > > stub that efi loads that has this bath to grubx64.efi codded in > > > it and that is what is causing this issue. > > > > It is very important to find and debug it because Oracle VirtualBox in > > UEFI mode installs and runs this guest just fine. So it must be some > > issue in bhyve itself. > > > > Here is the complete archive of everything the guest created in the EFI > > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > > can you find those confusing bits? > > I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, but > BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be some > kind of chain loader. And it brobably tries to read a efivariable, and if that variable is not set it defaults to grubx64.efi. This bootx64.efi is something the guest installed into the EFI partition, hence my assertion that the issue is with something the guest installed is some what valid. There are some 3rd party EFI boot managers that might help resolve this problem, or simply use the work around that I provided earlier until we can get efivars working in bhyve. > Watch the interactive session below. It does not however mean that there is > nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and runs this > guest just fine. > > FS0:\> cd EFI > FS0:\EFI\> ls > Directory of: FS0:\EFI\ > 04/04/2019 15:53 2,048 . > 04/04/2019 15:53 0 .. > 04/04/2019 16:26 2,048 centos > 04/06/2019 04:19 2,048 BOOT > 0 File(s) 0 bytes > 4 Dir(s) > FS0:\EFI\> cd BOOT > FS0:\EFI\BOOT\> ls > Directory of: FS0:\EFI\BOOT\ > 04/04/2019 16:18 2,048 . > 04/04/2019 16:18 2,048 .. > 08/31/2017 21:30 1,296,176 BOOTX64.EFI > 08/31/2017 21:30 79,048 fbx64.efi > 2 File(s) 1,375,224 bytes > 2 Dir(s) > FS0:\EFI\BOOT\> BOOTX64.EFI > Failed to set MokListRT: Invalid Parameter > Failed to open \EFI\BOOT\grubx64.efi - Not Found > Failed to load image \EFI\BOOT\grubx64.efi: Not Found > start_image() returned Not Found > FS0:\EFI\BOOT\> > > -- > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Sun Apr 7 08:02:37 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D76D2157B2AE for ; Sun, 7 Apr 2019 08:02:37 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0C85487996 for ; Sun, 7 Apr 2019 08:02:36 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=QFU4mZBNwN+Dq7dWqPvu4nUgb3CctUuVlDYQ6O3LYoU=; b=fjIFZUMN1Br3UkcFRdS8nkdJVA krW8dzGf3Rpv53IJedPRpSq0D03uHmNZFl9ymK15cSCOXDFPn1Kl4yCIkLJMjy9ey/fBQNfNWV+Z/ L30UfdwTfGxJRcC2pSXCiYN1BoBQAC5TyE5cKKcE64aV/as4Y6V8Ra76y35QJ6gkJi2o=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hD2l9-000Af9-Lh for freebsd-virtualization@freebsd.org; Sun, 07 Apr 2019 15:02:35 +0700 Date: Sun, 7 Apr 2019 15:02:35 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: running FreePBX SNG7 Official Distro Message-ID: <20190407080235.GA40361@admin.sibptus.ru> References: <20190407023743.GB99339@admin.sibptus.ru> <201904070419.x374JbIp048373@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EeQfGwPcQSOJBaQU" Content-Disposition: inline In-Reply-To: <201904070419.x374JbIp048373@gndrsh.dnsmgr.net> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 08:02:38 -0000 --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Rodney W. Grimes wrote: > > > > >=20 > > > > > You can usually use the host by doing mdconfig -f > > > >=20 > > > > Unfortunately mdconfig does not work with zvols: > > > >=20 > > > > root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 > > > > mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file > > >=20 > > > If its a zvol cant you just do > > > gpart show /dev/zvol/d02/vm/freepbx/disk0 > > >=20 > > > and=20 > > > mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 > >=20 > > No I can't if the zvol is in the "volmode=3Ddev" mode which is the defa= ult.=20 > >=20 > > This is the default for a reason: it's no good exposing scores of always > > coming and going guest geoms to the host system. I think you can even > > get a conflict of labels or something like that one day. >=20 > So it may take a few more commands but it should be > possible to do this from the host side using host > side tools without having to boot a guest to make > these corrections. I'm not aware of such commands. If anyone knows them please share with us. Moreover, I already asked a similar question in February under the topic "mounting/exporting/importing a zfs volume" and nobody gave a recipe. --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --EeQfGwPcQSOJBaQU Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcqa6bAAoJEA2k8lmbXsY0TfIIAJT3ciG0rk6ko/U74bwEenXy 318oH5L4YPr/G3C1k2htsbLrykGZzhRTOtwFBhzNExOEBzjL5X5Z6Nb7+0h+bCL0 ZB5brZqryFXNu8ANsk3h7d8s+LKt0anjq63uN0qfkSsrMRwmpY/pFAQ6I5P7ED95 ljhYv9iP+UBOLwQPTktTdNkoeCtCaAO7AIol/oz1CuNxn+fYqsGkoBeVCbOlRMem JxBe63X08FNo5YkEBXaPH04nhC4hHIFlFx6OkYp8hbu/nKAAKkxOTXSY3qpO+TWx xwm3FzA/ixO13wH5UuZpD4PqWt2BzYvzfryyRAFhHQjqGnlR++MCoNcD71sX8yk= =z+oj -----END PGP SIGNATURE----- --EeQfGwPcQSOJBaQU-- From owner-freebsd-virtualization@freebsd.org Sun Apr 7 08:14:46 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7548157B4B8 for ; Sun, 7 Apr 2019 08:14:46 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA98F87DAE for ; Sun, 7 Apr 2019 08:14:45 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=MQbf71YlQ40fnrRKpkqP8VIgJtDQoQDBdoFz9cwWc9Q=; b=QhENcF1KKD4xl3I2YqixuLd7JD aHFCWxkNuZpASnDhkLoZk6gp0cD7VJvAqsQr28Nj3yqMRi+8CJyef4Npc25Qo5BqLotdp9JdGvGpS pUxZi5JjEEnhD+0T1IrCTNgESEUxn+PfOnNPMlMaVzQW61K0WtytCW5mp3epKWaodufQ=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hD2wv-000Azq-4h for freebsd-virtualization@freebsd.org; Sun, 07 Apr 2019 15:14:45 +0700 Date: Sun, 7 Apr 2019 15:14:45 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: running FreePBX SNG7 Official Distro Message-ID: <20190407081445.GB40361@admin.sibptus.ru> References: <20190407031237.GA7489@admin.sibptus.ru> <201904070426.x374Q641048406@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZfOjI3PrQbgiZnxM" Content-Disposition: inline In-Reply-To: <201904070426.x374Q641048406@gndrsh.dnsmgr.net> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 08:14:47 -0000 --ZfOjI3PrQbgiZnxM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Rodney W. Grimes wrote: > > > > > > > I can guess that it looks for a FAT16 partition in the GPT wi= th the type > > > > > > > "efi" but the rest is a mystery for me. Why is it trying to f= ind > > > > > > > "grubx64.efi" and not the default "boot64.efi" (which is pres= ent), for > > > > > > > example? > > > > > >=20 > > > > > > I suspect that what ever guest you installed installed something > > > > > > else someplace, either within the eft partition, or possibly in > > > > > > the MBR? > > > > >=20 > > > > > Do you mean to say, the guest installing something else someplace= can > > > > > influence the boot sequence of bhyve efi? > > > >=20 > > > > The guest created all of the bits on that zvol, > > > > it can influence many things. There is probably a tiny initial > > > > stub that efi loads that has this bath to grubx64.efi codded in > > > > it and that is what is causing this issue. > > >=20 > > > It is very important to find and debug it because Oracle VirtualBox in > > > UEFI mode installs and runs this guest just fine. So it must be some > > > issue in bhyve itself. > > >=20 > > > Here is the complete archive of everything the guest created in the E= FI > > > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > > > can you find those confusing bits? > >=20 > > I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, but > > BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be some > > kind of chain loader. >=20 > And it brobably tries to read a efivariable, and if that variable > is not set it defaults to grubx64.efi. This bootx64.efi is something > the guest installed into the EFI partition, hence my assertion that > the issue is with something the guest installed is some what valid. Do you think the guest OS installer set some efi variable during the installation process, which bhyve did not save? That would explain a lot. > > > > > > Moreover, I waited (for a long time!) for the EFI interactive s= hell > > > > > > prompt and with a few commands: > > > > >=20 > > > > > Yes, the timeout is very long, and I do not know that we > > > > > document anyplace that if you wait long enough at a failed > > > > > boot you do get a EFI shell prompt eventually. > > > >=20 > > > > Can I press some key to escape to the EFI shell? > > > Not that I am aware of. > >=20 > > It's a major problem! There must be a well-known way to break the boot > > sequence any time and enter the EFI shell. >=20 > Agreed, hopefully those working on edk2 take note and either > chime in with what that way is, or create a bug and track > so that someone may fix this issue. Would it be useful to create a PR in the FreeBSD bugtracker with a feature request? > >=20 > > It is very important to find and debug it because Oracle VirtualBox in > > UEFI mode installs and runs this guest just fine. So it must be some > > issue in bhyve itself. >=20 > As I stated earlier bhyve is missing percistant efi variables, > and that is most likely the reason that VirtualBox just works > and bhyve does not. >=20 > Probably you well find in your VirtualBox directory a > file that is used to store efivars, that is where the I'll look into the VirtualBox directory tomorrow and report here. I was under the impression that efivars are stored in a configuration file in the EFI partition but I was probably wrong, they are kept in NVRAM somewhere, like BIOS settings, and not on a disk. --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --ZfOjI3PrQbgiZnxM Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcqbF1AAoJEA2k8lmbXsY05s8IAJPMWCzDz6BtHoPcziomjk5R 4xtzCKizy4NbZjpbg4jrG4Sw3E0BBtcQkJNlSwpXYj7/mq+NhgbXG9xtOLiSZYnd rzCo8WOjEv9TxoUg/2HRjiECiSjl2krUxk/qfOiUqaayrdy2eCvJCXL9AWE1K/aE bqWfBxrtfC+yIZI5OvU21IDOM3waZtaF8fXBxFFKM48M3SWw++O+4Z8oZxKeLyrE N2v4Vz2j5ZRbGNaW21RoL98JNH3/LHh7ZIDu/+FYdNpVShmZmd7rskZpl2a9ZCbW Y1YG4Ah00LvU2acLgCavnzrhZi2+siNlWE8I8HprzO37Ir7z8kQiVA9U7aeEB4w= =eKWW -----END PGP SIGNATURE----- --ZfOjI3PrQbgiZnxM-- From owner-freebsd-virtualization@freebsd.org Sun Apr 7 17:12:35 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C154A15655CB for ; Sun, 7 Apr 2019 17:12:35 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BC0469EE5 for ; Sun, 7 Apr 2019 17:12:35 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: by mail-lf1-x12f.google.com with SMTP id v14so7769780lfi.0 for ; Sun, 07 Apr 2019 10:12:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=UinYYOAMj1BxDE6Zmr1enIphNzZMoSIYxXKxAxmcz8U=; b=Z7KPv1tX1ckGlSGlhQE8Rf72SXLOX5nVV9Lsi3VNB7gSkl8pm8Z5l0NUjPVvcsEi/G YsdNRgaeVV04uRMF6dSuAEYHKjVLLrHSOhZXKc3c8GCQp8WOsQt0P84UfNfNp+wdmmQm PpFoi+oWcEJDMTXJ7sWU4qIJSOqBxa8dVVeM/JJnPbjWbXjSxNfjeO2zOuyZC8ay2ymo e3IwQeqVqnXN5iZ4qtKzRTY4Zt6dlpfi2KiYW3+3k89X4SI88oxkZNqNB24JBSo2yeh1 GlGg3dTeYeQZK3NsQ/9K9GBZDvhJ0+RohtJyT71L3OOvU+jLL4Yv2lfOMPpNVbnA7ZBS TSYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=UinYYOAMj1BxDE6Zmr1enIphNzZMoSIYxXKxAxmcz8U=; b=s6bvlvYNHAgKaaPNOvuCGIh6Ly3og+y6lN4Vb5fkvccTX524esfnzQbbBNGHjUJe8x qaEDyBfwNLb9DzaXFYCeFqx6Sj9DLAl1lXOGSK7PKIaFhkJ25auniPtS3kh58PeJMktW 8bGbYL66VIKLdFbS2eR3+962b+6QeeMg/svAWHc7HAkS7QhWVe4/nkjo8gNQ5P+/C1nJ xjT4CIBMgp25x6zRPTbB44FxrZ6I26LVTv4Iqxg/09tvGnp3VmsRRMtmWHXumT/hY5Mp 7mtF/2JQGFZD3m5BFPIrEhp/E+mCX/hKlLyy6IQ59hn4ZKpNfOGGBAgPlHPFC+G5x8V7 rJqw== X-Gm-Message-State: APjAAAX8r27Gt+fmUK6GGC6ibFloDmYPwHUVtBix4KwODlgF7HYx1M1l BOsXLwOWmPHLWGxB31//6g5j0uV4KNoo7PrlFnC9ROUs X-Google-Smtp-Source: APXvYqwcNGHgK4y75QXVxqfYpXSSo33adCfBEHTbDnIIWCKlq2nJspip2ZgmEeIlj6IOSD5g4+YIfPG9XlV693fB2P8= X-Received: by 2002:ac2:4551:: with SMTP id j17mr2681899lfm.141.1554657152898; Sun, 07 Apr 2019 10:12:32 -0700 (PDT) MIME-Version: 1.0 References: <20190407031237.GA7489@admin.sibptus.ru> <201904070426.x374Q641048406@gndrsh.dnsmgr.net> <20190407081445.GB40361@admin.sibptus.ru> In-Reply-To: <20190407081445.GB40361@admin.sibptus.ru> From: Subbsd Date: Sun, 7 Apr 2019 20:12:21 +0300 Message-ID: Subject: Re: running FreePBX SNG7 Official Distro To: Victor Sudakov Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 2BC0469EE5 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.94 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.94)[-0.945,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 17:12:35 -0000 On Sun, Apr 7, 2019 at 11:15 AM Victor Sudakov wrote: > I'll look into the VirtualBox directory tomorrow and report here. I was > under the impression that efivars are stored in a configuration file in > the EFI partition but I was probably wrong, they are kept in NVRAM > somewhere, like BIOS settings, and not on a disk. > > -- > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ I just wanted to confirm that this is indeed an EVIVARS problem, because through CBSD (which uses Refind [1]) FreePBX 1805-2 boots without any problems. Therefore, as Rodney said, you can try using a third-party EFI boot manager. And just wait until someone finds the time to add support for UEFI VARS in bhyve. [1] - http://www.rodsbooks.com/refind/ From owner-freebsd-virtualization@freebsd.org Sun Apr 7 17:32:00 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE0BD1565A5C for ; Sun, 7 Apr 2019 17:31:59 +0000 (UTC) (envelope-from jonathans.email@icloud.com) Received: from pv50p00im-ztdg10021201.me.com (pv50p00im-ztdg10021201.me.com [17.58.6.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D060A6A8F7 for ; Sun, 7 Apr 2019 17:31:57 +0000 (UTC) (envelope-from jonathans.email@icloud.com) Received: from jonathans-mbp.home (unknown [94.1.145.187]) by pv50p00im-ztdg10021201.me.com (Postfix) with ESMTPSA id A43FCA4000B for ; Sun, 7 Apr 2019 17:31:49 +0000 (UTC) From: Jonathan Rowley Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: unsubscribe Date: Sun, 7 Apr 2019 18:31:46 +0100 References: To: freebsd-virtualization@freebsd.org In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3445.102.3) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-04-07_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=11 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1812120000 definitions=main-1904070165 X-Rspamd-Queue-Id: D060A6A8F7 X-Spamd-Bar: -------- X-Spamd-Result: default: False [-8.64 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_COMPOSITE_RCVD_IN_DNSWL_MED_DWL_DNSWL_LOW(0.00)[]; FREEMAIL_FROM(0.00)[icloud.com]; R_SPF_ALLOW(-0.20)[+ip4:17.58.0.0/16]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DKIM_TRACE(0.00)[icloud.com:+]; RCVD_IN_DNSWL_MED(-0.20)[45.6.58.17.list.dnswl.org : 127.0.5.2]; DMARC_POLICY_ALLOW(-0.50)[icloud.com,quarantine]; MX_GOOD(-0.01)[mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, m x6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.co m, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud .com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icl oud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com]; NEURAL_HAM_SHORT(-0.93)[-0.932,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[icloud.com]; ASN(0.00)[asn:714, ipnet:17.58.0.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[icloud.com:s=04042017]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(0.00)[icloud.com.dwl.dnswl.org : 127.0.5.1]; IP_SCORE(-1.50)[ip: (-3.24), ipnet: 17.58.0.0/20(-2.10), asn: 714(-2.11), country: US(-0.06)]; WHITELIST_SPF_DKIM(-3.00)[icloud.com:d:+,icloud.com:s:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 17:32:00 -0000 > On 7 Apr 2019, at 13:00, freebsd-virtualization-request@freebsd.org = wrote: >=20 > Send freebsd-virtualization mailing list submissions to > freebsd-virtualization@freebsd.org >=20 > To subscribe or unsubscribe via the World Wide Web, visit > = https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > or, via email, send a message with subject or body 'help' to > freebsd-virtualization-request@freebsd.org >=20 > You can reach the person managing the list at > freebsd-virtualization-owner@freebsd.org >=20 > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-virtualization digest..." >=20 >=20 > Today's Topics: >=20 > 1. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 2. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 3. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes) > 4. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes) > 5. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 6. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) >=20 >=20 > ---------------------------------------------------------------------- >=20 > Message: 1 > Date: Sun, 7 Apr 2019 09:37:43 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407023743.GB99339@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>=20 >>>> You can usually use the host by doing mdconfig -f = >>>=20 >>> Unfortunately mdconfig does not work with zvols: >>>=20 >>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>=20 >> If its a zvol cant you just do >> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>=20 >> and=20 >> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >=20 > No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >=20 > This is the default for a reason: it's no good exposing scores of = always > coming and going guest geoms to the host system. I think you can even > get a conflict of labels or something like that one day. >=20 >>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>> prompt and with a few commands: >>>>=20 >>>> Yes, the timeout is very long, and I do not know that we >>>> document anyplace that if you wait long enough at a failed >>>> boot you do get a EFI shell prompt eventually. >>>=20 >>> Can I press some key to escape to the EFI shell? >> Not that I am aware of. >=20 > It's a major problem! There must be a well-known way to break the boot > sequence any time and enter the EFI shell. >=20 >>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), = for >>>>> example? >>>>=20 >>>> I suspect that what ever guest you installed installed something >>>> else someplace, either within the eft partition, or possibly in >>>> the MBR? >>>=20 >>> Do you mean to say, the guest installing something else someplace = can >>> influence the boot sequence of bhyve efi? >>=20 >> The guest created all of the bits on that zvol, >> it can influence many things. There is probably a tiny initial >> stub that efi loads that has this bath to grubx64.efi codded in >> it and that is what is causing this issue. >=20 > It is very important to find and debug it because Oracle VirtualBox in > UEFI mode installs and runs this guest just fine. So it must be some > issue in bhyve itself. >=20 > Here is the complete archive of everything the guest created in the = EFI > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > can you find those confusing bits? >=20 > The standard procedure should be as follows: >=20 > Automated detection relies on standardized file paths to the OS > loader, with the path varying depending on the computer architecture. > The format of the file path is defined as > /EFI/BOOT/BOOT.EFI; for > example, the file path to the OS loader on an x86-64 system is > /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture.=20= >=20 > Nothing about grub*.efi. But only bhyve is confused, VirtualBox is = not. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = >=20 > ------------------------------ >=20 > Message: 2 > Date: Sun, 7 Apr 2019 10:12:37 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407031237.GA7489@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Victor Sudakov wrote: >>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>> example? >>>>>=20 >>>>> I suspect that what ever guest you installed installed something >>>>> else someplace, either within the eft partition, or possibly in >>>>> the MBR? >>>>=20 >>>> Do you mean to say, the guest installing something else someplace = can >>>> influence the boot sequence of bhyve efi? >>>=20 >>> The guest created all of the bits on that zvol, >>> it can influence many things. There is probably a tiny initial >>> stub that efi loads that has this bath to grubx64.efi codded in >>> it and that is what is causing this issue. >>=20 >> It is very important to find and debug it because Oracle VirtualBox = in >> UEFI mode installs and runs this guest just fine. So it must be some >> issue in bhyve itself. >>=20 >> Here is the complete archive of everything the guest created in the = EFI >> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >> can you find those confusing bits? >=20 > I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but > BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some > kind of chain loader. >=20 > Watch the interactive session below. It does not however mean that = there is > nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and = runs this > guest just fine. >=20 > FS0:\> cd EFI > FS0:\EFI\> ls > Directory of: FS0:\EFI\ > 04/04/2019 15:53 2,048 . > 04/04/2019 15:53 0 .. > 04/04/2019 16:26 2,048 centos > 04/06/2019 04:19 2,048 BOOT > 0 File(s) 0 bytes > 4 Dir(s) > FS0:\EFI\> cd BOOT > FS0:\EFI\BOOT\> ls > Directory of: FS0:\EFI\BOOT\ > 04/04/2019 16:18 2,048 . > 04/04/2019 16:18 2,048 .. > 08/31/2017 21:30 1,296,176 BOOTX64.EFI > 08/31/2017 21:30 79,048 fbx64.efi > 2 File(s) 1,375,224 bytes > 2 Dir(s) > FS0:\EFI\BOOT\> BOOTX64.EFI > Failed to set MokListRT: Invalid Parameter > Failed to open \EFI\BOOT\grubx64.efi - Not Found > Failed to load image \EFI\BOOT\grubx64.efi: Not Found > start_image() returned Not Found > FS0:\EFI\BOOT\>=20 >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = >=20 > ------------------------------ >=20 > Message: 3 > Date: Sat, 6 Apr 2019 21:19:37 -0700 (PDT) > From: "Rodney W. Grimes" > To: Victor Sudakov > Cc: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <201904070419.x374JbIp048373@gndrsh.dnsmgr.net> > Content-Type: text/plain; charset=3DUS-ASCII >=20 >> Rodney W. Grimes wrote: >>>>>=20 >>>>> You can usually use the host by doing mdconfig -f = >>>>=20 >>>> Unfortunately mdconfig does not work with zvols: >>>>=20 >>>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>>=20 >>> If its a zvol cant you just do >>> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>>=20 >>> and=20 >>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >>=20 >> No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >>=20 >> This is the default for a reason: it's no good exposing scores of = always >> coming and going guest geoms to the host system. I think you can even >> get a conflict of labels or something like that one day. >=20 > So it may take a few more commands but it should be > possible to do this from the host side using host > side tools without having to boot a guest to make > these corrections. >=20 >>>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>>> prompt and with a few commands: >>>>>=20 >>>>> Yes, the timeout is very long, and I do not know that we >>>>> document anyplace that if you wait long enough at a failed >>>>> boot you do get a EFI shell prompt eventually. >>>>=20 >>>> Can I press some key to escape to the EFI shell? >>> Not that I am aware of. >>=20 >> It's a major problem! There must be a well-known way to break the = boot >> sequence any time and enter the EFI shell. >=20 > Agreed, hopefully those working on edk2 take note and either > chime in with what that way is, or create a bug and track > so that someone may fix this issue. >=20 >>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>> example? >>>>>=20 >>>>> I suspect that what ever guest you installed installed something >>>>> else someplace, either within the eft partition, or possibly in >>>>> the MBR? >>>>=20 >>>> Do you mean to say, the guest installing something else someplace = can >>>> influence the boot sequence of bhyve efi? >>>=20 >>> The guest created all of the bits on that zvol, >>> it can influence many things. There is probably a tiny initial >>> stub that efi loads that has this bath to grubx64.efi codded in >>> it and that is what is causing this issue. >>=20 >> It is very important to find and debug it because Oracle VirtualBox = in >> UEFI mode installs and runs this guest just fine. So it must be some >> issue in bhyve itself. >=20 > As I stated earlier bhyve is missing percistant efi variables, > and that is most likely the reason that VirtualBox just works > and bhyve does not. >=20 >> Here is the complete archive of everything the guest created in the = EFI >> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >> can you find those confusing bits? >=20 > Probably you well find in your VirtualBox directory a > file that is used to store efivars, that is where the > difference occurs. >=20 >> The standard procedure should be as follows: >>=20 >> Automated detection relies on standardized file paths to the OS >> loader, with the path varying depending on the computer architecture. >> The format of the file path is defined as >> /EFI/BOOT/BOOT.EFI; = for >> example, the file path to the OS loader on an x86-64 system is >> /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 = architecture.=20 >>=20 >> Nothing about grub*.efi. But only bhyve is confused, VirtualBox is = not. >=20 > bhyve is not really confused, it is simply missing a feature > that this guest depends on for its boot procedures. This is > a well known miss-feature, but I do not know of anyone actively > working on fixing it. >=20 >> Victor Sudakov, VAS4-RIPE, VAS47-RIPN >> 2:5005/49@fidonet http://vas.tomsk.ru/ > --=20 > Rod Grimes = rgrimes@freebsd.org >=20 >=20 > ------------------------------ >=20 > Message: 4 > Date: Sat, 6 Apr 2019 21:26:06 -0700 (PDT) > From: "Rodney W. Grimes" > To: Victor Sudakov > Cc: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <201904070426.x374Q641048406@gndrsh.dnsmgr.net> > Content-Type: text/plain; charset=3DUS-ASCII >=20 >> Victor Sudakov wrote: >>>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>>> example? >>>>>>=20 >>>>>> I suspect that what ever guest you installed installed something >>>>>> else someplace, either within the eft partition, or possibly in >>>>>> the MBR? >>>>>=20 >>>>> Do you mean to say, the guest installing something else someplace = can >>>>> influence the boot sequence of bhyve efi? >>>>=20 >>>> The guest created all of the bits on that zvol, >>>> it can influence many things. There is probably a tiny initial >>>> stub that efi loads that has this bath to grubx64.efi codded in >>>> it and that is what is causing this issue. >>>=20 >>> It is very important to find and debug it because Oracle VirtualBox = in >>> UEFI mode installs and runs this guest just fine. So it must be some >>> issue in bhyve itself. >>>=20 >>> Here is the complete archive of everything the guest created in the = EFI >>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >>> can you find those confusing bits? >>=20 >> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but >> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some >> kind of chain loader. >=20 > And it brobably tries to read a efivariable, and if that variable > is not set it defaults to grubx64.efi. This bootx64.efi is something > the guest installed into the EFI partition, hence my assertion that > the issue is with something the guest installed is some what valid. >=20 > There are some 3rd party EFI boot managers that might help > resolve this problem, or simply use the work around that > I provided earlier until we can get efivars working in > bhyve. >=20 >=20 >> Watch the interactive session below. It does not however mean that = there is >> nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and = runs this >> guest just fine. >>=20 >> FS0:\> cd EFI >> FS0:\EFI\> ls >> Directory of: FS0:\EFI\ >> 04/04/2019 15:53 2,048 . >> 04/04/2019 15:53 0 .. >> 04/04/2019 16:26 2,048 centos >> 04/06/2019 04:19 2,048 BOOT >> 0 File(s) 0 bytes >> 4 Dir(s) >> FS0:\EFI\> cd BOOT >> FS0:\EFI\BOOT\> ls >> Directory of: FS0:\EFI\BOOT\ >> 04/04/2019 16:18 2,048 . >> 04/04/2019 16:18 2,048 .. >> 08/31/2017 21:30 1,296,176 BOOTX64.EFI >> 08/31/2017 21:30 79,048 fbx64.efi >> 2 File(s) 1,375,224 bytes >> 2 Dir(s) >> FS0:\EFI\BOOT\> BOOTX64.EFI >> Failed to set MokListRT: Invalid Parameter >> Failed to open \EFI\BOOT\grubx64.efi - Not Found >> Failed to load image \EFI\BOOT\grubx64.efi: Not Found >> start_image() returned Not Found >> FS0:\EFI\BOOT\>=20 >>=20 >> --=20 >> Victor Sudakov, VAS4-RIPE, VAS47-RIPN >> 2:5005/49@fidonet http://vas.tomsk.ru/ >=20 > --=20 > Rod Grimes = rgrimes@freebsd.org >=20 >=20 > ------------------------------ >=20 > Message: 5 > Date: Sun, 7 Apr 2019 15:02:35 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407080235.GA40361@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>>>=20 >>>>>> You can usually use the host by doing mdconfig -f = >>>>>=20 >>>>> Unfortunately mdconfig does not work with zvols: >>>>>=20 >>>>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>>>=20 >>>> If its a zvol cant you just do >>>> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>>>=20 >>>> and=20 >>>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >>>=20 >>> No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >>>=20 >>> This is the default for a reason: it's no good exposing scores of = always >>> coming and going guest geoms to the host system. I think you can = even >>> get a conflict of labels or something like that one day. >>=20 >> So it may take a few more commands but it should be >> possible to do this from the host side using host >> side tools without having to boot a guest to make >> these corrections. >=20 > I'm not aware of such commands. If anyone knows them please share with = us. >=20 > Moreover, I already asked a similar question in February under the > topic "mounting/exporting/importing a zfs volume" and nobody gave a > recipe. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = >=20 > ------------------------------ >=20 > Message: 6 > Date: Sun, 7 Apr 2019 15:14:45 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407081445.GB40361@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>>>> "efi" but the rest is a mystery for me. Why is it trying to = find >>>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>>>> example? >>>>>>>=20 >>>>>>> I suspect that what ever guest you installed installed something >>>>>>> else someplace, either within the eft partition, or possibly in >>>>>>> the MBR? >>>>>>=20 >>>>>> Do you mean to say, the guest installing something else someplace = can >>>>>> influence the boot sequence of bhyve efi? >>>>>=20 >>>>> The guest created all of the bits on that zvol, >>>>> it can influence many things. There is probably a tiny initial >>>>> stub that efi loads that has this bath to grubx64.efi codded in >>>>> it and that is what is causing this issue. >>>>=20 >>>> It is very important to find and debug it because Oracle VirtualBox = in >>>> UEFI mode installs and runs this guest just fine. So it must be = some >>>> issue in bhyve itself. >>>>=20 >>>> Here is the complete archive of everything the guest created in the = EFI >>>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >>>> can you find those confusing bits? >>>=20 >>> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but >>> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some >>> kind of chain loader. >>=20 >> And it brobably tries to read a efivariable, and if that variable >> is not set it defaults to grubx64.efi. This bootx64.efi is something >> the guest installed into the EFI partition, hence my assertion that >> the issue is with something the guest installed is some what valid. >=20 > Do you think the guest OS installer set some efi variable during the > installation process, which bhyve did not save? That would explain a > lot. >=20 >>>>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>>>> prompt and with a few commands: >>>>>>=20 >>>>>> Yes, the timeout is very long, and I do not know that we >>>>>> document anyplace that if you wait long enough at a failed >>>>>> boot you do get a EFI shell prompt eventually. >>>>>=20 >>>>> Can I press some key to escape to the EFI shell? >>>> Not that I am aware of. >>>=20 >>> It's a major problem! There must be a well-known way to break the = boot >>> sequence any time and enter the EFI shell. >>=20 >> Agreed, hopefully those working on edk2 take note and either >> chime in with what that way is, or create a bug and track >> so that someone may fix this issue. >=20 > Would it be useful to create a PR in the FreeBSD bugtracker with a > feature request? >=20 >>>=20 >>> It is very important to find and debug it because Oracle VirtualBox = in >>> UEFI mode installs and runs this guest just fine. So it must be some >>> issue in bhyve itself. >>=20 >> As I stated earlier bhyve is missing percistant efi variables, >> and that is most likely the reason that VirtualBox just works >> and bhyve does not. >>=20 >> Probably you well find in your VirtualBox directory a >> file that is used to store efivars, that is where the >=20 > I'll look into the VirtualBox directory tomorrow and report here. I = was > under the impression that efivars are stored in a configuration file = in > the EFI partition but I was probably wrong, they are kept in NVRAM > somewhere, like BIOS settings, and not on a disk. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = >=20 > ------------------------------ >=20 > Subject: Digest Footer >=20 > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to = "freebsd-virtualization-unsubscribe@freebsd.org" >=20 >=20 > ------------------------------ >=20 > End of freebsd-virtualization Digest, Vol 436, Issue 7 > ****************************************************** From owner-freebsd-virtualization@freebsd.org Sun Apr 7 17:48:46 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E44551566215 for ; Sun, 7 Apr 2019 17:48:45 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 28DD86B2C9 for ; Sun, 7 Apr 2019 17:48:44 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x37HmhDO051358; Sun, 7 Apr 2019 10:48:43 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x37Hmh4q051357; Sun, 7 Apr 2019 10:48:43 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201904071748.x37Hmh4q051357@gndrsh.dnsmgr.net> Subject: Re: running FreePBX SNG7 Official Distro In-Reply-To: To: Subbsd Date: Sun, 7 Apr 2019 10:48:43 -0700 (PDT) CC: Victor Sudakov , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 28DD86B2C9 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-0.06 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.23)[-0.231,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.23)[-0.230,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: gndrsh.dnsmgr.net]; NEURAL_HAM_SHORT(-0.52)[-0.517,0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.03)[ip: (0.12), ipnet: 69.59.192.0/19(0.06), asn: 13868(0.04), country: US(-0.06)] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 17:48:46 -0000 > On Sun, Apr 7, 2019 at 11:15 AM Victor Sudakov wrote: > > > I'll look into the VirtualBox directory tomorrow and report here. I was > > under the impression that efivars are stored in a configuration file in > > the EFI partition but I was probably wrong, they are kept in NVRAM > > somewhere, like BIOS settings, and not on a disk. > > > > -- > > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > > 2:5005/49@fidonet http://vas.tomsk.ru/ > > I just wanted to confirm that this is indeed an EVIVARS problem, > because through CBSD (which uses Refind [1]) FreePBX 1805-2 > boots without any problems. Therefore, as Rodney said, you can try > using a third-party EFI boot manager. > And just wait until someone finds the time to add support for UEFI > VARS in bhyve. Thank you for the analysis and confirmation! > [1] - http://www.rodsbooks.com/refind/ -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Mon Apr 8 05:01:46 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D36F41574226 for ; Mon, 8 Apr 2019 05:01:46 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E166C8708E for ; Mon, 8 Apr 2019 05:01:45 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=u51pc4lC/taBcMmIDwlCtqbOb+itsbHjU8hxLFd7ogQ=; b=mwSN8UAKfA0AI7dh+drzhsjIGH HQOZDmpkXZaO2oLsxJjJSWqFQpu+fBzW4T87zIMEFRr6p/adYszhAh1iWjwOspLLMFP0BsbUjzgjE XAPG1HVuhIp1R0Dm+d5qER4hR+cOJcvDcw4/puibGCOK/K2w28oK6B9Okv9yp+iJr540=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hDMPg-000Exr-5I for freebsd-virtualization@freebsd.org; Mon, 08 Apr 2019 12:01:44 +0700 Date: Mon, 8 Apr 2019 12:01:44 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: running FreePBX SNG7 Official Distro Message-ID: <20190408050144.GA56879@admin.sibptus.ru> References: <20190407031237.GA7489@admin.sibptus.ru> <201904070426.x374Q641048406@gndrsh.dnsmgr.net> <20190407081445.GB40361@admin.sibptus.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qMm9M+Fa2AknHoGS" Content-Disposition: inline In-Reply-To: <20190407081445.GB40361@admin.sibptus.ru> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2019 05:01:47 -0000 --qMm9M+Fa2AknHoGS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Victor Sudakov wrote: > >=20 > > As I stated earlier bhyve is missing percistant efi variables, > > and that is most likely the reason that VirtualBox just works > > and bhyve does not. > >=20 > > Probably you well find in your VirtualBox directory a > > file that is used to store efivars, that is where the >=20 > I'll look into the VirtualBox directory tomorrow and report here.=20 I searched through my disk and was unable to find a persistant efivars storage in my VirtualBox 6.0 installation.=20 A Google search reveals some articles (rather dated I must admit) stating that VirtualBox does not support NVRAM emulation for storing efi variables:=20 https://www.virtualbox.org/ticket/14279 https://forums.virtualbox.org/viewtopic.php?t=3D61970=20 they recommend using startup.nsh instead. I wonder if bhyve's efi implementation supports startup.nsh. --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --qMm9M+Fa2AknHoGS Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcqtW4AAoJEA2k8lmbXsY0fNMIALZi+qMpQiu2WLhuabTg5enx 9jePQTnHmIFXh4UKm5QGz97cpJUisK1QEpDl9zMYNjM1yShqSzrm2PgGBM9umpyl KoDmzfSCOTmAPCxLJzXhwlOqb6pwupRvxiQR9xFdh1mm0tjIsuMXNr54MyiuTG3Y w09rdMTyk5n2WTG3W63SBCxqelJ03mGxlENmemdW1hJJgdeLGb6SqSAfBWvUUnvA YUQSLfoxihlkhhr/6qpY6YWJZ8OZHR7qmCqo+l6pxuwy43BuGvK2ITLb9FNHNM0M wupfDLbm2MWDGlrGSw5WDeCuvUOzELKYDhCj0FWPlIeyJcJPF4th0vlAsyEDeZY= =yXPS -----END PGP SIGNATURE----- --qMm9M+Fa2AknHoGS-- From owner-freebsd-virtualization@freebsd.org Mon Apr 8 07:39:04 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A75EE15778B1 for ; Mon, 8 Apr 2019 07:39:04 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 113F08C32F for ; Mon, 8 Apr 2019 07:39:03 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x387d14L054125; Mon, 8 Apr 2019 00:39:01 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x387d1T9054124; Mon, 8 Apr 2019 00:39:01 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201904080739.x387d1T9054124@gndrsh.dnsmgr.net> Subject: Re: running FreePBX SNG7 Official Distro In-Reply-To: <20190408050144.GA56879@admin.sibptus.ru> To: Victor Sudakov Date: Mon, 8 Apr 2019 00:39:01 -0700 (PDT) CC: freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 113F08C32F X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.984,0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2019 07:39:04 -0000 > Victor Sudakov wrote: > > > > > > As I stated earlier bhyve is missing percistant efi variables, > > > and that is most likely the reason that VirtualBox just works > > > and bhyve does not. > > > > > > Probably you well find in your VirtualBox directory a > > > file that is used to store efivars, that is where the > > > > I'll look into the VirtualBox directory tomorrow and report here. > > I searched through my disk and was unable to find a persistant efivars > storage in my VirtualBox 6.0 installation. > > A Google search reveals some articles (rather dated I must admit) > stating that VirtualBox does not support NVRAM emulation for storing efi > variables: > > https://www.virtualbox.org/ticket/14279 > https://forums.virtualbox.org/viewtopic.php?t=61970 My quick search turns up: https://www.virtualbox.org/svn/vbox/trunk/src/VBox/Devices/EFI/DevEFI.cpp aka source code thet implements efivars stored in nvram. > they recommend using startup.nsh instead. I wonder if bhyve's efi > implementation supports startup.nsh. I believe that is how the alternate boot selector someone pointed at gets hooked in. > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Mon Apr 8 07:46:36 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49C9F1578115 for ; Mon, 8 Apr 2019 07:46:36 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 986048CA13 for ; Mon, 8 Apr 2019 07:46:35 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=osxWd62nejoztAkp3UeLaMEYoULqzCBGFQ42sM6hrr0=; b=ASo6g0nGpmGMaWW0qO6Es+pr+4 7CgSM1SheqYyof9RSRePjDWbC+Fibm9Zm+nukmpw4wuTRJ6UXp+6uHi9YTlH9rUDXmAMQ6QR0VsG+ YoYFbHaq7ObQFZyVb5rArstya+r+tIkCmDjpgij2RfSyolUnh6vFnWTAOdQMfzNSY16I=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hDOzC-000Fbr-7L for freebsd-virtualization@freebsd.org; Mon, 08 Apr 2019 14:46:34 +0700 Date: Mon, 8 Apr 2019 14:46:34 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: running FreePBX SNG7 Official Distro Message-ID: <20190408074634.GA59930@admin.sibptus.ru> References: <20190408050144.GA56879@admin.sibptus.ru> <201904080739.x387d1T9054124@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline In-Reply-To: <201904080739.x387d1T9054124@gndrsh.dnsmgr.net> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2019 07:46:36 -0000 --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Rodney W. Grimes wrote: > > > >=20 > > > > As I stated earlier bhyve is missing percistant efi variables, > > > > and that is most likely the reason that VirtualBox just works > > > > and bhyve does not. > > > >=20 > > > > Probably you well find in your VirtualBox directory a > > > > file that is used to store efivars, that is where the > > >=20 > > > I'll look into the VirtualBox directory tomorrow and report here.=20 > >=20 > > I searched through my disk and was unable to find a persistant efivars > > storage in my VirtualBox 6.0 installation.=20 > >=20 > > A Google search reveals some articles (rather dated I must admit) > > stating that VirtualBox does not support NVRAM emulation for storing efi > > variables:=20 > >=20 > > https://www.virtualbox.org/ticket/14279 > > https://forums.virtualbox.org/viewtopic.php?t=3D61970=20 >=20 > My quick search turns up: > https://www.virtualbox.org/svn/vbox/trunk/src/VBox/Devices/EFI/DevEFI.cpp My quick search turns up that it's there in the code but it is not used, or it is not enabled by default, whatever (it's written in=20 https://forums.virtualbox.org/viewtopic.php?p=3D402022&sid=3D02333b87a8a2bb= a99383449fc08ca317#p402022 )=20 >=20 > aka source code thet implements efivars stored in nvram. Anyway, I could not find a file in my virtualbox directory which would look like a storage with NVRAM data. --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --h31gzZEtNLTqOjlF Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcqvxaAAoJEA2k8lmbXsY0HLYH/3qLJF7Kq1Uc4iw63YWGYe+Y dvyYXqJiqZL7hvcipDA2IQW0emPEgc0Vyi6kk80F0OlhgZTvDChX6L4hgYwB9VAm ZZhSDhWJLcQA/IimR53u84RvxFXOU1+yPqhNmuiDoPaLVfmeRS7/PVaQ2TVtSCkZ PhnkOgded02vffdkHBWlt/meh4VtSjEx4ySBF9QkeEh7lZKorx+wR7LXUEvxs8dA q3uNu+bjwma7pI7k5WGkn0a6HvZNH+RD1Dp2IHgaW4TaKkyE2le58J5/ncfDhwsY k5wWPbpIl7HGUVsE1XlJi5tg1cJoRoemEzV3R6cLryskMpfBWOPdEFhFFxqmaI0= =WNRJ -----END PGP SIGNATURE----- --h31gzZEtNLTqOjlF-- From owner-freebsd-virtualization@freebsd.org Mon Apr 8 09:52:58 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67534157AD8E for ; Mon, 8 Apr 2019 09:52:58 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6401B69FD6 for ; Mon, 8 Apr 2019 09:52:57 +0000 (UTC) (envelope-from mahesh86701997@gmail.com) Received: by mail-lf1-x133.google.com with SMTP id o199so1891507lfo.1 for ; Mon, 08 Apr 2019 02:52:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=+S+f/xLd5MTmrkgWxuGO678WLDziOZwkkjYMuBLTU34=; b=JLamkiFEvc9Gv+Tdk1c5SIXFoQkDI04YBdZJOretFQp8x4Fj8ADvBbQPtVDKU8jzOn N/qIRIm2LCm7+P0p8ZaKePDqMT8rkcQOtICcro1aq5K5zeCloWIeC4fc36I4TRqe0AMV yzutEZNhKN3OXL22JDbsCA+CgSEsBXxPA9bewJXc/X9c0tr2++lcxV2COHo05JQ2+7jF iQXEsJLpEgqBG4xL0uz0c1xROJT8H22oernsWuBEftpQESdrBstHryyyx97vKpz3Nf1F jTnoJ+N/XMi9jAIH1G8gIn6bKW6Dza2XZzUAMDZLF3zfPWLMk0o4i7YAAIgML9PWEsXZ Hp2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=+S+f/xLd5MTmrkgWxuGO678WLDziOZwkkjYMuBLTU34=; b=R2Oyy6Z3nXxb4VkzpVyRdaz5PgOhuAUXdmLowAdRWsraC7c+0cw9xYJLW/2oLrZGaQ ProgFl+PAhNwQSF6P5j2mR+1TEhTTaYO3l3m0iBOTW+QiJNwTDV6T/d2HObigSdwnUMX SuY0Ti94jO8p4YVuyEbnZHfdz+4fs+n3mdW1smzjnp65Z3PwcFkc0Obd4vr2aY2FAMNq x7mlVubds8vu/uJByxaomInplVxRf41/G+oOkN3LEA76V6LtcWotChvDYSdJpqQlvDCJ ydpu96mQeC+DS8JOJHNbMmLM0L2M2cnRMy7f6KhOsu7brUba+TZOLF3ZWO8hsypKoqWt FNuA== X-Gm-Message-State: APjAAAVhlppkMXo4iC13dXoQxmFZ04PPpOoQoiutq/R96G8shPBhsZ9N /kTndtGO7NNbZnJ2mUO7PMCoqgHCr9vVRql7o0mC+GRRodWIHQ== X-Google-Smtp-Source: APXvYqx24oFfWaeUsWYqZVQ+rN8sGXHq5NfxCrt3wfa+3LLCqz5RUhGe/bLNSCOr08eu3bSI9jib2Q2I9R0052gQMH0= X-Received: by 2002:a19:f81a:: with SMTP id a26mr15632622lff.34.1554717175736; Mon, 08 Apr 2019 02:52:55 -0700 (PDT) MIME-Version: 1.0 From: Mahesh Tudu Date: Mon, 8 Apr 2019 15:22:44 +0530 Message-ID: Subject: Regarding mentorship for GSOC 2019 To: freebsd-virtualization@freebsd.org X-Rspamd-Queue-Id: 6401B69FD6 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=JLamkiFE; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mahesh86701997@gmail.com designates 2a00:1450:4864:20::133 as permitted sender) smtp.mailfrom=mahesh86701997@gmail.com X-Spamd-Result: default: False [-6.75 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; IP_SCORE(-2.80)[ip: (-9.36), ipnet: 2a00:1450::/32(-2.38), asn: 15169(-2.18), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.95)[-0.949,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2019 09:52:58 -0000 Hello, I want to work for the project *Verification of Bhyve's instruction emulation *for GSOC 2019, but on the FreeBSD idea list for GSOC 2019 there is no mentor allotted for it till now. I also have written the proposal but I want it to be reviewed once by a mentor. If you know someone who can assist me on this project, then kindly let me know. Thank You. From owner-freebsd-virtualization@freebsd.org Mon Apr 8 12:18:56 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A68B157E988 for ; Mon, 8 Apr 2019 12:18:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2A1306F425 for ; Mon, 8 Apr 2019 12:18:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id D984C157E987; Mon, 8 Apr 2019 12:18:55 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C720E157E986 for ; Mon, 8 Apr 2019 12:18:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F3656F423 for ; Mon, 8 Apr 2019 12:18:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 986B2111DC for ; Mon, 8 Apr 2019 12:18:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x38CIsik053770 for ; Mon, 8 Apr 2019 12:18:54 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x38CIsO4053765 for virtualization@FreeBSD.org; Mon, 8 Apr 2019 12:18:54 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 230073] [hyper-v] Online Backup of FreeBSD 10.4 Guest VM fails, leaving VM in unresponsive state Date: Mon, 08 Apr 2019 12:18:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.4-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: drtr0jan@yandex.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2019 12:18:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230073 Boris Korzun changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |drtr0jan@yandex.ru --- Comment #1 from Boris Korzun --- I've faced with the issue on FreeBSD 11.2. The VM hangs while Microsoft SC DPM has been occupied by creating a backup = for another VM and has tried backup FreeBSD VM. Past backups of FreeBSD VM (every week, during year more) has been created successfully. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Apr 9 13:11:01 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BB661580B76 for ; Tue, 9 Apr 2019 13:11:01 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70DEA8CB06 for ; Tue, 9 Apr 2019 13:11:00 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=MHgIwvRIgayEWWagmx0JpN4tCJCKuyc619qFsVbXtuM=; b=VtZJAvk5i1shVTJ9VGqwFZ27JQ ocubSmPZpt+VaNW0bAyljOLg8xcLLVMnjR0xv4lEHNmzUpEoQOx4+n+CZLQE1piMmmh1Cv+XsvJ5n NMaxnovpaJjah9V8sriPXcXzf4IrzGVrWzvPoeVhrD7iEiTngfiXti5hPOj2d9Z3UmHU=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hDqWg-000Lp2-UU for freebsd-virtualization@freebsd.org; Tue, 09 Apr 2019 20:10:58 +0700 Date: Tue, 9 Apr 2019 20:10:58 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: running FreePBX SNG7 Official Distro Message-ID: <20190409131058.GA83554@admin.sibptus.ru> References: <20190408050144.GA56879@admin.sibptus.ru> <201904080739.x387d1T9054124@gndrsh.dnsmgr.net> <20190408074634.GA59930@admin.sibptus.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fdj2RfSjLxBAspz7" Content-Disposition: inline In-Reply-To: <20190408074634.GA59930@admin.sibptus.ru> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Apr 2019 13:11:01 -0000 --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Victor Sudakov wrote: > > > > I'll look into the VirtualBox directory tomorrow and report here.= =20 > > >=20 > > > I searched through my disk and was unable to find a persistant efivars > > > storage in my VirtualBox 6.0 installation.=20 > > >=20 > > > A Google search reveals some articles (rather dated I must admit) > > > stating that VirtualBox does not support NVRAM emulation for storing = efi > > > variables:=20 > > >=20 > > > https://www.virtualbox.org/ticket/14279 > > > https://forums.virtualbox.org/viewtopic.php?t=3D61970=20 > >=20 > > My quick search turns up: > > https://www.virtualbox.org/svn/vbox/trunk/src/VBox/Devices/EFI/DevEFI.c= pp >=20 > My quick search turns up that it's there in the code but it is not > used, or it is not enabled by default, whatever (it's written in=20 > https://forums.virtualbox.org/viewtopic.php?p=3D402022&sid=3D02333b87a8a2= bba99383449fc08ca317#p402022 )=20 >=20 A fairly recent note:=20 https://www.centos.org/forums/viewtopic.php?p=3D278745&sid=3De52bd17e42d253= 0f496749054b0cc174#p278745 --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --fdj2RfSjLxBAspz7 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcrJniAAoJEA2k8lmbXsY0vwcIAJbcxnz+GJoxQg1XzWcT8h3a SmTzXpCSHfWAs09G+cthhexgw935192A57wEdr77Fw0HV4+EkzxWfVhZhfXloOfs Wcktz23XsW0a3MmO1LrieXmVbQn2jQ2ZcTT10YmIoJ+yA+VdBA4VeF3J3NfxhiCB AA95aU/V2vL4zTu6DxWXMX36aHTsSRDIesnx+DD1+iX4nPjEg1CzLsR+GZ7ojWDR MT2qp73bdNnVZ/UUqeRMiaIZt4LsXJ0uDD71TvWPWsc8BK0T6BZ2a82QPO18jdA3 34n47JofHdNRLAWZCfzsl0e2XSgNQOuBb7gv+x3RzYU1zXrREIbv5qgh9eU29/0= =DsvW -----END PGP SIGNATURE----- --fdj2RfSjLxBAspz7-- From owner-freebsd-virtualization@freebsd.org Tue Apr 9 14:40:17 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E6351582A40 for ; Tue, 9 Apr 2019 14:40:17 +0000 (UTC) (envelope-from darkfiberiru@gmail.com) Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com [IPv6:2607:f8b0:4864:20::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 896DA8FC65 for ; Tue, 9 Apr 2019 14:40:16 +0000 (UTC) (envelope-from darkfiberiru@gmail.com) Received: by mail-oi1-x230.google.com with SMTP id i21so13712238oib.11 for ; Tue, 09 Apr 2019 07:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lEz4ZSK9ckvOUDSc1mrryrtpnPwS89T5sTbabddC4ns=; b=sivf3nw3Om+9j9YTrbx0u1HCIkNtSiIb9tQHCvFR1gM7X+q7zizkwlDdCLallqd1m1 +mIDe/Pi9eM9aj4i5nAm9CcrNvfqi/pTRMSmFpfPvwmIlvwLQdO8iovX52vnpQz7AoI6 V2Kw1ovLn0wkyzahYOJo52tafMei0HdQ3vQQibc3XdotqWEW/DtTC+vIociH985ynSZa kqJvNspa37OR1Ls40bTFKImUFpDFpD0fEnaaeecz5aSchJ0Ou9mcWjYkSkHswKS7Apmw s2+S6v3oOFBmob5caRbvz3cfEm9MB0qGnox1SlfoplDvwK0LPZvk/o1beiuxLIEcrTw3 rNrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lEz4ZSK9ckvOUDSc1mrryrtpnPwS89T5sTbabddC4ns=; b=rfAeSzZ/yoPR8M6fT+JIUtQ5cuQ0V0CPhAjixZnWzJxDT04J1Y5rvQO1zAORABf6pf tP7VODqTKQgxPgAK6namPulStP0DIxQCp0fvvdp3TJMtYjxrPrCnNka8BcmwCUWFEdvL kfrz3YI34kEAepKDzDVUHPhC0bHRdUiCfjEPQXITtEfePPpwalOjzsBmzYHRN+wdx+Rw VCfgsjm3tSSsi5e+VI3zVuSrixV8S4Gp3U7HXCcNjg7pz44prGRLI+usgLrxn7rEzXi1 gdOMbtpgwsjxLAgNu/poxSReVBPms48r6bDLT/aYFR4PF7K7ke75czIdsAtjxcf6JRmV bxPw== X-Gm-Message-State: APjAAAVwl+u75CiLw0Ry0WHXourDfRDz7kcdBFlnDyMRp3O26MzCS8vm xEgVLGLarZCk48931E1/z69b5Q4SA4CDa/N/8KA= X-Google-Smtp-Source: APXvYqxWQvMhv9Lcqmm6CEfJS2jmZGqw4ck0f52lWTxcycV+CrYMYsuTE8EuK52bfyMMvYqDD3f1BVhd0iBTBN2Wuz8= X-Received: by 2002:aca:c4cc:: with SMTP id u195mr20308192oif.40.1554820815580; Tue, 09 Apr 2019 07:40:15 -0700 (PDT) MIME-Version: 1.0 References: <20190328220234.GA52232@bsdpad.com> In-Reply-To: From: Nick Wolff Date: Tue, 9 Apr 2019 10:40:07 -0400 Message-ID: Subject: Re: GPU passthrough: mixed success on Linux, not yet on Windows To: Robert Crowston Cc: Ruslan Bukin , "freebsd-virtualization@freebsd.org" X-Rspamd-Queue-Id: 896DA8FC65 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=sivf3nw3; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of darkfiberiru@gmail.com designates 2607:f8b0:4864:20::230 as permitted sender) smtp.mailfrom=darkfiberiru@gmail.com X-Spamd-Result: default: False [-6.86 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; FREEMAIL_TO(0.00)[protonmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[0.3.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.86)[ip: (-9.10), ipnet: 2607:f8b0::/32(-2.95), asn: 15169(-2.18), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Apr 2019 14:40:17 -0000 Robert, I'm hoping that the set of commits done for https://reviews.freebsd.org/D19646 will fix the pcie reset problems. Apparently pcie wasn't auto retraining after reset. I don't know if connected to that review but wanted to let you know. On Wed, Apr 3, 2019 at 7:42 PM Robert Crowston via freebsd-virtualization < freebsd-virtualization@freebsd.org> wrote: > To get Windows to boot I think the only hacks I needed were in bhyve/mem.= c > (this is not production ready!) > > Diff'd against 12.0-release > > -- /tmp//sh-np.vFXFJd 2019-04-04 00:29:32.752990000 +0100 > +++ mem.c 2019-03-02 22:27:14.500906000 +0000 > @@ -101,20 +101,22 @@ > } > > static int > -mmio_rb_add(struct mmio_rb_tree *rbt, struct mmio_rb_range *new) > +mmio_rb_add(struct mmio_rb_tree *rbt, struct mmio_rb_range *new_element) > { > struct mmio_rb_range *overlap; > > - overlap =3D RB_INSERT(mmio_rb_tree, rbt, new); > + overlap =3D RB_INSERT(mmio_rb_tree, rbt, new_element); > > + printf("mmio_rb_add: %lx:%lx\n", new_element->mr_base, > new_element->mr_end); > + > if (overlap !=3D NULL) { > -#ifdef RB_DEBUG > - printf("overlap detected: new %lx:%lx, tree %lx:%lx\n", > - new->mr_base, new->mr_end, > +//#ifdef RB_DEBUG > + printf("overlap detected: new_element %lx:%lx, tree > %lx:%lx\n", > + new_element->mr_base, new_element->mr_end, > overlap->mr_base, overlap->mr_end); > -#endif > +//#endif > > - return (EEXIST); > +// return (EEXIST); > } > > return (0); > @@ -336,6 +338,8 @@ > assert((mr->flags & MEM_F_IMMUTABLE) =3D=3D 0); > RB_REMOVE(mmio_rb_tree, &mmio_rb_root, entry); > > + printf("unregister: %lx:%lx\n", mr->base, > mr->base+mr->size); > + > /* flush Per-vCPU cache */ > for (i=3D0; i < VM_MAXCPU; i++) { > if (mmio_hint[i] =3D=3D entry) > @@ -348,7 +352,12 @@ > if (entry) > free(entry); > > - return (err); > + if (err) > + fprintf( stderr, "Unregister mem errno %d for range > %lx:%lx.\n", err, > + memp->base, memp->base + memp->size ); > + > + return 0; > + //return (err); > } > > > > > =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original = Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 > On Thursday, 28 March 2019 22:02, Ruslan Bukin wrote: > > > Hi Robert: > > > > On Sun, Mar 17, 2019 at 04:22:29PM +0000, Robert Crowston via > freebsd-virtualization wrote: > > > > > Is it worth me continuing to hack away at these problems---of course > I'm happy to share anything I come up with---or is there an official > solution to GPU support in the pipe about to make my efforts redundant :)= ? > > > > Could you share your patch/hacks somewhere? > > I would like to try it with AMD graphics card and Windows. > > > > Ruslan > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Tue Apr 9 15:41:01 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 881CC153F588 for ; Tue, 9 Apr 2019 15:41:01 +0000 (UTC) (envelope-from crowston@protonmail.com) Received: from mail-40132.protonmail.ch (mail-40132.protonmail.ch [185.70.40.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.protonmail.ch", Issuer "SwissSign Server Silver CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 93F2E6B0A5 for ; Tue, 9 Apr 2019 15:41:00 +0000 (UTC) (envelope-from crowston@protonmail.com) Date: Tue, 09 Apr 2019 15:40:44 +0000 To: Nick Wolff From: Robert Crowston Cc: Ruslan Bukin , "freebsd-virtualization@freebsd.org" Reply-To: Robert Crowston Subject: Re: GPU passthrough: mixed success on Linux, not yet on Windows Message-ID: In-Reply-To: References: <20190328220234.GA52232@bsdpad.com> Feedback-ID: 2OVbcR1yHYpdkD8cgQllkFwcuMVZg_LiVMMPvptooFDfHD_03MuQO4ZaF626jWHZYFEhNR2cmIbZ53j4QGWMBQ==:Ext:ProtonMail MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=7.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,HTML_MESSAGE autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.protonmail.ch X-Rspamd-Queue-Id: 93F2E6B0A5 X-Spamd-Bar: ------- X-Spamd-Result: default: False [-7.76 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; HAS_REPLYTO(0.00)[crowston@protonmail.com]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; DKIM_TRACE(0.00)[protonmail.com:+]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; MX_GOOD(-0.01)[mailsec.protonmail.ch,mail.protonmail.ch]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:19905, ipnet:185.70.40.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[132.40.70.185.list.dnswl.org : 127.0.5.1]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=default]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; HAS_PHPMAILER_SIG(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-3.76)[ip: (-9.89), ipnet: 185.70.40.0/24(-4.90), asn: 19905(-3.92), country: US(-0.06)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Apr 2019 15:41:01 -0000 VGhhdOKAmXMgZ29vZCB0byBrbm93LCBJIHdhc27igJl0IGF3YXJlIG9mIHRoYXQuCgpTZXBhcmF0 ZWx5IEkgaGF2ZSBmb3VuZCBhIG51bWJlciBvZiBHUFVzIHRhaW50IHRoZWlyIFJPTXMgYWZ0ZXIg dGhleSBpbml0aWFsaXphdGlvbiwgcHJldmVudGluZyByZWluaXRpYWxpemF0aW9uLiBUaGVyZSBp cyBhIHJlbGF0ZWQgcHJvYmxlbSB0b28gd2l0aCB2aWRlbyBCSU9TIHNoYWRvd2luZyBpZiB5b3Ug d2FudCB0byB1c2UgdGhlIHByaW1hcnkgR1BVIGZvciBwYXNzIHRocm91Z2guIEkgYW0gbm90IHN1 cmUgaWYgdGhlIEQzIHJlc2V0IGNhbiBmaXggdGhhdC4KCkkgdW5kZXJzdGFuZCB0aGF0IHFlbXUg aGFzIGFuIG9wdGlvbiB0byBwcmVzZW50IGEgcHJlLWNhcHR1cmVkIHByaXN0aW5lIFJPTSBmcm9t IGZpbGUgYXMgdGhlIEdQVeKAmXMgUk9NIHRvIHRoZSBndWVzdC4gSSBhbSBsb29raW5nIGF0IHdo ZXRoZXIgdGhpcyBpcyBhIHBvc3NpYmxlIGFwcHJvYWNoIGluIGJoeXZlLgoKT24gVHVlLCBBcHIg OSwgMjAxOSBhdCAxNTo0MCwgTmljayBXb2xmZiA8ZGFya2ZpYmVyaXJ1QGdtYWlsLmNvbT4gd3Jv dGU6Cgo+IFJvYmVydCwKPgo+IEknbSBob3BpbmcgdGhhdCB0aGUgc2V0IG9mIGNvbW1pdHMgZG9u ZSBmb3IgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0QxOTY0NiB3aWxsIGZpeCB0aGUgcGNp ZSByZXNldCBwcm9ibGVtcy4gQXBwYXJlbnRseSBwY2llIHdhc24ndCBhdXRvIHJldHJhaW5pbmcg YWZ0ZXIgcmVzZXQuIEkgZG9uJ3Qga25vdyBpZiBjb25uZWN0ZWQgdG8gdGhhdCByZXZpZXcgYnV0 IHdhbnRlZCB0byBsZXQgeW91IGtub3cuCj4KPiBPbiBXZWQsIEFwciAzLCAyMDE5IGF0IDc6NDIg UE0gUm9iZXJ0IENyb3dzdG9uIHZpYSBmcmVlYnNkLXZpcnR1YWxpemF0aW9uIDxmcmVlYnNkLXZp cnR1YWxpemF0aW9uQGZyZWVic2Qub3JnPiB3cm90ZToKPgo+PiBUbyBnZXQgV2luZG93cyB0byBi b290IEkgdGhpbmsgdGhlIG9ubHkgaGFja3MgSSBuZWVkZWQgd2VyZSBpbiBiaHl2ZS9tZW0uYyAo dGhpcyBpcyBub3QgcHJvZHVjdGlvbiByZWFkeSEpCj4+Cj4+IERpZmYnZCBhZ2FpbnN0IDEyLjAt cmVsZWFzZQo+Pgo+PiAtLSAvdG1wLy9zaC1ucC52RlhGSmQgIDIwMTktMDQtMDQgMDA6Mjk6MzIu NzUyOTkwMDAwICswMTAwCj4+ICsrKyBtZW0uYyAgICAgICAyMDE5LTAzLTAyIDIyOjI3OjE0LjUw MDkwNjAwMCArMDAwMAo+PiBAQCAtMTAxLDIwICsxMDEsMjIgQEAKPj4gIH0KPj4KPj4gIHN0YXRp YyBpbnQKPj4gLW1taW9fcmJfYWRkKHN0cnVjdCBtbWlvX3JiX3RyZWUgKnJidCwgc3RydWN0IG1t aW9fcmJfcmFuZ2UgKm5ldykKPj4gK21taW9fcmJfYWRkKHN0cnVjdCBtbWlvX3JiX3RyZWUgKnJi dCwgc3RydWN0IG1taW9fcmJfcmFuZ2UgKm5ld19lbGVtZW50KQo+PiAgewo+PiAgICAgICAgIHN0 cnVjdCBtbWlvX3JiX3JhbmdlICpvdmVybGFwOwo+Pgo+PiAtICAgICAgIG92ZXJsYXAgPSBSQl9J TlNFUlQobW1pb19yYl90cmVlLCByYnQsIG5ldyk7Cj4+ICsgICAgICAgb3ZlcmxhcCA9IFJCX0lO U0VSVChtbWlvX3JiX3RyZWUsIHJidCwgbmV3X2VsZW1lbnQpOwo+Pgo+PiArICAgICAgIHByaW50 ZigibW1pb19yYl9hZGQ6ICVseDolbHggIiwgbmV3X2VsZW1lbnQtPm1yX2Jhc2UsIG5ld19lbGVt ZW50LT5tcl9lbmQpOwo+PiArCj4+ICAgICAgICAgaWYgKG92ZXJsYXAgIT0gTlVMTCkgewo+PiAt I2lmZGVmIFJCX0RFQlVHCj4+IC0gICAgICAgICAgICAgICBwcmludGYoIm92ZXJsYXAgZGV0ZWN0 ZWQ6IG5ldyAlbHg6JWx4LCB0cmVlICVseDolbHggIiwKPj4gLSAgICAgICAgICAgICAgICAgICAg ICBuZXctPm1yX2Jhc2UsIG5ldy0+bXJfZW5kLAo+PiArLy8jaWZkZWYgUkJfREVCVUcKPj4gKyAg ICAgICAgICAgICAgIHByaW50Zigib3ZlcmxhcCBkZXRlY3RlZDogbmV3X2VsZW1lbnQgJWx4OiVs eCwgdHJlZSAlbHg6JWx4ICIsCj4+ICsgICAgICAgICAgICAgICAgICAgICAgbmV3X2VsZW1lbnQt Pm1yX2Jhc2UsIG5ld19lbGVtZW50LT5tcl9lbmQsCj4+ICAgICAgICAgICAgICAgICAgICAgICAg b3ZlcmxhcC0+bXJfYmFzZSwgb3ZlcmxhcC0+bXJfZW5kKTsKPj4gLSNlbmRpZgo+PiArLy8jZW5k aWYKPj4KPj4gLSAgICAgICAgICAgICAgIHJldHVybiAoRUVYSVNUKTsKPj4gKy8vICAgICAgICAg ICAgIHJldHVybiAoRUVYSVNUKTsKPj4gICAgICAgICB9Cj4+Cj4+ICAgICAgICAgcmV0dXJuICgw KTsKPj4gQEAgLTMzNiw2ICszMzgsOCBAQAo+PiAgICAgICAgICAgICAgICAgYXNzZXJ0KChtci0+ ZmxhZ3MgJiBNRU1fRl9JTU1VVEFCTEUpID09IDApOwo+PiAgICAgICAgICAgICAgICAgUkJfUkVN T1ZFKG1taW9fcmJfdHJlZSwgJm1taW9fcmJfcm9vdCwgZW50cnkpOwo+Pgo+PiArICAgICAgICAg ICAgICAgcHJpbnRmKCJ1bnJlZ2lzdGVyOiAlbHg6JWx4ICIsIG1yLT5iYXNlLCBtci0+YmFzZStt ci0+c2l6ZSk7Cj4+ICsKPj4gICAgICAgICAgICAgICAgIC8qIGZsdXNoIFBlci12Q1BVIGNhY2hl ICovCj4+ICAgICAgICAgICAgICAgICBmb3IgKGk9MDsgaSA8IFZNX01BWENQVTsgaSsrKSB7Cj4+ ICAgICAgICAgICAgICAgICAgICAgICAgIGlmIChtbWlvX2hpbnRbaV0gPT0gZW50cnkpCj4+IEBA IC0zNDgsNyArMzUyLDEyIEBACj4+ICAgICAgICAgaWYgKGVudHJ5KQo+PiAgICAgICAgICAgICAg ICAgZnJlZShlbnRyeSk7Cj4+Cj4+IC0gICAgICAgcmV0dXJuIChlcnIpOwo+PiArICAgICAgIGlm IChlcnIpCj4+ICsgICAgICAgICAgICAgICBmcHJpbnRmKCBzdGRlcnIsICJVbnJlZ2lzdGVyIG1l bSBlcnJubyAlZCBmb3IgcmFuZ2UgJWx4OiVseC4gIiwgZXJyLAo+PiArICAgICAgICAgICAgICAg ICAgICAgICBtZW1wLT5iYXNlLCBtZW1wLT5iYXNlICsgbWVtcC0+c2l6ZSApOwo+PiArCj4+ICsg ICAgICAgcmV0dXJuIDA7Cj4+ICsgICAgICAgLy9yZXR1cm4gKGVycik7Cj4+ICB9Cj4+Cj4+IOKA kOKAkOKAkOKAkOKAkOKAkOKAkCBPcmlnaW5hbCBNZXNzYWdlIOKAkOKAkOKAkOKAkOKAkOKAkOKA kAo+PiBPbiBUaHVyc2RheSwgMjggTWFyY2ggMjAxOSAyMjowMiwgUnVzbGFuIEJ1a2luIDxickBi c2RwYWQuY29tPiB3cm90ZToKPj4KPj4+IEhpIFJvYmVydDoKPj4+Cj4+PiBPbiBTdW4sIE1hciAx NywgMjAxOSBhdCAwNDoyMjoyOVBNICswMDAwLCBSb2JlcnQgQ3Jvd3N0b24gdmlhIGZyZWVic2Qt dmlydHVhbGl6YXRpb24gd3JvdGU6Cj4+Pgo+Pj4gPiBJcyBpdCB3b3J0aCBtZSBjb250aW51aW5n IHRvIGhhY2sgYXdheSBhdCB0aGVzZSBwcm9ibGVtcy0tLW9mIGNvdXJzZSBJJ20gaGFwcHkgdG8g c2hhcmUgYW55dGhpbmcgSSBjb21lIHVwIHdpdGgtLS1vciBpcyB0aGVyZSBhbiBvZmZpY2lhbCBz b2x1dGlvbiB0byBHUFUgc3VwcG9ydCBpbiB0aGUgcGlwZSBhYm91dCB0byBtYWtlIG15IGVmZm9y dHMgcmVkdW5kYW50IDopPwo+Pj4KPj4+IENvdWxkIHlvdSBzaGFyZSB5b3VyIHBhdGNoL2hhY2tz IHNvbWV3aGVyZT8KPj4+IEkgd291bGQgbGlrZSB0byB0cnkgaXQgd2l0aCBBTUQgZ3JhcGhpY3Mg Y2FyZCBhbmQgV2luZG93cy4KPj4+Cj4+PiBSdXNsYW4KPj4KPj4gX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KPj4gZnJlZWJzZC12aXJ0dWFsaXphdGlvbkBm cmVlYnNkLm9yZyBtYWlsaW5nIGxpc3QKPj4gaHR0cHM6Ly9saXN0cy5mcmVlYnNkLm9yZy9tYWls bWFuL2xpc3RpbmZvL2ZyZWVic2QtdmlydHVhbGl6YXRpb24KPj4gVG8gdW5zdWJzY3JpYmUsIHNl bmQgYW55IG1haWwgdG8gImZyZWVic2QtdmlydHVhbGl6YXRpb24tdW5zdWJzY3JpYmVAZnJlZWJz ZC5vcmci From owner-freebsd-virtualization@freebsd.org Wed Apr 10 13:42:26 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C70F81583347 for ; Wed, 10 Apr 2019 13:42:25 +0000 (UTC) (envelope-from sandy@postagemeterquote.site) Received: from s177327.cloud.flynet.pro (alias9s177327.cloud.flynet.pro [193.47.34.23]) by mx1.freebsd.org (Postfix) with ESMTP id 02CCA714D0 for ; Wed, 10 Apr 2019 13:42:15 +0000 (UTC) (envelope-from sandy@postagemeterquote.site) Received: from alias1s177315.cloud.flynet.pro (alias18s177315.cloud.flynet.pro [193.47.34.219]) by s177327.cloud.flynet.pro (Postfix) with ESMTP id 8AB3F6828A for ; Wed, 10 Apr 2019 15:59:55 +0300 (IDT) Received: from m1.bizquotes.click (unknown [10.20.15.1]) by alias1s177315.cloud.flynet.pro (Postfix) with ESMTP id 97C4741D4C for ; Wed, 10 Apr 2019 15:57:18 +0300 (IDT) Received: from list.bizquotes.click (unknown [192.168.100.137]) by m1.bizquotes.click (Postfix) with ESMTP id BEC705136 for ; Wed, 10 Apr 2019 15:52:59 +0300 (IDT) DKIM-Filter: OpenDKIM Filter v2.11.0 m1.bizquotes.click BEC705136 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bizquotes.org; s=zmail; t=1554900779; bh=oXD+J/5Ru2jkliEM75WEH8sfKZ8IVrYWL27FdwZysYk=; h=To:Subject:Date:From:Reply-To:List-Unsubscribe:From; b=ZT1EI7VLRyCWN6WrAE2Zq8KFsa2Lj3DOioUnHrpH5j+zRRool+L6IPXxKs6azyNkJ c06gtXX4V99TMCi3ddhtNgjGOJlJZ79VO9qbpbST51x+U0AraHzemkjzQchTeX/NVi teUEmy2v/7uoNrXpTL8aBVvn1X5nTnMAx8VedTHY= To: freebsd-virtualization@freebsd.org Subject: Now Is Charlie's Choice's Chance To Save Up To 32% On Mail Message-ID: <1ea63a5a7559bdb1b03febdfa2c0069f@list.bizquotes.click> Date: Wed, 10 Apr 2019 12:42:40 +0000 From: "Sandy Hill" Reply-To: sandy@postagemeterquote.site MIME-Version: 1.0 X-Mailer-LID: 41,41 X-Mailer-RecptId: 34790047 X-Mailer-SID: 24034 X-Mailer-Sent-By: 1 X-Rspamd-Queue-Id: 02CCA714D0 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bizquotes.org header.s=zmail header.b=ZT1EI7VL; spf=pass (mx1.freebsd.org: domain of sandy@postagemeterquote.site designates 193.47.34.23 as permitted sender) smtp.mailfrom=sandy@postagemeterquote.site X-Spamd-Result: default: False [4.94 / 15.00]; HAS_REPLYTO(0.00)[sandy@postagemeterquote.site]; R_SPF_ALLOW(-0.20)[+Ip4:193.47.34.0/24]; TO_DN_NONE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[bizquotes.org:+]; MX_GOOD(-0.01)[mx20.postagemeterquote.site,mx10.postagemeterquote.site]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[country: RU(0.00)]; INTRODUCTION(2.00)[]; ASN(0.00)[asn:51724, ipnet:193.47.34.0/24, country:RU]; HAS_INTERSPIRE_SIG(1.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bizquotes.org:s=zmail]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.45)[0.451,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_PHPMAILER_SIG(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.93)[0.929,0]; DMARC_NA(0.00)[postagemeterquote.site]; NEURAL_SPAM_LONG(0.98)[0.980,0]; MIME_TRACE(0.00)[0:+,1:+] Content-Type: text/plain; format=flowed; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Apr 2019 13:42:26 -0000 Hi, My name is Sandy and I am with PostageMeterQuote[.]site, which keeps small businesses such as Charlie's Choice up to date on the best deals offered by nationwide postage meter vendors. Studies have shown that: * Small businesses can salvage up to 20% of postage costs a year by switching to a postage meter. * Small businesses that use a postage meter are up to 20% more productive than those who don't. Even if you are already using a postage meter at Charlie's Choice, it is always wise to compare the product you are currently using to that of other vendors. Are you paying too much? Are there additional features being offered for the same or a lesser price by other vendors? Please reply to this email with your postage meter needs to enjoy our free quote comparison service. Best regards, Sandy Hill Sandy@PostageMeterQuote.site Unsubscribe by reply back "REMOVE". From owner-freebsd-virtualization@freebsd.org Thu Apr 11 21:40:07 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9561156AF0D for ; Thu, 11 Apr 2019 21:40:07 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34F9C7789E for ; Thu, 11 Apr 2019 21:40:07 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hEhQK-000Hln-6n for freebsd-virtualization@freebsd.org; Thu, 11 Apr 2019 15:39:56 -0600 Date: Thu, 11 Apr 2019 15:39:56 -0600 From: The Doctor To: freebsd-virtualization@freebsd.org Subject: Windows 2019 server Message-ID: <20190411213956.GA67156@doctor.nl2k.ab.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: 34F9C7789E X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.17 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-0.58)[-0.577,0]; NEURAL_SPAM_MEDIUM(0.71)[0.709,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; NEURAL_SPAM_SHORT(0.87)[0.869,0]; MX_GOOD(-0.01)[cached: doctor.nl2k.ab.ca]; DMARC_POLICY_ALLOW(-0.50)[nl2k.ab.ca,quarantine]; INTRODUCTION(2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6171, ipnet:204.209.81.0/24, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-0.02)[country: CA(-0.09)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2019 21:40:07 -0000 Seems to be running slow on FreeBSD 12.0 p3. Just wondering if there are known issues. -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! From owner-freebsd-virtualization@freebsd.org Thu Apr 11 22:18:24 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 21CF3156B9AD for ; Thu, 11 Apr 2019 22:18:24 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com [IPv6:2a00:1450:4864:20::130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EBE4280839 for ; Thu, 11 Apr 2019 22:18:22 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: by mail-lf1-x130.google.com with SMTP id z9so5895158lfh.6 for ; Thu, 11 Apr 2019 15:18:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7ClPmTSLNdzQHBsRrQGqF5Z9NEjUpPmMEYIlZ5+9Yvk=; b=W9kutZS6dC7JiwZSsTDPpxH+Am7pikHdDEgYMZOFIGFZEn2HVtqjYGjm5o2funHO8I z/099eTXJt5NE5+rFs3EhJ9Q4/AcJ41LryWx7XybHir2kHutc99znM029fznJEjZrKZP WN5oA6QrhXGWz074adP3yeD6rAuBN4FC0q8xDFq8x95dAO/j52NZ4pRSZ15jixMM0O9Q PbwkL2iqxnzazkmnOQH1sM95iZJ0N0lH+rdbFWwJlsUHNDQXBDWRYr3XK3ivILXVAUof Bbcu+ECKmvzxLA4vuzXZlTdCiizaK48Q9AycJ74tQn6sMf1N5qpeUJ51+BHkW2iGqqpj tVGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7ClPmTSLNdzQHBsRrQGqF5Z9NEjUpPmMEYIlZ5+9Yvk=; b=n1haZQqfrrpr+7OHXznTa+2X0ppwyzdB0ZY9dk3N9yikNi7W0DEGFy1AXW7gVDHmL/ ZOBqiWESLvwNg1fc54m5Sh89+LuNm/14OFdFrWcWmYkwc+N7KJu87GxiuMaZFDCfZu3A 6Nqx92G0+RYCOqd8BDzXO32M4mWZJT+SD9XKKZC3olK8yR/tZKephRloq+SG/4AtPwQM qT54Hab9FRRLpnKvkXjeNhWYk8878+tBmtMH46ENAnHznvPmHW/0e0NtHIqton9ydaEs 43Wn4Qi9Kb36hl+X825Bz1jJBJ7Mw2TXif0WAucYjRn0y2EZBuAUoGC6ZL/fJ6tgQOJB rJAw== X-Gm-Message-State: APjAAAWYjLmOSx7eYSe36Rq5jTV8obRlN/rq+9j9g0xZCuC/W5hkesbV 65cGUx9g9Dbyy2JFQEHnDLYloVPTt8YKp6KRd1zJQGb+ X-Google-Smtp-Source: APXvYqzNaDLVMu7DjHBQlJKfeMHeMrYZ6O7qkrU2I6/MjuO/PuOwKyORM0E+HKMFag0OnWft5QIOJ/2ig+FYSTV+6xA= X-Received: by 2002:a19:7503:: with SMTP id y3mr27921828lfe.83.1555021101426; Thu, 11 Apr 2019 15:18:21 -0700 (PDT) MIME-Version: 1.0 References: <20190411213956.GA67156@doctor.nl2k.ab.ca> In-Reply-To: <20190411213956.GA67156@doctor.nl2k.ab.ca> From: Dustin Marquess Date: Thu, 11 Apr 2019 17:18:10 -0500 Message-ID: Subject: Re: Windows 2019 server To: The Doctor Cc: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: EBE4280839 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=W9kutZS6; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of dmarquess@gmail.com designates 2a00:1450:4864:20::130 as permitted sender) smtp.mailfrom=dmarquess@gmail.com X-Spamd-Result: default: False [-4.52 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.74)[-0.737,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.78)[ip: (-9.25), ipnet: 2a00:1450::/32(-2.39), asn: 15169(-2.18), country: US(-0.06)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[0.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; INTRODUCTION(2.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2019 22:18:24 -0000 I'm not sure if 12.0 has bhyve nmve support, but 10 & 2019 both seem to run MUCH faster when using nvme compared to ahci-hd. -Dustin On Thu, Apr 11, 2019 at 4:40 PM The Doctor via freebsd-virtualization wrote: > > Seems to be running slow on FreeBSD 12.0 p3. > > Just wondering if there are known issues. > -- > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@freebsd.org Thu Apr 11 22:54:35 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EC11156CAC7 for ; Thu, 11 Apr 2019 22:54:35 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A3B92821FC for ; Thu, 11 Apr 2019 22:54:33 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hEiaU-000PVX-Bs; Thu, 11 Apr 2019 16:54:30 -0600 Date: Thu, 11 Apr 2019 16:54:30 -0600 From: The Doctor To: Dustin Marquess Cc: FreeBSD virtualization Subject: Re: Windows 2019 server Message-ID: <20190411225430.GA96985@doctor.nl2k.ab.ca> References: <20190411213956.GA67156@doctor.nl2k.ab.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: A3B92821FC X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.07 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; NEURAL_HAM_LONG(-0.24)[-0.244,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; NEURAL_SPAM_MEDIUM(0.39)[0.386,0]; NEURAL_SPAM_SHORT(0.76)[0.760,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: doctor.nl2k.ab.ca]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[nl2k.ab.ca,quarantine]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; INTRODUCTION(2.00)[]; ASN(0.00)[asn:6171, ipnet:204.209.81.0/24, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-0.02)[country: CA(-0.09)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2019 22:54:35 -0000 On Thu, Apr 11, 2019 at 05:18:10PM -0500, Dustin Marquess wrote: > I'm not sure if 12.0 has bhyve nmve support, but 10 & 2019 both seem > to run MUCH faster when using nvme compared to ahci-hd. > > -Dustin > Here is the script I use to start the Windows server #!/bin/sh bhyve -c 4 -s 0,hostbridge -s 3,ahci-hd,/usr/vm/images/windows2019st -s 10,virtio-net,tap3 -s 31,lpc -l com1,/dev/nmdm9A -l com2,/dev/nmdm10A -s 29,fbuf,tcp=0.0.0.0:5958,w=1024,h=768,wait -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -m 4G -H -w windows2019st & So I should change ahci-cd to nvme? > On Thu, Apr 11, 2019 at 4:40 PM The Doctor via freebsd-virtualization > wrote: > > > > Seems to be running slow on FreeBSD 12.0 p3. > > > > Just wondering if there are known issues. > > -- > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! > > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! From owner-freebsd-virtualization@freebsd.org Fri Apr 12 03:10:18 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E01D1570B1F for ; Fri, 12 Apr 2019 03:10:18 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com [IPv6:2a00:1450:4864:20::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 042D68894D for ; Fri, 12 Apr 2019 03:10:17 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: by mail-lj1-x236.google.com with SMTP id r24so7479092ljg.3 for ; Thu, 11 Apr 2019 20:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=6j+0o0nNGwtzpUzzv03Qpp2TgeNWkjHE3YgLmH2lyV4=; b=pEFPUgr7Ltk29N1wJNSpJAIydpNJzdpATwOlz0bd7K+uEEve5ucRjjY4f6WSivtqFP 15jg9Hm5z53ttdQWWA6wKI5HeyoJRcdIW39V3WOAt3ZuHGOpVN6NmxhkDNZCiV30O98d owU2VErNqNF6Ds2mVrhGmGMR9ZH5bzSeQGN/Rs4SGDUDQ9PInOSoC06tH3L0YfkWXssw ZhQQ/XAtmzcvRkLpsdVxa9NPBoq80tLhAm00SMnalsx2OKFKyeQp+8bR6s/HpsfDzAqP e8UKfcu6hzhJm80UTXFV3OJU1YYTidcMy3xm7QN7TwEM1RDjCbMC9VjjgJqdQLWmPD4w VhiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=6j+0o0nNGwtzpUzzv03Qpp2TgeNWkjHE3YgLmH2lyV4=; b=cnSeT24G1qWHIFo1clCuYTiYKOVXmXqaHoLEACpjceQ04Cf0rCgDdaQPYG3D9KAwrX O49dJHaauokBJ6WCWXnhBYQEBdM68MurhP+LuUzaXp+YXiLRNAn3mduJpMEcow8vQ1lj 1Mi3nJVWYghXd056rTJT8/XyA450rFKM7oAF6iu9ViaSDOHSHy4yyDdaeX2XrACDE0i2 ZgKmXxlEsPP9iC7p4+VpdXm2qm9dCa5++iGAuxj7Xz7goQ5hG5YNkTrK439AzDfZXtde H4bWjqptNH7k9myuoqblMBsD9t2o6adBV7vOmYb84YvZzSVS4YaLrpbWTJqUdb6YubJX 4mbA== X-Gm-Message-State: APjAAAXvaitJJS3boOav5ue72KuJkJy+tYHtgRnbU7WcwsNM2rbljyW3 wjwYP5Em+Mse391zOPbdwq+bGFMmz22B8DzuvDMcMtxyIQA= X-Google-Smtp-Source: APXvYqzvwKzZV1fljY7YCtpXgkYT97Z+7M2CaBafFOjXaoLvS4lApjbH9N5jFrFmu7ntOcvPAFIvFpZOtIMG4ldgxzY= X-Received: by 2002:a2e:390c:: with SMTP id g12mr28437227lja.174.1555038615388; Thu, 11 Apr 2019 20:10:15 -0700 (PDT) MIME-Version: 1.0 References: <20190411213956.GA67156@doctor.nl2k.ab.ca> <20190411225430.GA96985@doctor.nl2k.ab.ca> In-Reply-To: <20190411225430.GA96985@doctor.nl2k.ab.ca> From: Dustin Marquess Date: Thu, 11 Apr 2019 22:10:03 -0500 Message-ID: Subject: Re: Windows 2019 server To: The Doctor Cc: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 042D68894D X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=pEFPUgr7; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of dmarquess@gmail.com designates 2a00:1450:4864:20::236 as permitted sender) smtp.mailfrom=dmarquess@gmail.com X-Spamd-Result: default: False [-4.67 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.81)[-0.812,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.87)[ip: (-9.70), ipnet: 2a00:1450::/32(-2.39), asn: 15169(-2.18), country: US(-0.06)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.983,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[6.3.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; INTRODUCTION(2.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Apr 2019 03:10:18 -0000 It's worth a shot at least to see if it works! But ya, basically the same as mine: bhyve -A -H -P -w -S -u -c sockets=3D1,cores=3D4,threads=3D2 -m ${RAM} \ -s 0,hostbridge \ -s 3,nvme,/dev/zvol/tank/vm/win2019 \ -s 4,ahci-cd,${VMROOT}/win2019/empty.iso,nocache,ro \ -s 5,passthru,130/0/0 \ -s 10,virtio-net,tap2 \ -s 20,virtio-rnd \ -s 31,lpc \ -l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd ${VM} -Dustin On Thu, Apr 11, 2019 at 5:54 PM The Doctor wrote= : > > On Thu, Apr 11, 2019 at 05:18:10PM -0500, Dustin Marquess wrote: > > I'm not sure if 12.0 has bhyve nmve support, but 10 & 2019 both seem > > to run MUCH faster when using nvme compared to ahci-hd. > > > > -Dustin > > > > Here is the script I use to start the Windows server > > > #!/bin/sh > bhyve -c 4 -s 0,hostbridge -s 3,ahci-hd,/usr/vm/images/windows2019st -s = 10,virtio-net,tap3 -s 31,lpc -l com1,/dev/nmdm9A -l com2,/dev/nmdm10A -s 29= ,fbuf,tcp=3D0.0.0.0:5958,w=3D1024,h=3D768,wait -l bootrom,/usr/local/share/= uefi-firmware/BHYVE_UEFI.fd -m 4G -H -w windows2019st & > > So I should change ahci-cd to nvme? > > > > On Thu, Apr 11, 2019 at 4:40 PM The Doctor via freebsd-virtualization > > wrote: > > > > > > Seems to be running slow on FreeBSD 12.0 p3. > > > > > > Just wondering if there are known issues. > > > -- > > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@= @nl2k.ab.ca > > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChr= ist rising! > > > https://www.empire.kred/ROOTNK?t=3D94a1f39b Look at Psalms 14 and 53= on Atheism > > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > > _______________________________________________ > > > freebsd-virtualization@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@= freebsd.org" > > -- > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2= k.ab.ca > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist = rising! > https://www.empire.kred/ROOTNK?t=3D94a1f39b Look at Psalms 14 and 53 on = Atheism > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! From owner-freebsd-virtualization@freebsd.org Fri Apr 12 12:52:08 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D181157C1DC for ; Fri, 12 Apr 2019 12:52:08 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 939C7726EB for ; Fri, 12 Apr 2019 12:52:07 +0000 (UTC) (envelope-from vas@mpeks.tomsk.su) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=uhE0qTrmIWqREOuzuB3/2FYVZSooXKdn8lAo9S7F6rY=; b=jL1lUuvlQR5AKPVvjzro3krmDj A2AvF4gaX/9FB5UCCy9vjHQTGd94gX/YZJfnUlx82ZgAFEEBY2Zk16bXTVOPHBXJiGFNXQxoSJD7O JyWOaqkgTeYL3CvQqRs2ucyq0YW4ozdkjTQeeEZUy+Xet0AqXyWqh1PXLC2eNUk7AXys=; Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hEvf3-000BBJ-Qm for freebsd-virtualization@freebsd.org; Fri, 12 Apr 2019 19:52:05 +0700 Date: Fri, 12 Apr 2019 19:52:05 +0700 From: Victor Sudakov To: freebsd-virtualization@freebsd.org Subject: Re: Windows 2019 server Message-ID: <20190412125205.GA42782@admin.sibptus.ru> References: <20190411213956.GA67156@doctor.nl2k.ab.ca> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="AhhlLboLdkugWU4S" Content-Disposition: inline In-Reply-To: <20190411213956.GA67156@doctor.nl2k.ab.ca> X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 User-Agent: Mutt/1.11.4 (2019-03-13) Sender: Victor Sudakov X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Apr 2019 12:52:08 -0000 --AhhlLboLdkugWU4S Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable The Doctor via freebsd-virtualization wrote: > Seems to be running slow on FreeBSD 12.0 p3. >=20 > Just wondering if there are known issues. I've just installed "Windows Server 2016" on 11.2-RELEASE-p9 and it is very slow. Windows 2012 R2 ran much faster AFAIR. I don't think this is the ahci-hd issue because the Novabench Windows benchmark in the guest VM shows read and write speeds over 1 GB/s, isn't it more than enough? --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --AhhlLboLdkugWU4S Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJcsIn1AAoJEA2k8lmbXsY0Bu4IALW4GU/mcTZxS0zVILrpg9W1 QQ2DBtY3vIfUoTlGjj9KPg7TxSOXEYoWbKRgrF8dNOg9IVuR3XF9CI/l72AozmLP U2f2xA6sirO8wbiDP8btP8UMX4S/XqE3zUY8X0snVB7CPrrrXVB5ecsUbVzQncVV EM6IBcvE8+aRdAbsp0IS7aqQbBm2KjsMkBVKJnh6aY6vGJ1TyV2brfBSzJT51es6 wR3JU3NeTa4vG3wyXN/WxHqI0K4w1fCFquofNE5Sdr6qDwifq/tyOkUJbzNj1CIX FHZj04fuXumsgybbSSAvZzlgMOb1MMnyrUYeZzh1zWkXr7kaXmqyv+siXpPuR+o= =xKQ6 -----END PGP SIGNATURE----- --AhhlLboLdkugWU4S-- From owner-freebsd-virtualization@freebsd.org Fri Apr 12 13:08:30 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6DEB4157C586 for ; Fri, 12 Apr 2019 13:08:30 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1E4A372D69 for ; Fri, 12 Apr 2019 13:08:28 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hEvur-0002XJ-Re; Fri, 12 Apr 2019 07:08:25 -0600 Date: Fri, 12 Apr 2019 07:08:25 -0600 From: The Doctor To: Dustin Marquess Cc: FreeBSD virtualization Subject: Re: Windows 2019 server Message-ID: <20190412130825.GA5886@doctor.nl2k.ab.ca> References: <20190411213956.GA67156@doctor.nl2k.ab.ca> <20190411225430.GA96985@doctor.nl2k.ab.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: 1E4A372D69 X-Spamd-Bar: / X-Spamd-Result: default: False [0.66 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.06)[-0.056,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; NEURAL_HAM_LONG(-0.22)[-0.218,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: doctor.nl2k.ab.ca]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[nl2k.ab.ca,quarantine]; NEURAL_HAM_SHORT(-0.24)[-0.239,0]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; INTRODUCTION(2.00)[]; ASN(0.00)[asn:6171, ipnet:204.209.81.0/24, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-0.02)[country: CA(-0.09)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Apr 2019 13:08:30 -0000 On Thu, Apr 11, 2019 at 10:10:03PM -0500, Dustin Marquess wrote: > It's worth a shot at least to see if it works! > > But ya, basically the same as mine: > > bhyve -A -H -P -w -S -u -c sockets=1,cores=4,threads=2 -m ${RAM} \ > -s 0,hostbridge \ > -s 3,nvme,/dev/zvol/tank/vm/win2019 \ > -s 4,ahci-cd,${VMROOT}/win2019/empty.iso,nocache,ro \ > -s 5,passthru,130/0/0 \ > -s 10,virtio-net,tap2 \ > -s 20,virtio-rnd \ > -s 31,lpc \ > -l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd ${VM} > > -Dustin I will try a bit of a mix. Also Do the same for the Linux emulations such as Scientific, Fedora, Centos, Debian and Ubuntu and ORacle? BTW rational for the -w , your -c content your passthru and the virtio-rnd? > > On Thu, Apr 11, 2019 at 5:54 PM The Doctor wrote: > > > > On Thu, Apr 11, 2019 at 05:18:10PM -0500, Dustin Marquess wrote: > > > I'm not sure if 12.0 has bhyve nmve support, but 10 & 2019 both seem > > > to run MUCH faster when using nvme compared to ahci-hd. > > > > > > -Dustin > > > > > > > Here is the script I use to start the Windows server > > > > > > #!/bin/sh > > bhyve -c 4 -s 0,hostbridge -s 3,ahci-hd,/usr/vm/images/windows2019st -s 10,virtio-net,tap3 -s 31,lpc -l com1,/dev/nmdm9A -l com2,/dev/nmdm10A -s 29,fbuf,tcp=0.0.0.0:5958,w=1024,h=768,wait -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -m 4G -H -w windows2019st & > > > > So I should change ahci-cd to nvme? > > > > > > > On Thu, Apr 11, 2019 at 4:40 PM The Doctor via freebsd-virtualization > > > wrote: > > > > > > > > Seems to be running slow on FreeBSD 12.0 p3. > > > > > > > > Just wondering if there are known issues. > > > > -- > > > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca > > > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! > > > > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism > > > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > > > _______________________________________________ > > > > freebsd-virtualization@freebsd.org mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > > > -- > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! > > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! From owner-freebsd-virtualization@freebsd.org Fri Apr 12 14:57:07 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C4D7157E92D for ; Fri, 12 Apr 2019 14:57:07 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DFD147686F for ; Fri, 12 Apr 2019 14:56:55 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x3CEunek076935; Fri, 12 Apr 2019 07:56:49 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x3CEumsi076934; Fri, 12 Apr 2019 07:56:48 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201904121456.x3CEumsi076934@gndrsh.dnsmgr.net> Subject: Re: Windows 2019 server In-Reply-To: <20190412130825.GA5886@doctor.nl2k.ab.ca> To: The Doctor Date: Fri, 12 Apr 2019 07:56:48 -0700 (PDT) CC: Dustin Marquess , FreeBSD virtualization X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: DFD147686F X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [5.64 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.76)[0.759,0]; RCVD_TLS_LAST(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.96)[0.961,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[gndrsh.dnsmgr.net,gndrsh.dnsmgr.net]; NEURAL_SPAM_LONG(1.00)[0.996,0]; IP_SCORE(0.03)[ip: (0.12), ipnet: 69.59.192.0/19(0.06), asn: 13868(0.04), country: US(-0.06)]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; INTRODUCTION(2.00)[]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Apr 2019 14:57:07 -0000 > On Thu, Apr 11, 2019 at 10:10:03PM -0500, Dustin Marquess wrote: > > It's worth a shot at least to see if it works! > > > > But ya, basically the same as mine: > > > > bhyve -A -H -P -w -S -u -c sockets=1,cores=4,threads=2 -m ${RAM} \ > > -s 0,hostbridge \ > > -s 3,nvme,/dev/zvol/tank/vm/win2019 \ > > -s 4,ahci-cd,${VMROOT}/win2019/empty.iso,nocache,ro \ > > -s 5,passthru,130/0/0 \ > > -s 10,virtio-net,tap2 \ > > -s 20,virtio-rnd \ > > -s 31,lpc \ > > -l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd ${VM} > > > > -Dustin > > I will try a bit of a mix. Also Do the same for the Linux emulations > such as Scientific, Fedora, Centos, Debian and Ubuntu and ORacle? > > BTW rational for the -w , your -c content your passthru and the virtio-rnd? The -c context is done most likely for license reasons, without specifying topology the default for -c 8 would be 8 sockets of 1 core of 1 thread, that would require a high end server win2019 license. By building this as a 8 threads in a single socket you get to use a single socket windows license. This is one of the major reasons that I added the complex topology specification to the -c option. > > On Thu, Apr 11, 2019 at 5:54 PM The Doctor wrote: > > > > > > On Thu, Apr 11, 2019 at 05:18:10PM -0500, Dustin Marquess wrote: > > > > I'm not sure if 12.0 has bhyve nmve support, but 10 & 2019 both seem > > > > to run MUCH faster when using nvme compared to ahci-hd. > > > > > > > > -Dustin > > > > > > > > > > Here is the script I use to start the Windows server > > > > > > > > > #!/bin/sh > > > bhyve -c 4 -s 0,hostbridge -s 3,ahci-hd,/usr/vm/images/windows2019st -s 10,virtio-net,tap3 -s 31,lpc -l com1,/dev/nmdm9A -l com2,/dev/nmdm10A -s 29,fbuf,tcp=0.0.0.0:5958,w=1024,h=768,wait -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -m 4G -H -w windows2019st & > > > > > > So I should change ahci-cd to nvme? > > > > > > > > > > On Thu, Apr 11, 2019 at 4:40 PM The Doctor via freebsd-virtualization > > > > wrote: > > > > > > > > > > Seems to be running slow on FreeBSD 12.0 p3. > > > > > > > > > > Just wondering if there are known issues. > > > > > -- > > > > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca > > > > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! > > > > > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism > > > > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > > > > _______________________________________________ > > > > > freebsd-virtualization@freebsd.org mailing list > > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > > > > > -- > > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca > > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! > > > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism > > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > -- > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Fri Apr 12 23:25:12 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1793B1588D24 for ; Fri, 12 Apr 2019 23:25:12 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D452991C44 for ; Fri, 12 Apr 2019 23:25:10 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: by mail-lf1-x12f.google.com with SMTP id r25so8662045lfn.13 for ; Fri, 12 Apr 2019 16:25:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=IAnROOu7r0kka112gi6+33AvbYA/zp6VuOTj6LlLC/E=; b=mzZGLkcAgalIm6dvN/MxrPA1N7Q9fjJPW+XEmucFKCfCZbIKusPVOBfCU09h1V0A3V oZeeuRSgzmgZUURMJafjHZDw23xm/ihWB5edIeFLg61mFRN4OYaL9PEwybOnFIX5V94W sucC0chbg3xy/ujYZ2FhtkbB5HFlfjVLt/+bCR6NNPN88kYFZZBCowi2IrMGjPafnCO5 AVKeYCfy7vr7tlvUMUSxLVmn2wqBcZDCS/92jJvFaVeId+YogfI3bHWDrQ5Tk+u/x31K 1wO2Q+6thulykE26W8YxcUyTJgoB7nMoaMy8NFqby9RDRhfEldEPG7ZuxoEqLddH5/pe o8+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=IAnROOu7r0kka112gi6+33AvbYA/zp6VuOTj6LlLC/E=; b=JEApWDHlHe0P83g+DXYVvFqBBriCTQfnJAP7JR2xK8HPHxpmUmgjtWtz4qV0y1Io9A UbC6123dzSCLEMnqGdOdw6VK/PgwZyqM6h8hgr7zkZwPQ3ULojIhX1CzyHjxYZuNqnSY 5hlXH4UHOHvkE158F3xiTzE8nZOgSsWFkiupnW+hl2jnnqdI32ppg3CmPzbZ56xf98Hr 9ZxZ5ouUFSy08PBil03fqc991i1aoPUv5bUlEQaORjyuaOhHmMZDglC5sZTPZ8wD9/KD LVZ8Zn/Obm0XzFLJwUUtzeubwD0ALpOneZjp+200z0y4QoBNVglhi/LOsdP5iilebj52 4+0Q== X-Gm-Message-State: APjAAAUgpDJbychccsf5SrYLf478QFppLLyOKCsyCQVdppyDKaAvd94d wbG2p9a++9QUKmDfKE0NWllGJY0lJj7u3NPT8t/6mnXBEIY= X-Google-Smtp-Source: APXvYqxIdt0QBOSUc/Yld+TgsUYLbC2GyPAU4KiAe+68qLKX0yQzgaki0xN+rNbemfLLi9+gNNBgSofPK1BGJo9oQ/4= X-Received: by 2002:ac2:4192:: with SMTP id z18mr30623338lfh.96.1555111508381; Fri, 12 Apr 2019 16:25:08 -0700 (PDT) MIME-Version: 1.0 References: <20190412130825.GA5886@doctor.nl2k.ab.ca> <201904121456.x3CEumsi076934@gndrsh.dnsmgr.net> In-Reply-To: <201904121456.x3CEumsi076934@gndrsh.dnsmgr.net> From: Dustin Marquess Date: Fri, 12 Apr 2019 18:24:57 -0500 Message-ID: Subject: Re: Windows 2019 server To: "Rodney W. Grimes" Cc: The Doctor , FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D452991C44 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=mzZGLkcA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of dmarquess@gmail.com designates 2a00:1450:4864:20::12f as permitted sender) smtp.mailfrom=dmarquess@gmail.com X-Spamd-Result: default: False [-4.78 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.79)[ip: (-9.33), ipnet: 2a00:1450::/32(-2.39), asn: 15169(-2.19), country: US(-0.06)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[f.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; INTRODUCTION(2.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Apr 2019 23:25:12 -0000 Indeed! As for the other two... the virtio-rnd I pass through to all of my capable VMs, as VMs usually have a tougher time to collect entropy, so I figure it can't hurt. The PCI passthrough I just forgot to remove from my example before I sent it. I have a dual-port 16C950 PCIe card that I passthrough to Windows to run u-blox u-center :). -Dustin On Fri, Apr 12, 2019 at 9:57 AM Rodney W. Grimes wrote: > > > On Thu, Apr 11, 2019 at 10:10:03PM -0500, Dustin Marquess wrote: > > > It's worth a shot at least to see if it works! > > > > > > But ya, basically the same as mine: > > > > > > bhyve -A -H -P -w -S -u -c sockets=3D1,cores=3D4,threads=3D2 -m ${RAM= } \ > > > -s 0,hostbridge \ > > > -s 3,nvme,/dev/zvol/tank/vm/win2019 \ > > > -s 4,ahci-cd,${VMROOT}/win2019/empty.iso,nocache,ro \ > > > -s 5,passthru,130/0/0 \ > > > -s 10,virtio-net,tap2 \ > > > -s 20,virtio-rnd \ > > > -s 31,lpc \ > > > -l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd= ${VM} > > > > > > -Dustin > > > > I will try a bit of a mix. Also Do the same for the Linux emulations > > such as Scientific, Fedora, Centos, Debian and Ubuntu and ORacle? > > > > BTW rational for the -w , your -c content your passthru and the virtio= -rnd? > > The -c context is done most likely for license reasons, > without specifying topology the default for -c 8 would > be 8 sockets of 1 core of 1 thread, that would require > a high end server win2019 license. By building this > as a 8 threads in a single socket you get to use a > single socket windows license. This is one of the major > reasons that I added the complex topology specification > to the -c option. > > > > On Thu, Apr 11, 2019 at 5:54 PM The Doctor = wrote: > > > > > > > > On Thu, Apr 11, 2019 at 05:18:10PM -0500, Dustin Marquess wrote: > > > > > I'm not sure if 12.0 has bhyve nmve support, but 10 & 2019 both s= eem > > > > > to run MUCH faster when using nvme compared to ahci-hd. > > > > > > > > > > -Dustin > > > > > > > > > > > > > Here is the script I use to start the Windows server > > > > > > > > > > > > #!/bin/sh > > > > bhyve -c 4 -s 0,hostbridge -s 3,ahci-hd,/usr/vm/images/windows2019s= t -s 10,virtio-net,tap3 -s 31,lpc -l com1,/dev/nmdm9A -l com2,/dev/nmdm10A= -s 29,fbuf,tcp=3D0.0.0.0:5958,w=3D1024,h=3D768,wait -l bootrom,/usr/local/= share/uefi-firmware/BHYVE_UEFI.fd -m 4G -H -w windows2019st & > > > > > > > > So I should change ahci-cd to nvme? > > > > > > > > > > > > > On Thu, Apr 11, 2019 at 4:40 PM The Doctor via freebsd-virtualiza= tion > > > > > wrote: > > > > > > > > > > > > Seems to be running slow on FreeBSD 12.0 p3. > > > > > > > > > > > > Just wondering if there are known issues. > > > > > > -- > > > > > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici d= octor@@nl2k.ab.ca > > > > > > Yahweh, Queen & country!Never Satan President Republic!Beware A= ntiChrist rising! > > > > > > https://www.empire.kred/ROOTNK?t=3D94a1f39b Look at Psalms 14 = and 53 on Atheism > > > > > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > > > > > _______________________________________________ > > > > > > freebsd-virtualization@freebsd.org mailing list > > > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualizati= on > > > > > > To unsubscribe, send any mail to "freebsd-virtualization-unsubs= cribe@freebsd.org" > > > > > > > > -- > > > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici docto= r@@nl2k.ab.ca > > > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiC= hrist rising! > > > > https://www.empire.kred/ROOTNK?t=3D94a1f39b Look at Psalms 14 and = 53 on Atheism > > > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > > > -- > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@n= l2k.ab.ca > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChris= t rising! > > https://www.empire.kred/ROOTNK?t=3D94a1f39b Look at Psalms 14 and 53 o= n Atheism > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" > > > > -- > Rod Grimes rgrimes@freebs= d.org