From owner-freebsd-virtualization@freebsd.org Thu Sep 10 07:30:43 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AD9653EDBAB for ; Thu, 10 Sep 2020 07:30:43 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from vito-vars.onthenet.com.au (vito-vars.OntheNet.com.au [203.13.68.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bn9Xf6rtQz4ZpX for ; Thu, 10 Sep 2020 07:30:42 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by vito-vars.onthenet.com.au (Postfix) with ESMTPS id CF1062083928 for ; Thu, 10 Sep 2020 17:30:37 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id B77D220AE980 for ; Thu, 10 Sep 2020 17:30:37 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id ACC05209C021 for ; Thu, 10 Sep 2020 17:30:37 +1000 (AEST) Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 5DItxeJT2tyj for ; Thu, 10 Sep 2020 17:30:37 +1000 (AEST) Received: from MacBook-Air-4.local (CPE-120-29-42-110.dsl.OntheNet.net [120.29.42.110]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 0A41B209C01A; Thu, 10 Sep 2020 17:30:37 +1000 (AEST) Subject: Re: Nested Virtualization Under KVM Hypervisor To: Pete Wright Cc: FreeBSD virtualization References: <5eacd0a5-89bd-cc98-6c5a-d5490e33b100@freebsd.org> <5e62480f-14cb-5963-f753-8562aab381e3@nomadlogic.org> From: Peter Grehan Message-ID: Date: Thu, 10 Sep 2020 17:30:36 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <5e62480f-14cb-5963-f753-8562aab381e3@nomadlogic.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=bM+N3M2Z c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=jXL/WSj5nX36MkVNSbrI6Q==:17 a=IkcTkHD0fZMA:10 a=reM5J-MqmosA:10 a=6I5d2MoRAAAA:8 a=DYosF9_QlMOk_I5_GGcA:9 a=QEXdDO2ut3YA:10 a=IjZwj45LgO3ly-622nXo:22 X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=LtAJNkVc c=1 sm=1 tr=0 a=eF5wA05EhuPcU0j7PNT2Vg==:117 a=jXL/WSj5nX36MkVNSbrI6Q==:17 a=IkcTkHD0fZMA:10 a=reM5J-MqmosA:10 a=6I5d2MoRAAAA:8 a=DYosF9_QlMOk_I5_GGcA:9 a=QEXdDO2ut3YA:10 a=IjZwj45LgO3ly-622nXo:22 X-Rspamd-Queue-Id: 4Bn9Xf6rtQz4ZpX X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:9313, ipnet:203.13.68.0/23, country:AU]; local_wl_from(0.00)[freebsd.org] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Sep 2020 07:30:43 -0000 Hi Pete, > Sep 10 03:25:01 bells kernel: vmx_init: processor does not support=20 > desired basic capabilities ^^^ This is what's causing the issues, and is covered in the first=20 bug report I mentioned=20 (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203994) Would you be able to dump the MSRs for the FreeBSD guest as mentioned=20 in the bug report ? > I also received this input from the ISP, they are running this kernel: > =C2=A04.15.0-47-generic #50~16.04.1-Ubuntu SMP I'm now wondering if this is too old a kernel :( > reading through that ticket i set this in loader.conf: > hw.vmm.vmx.use_apic_vid=3D0 >=20 > and verified via sysctl: > $ sysctl hw.vmm.vmx.cap.virtual_interrupt_delivery > hw.vmm.vmx.cap.virtual_interrupt_delivery: 0 > $ >=20 > since you mention this is a possible bug with KVM is there some useful=20 > information i can relay to my ISP (they are a BSD friendly vendor) that= =20 > they can try on their end as a workaround, or perhaps something they=20 > ping the linux/kvm community about? As above, this isn't the issue for this particular case: it manifests=20 as a hypervisor guest immediately exiting when started. later, Peter.