From owner-freebsd-virtualization@freebsd.org Sun Oct 18 21:00:40 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 082F1A181A8 for ; Sun, 18 Oct 2015 21:00:40 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D75A78AB for ; Sun, 18 Oct 2015 21:00:39 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9IL0dod089450 for ; Sun, 18 Oct 2015 21:00:39 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201510182100.t9IL0dod089450@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-virtualization@FreeBSD.org Subject: Problem reports for freebsd-virtualization@FreeBSD.org that need special attention X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 Date: Sun, 18 Oct 2015 21:00:39 +0000 Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2015 21:00:40 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 202321 | [bhyve,patch] More verbose error reporting in bhy New | 202322 | [bhyve,patch] add option to have bhyve write its 2 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Tue Oct 20 11:18:43 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1BF3A19CCB for ; Tue, 20 Oct 2015 11:18:43 +0000 (UTC) (envelope-from bsam@passap.ru) Received: from forward18p.cmail.yandex.net (forward18p.cmail.yandex.net [IPv6:2a02:6b8:0:1465::ab]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9AA0B81 for ; Tue, 20 Oct 2015 11:18:43 +0000 (UTC) (envelope-from bsam@passap.ru) Received: from smtp17.mail.yandex.net (smtp17.mail.yandex.net [IPv6:2a02:6b8:0:1402::118]) by forward18p.cmail.yandex.net (Yandex) with ESMTP id 5114E2133E for ; Tue, 20 Oct 2015 14:18:30 +0300 (MSK) Received: from smtp17.mail.yandex.net (localhost [127.0.0.1]) by smtp17.mail.yandex.net (Yandex) with ESMTP id 15D5019003C1 for ; Tue, 20 Oct 2015 14:18:29 +0300 (MSK) Received: by smtp17.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id NppDBAm66x-ITmiB3xb; Tue, 20 Oct 2015 14:18:29 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-ForeignMX: US Subject: Re: libvirt + zvol + bhyve: no storage pool with matching name To: freebsd-virtualization@freebsd.org References: <562182B3.9040601@passap.ru> From: Boris Samorodov Message-ID: <56262305.906@passap.ru> Date: Tue, 20 Oct 2015 14:18:29 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <562182B3.9040601@passap.ru> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 11:18:44 -0000 17.10.2015 02:05, Boris Samorodov пишет: > Hi All, > > I try to use libvirt to run a zvol-backed vm. The vm starts just fine > if run from a command line. But libvirt does not create a domain > (raw file images work though). Here is the output: > ----- > % virsh create test.xml > error: Failed to create domain from test.xml > error: Storage pool not found: no storage pool with matching name 'zroot' > ----- > > The relevant part from the config: > ----- > > OK, I figured this out. Source pool here is a libvirt pool, not a zfs one. > > > ----- > > The host: > ----- > zfs get type,volmode zroot/test > NAME PROPERTY VALUE SOURCE > zroot/test type volume - > zroot/test volmode dev local > > % uname -a > FreeBSD bb055.bsnet 11.0-CURRENT FreeBSD 11.0-CURRENT #2 r289095: Sat > Oct 10 05:10:17 MSK 2015 bsam@bb055.bsnet:/usr/obj/usr/src/sys/BB64X > amd64 > > % pkg info -x virt > libvirt-1.2.20 > ----- > > Is it just me, a bug or something else? > > Thanks! -- WBR, Boris Samorodov (bsam) FreeBSD Committer, http://www.FreeBSD.org The Power To Serve From owner-freebsd-virtualization@freebsd.org Tue Oct 20 23:48:06 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8481FA15C6C for ; Tue, 20 Oct 2015 23:48:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 711A71383 for ; Tue, 20 Oct 2015 23:48:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9KNm6AV032449 for ; Tue, 20 Oct 2015 23:48:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203884] bhyve exits with assert when vcpu already in use Date: Tue, 20 Oct 2015 23:48:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 23:48:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203884 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org Keywords| |patch -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Tue Oct 20 23:49:38 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BBF7A15D3A for ; Tue, 20 Oct 2015 23:49:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 083621628 for ; Tue, 20 Oct 2015 23:49:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9KNnbXC034024 for ; Tue, 20 Oct 2015 23:49:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203643] [bhyve] NetBSD causes bhyve to exit on AMD processors Date: Tue, 20 Oct 2015 23:49:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 23:49:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203643 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|emulation@FreeBSD.org |freebsd-virtualization@Free | |BSD.org --- Comment #1 from Mark Linimon --- Canonicalize assignment. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Tue Oct 20 23:50:07 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB70EA15D9F for ; Tue, 20 Oct 2015 23:50:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A7E0417CF for ; Tue, 20 Oct 2015 23:50:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9KNo7xc034625 for ; Tue, 20 Oct 2015 23:50:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 200859] [bhyve] vbsc_ident buffer accessed out of bounds Date: Tue, 20 Oct 2015 23:50:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 23:50:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200859 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org --- Comment #2 from Mark Linimon --- Canonicalize assignment. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Tue Oct 20 23:50:23 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB264A15DF7 for ; Tue, 20 Oct 2015 23:50:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D7C00188A for ; Tue, 20 Oct 2015 23:50:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9KNoN0p034950 for ; Tue, 20 Oct 2015 23:50:23 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 200397] [bhyve] bhyve crashes in Linux KVM on AMD Host Date: Tue, 20 Oct 2015 23:50:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 23:50:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200397 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org --- Comment #4 from Mark Linimon --- Canonicalize assignment. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Wed Oct 21 02:19:26 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7997BA1A501 for ; Wed, 21 Oct 2015 02:19:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5BF34C20 for ; Wed, 21 Oct 2015 02:19:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9L2JQPK054296 for ; Wed, 21 Oct 2015 02:19:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203884] bhyve exits with assert when vcpu already in use Date: Wed, 21 Oct 2015 02:19:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: will@worrbase.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 02:19:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203884 --- Comment #2 from will@worrbase.com --- Wow, that's the most embarrassing mistake I've made to date :) I've updated the patch not to be braindead. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Wed Oct 21 02:19:47 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F749A1A51D for ; Wed, 21 Oct 2015 02:19:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B2F3C70 for ; Wed, 21 Oct 2015 02:19:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9L2JlYN059911 for ; Wed, 21 Oct 2015 02:19:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203884] bhyve exits with assert when vcpu already in use Date: Wed, 21 Oct 2015 02:19:47 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: will@worrbase.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 02:19:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203884 will@worrbase.com changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162227|0 |1 is obsolete| | --- Comment #3 from will@worrbase.com --- Created attachment 162264 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=162264&action=edit bhyverun.c patch -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Wed Oct 21 04:29:44 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25F0AA1A875 for ; Wed, 21 Oct 2015 04:29:44 +0000 (UTC) (envelope-from corys@ixsystems.com) Received: from barracuda.ixsystems.com (mail.ixsystems.com [69.198.165.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F19B7CC5 for ; Wed, 21 Oct 2015 04:29:43 +0000 (UTC) (envelope-from corys@ixsystems.com) X-ASG-Debug-ID: 1445401782-08ca040e84830e0001-XWyayz Received: from mail.iXsystems.com ([10.2.55.1]) by barracuda.ixsystems.com with ESMTP id BVggEfD92u6ZpBzY (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 20 Oct 2015 21:29:42 -0700 (PDT) X-Barracuda-Envelope-From: corys@ixsystems.com X-Barracuda-RBL-Trusted-Forwarder: 10.2.55.1 X-ASG-Whitelist: Client Received: from roundcube.ixsystems.com (unknown [10.2.55.10]) by mail.iXsystems.com (Postfix) with ESMTPA id 61AE584AE9 for ; Tue, 20 Oct 2015 21:29:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1445401782; bh=FARsa3KQY/s3p/bb90kfQa04hnjNi+wikPINOVFhD0o=; h=Date:From:To:Subject:In-Reply-To:References; b=Jqa6A1DZOg050f0whwKK+ELJcGocwqExWeKcUt7+FoVHQ2csqaeC9Up9HZp9Mn/Kw EHAYtX8Lh2dIaHPnwp8n9dDOupVpyWIJm0SJDiq3foW3KAoCB4Z5qXlHdOcw5Z/qf+ IyiP625ugIgyNpnrK++55jGnhgZhVcHnhT58OvWw= X-Barracuda-RBL-Trusted-Forwarder: 10.2.55.10 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 20 Oct 2015 21:40:42 -0700 From: Cory Smelosky To: freebsd-virtualization@freebsd.org Subject: Issues with bhyve under KVM In-Reply-To: <061b9d92bf351baa9263279e86513861@ixsystems.com> X-ASG-Orig-Subj: Issues with bhyve under KVM References: <061b9d92bf351baa9263279e86513861@ixsystems.com> Message-ID: <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> X-Sender: corys@ixsystems.com User-Agent: Roundcube Webmail/1.1.3 X-Barracuda-Connect: UNKNOWN[10.2.55.1] X-Barracuda-Start-Time: 1445401782 X-Barracuda-Encrypted: DHE-RSA-CAMELLIA256-SHA X-Barracuda-URL: https://10.2.0.41:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at ixsystems.com X-Barracuda-BRTS-Status: 1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 04:29:44 -0000 Hey all, I am utterly confused by this one: I am unable to load vmm.ko nested in KVM. (10.2 on a 2x E5-2620v2 board, linux kernel 4.x for host, kvm/qemu 2.4, proxmox 4.0. Nested bool is set true in module config). `dmesg` prints a message saying that VT-x is "disabled by BIOS" (I passed -L and -bios pointed at OVMF just in case it was a strange SEABIOS compile option) to no change. -cpu is set to host, I have also tried passing -cpu qemu64,+vmx manually to no change. (`kldload vmm` does agree - it says features are missing) Linux appears to have no problems with bringing kvm up nested (I tested kvm -enable-kvm as well to no complaints). What am I missing? model name : Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz stepping : 4 microcode : 0x1 cpu MHz : 2099.998 cache size : 15360 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm arat xsaveopt tpr_shadow vnmi flexpriority ept fsgsbase tsc_adjust smep erms (note: this is the VM) Verbose boot info: Oct 14 06:13:10 kernel: CPU: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (2100.02-MHz K8-class CPU) Oct 14 06:13:10 kernel: Origin="GenuineIntel" Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 Oct 14 06:13:10 kernel: Features=0xf83fbff Oct 14 06:13:10 kernel: Features2=0xffba2223 Oct 14 06:13:10 kernel: AMD Features=0x2c100800 Oct 14 06:13:10 kernel: AMD Features2=0x1 Oct 14 06:13:10 kernel: Structured Extended Features=0x202 Oct 14 06:13:10 kernel: XSAVE Features=0x1 Oct 14 06:13:10 kernel: VT-x: (disabled in BIOS) Basic Features=0x981000 Oct 14 06:13:10 kernel: Pin-Based Controls=0xff Oct 14 06:13:10 kernel: Primary Processor Controls=0xf7f9fffe Oct 14 06:13:10 kernel: Secondary Processor Controls=0x3db Oct 14 06:13:10 kernel: Exit Controls=0x981000 Oct 14 06:13:10 kernel: Entry Controls=0x981000 Oct 14 06:13:10 kernel: EPT Features=0x4114040 Oct 14 06:13:10 kernel: VPID Features=0x0 Oct 14 06:13:10 kernel: Data TLB: 1 GByte pages, 4-way set associative, 4 entries Oct 14 06:13:10 kernel: Data TLB: 4 KB pages, 4-way set associative, 64 entries Oct 14 06:13:10 kernel: Instruction TLB: 2M/4M pages, fully associative, 8 entries Oct 14 06:13:10 kernel: Instruction TLB: 4KByte pages, 4-way set associative, 64 entries Oct 14 06:13:10 kernel: 64-Byte prefetching Oct 14 06:13:10 kernel: Shared 2nd-Level TLB: 4 KByte pages, 4-way associative, 512 entries Oct 14 06:13:10 kernel: L2 cache: 256 kbytes, 8-way associative, 64 bytes/line Oct 14 06:13:10 kernel: Hypervisor: Origin = "KVMKVMKVM" Oct 14 06:13:10 kernel: real memory = 4294967296 (4096 MB) Oct 14 06:13:10 kernel: Physical memory chunk(s): Oct 14 06:13:19 kernel: domain=0, bus=0, slot=31, func=3 Oct 14 06:13:19 kernel: class=0c-05-00, hdrtype=0x00, mfdev=1 Oct 14 06:13:19 kernel: cmdreg=0x0103, statreg=0x0000, cachelnsz=0 (dwords) Oct 14 06:13:19 kernel: lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) Oct 14 06:13:19 kernel: intpin=a, irq=16 Oct 14 06:13:19 kernel: pci0:0:31:3: reprobing on driver added Oct 14 06:13:19 kernel: pci1: driver added Oct 14 06:13:19 kernel: pci2: driver added Oct 14 06:13:19 kernel: pci3: driver added Oct 14 06:13:19 kernel: pci4: driver added Oct 14 06:13:19 kernel: pci5: driver added Oct 14 06:13:19 kernel: pci6: driver added Oct 14 06:13:19 kernel: pci7: driver added Oct 14 06:13:19 kernel: pci8: driver added Oct 14 06:13:19 kernel: vmx_init: processor does not support desired basic capabilities Oct 14 06:13:19 kernel: module_register_init: MOD_LOAD (vmm, 0xffffffff81a16fd0, 0) error 2 Thanks! -- -- Cory Smelosky iXsystems From owner-freebsd-virtualization@freebsd.org Wed Oct 21 06:57:52 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98EDEA1AF00 for ; Wed, 21 Oct 2015 06:57:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 85D7810C0 for ; Wed, 21 Oct 2015 06:57:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9L6vqUG086075 for ; Wed, 21 Oct 2015 06:57:52 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203884] bhyve exits with assert when vcpu already in use Date: Wed, 21 Oct 2015 06:57:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 06:57:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203884 --- Comment #4 from Garrett Cooper,425-314-3911 --- (In reply to will from comment #2) Much better :). grehan/neel: what do you think about the attached patch? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Thu Oct 22 05:35:51 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0BDAA1BE47 for ; Thu, 22 Oct 2015 05:35:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD0351F45 for ; Thu, 22 Oct 2015 05:35:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9M5ZpqM069652 for ; Thu, 22 Oct 2015 05:35:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203884] bhyve exits with assert when vcpu already in use Date: Thu, 22 Oct 2015 05:35:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: grehan@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 05:35:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203884 --- Comment #5 from Peter Grehan --- Looks fine to me. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Thu Oct 22 06:15:40 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC132A1C334 for ; Thu, 22 Oct 2015 06:15:40 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7E99C1BB5 for ; Thu, 22 Oct 2015 06:15:40 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTP id 651EC114EB for ; Thu, 22 Oct 2015 16:15:33 +1000 (EST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 57A7C28202A for ; Thu, 22 Oct 2015 16:15:33 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ffxMSEMtpztq for ; Thu, 22 Oct 2015 16:15:33 +1000 (AEST) Received: from Peters-MacBook-Pro.local (unknown [64.245.0.210]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id BEFAF2811A5; Thu, 22 Oct 2015 16:15:30 +1000 (AEST) Subject: Re: Issues with bhyve under KVM To: Cory Smelosky References: <061b9d92bf351baa9263279e86513861@ixsystems.com> <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> Cc: freebsd-virtualization@freebsd.org From: Peter Grehan Message-ID: <56287F01.7060308@freebsd.org> Date: Wed, 21 Oct 2015 23:15:29 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 06:15:40 -0000 Hi Cory, > `dmesg` prints a message saying that VT-x is "disabled by BIOS" (I > passed -L and -bios pointed at OVMF just in case it was a strange > SEABIOS compile option) to no change. -cpu is set to host, I have also > tried passing -cpu qemu64,+vmx manually to no change. That's not strictly true, and in this case harmless. I'll make sure that is fixed. > Oct 14 06:13:19 kernel: vmx_init: processor does not support desired > basic capabilities bhyve expects that the CPU supports INS/OUTS exits. This is signalled in bit 54 of the VMX_BASIC MSR. I'll have a talk with Neel about the implications of ignoring this bit. later, Peter. From owner-freebsd-virtualization@freebsd.org Thu Oct 22 07:40:07 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89F33A1B17C for ; Thu, 22 Oct 2015 07:40:07 +0000 (UTC) (envelope-from mr.xanto@gmail.com) Received: from mail-io0-x22a.google.com (mail-io0-x22a.google.com [IPv6:2607:f8b0:4001:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A21B1639 for ; Thu, 22 Oct 2015 07:40:07 +0000 (UTC) (envelope-from mr.xanto@gmail.com) Received: by ioll68 with SMTP id l68so83493924iol.3 for ; Thu, 22 Oct 2015 00:40:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=pPzDT18EP3BXYchZg72YedoaoGANWY1RtYX/b5hnV2o=; b=rU4TKLqjM5edCsNhsSYEHQ5kqg8uqx1+kzX8b8HOIEe/Mcsoi7yjOTVBr6+F1LQXf/ 4q1LvRgIYNWP+FeMD7hqnCRioRpZKniROjEDAjO7Cpni/BLEUaZ+s9kMdPp5rQokiMp5 JoOuDFvbFDPXIDn+SCkdV/OmK+GI5A0a9NeQ3CzUsU+CMnEP13MgWtJytJHKXkoH/QUY +X2bTS1v0UCXkTlL0LTl2PLiYbAe84f6ts29myBH4pYnbZfGVqMzmukXlO7Jbtlz6h1B OodJziEPj1ZKLMSiAKff4Fz2eTxc1b6hzuILTD2Rqh59KrmMCHPuJ5SaP68Oy/fff6bM lKkg== MIME-Version: 1.0 X-Received: by 10.107.17.18 with SMTP id z18mr7825741ioi.28.1445499606395; Thu, 22 Oct 2015 00:40:06 -0700 (PDT) Received: by 10.79.106.197 with HTTP; Thu, 22 Oct 2015 00:40:06 -0700 (PDT) Date: Thu, 22 Oct 2015 10:40:06 +0300 Message-ID: Subject: Loader tunables for virtio From: Mamontov Roman To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 07:40:07 -0000 Hello. I try to change some tunables for vtnet Ethernet driver and set variable: *hw.vtnet.csum**_* *disable=1**hw.vtnet.tso**_* *disable=1**hw.vtnet.lro**_* *disable=1**hw.vtnet.mq **_* *disable=1* in /boot/loader.conf file. After reboot I try to read value from sysctl command, but no luck: #sysctl hw.vtnet sysctl: unknown oid 'hw.vtnet' How I can understand, that these variables set to correct values? Uname: 11.0-CURRENT r289627. P.S. I just looking why em0 not work (not see packets from host system in KVM qemu, watchdog timeout -- resetting) after update from 9.3 RELEASE. From owner-freebsd-virtualization@freebsd.org Thu Oct 22 08:37:46 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 301D8A1BFAF for ; Thu, 22 Oct 2015 08:37:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D42D11B0 for ; Thu, 22 Oct 2015 08:37:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9M8bjiA006839 for ; Thu, 22 Oct 2015 08:37:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203884] bhyve exits with assert when vcpu already in use Date: Thu, 22 Oct 2015 08:37:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 08:37:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203884 --- Comment #6 from commit-hook@freebsd.org --- A commit references this bug: Author: ngie Date: Thu Oct 22 08:37:11 UTC 2015 New revision: 289746 URL: https://svnweb.freebsd.org/changeset/base/289746 Log: Exit with a user-friendly message instead of tripping an assert if vm_activate_cpu(..) fails when called from fbsdrun_addcpu(..) MFC after: 1 week PR: 203884 Reviewed by: grehan Submitted by: William Orr Changes: head/usr.sbin/bhyve/bhyverun.c -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Thu Oct 22 08:38:09 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FC61A1BFD7 for ; Thu, 22 Oct 2015 08:38:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8CE6E1221 for ; Thu, 22 Oct 2015 08:38:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id t9M8c9dD007284 for ; Thu, 22 Oct 2015 08:38:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 203884] bhyve exits with assert when vcpu already in use Date: Thu, 22 Oct 2015 08:38:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 08:38:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203884 Garrett Cooper,425-314-3911 changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-virtualization@freebsd.org Thu Oct 22 17:21:11 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4262A1C8A5 for ; Thu, 22 Oct 2015 17:21:11 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from webmail2.jnielsen.net (webmail2.jnielsen.net [50.114.224.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "webmail2.jnielsen.net", Issuer "freebsdsolutions.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C986D1C35 for ; Thu, 22 Oct 2015 17:21:11 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from [10.10.1.196] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by webmail2.jnielsen.net (8.15.1/8.15.1) with ESMTPSA id t9MHH6xj093134 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 22 Oct 2015 11:17:09 -0600 (MDT) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail2.jnielsen.net: Host office.betterlinux.com [199.58.199.60] claimed to be [10.10.1.196] From: John Nielsen Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: error running grub-bhyve with -S Message-Id: <71AAD86C-6966-46AA-BD73-2D0CE1E2C213@jnielsen.net> Date: Thu, 22 Oct 2015 11:17:06 -0600 To: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 17:21:12 -0000 I=E2=80=99m running bhyve on an Intel machine running FreeBSD = 11-CURRENT, updated a few days ago. I have a Debian 8.2 VM that has been = running fine but now I=E2=80=99d like to add a PCI pass-through device. = Unfortunately, when I add the required =E2=80=9C-S=E2=80=9D flag to my = grub-bhyve command line it doesn=E2=80=99t work: # bhyvectl =E2=80=94vm=3Dvm0 --destroy # grub-bhyve -m /images/vm0-device.map -M 1024 -r hd1 -S vm0 Could not setup memory for VM Error in initializing VM Obviously, running =E2=80=9Cbhyve=E2=80=9D after that fails as well. What would cause that error and what can I do about it? Thanks! JN From owner-freebsd-virtualization@freebsd.org Thu Oct 22 17:27:18 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E817A1CB60 for ; Thu, 22 Oct 2015 17:27:18 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0138A1554 for ; Thu, 22 Oct 2015 17:27:17 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTP id 1F455115C5 for ; Fri, 23 Oct 2015 03:27:15 +1000 (EST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 19217280F61 for ; Fri, 23 Oct 2015 03:27:15 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xH2zHY-9imjJ for ; Fri, 23 Oct 2015 03:27:15 +1000 (AEST) Received: from Peters-MacBook-Pro.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 025C7280F52; Fri, 23 Oct 2015 03:27:12 +1000 (AEST) Subject: Re: error running grub-bhyve with -S To: John Nielsen References: <71AAD86C-6966-46AA-BD73-2D0CE1E2C213@jnielsen.net> Cc: freebsd-virtualization@freebsd.org From: Peter Grehan Message-ID: <56291C6F.9070405@freebsd.org> Date: Thu, 22 Oct 2015 10:27:11 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <71AAD86C-6966-46AA-BD73-2D0CE1E2C213@jnielsen.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 17:27:18 -0000 > # bhyvectl —vm=vm0 --destroy > # grub-bhyve -m /images/vm0-device.map -M 1024 -r hd1 -S vm0 > Could not setup memory for VM > Error in initializing VM The -S option will force allocation of guest memory (required by passthru). Is there 1G of free mem available on the machine when grub-bhyve was run ? later, Peter. From owner-freebsd-virtualization@freebsd.org Thu Oct 22 17:39:41 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6EB43A1CD2E for ; Thu, 22 Oct 2015 17:39:41 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from webmail2.jnielsen.net (webmail2.jnielsen.net [50.114.224.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "webmail2.jnielsen.net", Issuer "freebsdsolutions.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A8F41A50; Thu, 22 Oct 2015 17:39:40 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from [10.10.1.196] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by webmail2.jnielsen.net (8.15.1/8.15.1) with ESMTPSA id t9MHdb0i017290 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 22 Oct 2015 11:39:39 -0600 (MDT) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail2.jnielsen.net: Host office.betterlinux.com [199.58.199.60] claimed to be [10.10.1.196] Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: error running grub-bhyve with -S From: John Nielsen In-Reply-To: <56291C6F.9070405@freebsd.org> Date: Thu, 22 Oct 2015 11:39:37 -0600 Cc: freebsd-virtualization@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <8B7A0F48-9307-405A-A7F3-5E1B579DAA34@jnielsen.net> References: <71AAD86C-6966-46AA-BD73-2D0CE1E2C213@jnielsen.net> <56291C6F.9070405@freebsd.org> To: Peter Grehan X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 17:39:41 -0000 On Oct 22, 2015, at 11:27 AM, Peter Grehan wrote: >> # bhyvectl =E2=80=94vm=3Dvm0 --destroy >> # grub-bhyve -m /images/vm0-device.map -M 1024 -r hd1 -S vm0 >> Could not setup memory for VM >> Error in initializing VM >=20 > The -S option will force allocation of guest memory (required by = passthru). Is there 1G of free mem available on the machine when = grub-bhyve was run ? Hi, thanks for the response. Yes, the machine had something like 6GB = free (not including cache/buf). I also tried running grub-bhyve with = smaller memory values down to and including 32MB (just to see if it = would work, not because I expect my VM to run with that), and without a = -M flag (which, IIRC, defaults to 256MB). I always got the same error. JN From owner-freebsd-virtualization@freebsd.org Fri Oct 23 05:10:01 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5149EA19FF4 for ; Fri, 23 Oct 2015 05:10:01 +0000 (UTC) (envelope-from corys@ixsystems.com) Received: from barracuda.ixsystems.com (mail.ixsystems.com [69.198.165.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 310C71876 for ; Fri, 23 Oct 2015 05:10:00 +0000 (UTC) (envelope-from corys@ixsystems.com) X-ASG-Debug-ID: 1445576999-08ca040e85b15e0001-XWyayz Received: from mail.iXsystems.com ([10.2.55.1]) by barracuda.ixsystems.com with ESMTP id mqwCAMnuqzyrED6S (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 22 Oct 2015 22:09:59 -0700 (PDT) X-Barracuda-Envelope-From: corys@ixsystems.com X-Barracuda-RBL-Trusted-Forwarder: 10.2.55.1 X-ASG-Whitelist: Client Received: from roundcube.ixsystems.com (unknown [10.2.55.10]) by mail.iXsystems.com (Postfix) with ESMTPA id C835F85C72 for ; Thu, 22 Oct 2015 22:09:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1445576998; bh=XRp+Fzq7ZD8cPCHOoEV/egt9UcnYaFJekQL5G4IOaZg=; h=Date:From:To:Subject:In-Reply-To:References; b=TITW2ZXvJTpoYLsgIejcWkm3cpuc98W/5mi9nd0STZGgjdy8wrFarQ+/oA+UmzmBZ cSxMvBlPTeM9y2/ObiQ39ZBuvP2Kedcovc+6HSMV91xSgTVmOdwn9+j3byrrhLW1Ri 1sjdOGwQRRCsViRRZhO0VrbdncRVqOZZAw8RtmQ0= X-Barracuda-RBL-Trusted-Forwarder: 10.2.55.10 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 22 Oct 2015 22:21:02 -0700 From: Cory Smelosky To: freebsd-virtualization@freebsd.org Subject: Re: Issues with bhyve under KVM In-Reply-To: <56287F01.7060308@freebsd.org> X-ASG-Orig-Subj: Re: Issues with bhyve under KVM References: <061b9d92bf351baa9263279e86513861@ixsystems.com> <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> <56287F01.7060308@freebsd.org> Message-ID: <941285e78b5cd6a3a9283ef1f0068893@ixsystems.com> X-Sender: corys@ixsystems.com User-Agent: Roundcube Webmail/1.1.3 X-Barracuda-Connect: UNKNOWN[10.2.55.1] X-Barracuda-Start-Time: 1445576999 X-Barracuda-Encrypted: DHE-RSA-CAMELLIA256-SHA X-Barracuda-URL: https://10.2.0.41:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at ixsystems.com X-Barracuda-BRTS-Status: 1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 05:10:01 -0000 On 2015-10-21 23:15, Peter Grehan wrote: > Hi Cory, > >> `dmesg` prints a message saying that VT-x is "disabled by BIOS" (I >> passed -L and -bios pointed at OVMF just in case it was a strange >> SEABIOS compile option) to no change. -cpu is set to host, I have >> also >> tried passing -cpu qemu64,+vmx manually to no change. > > That's not strictly true, and in this case harmless. I'll make sure > that is fixed. Ahhh. Yeah - misleading errors are annoying! > >> Oct 14 06:13:19 kernel: vmx_init: processor does not support desired >> basic capabilities > > bhyve expects that the CPU supports INS/OUTS exits. This is signalled > in bit 54 of the VMX_BASIC MSR. I'll have a talk with Neel about the > implications of ignoring this bit. Hmm. Does Linux KVM ignore it? (does -host not emulate all MSRs?) > > later, > > Peter. -- -- Cory Smelosky iXsystems From owner-freebsd-virtualization@freebsd.org Fri Oct 23 05:17:26 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17B7DA16196 for ; Fri, 23 Oct 2015 05:17:26 +0000 (UTC) (envelope-from peterg@ptree32.com.au) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id BA0D41CAE for ; Fri, 23 Oct 2015 05:17:25 +0000 (UTC) (envelope-from peterg@ptree32.com.au) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTP id 34C751160B for ; Fri, 23 Oct 2015 15:17:23 +1000 (EST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 2937A28202C for ; Fri, 23 Oct 2015 15:17:23 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 83hX_zYw9VUE for ; Fri, 23 Oct 2015 15:17:23 +1000 (AEST) Received: from Peters-MacBook-Pro.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id B425C2804FF; Fri, 23 Oct 2015 15:17:20 +1000 (AEST) Subject: Re: Issues with bhyve under KVM To: Cory Smelosky , freebsd-virtualization@freebsd.org References: <061b9d92bf351baa9263279e86513861@ixsystems.com> <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> <56287F01.7060308@freebsd.org> <941285e78b5cd6a3a9283ef1f0068893@ixsystems.com> From: Peter Grehan Message-ID: <5629C2DE.4040908@ptree32.com.au> Date: Thu, 22 Oct 2015 22:17:18 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <941285e78b5cd6a3a9283ef1f0068893@ixsystems.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 05:17:26 -0000 Hi Cory, >> bhyve expects that the CPU supports INS/OUTS exits. This is signalled >> in bit 54 of the VMX_BASIC MSR. I'll have a talk with Neel about the >> implications of ignoring this bit. > > Hmm. Does Linux KVM ignore it? It must do (as did bhyve for a long time). > (does -host not emulate all MSRs?) Probably just not that bit. You might want to try the this patch which comments out that test, and see if it gets you any further. https://people.freebsd.org/~grehan/patches/vmx_ins.diff later, Peter. From owner-freebsd-virtualization@freebsd.org Fri Oct 23 05:36:39 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 280ADA164F3 for ; Fri, 23 Oct 2015 05:36:39 +0000 (UTC) (envelope-from corys@ixsystems.com) Received: from barracuda.ixsystems.com (mail.ixsystems.com [69.198.165.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 07D0382C for ; Fri, 23 Oct 2015 05:36:38 +0000 (UTC) (envelope-from corys@ixsystems.com) X-ASG-Debug-ID: 1445578598-08ca040e84b1860001-XWyayz Received: from mail.iXsystems.com ([10.2.55.1]) by barracuda.ixsystems.com with ESMTP id efmn6HxGa6zx9xVX (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 22 Oct 2015 22:36:38 -0700 (PDT) X-Barracuda-Envelope-From: corys@ixsystems.com X-Barracuda-RBL-Trusted-Forwarder: 10.2.55.1 X-ASG-Whitelist: Client Received: from roundcube.ixsystems.com (unknown [10.2.55.10]) by mail.iXsystems.com (Postfix) with ESMTPA id C078B85CE0 for ; Thu, 22 Oct 2015 22:36:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1445578597; bh=1rTy2Ulr8DtwZtJ6kzZa28cbfuyQBXO2iJPy7fQekwg=; h=Date:From:To:Subject:In-Reply-To:References; b=PR+brBQhUCdRW0RN1BYqF+zfAhZu9fPEGaZ9wK3ZoZOHYBW5zNtdHneISkGbCn7Md 0y5tfIl5IvIGz9mW38tFsGk6lqdZesxgRo8e/qWFbgVWQxkKKZTN14bFcsxUH0EJDp BC4JhdkyAvAsj3WmpiOypcinHT09FhAkDmlW3Fd8= X-Barracuda-RBL-Trusted-Forwarder: 10.2.55.10 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 22 Oct 2015 22:47:41 -0700 From: Cory Smelosky To: freebsd-virtualization@freebsd.org Subject: Re: Issues with bhyve under KVM In-Reply-To: <5629C2DE.4040908@ptree32.com.au> X-ASG-Orig-Subj: Re: Issues with bhyve under KVM References: <061b9d92bf351baa9263279e86513861@ixsystems.com> <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> <56287F01.7060308@freebsd.org> <941285e78b5cd6a3a9283ef1f0068893@ixsystems.com> <5629C2DE.4040908@ptree32.com.au> Message-ID: <7a8d9aeee8a4fa4bdd1afbc850e08543@ixsystems.com> X-Sender: corys@ixsystems.com User-Agent: Roundcube Webmail/1.1.3 X-Barracuda-Connect: UNKNOWN[10.2.55.1] X-Barracuda-Start-Time: 1445578598 X-Barracuda-Encrypted: DHE-RSA-CAMELLIA256-SHA X-Barracuda-URL: https://10.2.0.41:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at ixsystems.com X-Barracuda-BRTS-Status: 1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 05:36:39 -0000 On 2015-10-22 22:17, Peter Grehan wrote: > Hi Cory, > >>> bhyve expects that the CPU supports INS/OUTS exits. This is >>> signalled >>> in bit 54 of the VMX_BASIC MSR. I'll have a talk with Neel about the >>> implications of ignoring this bit. >> >> Hmm. Does Linux KVM ignore it? > > It must do (as did bhyve for a long time). Ahhhh. > >> (does -host not emulate all MSRs?) > > Probably just not that bit. You might want to try the this patch > which comments out that test, and see if it gets you any further. > https://people.freebsd.org/~grehan/patches/vmx_ins.diff --- all_subdir_vmm --- /usr/src/sys/modules/vmm/../../amd64/vmm/intel/vmx.c:535:11: error: unused variable 'basic' [-Werror,-Wunused-variable] uint64_t basic, fixed0, fixed1, feature_control; ^ 1 error generated. Aww. I need to actually run a later SVN release instead of 286666. Tried to save some time. ;) > > later, > > Peter. -- -- Cory Smelosky iXsystems From owner-freebsd-virtualization@freebsd.org Fri Oct 23 05:38:43 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61676A1654D for ; Fri, 23 Oct 2015 05:38:43 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 20F14899 for ; Fri, 23 Oct 2015 05:38:43 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTP id 93CCB1160D for ; Fri, 23 Oct 2015 15:38:40 +1000 (EST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 89EF9282029 for ; Fri, 23 Oct 2015 15:38:40 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ALkc0psrUlsb for ; Fri, 23 Oct 2015 15:38:40 +1000 (AEST) Received: from Peters-MacBook-Pro.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id A55F2280F61; Fri, 23 Oct 2015 15:38:37 +1000 (AEST) Subject: Re: Issues with bhyve under KVM To: Cory Smelosky References: <061b9d92bf351baa9263279e86513861@ixsystems.com> <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> <56287F01.7060308@freebsd.org> <941285e78b5cd6a3a9283ef1f0068893@ixsystems.com> <5629C2DE.4040908@ptree32.com.au> <7a8d9aeee8a4fa4bdd1afbc850e08543@ixsystems.com> Cc: freebsd-virtualization@freebsd.org From: Peter Grehan Message-ID: <5629C7DD.70606@freebsd.org> Date: Thu, 22 Oct 2015 22:38:37 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <7a8d9aeee8a4fa4bdd1afbc850e08543@ixsystems.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 05:38:43 -0000 Hi Cory, >> Probably just not that bit. You might want to try the this patch >> which comments out that test, and see if it gets you any further. >> https://people.freebsd.org/~grehan/patches/vmx_ins.diff > > --- all_subdir_vmm --- > /usr/src/sys/modules/vmm/../../amd64/vmm/intel/vmx.c:535:11: error: > unused variable 'basic' [-Werror,-Wunused-variable] > uint64_t basic, fixed0, fixed1, feature_control; > ^ > 1 error generated. > > Aww. I need to actually run a later SVN release instead of 286666. Tried > to save some time. ;) Apologies - I didn't compile-test the patch, but the basic idea is to comment out that block. You can delete the unused variable 'basic' which should get you going. later, Peter. From owner-freebsd-virtualization@freebsd.org Fri Oct 23 05:47:29 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EE8EA16753 for ; Fri, 23 Oct 2015 05:47:29 +0000 (UTC) (envelope-from corys@ixsystems.com) Received: from barracuda.ixsystems.com (mail.ixsystems.com [69.198.165.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1E33CA5 for ; Fri, 23 Oct 2015 05:47:28 +0000 (UTC) (envelope-from corys@ixsystems.com) X-ASG-Debug-ID: 1445579247-08ca040e85b19c0001-XWyayz Received: from mail.iXsystems.com ([10.2.55.1]) by barracuda.ixsystems.com with ESMTP id 8sQAHccWwwAT21VR (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 22 Oct 2015 22:47:27 -0700 (PDT) X-Barracuda-Envelope-From: corys@ixsystems.com X-Barracuda-RBL-Trusted-Forwarder: 10.2.55.1 X-ASG-Whitelist: Client Received: from roundcube.ixsystems.com (unknown [10.2.55.10]) by mail.iXsystems.com (Postfix) with ESMTPA id 6106585D23 for ; Thu, 22 Oct 2015 22:47:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1445579247; bh=Ay8lcnpYfWPb42CX31RHJJxkdCvJwfj/kOqpoN162UA=; h=Date:From:To:Subject:In-Reply-To:References; b=o67JfgsGoi35frz9j1t9zAF5SHGxeuJY2O7zl8zcy4ZSG3ZRRX09OZjzm7OpSt+eO bkC9/cDKb7WBs5yConaC7mohJpXYh8AR0QeFr4MlX3Oru5EkZUfgJHjTjkqzG7I/jY aW5doW0FV8+fUGWIdd5nIUylGTtVnMQPJhMrnz9k= X-Barracuda-RBL-Trusted-Forwarder: 10.2.55.10 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 22 Oct 2015 22:58:30 -0700 From: Cory Smelosky To: freebsd-virtualization@freebsd.org Subject: Re: Issues with bhyve under KVM In-Reply-To: <5629C7DD.70606@freebsd.org> X-ASG-Orig-Subj: Re: Issues with bhyve under KVM References: <061b9d92bf351baa9263279e86513861@ixsystems.com> <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> <56287F01.7060308@freebsd.org> <941285e78b5cd6a3a9283ef1f0068893@ixsystems.com> <5629C2DE.4040908@ptree32.com.au> <7a8d9aeee8a4fa4bdd1afbc850e08543@ixsystems.com> <5629C7DD.70606@freebsd.org> Message-ID: <4b1f91b885ef17046374304e219a230a@ixsystems.com> X-Sender: corys@ixsystems.com User-Agent: Roundcube Webmail/1.1.3 X-Barracuda-Connect: UNKNOWN[10.2.55.1] X-Barracuda-Start-Time: 1445579247 X-Barracuda-Encrypted: DHE-RSA-CAMELLIA256-SHA X-Barracuda-URL: https://10.2.0.41:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at ixsystems.com X-Barracuda-BRTS-Status: 1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 05:47:29 -0000 On 2015-10-22 22:38, Peter Grehan wrote: > Hi Cory, > >>> Probably just not that bit. You might want to try the this patch >>> which comments out that test, and see if it gets you any further. >>> https://people.freebsd.org/~grehan/patches/vmx_ins.diff >> >> --- all_subdir_vmm --- >> /usr/src/sys/modules/vmm/../../amd64/vmm/intel/vmx.c:535:11: error: >> unused variable 'basic' [-Werror,-Wunused-variable] >> uint64_t basic, fixed0, fixed1, feature_control; >> ^ >> 1 error generated. >> >> Aww. I need to actually run a later SVN release instead of 286666. >> Tried >> to save some time. ;) > > Apologies - I didn't compile-test the patch, but the basic idea is to > comment out that block. You can delete the unused variable 'basic' > which should get you going. Yup. vmx_init: ept initialization failed (22) module_register_init: MOD_LOAD (vmm, 0xffffffff81e16fd0, 0) error 22 A more useful error, but still incorrect I am pretty sure. > > later, > > Peter. -- -- Cory Smelosky iXsystems From owner-freebsd-virtualization@freebsd.org Fri Oct 23 18:27:14 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 764D9A1DEF9 for ; Fri, 23 Oct 2015 18:27:14 +0000 (UTC) (envelope-from joe@getsomewhere.net) Received: from prak.gameowls.com (prak.gameowls.com [IPv6:2001:19f0:5c00:950b:5400:ff:fe14:46b7]) by mx1.freebsd.org (Postfix) with ESMTP id 4482B920 for ; Fri, 23 Oct 2015 18:27:14 +0000 (UTC) (envelope-from joe@getsomewhere.net) Received: from [IPv6:2001:470:c412:beef:135:a9e:3ef5:fbe7] (unknown [IPv6:2001:470:c412:beef:135:a9e:3ef5:fbe7]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by prak.gameowls.com (Postfix) with ESMTPSA id 276921852C for ; Fri, 23 Oct 2015 13:27:06 -0500 (CDT) From: Joseph Love Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: some Docker cmd options not working? Message-Id: <0DBFBC10-4B04-4664-9EEE-26592B9C41DE@getsomewhere.net> Date: Fri, 23 Oct 2015 13:27:04 -0500 To: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 18:27:14 -0000 Hi, I=E2=80=99ve been poking at using docker to run some linux apps. In = doing so, I=E2=80=99ve been looking at a couple features that I cannot = tell if they=E2=80=99re failing, or just not implemented, or if = there=E2=80=99s some other reason that it isn=E2=80=99t working. The two options I=E2=80=99ve tried sofar that haven=E2=80=99t seemed to = have any effect are these -v & -u. eg: docker run -i -v /tmp:/mnt =E2=80=94name=3Dtest centos docker exec -u testuser test /bin/sh I=E2=80=99ve been using the centos docker image as a basis for trying a = few things, so the paths used in the -v command exist, both the path = specified for the host, as well as that for the jailed guest. In the = case of the user flag, -u, the user definitely exists in the guest. If = I run a program which will run for a bit, and check the hosts=E2=80=99s = =E2=80=9Cps aux=E2=80=9D list, the user it runs as is root, rather than = the user/userid I specified. I=E2=80=99m mostly just curious if these parameters aren=E2=80=99t = implemented, or if I=E2=80=99m trying something wrong. Thanks, -Joe From owner-freebsd-virtualization@freebsd.org Sat Oct 24 03:09:54 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 858A7A18F04 for ; Sat, 24 Oct 2015 03:09:54 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 46C841996 for ; Sat, 24 Oct 2015 03:09:53 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTP id 02472115A4 for ; Sat, 24 Oct 2015 13:09:46 +1000 (EST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id EE62C280F80 for ; Sat, 24 Oct 2015 13:09:45 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MR9yG0BDjeBb for ; Sat, 24 Oct 2015 13:09:45 +1000 (AEST) Received: from Peters-MacBook-Pro.local (unknown [64.245.0.210]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id EC14C280500; Sat, 24 Oct 2015 13:09:43 +1000 (AEST) Subject: Re: Issues with bhyve under KVM To: Cory Smelosky References: <061b9d92bf351baa9263279e86513861@ixsystems.com> <78cd6c472dfa583c81442d766e620bf7@ixsystems.com> <56287F01.7060308@freebsd.org> <941285e78b5cd6a3a9283ef1f0068893@ixsystems.com> <5629C2DE.4040908@ptree32.com.au> <7a8d9aeee8a4fa4bdd1afbc850e08543@ixsystems.com> <5629C7DD.70606@freebsd.org> <4b1f91b885ef17046374304e219a230a@ixsystems.com> From: Peter Grehan Cc: freebsd-virtualization@freebsd.org Message-ID: <562AF675.9050500@freebsd.org> Date: Fri, 23 Oct 2015 20:09:41 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <4b1f91b885ef17046374304e219a230a@ixsystems.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2015 03:09:54 -0000 Hi Cory, > vmx_init: ept initialization failed (22) > module_register_init: MOD_LOAD (vmm, 0xffffffff81e16fd0, 0) error 22 > > A more useful error, but still incorrect I am pretty sure. This one is because KVM doesn't support the invvpid instruction when running nested. The bhyve ept init code requires for this, though it may not be strictly necessary. I've created PR 203994 to track the issues for running nested under KVM. later, Peter. From owner-freebsd-virtualization@freebsd.org Sat Oct 24 03:13:26 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 348D0A1B0B3 for ; Sat, 24 Oct 2015 03:13:26 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id EBE881C2A for ; Sat, 24 Oct 2015 03:13:25 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTP id E705A11607 for ; Sat, 24 Oct 2015 13:13:24 +1000 (EST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id DE345280F80 for ; Sat, 24 Oct 2015 13:13:24 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UGBgyMa6HDE0 for ; Sat, 24 Oct 2015 13:13:24 +1000 (AEST) Received: from Peters-MacBook-Pro.local (unknown [64.245.0.210]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 09041280500; Sat, 24 Oct 2015 13:13:21 +1000 (AEST) Subject: Re: error running grub-bhyve with -S To: John Nielsen References: <71AAD86C-6966-46AA-BD73-2D0CE1E2C213@jnielsen.net> <56291C6F.9070405@freebsd.org> <8B7A0F48-9307-405A-A7F3-5E1B579DAA34@jnielsen.net> Cc: freebsd-virtualization@freebsd.org From: Peter Grehan Message-ID: <562AF750.5030303@freebsd.org> Date: Fri, 23 Oct 2015 20:13:20 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <8B7A0F48-9307-405A-A7F3-5E1B579DAA34@jnielsen.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2015 03:13:26 -0000 >> The -S option will force allocation of guest memory (required by >> passthru). Is there 1G of free mem available on the machine when >> grub-bhyve was run ? > > Hi, thanks for the response. Yes, the machine had something like 6GB > free (not including cache/buf). I also tried running grub-bhyve with > smaller memory values down to and including 32MB (just to see if it > would work, not because I expect my VM to run with that), and without > a -M flag (which, IIRC, defaults to 256MB). I always got the same > error. I can get that error if vmm.ko isn't loaded, but in that case the bhyvectl command would also error out and it doesn't appear to in your case. Some questions: Is this on FreeBSD CURRENT ? Does grub-bhyve work without the -S option ? Or, with -S but a different VM name ? Does the issue persist after a reboot ? later, Peter.