From owner-freebsd-virtualization@freebsd.org Sun Apr 22 21:00:10 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E658FBD397 for ; Sun, 22 Apr 2018 21:00:10 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id DA062756AD for ; Sun, 22 Apr 2018 21:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 97106FBD396; Sun, 22 Apr 2018 21:00:09 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 84E02FBD395 for ; Sun, 22 Apr 2018 21:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 292AE756A9 for ; Sun, 22 Apr 2018 21:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 53D2B23346 for ; Sun, 22 Apr 2018 21:00:08 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3ML08Kx025634 for ; Sun, 22 Apr 2018 21:00:08 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3ML08Pr025625 for virtualization@FreeBSD.org; Sun, 22 Apr 2018 21:00:08 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201804222100.w3ML08Pr025625@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 22 Apr 2018 21:00:08 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Apr 2018 21:00:10 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 226583 | FreeBSD VM on Hyper-V/Azure can't properly detect 1 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Tue Apr 24 03:06:31 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A700FB5FCA for ; Tue, 24 Apr 2018 03:06:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BFE3275EA6 for ; Tue, 24 Apr 2018 03:06:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 80651FB5FBB; Tue, 24 Apr 2018 03:06:30 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35D2CFB5FBA for ; Tue, 24 Apr 2018 03:06:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BAB6E75EA1 for ; Tue, 24 Apr 2018 03:06:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id EC6E112788 for ; Tue, 24 Apr 2018 03:06:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3O36SK9043589 for ; Tue, 24 Apr 2018 03:06:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3O36Sfb043581 for virtualization@FreeBSD.org; Tue, 24 Apr 2018 03:06:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 226583] FreeBSD VM on Hyper-V/Azure can't properly detected SCSI disk da2 Date: Tue, 24 Apr 2018 03:06:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 03:06:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D226583 --- Comment #7 from commit-hook@freebsd.org --- A commit references this bug: Author: dexuan Date: Tue Apr 24 03:06:05 UTC 2018 New revision: 332903 URL: https://svnweb.freebsd.org/changeset/base/332903 Log: MFC: 332385 r332385: hyperv/storvsc: storvsc_io_done(): do not use CAM_SEL_TIMEOUT CAM_SEL_TIMEOUT was introduced in https://reviews.freebsd.org/D7521 (r304251), which claimed: "VM shall response to CAM layer with CAM_SEL_TIMEOUT to filter those invalid LUNs. Never use CAM_DEV_NOT_THERE which will block LUN scan for LUN number higher than 7." But it turns out this is not correct: I think what really filters the invalid LUNs in r304251 is that: before r304251, we could set the CAM_REQ_CMP without checking vm_srb->srb_status at all: ccb->ccb_h.status |=3D CAM_REQ_CMP. r304251 checks vm_srb->srb_status and sets ccb->ccb_h.status properly, so the invalid LUNs are filtered. I changed my code version to r304251 but replaced the CAM_SEL_TIMEOUT with CAM_DEV_NOT_THERE, and I confirmed the invalid LUNs can also be filtered, and I successfully hot-added and hot-removed 8 disks to/from the VM without any issue. CAM_SEL_TIMEOUT has an unwanted side effect -- see cam_periph_error(): For a selection timeout, we consider all of the LUNs on the target to be gone. If the status is CAM_DEV_NOT_THERE, then we only get rid of the device(s) specified by the path in the original CCB. This means: for a VM with a valid LUN on 3:0:0:0, when the VM inquires 3:0:0:1 and the host reports 3:0:0:1 doesn't exist and storvsc returns CAM_SEL_TIMEOUT to the CAM layer, CAM will detech 3:0:0:0 as well: th= is is the bug I reported recently: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D226583 PR: 226583 Reviewed by: mav Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D14690 Changes: _U stable/11/ stable/11/sys/dev/hyperv/storvsc/hv_storvsc_drv_freebsd.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Apr 24 03:08:35 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BD70FB625C for ; Tue, 24 Apr 2018 03:08:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A29757610E for ; Tue, 24 Apr 2018 03:08:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 61AACFB6251; Tue, 24 Apr 2018 03:08:34 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 221B0FB6248 for ; Tue, 24 Apr 2018 03:08:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A960B76108 for ; Tue, 24 Apr 2018 03:08:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C9B6A1278F for ; Tue, 24 Apr 2018 03:08:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3O38W4K022369 for ; Tue, 24 Apr 2018 03:08:32 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3O38WQZ022355 for virtualization@FreeBSD.org; Tue, 24 Apr 2018 03:08:32 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 226583] FreeBSD VM on Hyper-V/Azure can't properly detected SCSI disk da2 Date: Tue, 24 Apr 2018 03:08:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 03:08:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D226583 --- Comment #8 from commit-hook@freebsd.org --- A commit references this bug: Author: dexuan Date: Tue Apr 24 03:07:49 UTC 2018 New revision: 332904 URL: https://svnweb.freebsd.org/changeset/base/332904 Log: MFC: 332385 r332385: hyperv/storvsc: storvsc_io_done(): do not use CAM_SEL_TIMEOUT CAM_SEL_TIMEOUT was introduced in https://reviews.freebsd.org/D7521 (r304251), which claimed: "VM shall response to CAM layer with CAM_SEL_TIMEOUT to filter those invalid LUNs. Never use CAM_DEV_NOT_THERE which will block LUN scan for LUN number higher than 7." But it turns out this is not correct: I think what really filters the invalid LUNs in r304251 is that: before r304251, we could set the CAM_REQ_CMP without checking vm_srb->srb_status at all: ccb->ccb_h.status |=3D CAM_REQ_CMP. r304251 checks vm_srb->srb_status and sets ccb->ccb_h.status properly, so the invalid LUNs are filtered. I changed my code version to r304251 but replaced the CAM_SEL_TIMEOUT with CAM_DEV_NOT_THERE, and I confirmed the invalid LUNs can also be filtered, and I successfully hot-added and hot-removed 8 disks to/from the VM without any issue. CAM_SEL_TIMEOUT has an unwanted side effect -- see cam_periph_error(): For a selection timeout, we consider all of the LUNs on the target to be gone. If the status is CAM_DEV_NOT_THERE, then we only get rid of the device(s) specified by the path in the original CCB. This means: for a VM with a valid LUN on 3:0:0:0, when the VM inquires 3:0:0:1 and the host reports 3:0:0:1 doesn't exist and storvsc returns CAM_SEL_TIMEOUT to the CAM layer, CAM will detech 3:0:0:0 as well: th= is is the bug I reported recently: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D226583 PR: 226583 Reviewed by: mav Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D14690 Changes: _U stable/10/ stable/10/sys/dev/hyperv/storvsc/hv_storvsc_drv_freebsd.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Apr 24 03:10:13 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC773FB63FF for ; Tue, 24 Apr 2018 03:10:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 5CE9D7624E for ; Tue, 24 Apr 2018 03:10:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 16CFFFB63FA; Tue, 24 Apr 2018 03:10:13 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 043EAFB63F9 for ; Tue, 24 Apr 2018 03:10:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8AB537624A for ; Tue, 24 Apr 2018 03:10:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A311312798 for ; Tue, 24 Apr 2018 03:10:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3O3ABPc082237 for ; Tue, 24 Apr 2018 03:10:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3O3ABcb082234 for virtualization@FreeBSD.org; Tue, 24 Apr 2018 03:10:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 226583] FreeBSD VM on Hyper-V/Azure can't properly detected SCSI disk da2 Date: Tue, 24 Apr 2018 03:10:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: decui@microsoft.com X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 03:10:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D226583 Dexuan Cui changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Open |Closed Resolution|--- |FIXED --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Apr 24 03:11:49 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E074FB6635 for ; Tue, 24 Apr 2018 03:11:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 29B9E769CB for ; Tue, 24 Apr 2018 03:11:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DCE96FB6634; Tue, 24 Apr 2018 03:11:48 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57F96FB6633 for ; Tue, 24 Apr 2018 03:11:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D555A769A1 for ; Tue, 24 Apr 2018 03:11:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 23F4D127CB for ; Tue, 24 Apr 2018 03:11:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3O3BlTg021318 for ; Tue, 24 Apr 2018 03:11:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3O3Bk7C021290 for virtualization@FreeBSD.org; Tue, 24 Apr 2018 03:11:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 226583] FreeBSD VM on Hyper-V/Azure can't properly detected SCSI disk da2 Date: Tue, 24 Apr 2018 03:11:47 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: dexuan@FreeBSD.org X-Bugzilla-Flags: mfc-stable10+ mfc-stable11+ X-Bugzilla-Changed-Fields: assigned_to flagtypes.name keywords cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 03:11:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D226583 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|virtualization@FreeBSD.org |dexuan@FreeBSD.org Flags|mfc-stable10?, |mfc-stable10+, |mfc-stable11? |mfc-stable11+ Keywords|needs-qa | CC| |virtualization@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Apr 24 07:38:44 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CE0AFBECB7; Tue, 24 Apr 2018 07:38:44 +0000 (UTC) (envelope-from dariusmihaim@gmail.com) Received: from mail-qt0-x22f.google.com (mail-qt0-x22f.google.com [IPv6:2607:f8b0:400d:c0d::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AECA575872; Tue, 24 Apr 2018 07:38:43 +0000 (UTC) (envelope-from dariusmihaim@gmail.com) Received: by mail-qt0-x22f.google.com with SMTP id b13-v6so20799932qtp.12; Tue, 24 Apr 2018 00:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=2DXXlDRKsrwzLK/tRx+Q+YLGWyIC85AqPNYp9DGPtYc=; b=RS9PhyIaz/Jnc+TKcWZHj/epp2WSRfn9fZL2O2b894WcnTLIiq7P+eLDvvi2NR3OyV hED0WzkVRcgEToadZn2mkQ28UE72g1cO79I95Qopt/6JMNSv1SSnqO4tzBpeHN9WT/Lj SyeNQVu4DtBC1DBGud/62EZ35eux+mr2yCvmakLPaGi9kPiNRBLJ0Zn6zGhQFIDoWPa1 fr58Np+3TqAPvyJmV38ycRISH0UownVjIujN8e1kAnSgAijArK7TOAw9xkGAhnAzE14v hWYg/oXChaeAcxUIq3B6OqgRYdBKP09KY5jd5pr5t8Mf5Qif55KYx3Xh//n9WsLlJ8QL SsIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=2DXXlDRKsrwzLK/tRx+Q+YLGWyIC85AqPNYp9DGPtYc=; b=guHz/83IsIeriZ1/MZ7b19Fv+lm6OBej/MOUn0S1g3taE2Or9DuN/+8qqyqJAqGids eG2dCZAJf9+OIF2dNvLEWOwss4gBv+dqnwpISv/AX/QNekfo+W9zsypMaNGNd+lRjC/A YaDhf2PRtrj9BMz6/cMwXQl4poZKwrbY9pvVD15u0WgfjShG0VFJqsdpN7FgyZIPFnks RSXsbUxVxBOkprCe90Bg4qaXoO/4cE7F2bpASH0x1QvhVx54CmC7AZvQ+T7KFfQPFyc4 I4ftF/1xCqRP7CLRC+Izinj3PQ9KL2x8rK3glv0UNCB1VrE0Xcev6X4XL556SIcwJNGn g4Iw== X-Gm-Message-State: ALQs6tCJCAveYYreAKynMQ4MI/XB3U5oiHIoB9NTs1FaPYJn468chk2z dvhbL7zOgwD75hPFECXR1FufNu3/nyxeGm2hhzHFpw== X-Google-Smtp-Source: AB8JxZqR6z568dHmKl6w/X4NvKGiqadEdVTzQnwF4DXxS+M3W5jvyGuDdalgMqrXMkwg20+BLQ0i5SbKyHzovy9UteE= X-Received: by 2002:ac8:95a:: with SMTP id z26-v6mr27707237qth.201.1524555523201; Tue, 24 Apr 2018 00:38:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.200.46.185 with HTTP; Tue, 24 Apr 2018 00:38:02 -0700 (PDT) From: Darius Mihai Date: Tue, 24 Apr 2018 10:38:02 +0300 Message-ID: Subject: Build fails after rebasing attempt To: freebsd-arm@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 07:38:44 -0000 Hi all, After attempting to rebase the bhyve on ARM project using the latest master commits, the building process fails with the following error(s): --- all_subdir_vmm --- make[4]: Graph cycles through `hyp_assym.h' make[4]: Graph cycles through `hyp_genassym.o' `hyp_genassym.c' is up to date. `vmm.c' is up to date. `vmm_dev.c' is up to date. `vmm_instruction_emul.c' is up to date. `vmm_mem.c' is up to date. `mmu.c' is up to date. `vmm_stat.c' is up to date. `arm.c' is up to date. `vgic.c' is up to date. `vtimer.c' is up to date. `hyp.S' is up to date. `afterdepend' was not built (made 1, flags 2019, type 3018001)! `afterdepend' has .ORDER dependency against .depend (made 1, flags 301b, type 3020001) `opt_global.h' was not built (made 0, flags 2009, type 3000000)! *** [all_subdir_vmm] Error code 1 Since the buld did not fail prior to rebasing the code, I am not entirely sure how I should fix this. If anyone wishes to take a look at the code, the error seems to be thrown when trying to build the vmm kernel object (sources are in the directory at [1]) for the guest operating system (configuration file is at [2] and dts unde the directory at [3] - I am aware that the location changed, but the error is not caused by a missing file in the configuration). The configuration for the make buildkernel command is TARGET_ARCH=armv6, TARGET=arm, MAKEOBJDIRPREFIX=$HOME/freebsd-obj/; KERNCONF=FVP_VE_CORTEX_A15x1_GUEST is sent as a command line parameter. [1]: https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyvearm/sys/arm/vmm [2]: https://github.com/FreeBSD-UPB/freebsd/blob/projects/bhyvearm/sys/arm/conf/FVP_VE_CORTEX_A15x1_GUEST [3]: https://github.com/FreeBSD-UPB/freebsd/tree/projects/bhyvearm/sys/boot/fdt/dts/arm Thanks, Darius From owner-freebsd-virtualization@freebsd.org Wed Apr 25 18:44:26 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8F3FFAC864 for ; Wed, 25 Apr 2018 18:44:26 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0042.outbound.protection.outlook.com [104.47.1.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D02AD6938A for ; Wed, 25 Apr 2018 18:44:25 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=WF/h53xSuwy6p8ZGagKYgvGsYP2dNkTTdXtWoBqJBSk=; b=ShZ69foCfkxIjbvC0JpslxCHS+20kFLnXJpIufVvPoVU4bPuFlIBgqTrWc7Zr6ZoHeIUauCIcLukPAMdeVIuYVPzytGYqJaWZE4g/S7Dp3jbD5cAWLFFr1F2GTnCpfKen/pPFuXnP4bbzwIjN1VY2LpZz08PXwCsgs7fM5JBRak= Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com (10.170.251.141) by HE1PR0102MB2843.eurprd01.prod.exchangelabs.com (10.170.253.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.13; Wed, 25 Apr 2018 18:44:22 +0000 Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152]) by HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152%13]) with mapi id 15.20.0696.020; Wed, 25 Apr 2018 18:44:22 +0000 From: Paul Esson To: "freebsd-virtualization@freebsd.org" Subject: bhyve networking Thread-Topic: bhyve networking Thread-Index: AdPcwrhbbh0fzB+SRv2CNcsNb6p8OQ== Date: Wed, 25 Apr 2018 18:44:22 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-originating-ip: [81.136.184.115] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0102MB2843; 7:Cnmayyxz98UKCRw0DkWKBHgbpaqklkj8EblZKtdARNbLYEXYzhY2ZnHy6n7bNyRj0zYDzhmwRq0KvJVlOoGHBDqu59g9U9wlVE9E291rr8ZMTr//iKDx3DG5e1OKahqSTumjz+CG66BXBR4RL2STqAVaBKi4fGzwpNG/TSn2bWQijpsoyoBYxq06pU1YaamOYf4IV19TY7xfdQRsNfnZqMLIZZ37+gwWUzIbgFpCs56j+XJDwlrbTWbcsnD/Y+Zj x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0102MB2843; x-ms-traffictypediagnostic: HE1PR0102MB2843: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(788757137089)(21748063052155); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3002001)(3231232)(944501410)(52105095)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123560045)(20161123562045)(6072148)(201708071742011); SRVR:HE1PR0102MB2843; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0102MB2843; x-forefront-prvs: 06530126A4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(39850400004)(39380400002)(366004)(376002)(346002)(189003)(199004)(316002)(9326002)(478600001)(54896002)(6306002)(25786009)(53936002)(9686003)(2351001)(5640700003)(6436002)(5630700001)(6116002)(3846002)(68736007)(55016002)(790700001)(97736004)(14454004)(476003)(3280700002)(74316002)(8676002)(486006)(2906002)(6916009)(33656002)(186003)(26005)(86362001)(2501003)(5250100002)(2900100001)(102836004)(105586002)(106356001)(44832011)(7736002)(5660300001)(3660700001)(81156014)(7116003)(66066001)(99286004)(3480700004)(8936002)(81166006)(221733001)(6506007)(7696005); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0102MB2843; H:HE1PR0102MB2588.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: yg3iQuPpca+s2qWoq6NWxYtpqPOI3fS0hZpbxMO0VQBwZk6aaXLZIFVR0iv/gT/OqyEYDh00cgnHumfMxywomD7y9BM/egFkMEOz+vOWSPcs3cb5YLjvlwHBPtiu/Ejvkozt34VA86fiFaiQyf965ijMRICIBpvxsIbSy6B911+lbUY3cGW2VrxpWe0ZCTWm spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 28c08b98-5321-42ab-a6cd-08d5aadc8fac X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: 28c08b98-5321-42ab-a6cd-08d5aadc8fac X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Apr 2018 18:44:22.1214 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0102MB2843 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 18:44:27 -0000 Hi Folks, I'm struggling with networking when using vm-bhyve on FreeBSD 11.1-RELEASE.= I have two NICs and have configured the first (igb0) on a management netwo= rk and want to use the second (igb1) for VMs. However, I can't get any VM = to communicate through the virtual switch if I have igb1 added to it. If I= take the NIC out of the switch and configure an ipv4 address on it I can r= each other hosts on the relevant subnet so I believe the poet set-up is val= id. If I replace igb1 in the switch with igb0, I can then configure VMs on= my management network and they have network connectivity. Can I only use = an interface that has an IP address configured at the host level before add= ing it to the switch? I've used other Hypervisors successfully by simply = presenting a trunk port to the virtual switch, but this does not seem to wo= rk with vm-bhyve either. Regards, Paul Esson From owner-freebsd-virtualization@freebsd.org Wed Apr 25 19:07:02 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A38E8FAD0DC for ; Wed, 25 Apr 2018 19:07:02 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 21A3B6D9D1 for ; Wed, 25 Apr 2018 19:07:01 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w3PJ6xHH099857; Wed, 25 Apr 2018 21:06:59 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 13089781; Wed, 25 Apr 2018 21:06:59 +0200 (CEST) Message-ID: <5AE0D1D2.9010201@omnilan.de> Date: Wed, 25 Apr 2018 21:06:58 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Paul Esson CC: "freebsd-virtualization@freebsd.org" Subject: Re: bhyve networking References: In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Greylist: ACL 130 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Wed, 25 Apr 2018 21:06:59 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 19:07:02 -0000 Bezüglich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > Hi Folks, > > I'm struggling with networking when using vm-bhyve on FreeBSD 11.1-RELEASE. I have two NICs and have configured the first (igb0) on a management network and want to use the second (igb1) for VMs. However, I can't get any VM to communicate through the virtual switch if I have igb1 added to it. If I take the NIC out of the switch and configure an ipv4 address on it I can reach other hosts on the relevant subnet so I believe the poet set-up is valid. If I replace igb1 in the switch with igb0, I can then configure VMs on my management network and they have network connectivity. Can I only use Hello, a example of the command you used was nice. I guess you're using if_bridge(4) – the example would clarify. But there's ng_bridge(4) and vale(4) also, and others are using OpenVSwitch... > an interface that has an IP address configured at the host level before adding it to the switch? I've used other No. But the interface has to be in promisc mode. And some offloading functions must be disabled, but in case of if_bridge(4), it's done automagically (and reverted if you remove the interface again). Hth, -harry From owner-freebsd-virtualization@freebsd.org Wed Apr 25 20:12:22 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D51DFAF21F for ; Wed, 25 Apr 2018 20:12:22 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0608.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe1f::608]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 473217CFEC for ; Wed, 25 Apr 2018 20:12:21 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=j25rnFZQLoCeE6SwF00kRHjVf9rgPt8K6vQtlrZ4tHg=; b=IH79c2yA60KB8Adxi3U0hmCUV9KiOMBYmSZ9jKTNSlrKawgXBWIjr0RLOL2U5z8Y7xRYy+EM5mXsHW5sp0woFRUG7QB4lRnbMssXD8sZ9IFhAJqxjgctY7sBgu+EpNz9yOH1loiVKKCLIzqoJQ4etQ2C2H6sSiD6CNN153Hi/QI= Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com (10.170.251.141) by HE1PR0102MB3147.eurprd01.prod.exchangelabs.com (10.167.124.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.15; Wed, 25 Apr 2018 20:12:16 +0000 Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152]) by HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152%13]) with mapi id 15.20.0696.020; Wed, 25 Apr 2018 20:12:16 +0000 From: Paul Esson To: Harry Schmalzbauer CC: "freebsd-virtualization@freebsd.org" Subject: Re: bhyve networking Thread-Topic: bhyve networking Thread-Index: AdPcwrhbbh0fzB+SRv2CNcsNb6p8OQABdzsAAAJH2bQ= Date: Wed, 25 Apr 2018 20:12:16 +0000 Message-ID: References: , <5AE0D1D2.9010201@omnilan.de> In-Reply-To: <5AE0D1D2.9010201@omnilan.de> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-originating-ip: [13.95.219.91] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0102MB3147; 7:nqeUgGOIPQCmRqsXFKHFawmTiwBoOPpYoz6/sBZ6B1Bh1CZG2o0Ay17qEuro6JA2qXTdw5+DyYr9v+Veja5FzIjUcf/r3TqMiABmCBegTbvHkXL09H+e7vWGx7mznv74H1mC8XrkzhjrRHul16t104VCrzgsG3WPR4uIvOqfodai1QgLsgsInT0OLwXZB6ELbOpDYipRjOasS12UG5CEShnPjta0H/LDeEuVIlAnq62WTcgAyCJe0RUCOsQNBSQR x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0102MB3147; x-ms-traffictypediagnostic: HE1PR0102MB3147: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(788757137089)(81439100147899)(21532816269658); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3002001)(3231232)(944501410)(52105095)(6041310)(20161123558120)(20161123560045)(20161123564045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:HE1PR0102MB3147; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0102MB3147; x-forefront-prvs: 06530126A4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(39850400004)(366004)(39380400002)(346002)(376002)(189003)(199004)(105586002)(6436002)(59450400001)(106356001)(7116003)(6506007)(3480700004)(229853002)(81156014)(7696005)(76176011)(81166006)(2900100001)(74316002)(5660300001)(86362001)(186003)(102836004)(2906002)(25786009)(68736007)(11346002)(6246003)(53546011)(54896002)(6306002)(53936002)(8936002)(9686003)(55016002)(446003)(8676002)(486006)(3280700002)(14454004)(478600001)(44832011)(5250100002)(476003)(221733001)(316002)(99286004)(26005)(6116002)(6916009)(3846002)(66066001)(33656002)(7736002)(3660700001)(97736004)(4326008)(111123002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0102MB3147; H:HE1PR0102MB2588.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:3; A:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: +GLwzU7UMKKSnxxoWuFl/sFQJLjWYqw0aX8gsxqj9NZV+AubX4kG8KUC3dwKy4HkHkESIepR8ol+w4vmGBAA1dMqrn8Sxjg94sjPWxO8X3PdjpCMyGimKTHp26soxTqSktTJ+1qVjcrI6h3sr3N5r/b0Zb5FLMtjT/rXQiy/8c2c6NgAm/HneWif0YEu+zvC spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: afd099fd-3fbc-46d9-cf00-08d5aae8d76b X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: afd099fd-3fbc-46d9-cf00-08d5aae8d76b X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Apr 2018 20:12:16.5313 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0102MB3147 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 20:12:22 -0000 Hi Harry, I=92m simply using the =93vm=94 utility as in vm switch create public vm switch add public igb1 That must make underlying calls to if config or equivalent as the bridge an= d tap interfaces are created automatically. The vm template file has these relevant parameters network0_type=3D=93virtio-net=94 network0_switch=3D=93public=94 I=92ve done nothing to the igb1 interface other than connect it to a physic= al switch on the appropriate VLAN. Regards, Paul Esson | Redstor Limited t +44 (0)118 951 5235 m +44 (0)776 690 6514 e paul.esson@redstor.com ________________________________ From: Harry Schmalzbauer Sent: Wednesday, April 25, 2018 8:06:58 PM To: Paul Esson Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve networking Bez=FCglich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > Hi Folks, > > I'm struggling with networking when using vm-bhyve on FreeBSD 11.1-RELEAS= E. I have two NICs and have configured the first (igb0) on a management net= work and want to use the second (igb1) for VMs. However, I can't get any V= M to communicate through the virtual switch if I have igb1 added to it. If= I take the NIC out of the switch and configure an ipv4 address on it I can= reach other hosts on the relevant subnet so I believe the poet set-up is v= alid. If I replace igb1 in the switch with igb0, I can then configure VMs = on my management network and they have network connectivity. Can I only us= e Hello, a example of the command you used was nice. I guess you're using if_bridge(4) =96 the example would clarify. But there's ng_bridge(4) and vale(4) also, and others are using OpenVSwitch... > an interface that has an IP address configured at the host level before = adding it to the switch? I've used other No. But the interface has to be in promisc mode. And some offloading functions must be disabled, but in case of if_bridge(4), it's done automagically (and reverted if you remove the interface again). Hth, -harry From owner-freebsd-virtualization@freebsd.org Wed Apr 25 20:34:07 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB82AFAF852 for ; Wed, 25 Apr 2018 20:34:06 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3EA868267D for ; Wed, 25 Apr 2018 20:34:05 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w3PKXvOp097828; Wed, 25 Apr 2018 13:33:57 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w3PKXv8M097827; Wed, 25 Apr 2018 13:33:57 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> Subject: Re: bhyve networking In-Reply-To: To: Paul Esson Date: Wed, 25 Apr 2018 13:33:57 -0700 (PDT) CC: Harry Schmalzbauer , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 20:34:07 -0000 [ Charset windows-1252 unsupported, converting... ] > Hi Harry, > I?m simply using the ?vm? utility as in > > vm switch create public > vm switch add public igb1 > > That must make underlying calls to if config or equivalent as the bridge and tap interfaces are created automatically. > > The vm template file has these relevant parameters > > network0_type=?virtio-net? > network0_switch=?public? > > I?ve done nothing to the igb1 interface other than connect it to a physical switch on the appropriate VLAN. How have you configured your vtnet devices inside the guest? If your pass a "trunked" ethernet device to a guest the guest is going to need to run vlan decapuslation. I do this here, and it works fine. vtnet0: flags=8943 metric 0 mtu 1500 options=80028 ether 58:9c:fc:0e:8b:ec nd6 options=29 media: Ethernet 10Gbase-T status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 nd6 options=21 groups: lo vlan48: flags=8843 metric 0 mtu 1500 ether 58:9c:fc:0e:8b:ec inet 192.168.48.38 netmask 0xffffff00 broadcast 192.168.48.255 nd6 options=29 media: Ethernet 10Gbase-T status: active vlan: 48 vlanpcp: 0 parent interface: vtnet0 groups: vlan ... > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > > Hi Folks, > > > > I'm struggling with networking when using vm-bhyve on FreeBSD 11.1-RELEASE. I have two NICs and have configured the first (igb0) on a management network and want to use the second (igb1) for VMs. However, I can't get any VM to communicate through the virtual switch if I have igb1 added to it. If I take the NIC out of the switch and configure an ipv4 address on it I can reach other hosts on the relevant subnet so I believe the poet set-up is valid. If I replace igb1 in the switch with igb0, I can then configure VMs on my management network and they have network connectivity. Can I only use > > Hello, > > a example of the command you used was nice. > I guess you're using if_bridge(4) ? the example would clarify. > But there's ng_bridge(4) and vale(4) also, and others are using > OpenVSwitch... > > > an interface that has an IP address configured at the host level before adding it to the switch? I've used other > > No. > But the interface has to be in promisc mode. And some offloading > functions must be disabled, but in case of if_bridge(4), it's done > automagically (and reverted if you remove the interface again). > > Hth, > > -harry > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Wed Apr 25 21:15:56 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D9C77FB0AB2 for ; Wed, 25 Apr 2018 21:15:55 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-ve1eur03on0607.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe09::607]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFBD66BD90 for ; Wed, 25 Apr 2018 21:15:54 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=eNMeY4Vq31Oi5Gk4nw2eaxgylQIhqLivgZVFowLsl64=; b=W9qhw7VQLHiX6cszdNStaeFmOI1NhfBXA0zpjkaTJgnuXYoWq2EbF2Wpu6u9RBecw5Nu2dE/Tg1gzX7R7EN3dS9fRfhDC2RwzCr8pMeuj20V0vjilmiQRrOhcqcwnJKhQCND+Y17KTqWSHhEUWrWKw435WzlHjG1EZk7WAA3qyM= Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com (10.170.251.141) by HE1PR0102MB3274.eurprd01.prod.exchangelabs.com (10.167.124.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.13; Wed, 25 Apr 2018 21:15:51 +0000 Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152]) by HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152%13]) with mapi id 15.20.0696.020; Wed, 25 Apr 2018 21:15:51 +0000 From: Paul Esson To: "Rodney W. Grimes" CC: Harry Schmalzbauer , "freebsd-virtualization@freebsd.org" Subject: Re: bhyve networking Thread-Topic: bhyve networking Thread-Index: AdPcwrhbbh0fzB+SRv2CNcsNb6p8OQABdzsAAAJH2bQAAMHYgAABdrkY Date: Wed, 25 Apr 2018 21:15:51 +0000 Message-ID: References: , <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-originating-ip: [13.95.219.91] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0102MB3274; 7:At9mcFfddXJNOSzS8SRCqZ/WdGrH4AxjDL90Ra8wacDdEjTxI78uNH7rkdcZxDpd+u+SdIZKyPiTJlWAUP4xG1DDMtuOTXjKWN84rp6LgX0SN84hwC9gsHg9uRyxUvRffKKXpmN24/nUPVps3ksiV9zLGxTicyPN6++xM8fCUkmzAaLCuNf+znlb2ni9UDTn630ivREqd8Jy/SYXMhGeGJSWB1XvmVpja5OdbiKYPMX3mc9CiWh8T1Wm5yyahfCA x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0102MB3274; x-ms-traffictypediagnostic: HE1PR0102MB3274: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(788757137089)(81439100147899)(75325880899374)(21532816269658)(17755550239193); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(3002001)(10201501046)(93006095)(93001095)(3231232)(944501410)(52105095)(6041310)(20161123558120)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(6072148)(201708071742011); SRVR:HE1PR0102MB3274; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0102MB3274; x-forefront-prvs: 06530126A4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(376002)(39850400004)(396003)(366004)(39380400002)(199004)(189003)(2906002)(59450400001)(2900100001)(3480700004)(3660700001)(221733001)(53546011)(25786009)(11346002)(26005)(105586002)(44832011)(966005)(446003)(6506007)(86362001)(7116003)(106356001)(476003)(3280700002)(606006)(478600001)(316002)(14454004)(74316002)(81156014)(5250100002)(4326008)(55016002)(97736004)(6116002)(6306002)(66066001)(99286004)(6246003)(3846002)(54906003)(6436002)(6916009)(5660300001)(9686003)(33656002)(229853002)(102836004)(486006)(8936002)(68736007)(53936002)(7736002)(236005)(8676002)(186003)(54896002)(81166006)(76176011)(7696005)(111123002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0102MB3274; H:HE1PR0102MB2588.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: dZoj660Gr72UEsMKK/2uM1KCszdMKjvPMl3BFl2T8CqIRAraQwhTwXsMjOE9+jPd7txtpr+K5pcr/FEDuAT27wzieTLG5ELBmd0/0TncXVNUUrlE1tw0PbtnFa94gq1ur23Qbt4F0FSnl9+rhgnQJTAVweOjjr6i5vTXbDj+3zJ0La5RKFsFrI2XO0WAA2IX spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 13616142-acf4-4519-7005-08d5aaf1b988 X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: 13616142-acf4-4519-7005-08d5aaf1b988 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Apr 2018 21:15:51.8578 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0102MB3274 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 21:15:56 -0000 Hi Rod, Can you share a command line for that? I also tried presenting an access p= ort from my switch on a specific VLAN - not trimmed. Would I still have to= tag the interface on the guest in that scenario? Regards, Paul Esson | Redstor Limited t +44 (0)118 951 5235 m +44 (0)776 690 6514 e paul.esson@redstor.com ________________________________ From: Rodney W. Grimes Sent: Wednesday, April 25, 2018 9:33:57 PM To: Paul Esson Cc: Harry Schmalzbauer; freebsd-virtualization@freebsd.org Subject: Re: bhyve networking [ Charset windows-1252 unsupported, converting... ] > Hi Harry, > I?m simply using the ?vm? utility as in > > vm switch create public > vm switch add public igb1 > > That must make underlying calls to if config or equivalent as the bridge = and tap interfaces are created automatically. > > The vm template file has these relevant parameters > > network0_type=3D?virtio-net? > network0_switch=3D?public? > > I?ve done nothing to the igb1 interface other than connect it to a physic= al switch on the appropriate VLAN. How have you configured your vtnet devices inside the guest? If your pass = a "trunked" ethernet device to a guest the guest is going to need to run vlan decapuslation. I do this= here, and it works fine. vtnet0: flags=3D8943 metric= 0 mtu 1500 options=3D80028 ether 58:9c:fc:0e:8b:ec nd6 options=3D29 media: Ethernet 10Gbase-T status: active lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 nd6 options=3D21 groups: lo vlan48: flags=3D8843 metric 0 mtu 1= 500 ether 58:9c:fc:0e:8b:ec inet 192.168.48.38 netmask 0xffffff00 broadcast 192.168.48.255 nd6 options=3D29 media: Ethernet 10Gbase-T status: active vlan: 48 vlanpcp: 0 parent interface: vtnet0 groups: vlan ... > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > > Hi Folks, > > > > I'm struggling with networking when using vm-bhyve on FreeBSD 11.1-RELE= ASE. I have two NICs and have configured the first (igb0) on a management n= etwork and want to use the second (igb1) for VMs. However, I can't get any= VM to communicate through the virtual switch if I have igb1 added to it. = If I take the NIC out of the switch and configure an ipv4 address on it I c= an reach other hosts on the relevant subnet so I believe the poet set-up is= valid. If I replace igb1 in the switch with igb0, I can then configure VM= s on my management network and they have network connectivity. Can I only = use > > Hello, > > a example of the command you used was nice. > I guess you're using if_bridge(4) ? the example would clarify. > But there's ng_bridge(4) and vale(4) also, and others are using > OpenVSwitch... > > > an interface that has an IP address configured at the host level befor= e adding it to the switch? I've used other > > No. > But the interface has to be in promisc mode. And some offloading > functions must be disabled, but in case of if_bridge(4), it's done > automagically (and reverted if you remove the interface again). > > Hth, > > -harry > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" > > -- Rod Grimes rgrimes@freebsd.= org From owner-freebsd-virtualization@freebsd.org Wed Apr 25 21:22:37 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DC75FB0CB1 for ; Wed, 25 Apr 2018 21:22:37 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C92906D349 for ; Wed, 25 Apr 2018 21:22:36 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-it0-x236.google.com with SMTP id e20-v6so21170678itc.1 for ; Wed, 25 Apr 2018 14:22:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=LXocDLdejXVUKknM3NfJeiG/DZWudMcscEQnFr2Cyak=; b=Z1SJwjDJwNgHHOi30LvFga20XwpYg7Wab4fqsbx1lOvXR0OW7e0geRbOSB6D5F+WCR 5GLSXlOk9ncsOFKVYbj4u1FM0YbhWxS5AFC6JassOrnKZJLQa62Hc9N71xXNigrXHa0+ QxQkNzZPCezkgz3KvpNP1QUffjQoefXkj17uSnAbRGmMGy0mHO7O8z6ipx+nmLNqO7f/ PxlklvVv7lF4tYAZhWbhkU3P0WeKRjNTeawW6nGEqHn2rUPKiLeKiLK+Qfg86oLI1ESc GiRKFupocEYDPLWBxo0xIHSyKE3DLod7t45KY51DyqqRFhpBrZyGz1tzf+hQCxgEMFFh UiDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=LXocDLdejXVUKknM3NfJeiG/DZWudMcscEQnFr2Cyak=; b=V9EUzS7XTBSe31tQ5FII97Dbn187gDAqdl9el1nbcSdbOgGyu3258VV6DovPsCL+AT zK6xcXC+hSMN/B6zBBYFFCas+HcmfwZciNmw2SUVMe/CQQzzvSRxLWqtKBMHjStRG8hM 45ouKwgZZwGIsvVEd3A74HUhruLT03sLE+piZaZBi/rvh2VJ1xX6/3EK/7EJZ/kfulZl VFOw56pvNNE7l4mTM/dvfm9Q0ZpgvKWJGQsJFQ+90htlBUNKbIuW2/EnDT9zSH65e1o/ wULQ1IePsTJ437b+CmPfksrHeRTj1JLF5fHHd95X72x0tpRDDAkEb4U6ePZMo4DGXibu 2SUQ== X-Gm-Message-State: ALQs6tDZJ2++wHZsgGjtS+llv0tT6o66srk7CswajGjNakADF/MdywAs Pqar7nJZ4PuPadDITp6ZYtEDS3Gq4VpfRU8jlNCk0A== X-Google-Smtp-Source: AIpwx4/OVm8r25sBOpMJld5NVqOkOCjcfnRmirsWYrBTb82zf3SBsQCdfG+dTTNDiNfdIjx31E29KtDKvyjmEd/vlrU= X-Received: by 2002:a24:d34c:: with SMTP id n73-v6mr22799654itg.104.1524691356102; Wed, 25 Apr 2018 14:22:36 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a02:8d57:0:0:0:0:0 with HTTP; Wed, 25 Apr 2018 14:22:35 -0700 (PDT) In-Reply-To: References: <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> From: Adam Date: Wed, 25 Apr 2018 16:22:35 -0500 Message-ID: Subject: Re: bhyve networking To: Paul Esson Cc: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 21:22:37 -0000 On Wed, Apr 25, 2018 at 4:15 PM, Paul Esson wrote: > Hi Rod, > Can you share a command line for that? https://www.freebsd.org/doc/handbook/network-vlan.html > I also tried presenting an access port from my switch on a specific VLAN - > not trimmed. Would I still have to tag the interface on the guest in that > scenario? > No. What did tcpdump see there? -- Adam From owner-freebsd-virtualization@freebsd.org Wed Apr 25 21:31:32 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B08DFB0E60 for ; Wed, 25 Apr 2018 21:31:32 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8EF956F2FA for ; Wed, 25 Apr 2018 21:31:31 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w3PLVTSR098069; Wed, 25 Apr 2018 14:31:29 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w3PLVTs3098068; Wed, 25 Apr 2018 14:31:29 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201804252131.w3PLVTs3098068@pdx.rh.CN85.dnsmgr.net> Subject: Re: bhyve networking In-Reply-To: To: Paul Esson Date: Wed, 25 Apr 2018 14:31:29 -0700 (PDT) CC: "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 21:31:32 -0000 > Hi Rod, > Can you share a command line for that? Its not just a single command, but you want these in /etc/rc.conf of the GUEST: network_interfaces="lo0" cloned_interfaces="vlan48" ifconfig_lo0=" inet 127.0.0.1 netmask 0xff000000" ifconfig_vtnet0=" up" ifconfig_vlan48=" inet 192.168.48.38 netmask 0xffffff00 vlan 48 vlandev vtnet0" That may be your issue... is your vtnet0 "up" in the guest. It would help a whole lot to share more of the info about your system, from commands, not from "vm-bhyve" settings. Like ifconfig -a on the host and the guest would be a starting point. > I also tried presenting an access port from my switch on a specific VLAN - not trimmed. Trimmed? You mean you set the switch port to untagged mode, and had the switch tag/untag the packets to a specific vlan. Be sure you also set the default incoming tag at the switch if you did this, some switches do not follow the vlan setting. > Would I still have to tag the interface on the guest in that scenario? No. If I understand what I think you meant by trimmed. > Regards, > > > Paul Esson | Redstor Limited > t +44 (0)118 951 5235 > m +44 (0)776 690 6514 > e paul.esson@redstor.com > > ________________________________ > From: Rodney W. Grimes > Sent: Wednesday, April 25, 2018 9:33:57 PM > To: Paul Esson > Cc: Harry Schmalzbauer; freebsd-virtualization@freebsd.org > Subject: Re: bhyve networking > > [ Charset windows-1252 unsupported, converting... ] > > Hi Harry, > > I?m simply using the ?vm? utility as in > > > > vm switch create public > > vm switch add public igb1 > > > > That must make underlying calls to if config or equivalent as the bridge and tap interfaces are created automatically. > > > > The vm template file has these relevant parameters > > > > network0_type=?virtio-net? > > network0_switch=?public? > > > > I?ve done nothing to the igb1 interface other than connect it to a physical switch on the appropriate VLAN. > > How have you configured your vtnet devices inside the guest? If your pass a "trunked" ethernet device > to a guest the guest is going to need to run vlan decapuslation. I do this here, and it works fine. > > vtnet0: flags=8943 metric 0 mtu 1500 > options=80028 > ether 58:9c:fc:0e:8b:ec > nd6 options=29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet 127.0.0.1 netmask 0xff000000 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > nd6 options=21 > groups: lo > vlan48: flags=8843 metric 0 mtu 1500 > ether 58:9c:fc:0e:8b:ec > inet 192.168.48.38 netmask 0xffffff00 broadcast 192.168.48.255 > nd6 options=29 > media: Ethernet 10Gbase-T > status: active > vlan: 48 vlanpcp: 0 parent interface: vtnet0 > groups: vlan > > ... > > > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > > > Hi Folks, > > > > > > I'm struggling with networking when using vm-bhyve on FreeBSD 11.1-RELEASE. I have two NICs and have configured the first (igb0) on a management network and want to use the second (igb1) for VMs. However, I can't get any VM to communicate through the virtual switch if I have igb1 added to it. If I take the NIC out of the switch and configure an ipv4 address on it I can reach other hosts on the relevant subnet so I believe the poet set-up is valid. If I replace igb1 in the switch with igb0, I can then configure VMs on my management network and they have network connectivity. Can I only use > > > > Hello, > > > > a example of the command you used was nice. > > I guess you're using if_bridge(4) ? the example would clarify. > > But there's ng_bridge(4) and vale(4) also, and others are using > > OpenVSwitch... > > > > > an interface that has an IP address configured at the host level before adding it to the switch? I've used other > > > > No. > > But the interface has to be in promisc mode. And some offloading > > functions must be disabled, but in case of if_bridge(4), it's done > > automagically (and reverted if you remove the interface again). > > > > Hth, > > > > -harry > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > > > > > -- > Rod Grimes rgrimes@freebsd.org > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Wed Apr 25 21:37:31 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE3F4FB101C for ; Wed, 25 Apr 2018 21:37:30 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-am5eur03on060f.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe08::60f]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D38227158D for ; Wed, 25 Apr 2018 21:37:29 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=rGJfBjOTO7ABzs3rCQQHXEuP7aK/FKjrw3OBmOUxDu8=; b=ovtIoXoRVf2vkuWqdm0X1kc3Xg56HzixkPTCg0Wt9bdZCjkFrM5E8+AxAJkCp+FmganuQj4zCQH/Qjr2HKiAenL3gGR5gwrQ7Hl9RCbN9jGVPjG5xQHsmL8C+PkWgeJNC9f/JgHB4kp6qEkhWrb584xZuRDwqpBJ3jbbOyIrnxc= Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com (10.170.251.141) by HE1PR0102MB3225.eurprd01.prod.exchangelabs.com (10.167.124.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.13; Wed, 25 Apr 2018 21:37:27 +0000 Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152]) by HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152%13]) with mapi id 15.20.0696.020; Wed, 25 Apr 2018 21:37:27 +0000 From: Paul Esson To: "Rodney W. Grimes" CC: "freebsd-virtualization@freebsd.org" Subject: Re: bhyve networking Thread-Topic: bhyve networking Thread-Index: AdPcwrhbbh0fzB+SRv2CNcsNb6p8OQABdzsAAAJH2bQAAMHYgAABdrkYAACLqoAAADVg/Q== Date: Wed, 25 Apr 2018 21:37:27 +0000 Message-ID: References: , <201804252131.w3PLVTs3098068@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201804252131.w3PLVTs3098068@pdx.rh.CN85.dnsmgr.net> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-originating-ip: [40.68.209.210] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0102MB3225; 7:q3l9nHS5XCFVazMkUXoXGX0vEc+qYOxoD8+J7E+dK/kfOS1XPcD3ajfpgskBnjaMZciMNAxIvtKsbcmA8oX05GB8rrkcJNiiO9FcERsi/fFiJXbUFtxiczfMJVdx8eNNzWpDRpOvt3RJlmkPt9q1RtA1R43jPDfcujJGFK62RHMBHjUBEvixXZ4uDwlH8QytRXOqc6Klo9i8maOV6lSmxQDWuy0uXeUilVaCfZUKmPqsc6fWFN9cMP0uxe7qyrug x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0102MB3225; x-ms-traffictypediagnostic: HE1PR0102MB3225: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(209352067349851)(788757137089)(81439100147899)(75325880899374)(21532816269658)(17755550239193); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(3002001)(3231232)(944501410)(52105095)(93006095)(93001095)(10201501046)(6041310)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123562045)(6072148)(201708071742011); SRVR:HE1PR0102MB3225; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0102MB3225; x-forefront-prvs: 06530126A4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(366004)(39850400004)(376002)(396003)(39380400002)(199004)(189003)(81166006)(97736004)(8676002)(54896002)(7736002)(11346002)(486006)(476003)(6506007)(6246003)(106356001)(8936002)(3660700001)(6306002)(59450400001)(86362001)(105586002)(81156014)(446003)(55016002)(53546011)(74316002)(3280700002)(4326008)(221733001)(6916009)(5250100002)(316002)(7696005)(25786009)(229853002)(99286004)(76176011)(66066001)(606006)(14454004)(44832011)(26005)(68736007)(3480700004)(186003)(6116002)(5660300001)(33656002)(7116003)(478600001)(2900100001)(3846002)(2906002)(6436002)(102836004)(53936002)(966005)(236005)(9686003)(111123002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0102MB3225; H:HE1PR0102MB2588.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: pjoZvdypmHsideyGfw5wL2qUobEzGXE7dxBFKee2Pq91LCT4qiAFouj81BLfaGRq8a5MrChA0MXst8NAKLjZU4fDxTK3lc9r32uM2IhdguHMZcFnqbU38RG9MP0yLS++fMkzrmNsrbzEKB+WPSSIjiRjU8QtTewDGwGHwBIrO3A3G9+9WVAk6LwaMqRnto1I spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 81642689-ca3b-4917-d455-08d5aaf4bdad X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81642689-ca3b-4917-d455-08d5aaf4bdad X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Apr 2018 21:37:27.3352 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0102MB3225 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 21:37:31 -0000 Hi Rod, Apologies, predictive testing - for trimmed read trunked. I'll get some co= mmand output when I get back online to system tomorrow. Regards, Paul Esson | Redstor Limited t +44 (0)118 951 5235 m +44 (0)776 690 6514 e paul.esson@redstor.com ________________________________ From: Rodney W. Grimes Sent: Wednesday, April 25, 2018 10:31:29 PM To: Paul Esson Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve networking > Hi Rod, > Can you share a command line for that? Its not just a single command, but you want these in /etc/rc.conf of the GU= EST: network_interfaces=3D"lo0" cloned_interfaces=3D"vlan48" ifconfig_lo0=3D" inet 127.0.0.1 netmask 0xff000000" ifconfig_vtnet0=3D" up" ifconfig_vlan48=3D" inet 192.168.48.38 netmask 0xffffff00 vlan 4= 8 vlandev vtnet0" That may be your issue... is your vtnet0 "up" in the guest. It would help a whole lot to share more of the info about your system, from commands, not from "vm-bhyve" settings. Like ifconfig -a on the host and the guest would be a starting point. > I also tried presenting an access port from my switch on a specific VLAN = - not trimmed. Trimmed? You mean you set the switch port to untagged mode, and had the switch tag/untag the packets to a specific vlan. Be sure you also set the default incoming tag at the switch if you did this= , some switches do not follow the vlan setting. > Would I still have to tag the interface on the guest in that scenario? No. If I understand what I think you meant by trimmed. > Regards, > > > Paul Esson | Redstor Limited > t +44 (0)118 951 5235 > m +44 (0)776 690 6514 > e paul.esson@redstor.com > > ________________________________ > From: Rodney W. Grimes > Sent: Wednesday, April 25, 2018 9:33:57 PM > To: Paul Esson > Cc: Harry Schmalzbauer; freebsd-virtualization@freebsd.org > Subject: Re: bhyve networking > > [ Charset windows-1252 unsupported, converting... ] > > Hi Harry, > > I?m simply using the ?vm? utility as in > > > > vm switch create public > > vm switch add public igb1 > > > > That must make underlying calls to if config or equivalent as the bridg= e and tap interfaces are created automatically. > > > > The vm template file has these relevant parameters > > > > network0_type=3D?virtio-net? > > network0_switch=3D?public? > > > > I?ve done nothing to the igb1 interface other than connect it to a phys= ical switch on the appropriate VLAN. > > How have you configured your vtnet devices inside the guest? If your pas= s a "trunked" ethernet device > to a guest the guest is going to need to run vlan decapuslation. I do th= is here, and it works fine. > > vtnet0: flags=3D8943 metr= ic 0 mtu 1500 > options=3D80028 > ether 58:9c:fc:0e:8b:ec > nd6 options=3D29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=3D8049 metric 0 mtu 16384 > options=3D600003 > inet 127.0.0.1 netmask 0xff000000 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > nd6 options=3D21 > groups: lo > vlan48: flags=3D8843 metric 0 mtu= 1500 > ether 58:9c:fc:0e:8b:ec > inet 192.168.48.38 netmask 0xffffff00 broadcast 192.168.48.255 > nd6 options=3D29 > media: Ethernet 10Gbase-T > status: active > vlan: 48 vlanpcp: 0 parent interface: vtnet0 > groups: vlan > > ... > > > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > > > Hi Folks, > > > > > > I'm struggling with networking when using vm-bhyve on FreeBSD 11.1-RE= LEASE. I have two NICs and have configured the first (igb0) on a management= network and want to use the second (igb1) for VMs. However, I can't get a= ny VM to communicate through the virtual switch if I have igb1 added to it.= If I take the NIC out of the switch and configure an ipv4 address on it I= can reach other hosts on the relevant subnet so I believe the poet set-up = is valid. If I replace igb1 in the switch with igb0, I can then configure = VMs on my management network and they have network connectivity. Can I onl= y use > > > > Hello, > > > > a example of the command you used was nice. > > I guess you're using if_bridge(4) ? the example would clarify. > > But there's ng_bridge(4) and vale(4) also, and others are using > > OpenVSwitch... > > > > > an interface that has an IP address configured at the host level bef= ore adding it to the switch? I've used other > > > > No. > > But the interface has to be in promisc mode. And some offloading > > functions must be disabled, but in case of if_bridge(4), it's done > > automagically (and reverted if you remove the interface again). > > > > Hth, > > > > -harry > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" > > > > > > -- > Rod Grimes rgrimes@freebs= d.org > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" > -- Rod Grimes rgrimes@freebsd.= org From owner-freebsd-virtualization@freebsd.org Thu Apr 26 05:12:49 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B72B6FC2BC2 for ; Thu, 26 Apr 2018 05:12:49 +0000 (UTC) (envelope-from patrick.mooney@joyent.com) Received: from mail-pg0-x22d.google.com (mail-pg0-x22d.google.com [IPv6:2607:f8b0:400e:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44EC674946 for ; Thu, 26 Apr 2018 05:12:49 +0000 (UTC) (envelope-from patrick.mooney@joyent.com) Received: by mail-pg0-x22d.google.com with SMTP id e12so14999982pgn.9 for ; Wed, 25 Apr 2018 22:12:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joyent.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=xIzHilfz1XbEnDs6QL40+1x/acWM5RUmjXkKxW9bz84=; b=Q/x3KrTAhNLVZjEXAOML/3nXixBRKfz/OpiWviYSV0ZlyOkJ9Amvior3CRKnudq+fE DHGRjy1ZEdvIKuemq/4GvGwjlaLxnyXz9rCkQ6p9hygHclQ5MZVU79Mg0WgkU1Fz99WZ tyTUyrUM5Wb7eyVWWh+XUPuo5aaRXuTx5KaJ4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=xIzHilfz1XbEnDs6QL40+1x/acWM5RUmjXkKxW9bz84=; b=YK/cf7RjIxDhetjjgNBPUS+uBhWurcSB6JC/c6flv/8iupAgBt7XU2i09uvR2AzJsj 26ZISeuA2UJa87tVpwMe3kVcQMvYmntzjjUyg/fef/wwei3Hq1U8hNmJEt+zw5FDEA2b RU5hbLn7TDu7lUGY9Lnc2PaG6NZM1T2EaCSSQzH3L2ECrqPNNrDK/UmVChbqhTER1erV 3zTj6XcxFJZTXMzp/0A2rC09xqjJyIBigYqT6B1t+jjjzmKX6DvhS8weIAd2wzdt90wW Lp7rrKHkjfrMCYnUW+FmsALD+FQojGrbM6Z28pqeEuY7KY2Nx3VB6z1DePZ8z1qty/R+ i8YQ== X-Gm-Message-State: ALQs6tDjneI6sfgFGX/IYr8a1Ij7HvaBWsSQkXuTXX76IY57mLwv973q EkMfFkGQmKu7ueOBcoPk89l9Kgql+8OxUhY0FTmoMTxXUEI= X-Google-Smtp-Source: AB8JxZrRPMQM0vdyvzoY7O/yPqlnFdhLHCSm4ZQOgpQGyHhnKKFz5yO07BUfSMU9yZBNKyHfmDe15V8xqU+shpkto4Y= X-Received: by 2002:a17:902:bc48:: with SMTP id t8-v6mr48718plz.133.1524719568103; Wed, 25 Apr 2018 22:12:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.177.197 with HTTP; Wed, 25 Apr 2018 22:12:47 -0700 (PDT) From: Patrick Mooney Date: Thu, 26 Apr 2018 00:12:47 -0500 Message-ID: Subject: Greetings from SmartOS To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 05:12:49 -0000 Hi folks, I'm a member of the team at Joyent working to port bhyve to SmartOS (and eventually illumos). While illumos already has a port of KVM/qemu, the design goals and community around bhyve appear to be a much better match for us. Like ZFS, DTrace, and most recently the BSD loader, this seems to be an area where we can benefit through collaboration. Starting this autumn with the initial port Pluribus crafted in their OpenSolaris fork, we've made good progress on pulling in trunk bhyve from FreeBSD and integrating it into illumos-joyent. Test results have been positive and we're hoping to see it deployed in a wider fashion soon. Now that the project is stabilizing on our end, we would like to reach out about upstreaming fixes or feature work back into FreeBSD. One small fix to posted interrupt handling on Intel (https://smartos.org/bugview/OS-6829) might be a good starting point for us to walk through the process of review and upstreaming. Looking forward to working with you further, - Patrick Mooney From owner-freebsd-virtualization@freebsd.org Thu Apr 26 05:34:05 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 983FAFC3ED8 for ; Thu, 26 Apr 2018 05:34:05 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 344E4794C8 for ; Thu, 26 Apr 2018 05:34:04 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (220-253-154-11.dyn.iinet.net.au [220.253.154.11]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id w3Q5XrQF074753 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 25 Apr 2018 22:33:57 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: Greetings from SmartOS To: Patrick Mooney References: Cc: FreeBSD virtualization mailing list From: Julian Elischer Message-ID: <24af57a1-e907-5680-b471-310366ef6ab8@freebsd.org> Date: Thu, 26 Apr 2018 13:33:48 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 05:34:05 -0000 On 26/4/18 1:12 pm, Patrick Mooney wrote: > Hi folks, > > I'm a member of the team at Joyent working to port bhyve to SmartOS > (and eventually illumos). While illumos already has a port of > KVM/qemu, the design goals and community around bhyve appear to be a > much better match for us. Like ZFS, DTrace, and most recently the BSD > loader, this seems to be an area where we can benefit through > collaboration. welcome From this timezone (awake at this time). I'm sure others will join as the sun makes it's Journey westward. We've found that the various sun-inspired groups have been good development partners. It makes sense to share work.. developers are a scarce resource. > > Starting this autumn with the initial port Pluribus crafted in their who's autumn? Autumn started 7 weeks ago here. That's quick work! :-) > OpenSolaris fork, we've made good progress on pulling in trunk bhyve > from FreeBSD and integrating it into illumos-joyent. Test results > have been positive and we're hoping to see it deployed in a wider > fashion soon. > > Now that the project is stabilizing on our end, we would like to reach > out about upstreaming fixes or feature work back into FreeBSD. One > small fix to posted interrupt handling on Intel > (https://smartos.org/bugview/OS-6829) might be a good starting point > for us to walk through the process of review and upstreaming. > > > Looking forward to working with you further, > > - Patrick Mooney > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" Julian > From owner-freebsd-virtualization@freebsd.org Thu Apr 26 07:39:18 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24C69FA43A5 for ; Thu, 26 Apr 2018 07:39:18 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9597C77EED for ; Thu, 26 Apr 2018 07:39:17 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w3Q7dFfT008519; Thu, 26 Apr 2018 09:39:15 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id D3C738A6; Thu, 26 Apr 2018 09:39:14 +0200 (CEST) Message-ID: <5AE18222.6000900@omnilan.de> Date: Thu, 26 Apr 2018 09:39:14 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Paul Esson CC: "freebsd-virtualization@freebsd.org" Subject: Re: bhyve networking References: , <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Greylist: ACL 130 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Thu, 26 Apr 2018 09:39:15 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 07:39:18 -0000 Bezüglich Paul Esson's Nachricht vom 25.04.2018 23:15 (localtime): > Hi Rod, > Can you share a command line for that? I also tried presenting an > access port from my switch on a specific VLAN - not trimmed. Would I > still have to tag the interface on the guest in that scenario? Hmm, I lost the overview – I'm not familar with 'vm'. To filter a specific id (tag/untag frames) inside the guest: 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnn' 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnm' At boot time by rc(8): vlans_vtnet0="vtnet_dmz vtnet_dmz2" create_args_vtnet_dmz="vlan nnnn" create_args_vtnet_dmz2="vlan nnnm" [To optionally also rename the vlan interfaces after manually creating cloned vlan interfaces, which is what the rc.conf(5) example does: ifconfig rename vlan0 vtnet_dmz; ifconfig rename vlan0 vtnet_dmz2; ] Hth, -harry From owner-freebsd-virtualization@freebsd.org Thu Apr 26 12:40:43 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67F9CFAC94D for ; Thu, 26 Apr 2018 12:40:43 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00044.outbound.protection.outlook.com [40.107.0.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE5A879BA5 for ; Thu, 26 Apr 2018 12:40:42 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=PYVfbdYLtqJWC5mnZPOklj7UzvUHL16yrJA4M2Wz+0Q=; b=hXXawKmachV2FjoIuHkytdnNdcXQ849tuT3VjZLKXOVQAxF5QMhbBMw8fZ4bwbD5Sy/FkaqlUrzPf/aCrL01aYVjg8o76FhxU/Gh7vGGIp//5NI3BinFnMf/IxitysRP0T5oL2ZRg3FSLnuHRod9rn3IzTBygcTZ6ZN8hr1zyxE= Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com (10.170.251.141) by HE1PR0102MB2764.eurprd01.prod.exchangelabs.com (10.170.253.13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.13; Thu, 26 Apr 2018 12:40:39 +0000 Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152]) by HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152%13]) with mapi id 15.20.0715.018; Thu, 26 Apr 2018 12:40:39 +0000 From: Paul Esson To: Harry Schmalzbauer CC: "freebsd-virtualization@freebsd.org" Subject: RE: bhyve networking Thread-Topic: bhyve networking Thread-Index: AdPcwrhbbh0fzB+SRv2CNcsNb6p8OQABdzsAAAJH2bQAAMHYgAABdrkYABXFYAAABzXgsA== Date: Thu, 26 Apr 2018 12:40:38 +0000 Message-ID: References: , <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> <5AE18222.6000900@omnilan.de> In-Reply-To: <5AE18222.6000900@omnilan.de> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-originating-ip: [81.136.184.115] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0102MB2764; 7:l/wn/rrfp+xVLSTMG4vPua7A9AmcHc30G8ljbZyO55lYeQH5bmXzbUzCl85odphLZMipYeJYRpaMApKmXTv/v05fDMHz5hT5LPGutP5MdeDdFXXgdhzuVLxnj8O4jkaytq67czCzctbwonmuWgMX86DIk158mjRTIjXsWGxfC4hDko6tYGRHfX1ctdgC6oXp/jGD3CUTmzLy18kiOa48WHEYu7+TV8FizMGoz+yHr97hXo5rJPqNqJAdJmN+tQMe x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0102MB2764; x-ms-traffictypediagnostic: HE1PR0102MB2764: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(788757137089)(75325880899374)(127952516941037)(21532816269658)(17755550239193); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3231232)(944501410)(52105095)(3002001)(6041310)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123564045)(6072148)(201708071742011); SRVR:HE1PR0102MB2764; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0102MB2764; x-forefront-prvs: 0654257CF5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(346002)(39850400004)(376002)(39380400002)(396003)(13464003)(199004)(189003)(305945005)(55016002)(3280700002)(2906002)(3660700001)(86362001)(66066001)(478600001)(59450400001)(6246003)(5250100002)(221733001)(966005)(2900100001)(25786009)(6116002)(229853002)(3846002)(99286004)(6436002)(186003)(4326008)(446003)(74316002)(15974865002)(476003)(81166006)(76176011)(6916009)(8676002)(7116003)(14454004)(93886005)(7696005)(105586002)(7736002)(6306002)(9686003)(44832011)(97736004)(26005)(33656002)(53546011)(106356001)(5660300001)(486006)(8936002)(53936002)(102836004)(81156014)(316002)(11346002)(68736007)(3480700004)(53946003)(6506007)(111123002)(21314002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0102MB2764; H:HE1PR0102MB2588.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: +YDl8RGaEvbXTHG5d2Oki4Av7XQQLcx+v4H+zbFd9H+ML0ncRYpsJfctv0TP2oEI+xJJktnkSU4cvL4uaKJjhQ/eSkfmcNB7pJq1JDiQ6JKnbReEIoF0zO7pOSnKFsnXFSiG/bVm2H4kPTnE5UMHt9WxUoPAwwyZ0eNSlvVfLyPmmh2LHTVqBebg+n/YC0lR spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 7dd36791-e4df-4290-11ee-08d5ab72ead0 X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7dd36791-e4df-4290-11ee-08d5ab72ead0 X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Apr 2018 12:40:39.6138 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0102MB2764 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 12:40:43 -0000 Hi Folks, Apologies for the lack of detail on my first post. To recap, I am attempti= ng to set-up a guest using vm-bhyve. I have a Dell PER730xd server with qu= ad-port INTEL 350 NIC. The first two ports have been configured on a) a ma= nagement LAN for the host and b) an application LAN for the guests. FreeBSD 11.1-RELEASE-p9 Dell PowerEdge R730xd - INTEL i350 NICs NIC-1 igb0 24:6E:96:B4:61:CC VLAN92 ge-6/0/11 (Host) NIC-2 igb1 24:6E:96:B4:61:CD VLAN101 ge-6/0/18 (Guests) - not a trunk Both interfaces are active as viewed from the host, but I have only assigne= d an ipv4 address to igb0 for management of the host igb0: flags=3D8843 metric 0 mtu 150= 0 options=3D6403bb ether 24:6e:96:b4:61:cc hwaddr 24:6e:96:b4:61:cc inet 172.16.92.20 netmask 0xffffff00 broadcast 172.16.92.255 nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active igb1: flags=3D8c02 metric 0 mtu 1500 = options=3D6403bb ether 24:6e:96:b4:61:cd hwaddr 24:6e:96:b4:61:cd nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active =20 If I assign a temporary address to igb1 I can then ping other computers on = the guests subnet - I've had to hide the address as the network is restrict= ed. # ifconfig igb1 inet xx.xxx.xxx.xx/25 up # ping xx.xxx.xxx.xx PING xx.xxx.xxx.xx (xx.xxx.xxx.xx): 56 data bytes 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D0 ttl=3D64 time=3D0.145 ms 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D1 ttl=3D64 time=3D0.080 ms 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D2 ttl=3D64 time=3D0.078 ms 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D3 ttl=3D64 time=3D0.077 ms 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D4 ttl=3D64 time=3D0.076 ms I then used the "vm" command to create a virtual switch and add interface i= gb1 to it. This automatically created the bridge interface. root@dc1-olbp-hn-01:~ # vm switch create public root@dc1-olbp-hn-01:~ # vm switch add public igb1 root@dc1-olbp-hn-01:~ # vm switch info public ------------------------ Virtual Switch: public ------------------------ type: auto ident: bridge0 vlan: - nat: - physical-ports: igb1 bytes-in: 0 (0.000B) bytes-out: 0 (0.000B) Finally, I created a guest VM and gave its NIC the same ipv4 address detail= s I used previously to test igb1 from the host. This automatically created= the tap interface. igb0: flags=3D8843 metric 0 mtu 150= 0 options=3D6403bb ether 24:6e:96:b4:61:cc hwaddr 24:6e:96:b4:61:cc inet 172.16.92.20 netmask 0xffffff00 broadcast 172.16.92.255 nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active igb1: flags=3D8d02 metric 0 mt= u 1500 options=3D6403bb ether 24:6e:96:b4:61:cd hwaddr 24:6e:96:b4:61:cd nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo bridge0: flags=3D8843 metric 0 mtu = 1500 description: vm-public ether 02:ee:ce:b0:6a:00 nd6 options=3D1 groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap0 flags=3D143 ifmaxaddr 0 port 7 priority 128 path cost 2000000 member: igb1 flags=3D143 ifmaxaddr 0 port 2 priority 128 path cost 20000 tap0: flags=3D8943 metric 0= mtu 1500 description: vmnet-testvm-0-public options=3D80000 ether 00:bd:dd:51:0a:00 hwaddr 00:bd:dd:51:0a:00 nd6 options=3D29 media: Ethernet autoselect status: active groups: tap Opened by PID 1791 >From the guest VM I can see that the interface vtnet0 is up and has the rel= evant ipv4 address information. However, I cannot communicate with any oth= er computer on the guest subnet or beyond. vtnet0: flags=3D8943 metric= 0 mtu 1500 options=3D80028 ether 58:9c:fc:08:4a:20 hwaddr 58:9c:fc:08:4a:20 inet xx.xxx.xxx.xx netmask 0xffffff80 broadcast xx.xxx.xxx.xx nd6 options=3D29 media: Ethernet 10Gbase-T status: active lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo The guest configuration file has the following network details network0_type=3D"virtio-net" network0_switch=3D"public" >From the vm-bhyve.log I see the following Apr 26 07:59:23: initialising Apr 26 07:59:23: [loader: bhyveload] Apr 26 07:59:23: [uefi: no] Apr 26 07:59:23: [cpu: 1] Apr 26 07:59:23: [memory: 256M] Apr 26 07:59:23: [hostbridge: standard] Apr 26 07:59:23: [com ports: com1] Apr 26 07:59:23: [uuid: 417cfb63-491f-11e8-949b-246e96b461cc] Apr 26 07:59:23: [utctime: no] Apr 26 07:59:23: [debug mode: no] Apr 26 07:59:23: [primary disk: disk0] Apr 26 07:59:23: [primary disk dev: sparse-zvol] Apr 26 07:59:23: generated static mac 58:9c:fc:08:4a:20 (based on 'testvm:0= :1524725963:0') Apr 26 07:59:23: initialising network device tap0 Apr 26 07:59:23: adding tap0 -> bridge0 (public) Apr 26 07:59:23: booting Should I have to supply ipv4 details anywhere other than the guest's own vt= net0 interface? If I re-configure the switch to remove the igb1 interface = and add igb0 instead, then change the guest ipv4 address details to the man= agement network (172.16.92.0/24), I can connect to other computers on that = subnet and beyond. =20 vtnet0: flags=3D8943 metric= 0 mtu 1500 options=3D80028 ether 58:9c:fc:08:4a:20 hwaddr 58:9c:fc:08:4a:20 inet 172.16.92.21 netmask 0xffffff80 broadcast 172.16.92.127 nd6 options=3D29 media: Ethernet 10Gbase-T status: active lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo $ ping 172.16.92.11 PING 172.16.92.11 (172.16.92.11): 56 data bytes 64 bytes from 172.16.92.11: icmp_seq=3D0 ttl=3D64 time=3D0.416 ms 64 bytes from 172.16.92.11: icmp_seq=3D1 ttl=3D64 time=3D0.371 ms 64 bytes from 172.16.92.11: icmp_seq=3D2 ttl=3D64 time=3D0.369 ms --- 172.16.92.11 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 0.369/0.385/0.416/0.022 ms $ telnet bbc.co.uk 443 Trying 151.101.192.81... Connected to bbc.co.uk. Escape character is '^]'. Regards, Paul Esson | Redstor Limited t +44 (0)118 951 5235 | m +44 (0)776 690 6514 e paul.esson@redstor.com www.redstor.com -----Original Message----- From: Rodney W. Grimes =20 Sent: 25 April 2018 22:31 To: Paul Esson Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve networking > Hi Rod, > Can you share a command line for that?=20 Its not just a single command, but you want these in /etc/rc.conf of the GU= EST: network_interfaces=3D"lo0" cloned_interfaces=3D"vlan48" ifconfig_lo0=3D" inet 127.0.0.1 netmask 0xff000000" ifconfig_vtnet0=3D" up" ifconfig_vlan48=3D" inet 192.168.48.38 netmask 0xffffff00 vlan 4= 8 vlandev vtnet0" That may be your issue... is your vtnet0 "up" in the guest. It would help a whole lot to share more of the info about your system, from= commands, not from "vm-bhyve" settings. Like ifconfig -a on the host and the guest would be a starting point. > I also tried presenting an access port from my switch on a specific VLAN = - not trimmed. Trimmed? You mean you set the switch port to untagged mode, and had the sw= itch tag/untag the packets to a specific vlan. Be sure you also set the default incoming tag at the switch if you did this= , some switches do not follow the vlan setting. > Would I still have to tag the interface on the guest in that scenario? No. If I understand what I think you meant by trimmed. > Regards, >=20 >=20 > Paul Esson | Redstor Limited > t +44 (0)118 951 5235 > m +44 (0)776 690 6514 > e paul.esson@redstor.com >=20 > ________________________________ > From: Rodney W. Grimes > Sent: Wednesday, April 25, 2018 9:33:57 PM > To: Paul Esson > Cc: Harry Schmalzbauer; freebsd-virtualization@freebsd.org > Subject: Re: bhyve networking >=20 > [ Charset windows-1252 unsupported, converting... ] > > Hi Harry, > > I?m simply using the ?vm? utility as in > > > > vm switch create public > > vm switch add public igb1 > > > > That must make underlying calls to if config or equivalent as the bridg= e and tap interfaces are created automatically. > > > > The vm template file has these relevant parameters > > > > network0_type=3D?virtio-net? > > network0_switch=3D?public? > > > > I?ve done nothing to the igb1 interface other than connect it to a phys= ical switch on the appropriate VLAN. >=20 > How have you configured your vtnet devices inside the guest? If your=20 > pass a "trunked" ethernet device to a guest the guest is going to need to= run vlan decapuslation. I do this here, and it works fine. >=20 > vtnet0: flags=3D8943 metr= ic 0 mtu 1500 > options=3D80028 > ether 58:9c:fc:0e:8b:ec > nd6 options=3D29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=3D8049 metric 0 mtu 16384 > options=3D600003 > inet 127.0.0.1 netmask 0xff000000 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > nd6 options=3D21 > groups: lo > vlan48: flags=3D8843 metric 0 mtu= 1500 > ether 58:9c:fc:0e:8b:ec > inet 192.168.48.38 netmask 0xffffff00 broadcast 192.168.48.255 > nd6 options=3D29 > media: Ethernet 10Gbase-T > status: active > vlan: 48 vlanpcp: 0 parent interface: vtnet0 > groups: vlan >=20 > ... > > > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > > > Hi Folks, > > > > > > I'm struggling with networking when using vm-bhyve on FreeBSD=20 > > > 11.1-RELEASE. I have two NICs and have configured the first (igb0)=20 > > > on a management network and want to use the second (igb1) for VMs. =20 > > > However, I can't get any VM to communicate through the virtual=20 > > > switch if I have igb1 added to it. If I take the NIC out of the=20 > > > switch and configure an ipv4 address on it I can reach other hosts=20 > > > on the relevant subnet so I believe the poet set-up is valid. If=20 > > > I replace igb1 in the switch with igb0, I can then configure VMs=20 > > > on my management network and they have network connectivity. Can=20 > > > I only use > > > > Hello, > > > > a example of the command you used was nice. > > I guess you're using if_bridge(4) ? the example would clarify. > > But there's ng_bridge(4) and vale(4) also, and others are using=20 > > OpenVSwitch... > > > > > an interface that has an IP address configured at the host level bef= ore adding it to the switch? I've used other > > > > No. > > But the interface has to be in promisc mode. And some offloading=20 > > functions must be disabled, but in case of if_bridge(4), it's done=20 > > automagically (and reverted if you remove the interface again). > > > > Hth, > > > > -harry > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list=20 > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" > > > > >=20 > -- > Rod Grimes rgrimes@freebs= d.org > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" >=20 --=20 Rod Grimes rgrimes@freebsd.= org Paul Esson=A0=A0|=A0=A0Redstor Limited t=A0=A0+44 (0)118 951 5235=A0=A0|=A0=A0=A0m=A0=A0+44 (0)776 690 6514 e=A0=A0paul.esson@redstor.com www.redstor.com -----Original Message----- From: Harry Schmalzbauer =20 Sent: 26 April 2018 08:39 To: Paul Esson Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve networking Bez=FCglich Paul Esson's Nachricht vom 25.04.2018 23:15 (localtime): > Hi Rod, > Can you share a command line for that? I also tried presenting an=20 > access port from my switch on a specific VLAN - not trimmed. Would I=20 > still have to tag the interface on the guest in that scenario? Hmm, I lost the overview - I'm not familar with 'vm'. To filter a specific id (tag/untag frames) inside the guest: 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnn' 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnm' At boot time by rc(8): vlans_vtnet0=3D"vtnet_dmz vtnet_dmz2" create_args_vtnet_dmz=3D"vlan nnnn" create_args_vtnet_dmz2=3D"vlan nnnm" [To optionally also rename the vlan interfaces after manually creating clon= ed vlan interfaces, which is what the rc.conf(5) example does: ifconfig rename vlan0 vtnet_dmz; ifconfig rename vlan0 vtnet_dmz2; ] Hth, -harry From owner-freebsd-virtualization@freebsd.org Thu Apr 26 13:32:25 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8B86FAD5AF for ; Thu, 26 Apr 2018 13:32:24 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0042.outbound.protection.outlook.com [104.47.2.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 30BC6837B5 for ; Thu, 26 Apr 2018 13:32:23 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=H+FLOruqXX6cwY+1ncXo8sd4d9kWAOueMM5pYbAKrjs=; b=mHGEP1rmRba4hYI1vxqZrmb5QlpVxhL1IeZLQMFtaGaSL6RGPQxI/gkxA7Cn61Wrmm8bczKXa+U7iggagYlxiksh2etoG/nCU+KvB+O55tkhta9gbVy+M0ukJM0A4spx3QhXOKeggyXgCdUQPHn/DcVMZ0YO63LWz+xPt4CZ8wY= Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com (10.170.251.141) by HE1PR0102MB3321.eurprd01.prod.exchangelabs.com (10.167.124.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.15; Thu, 26 Apr 2018 13:32:20 +0000 Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152]) by HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::dd48:cd01:a067:e152%13]) with mapi id 15.20.0715.018; Thu, 26 Apr 2018 13:32:20 +0000 From: Paul Esson To: Harry Schmalzbauer CC: "freebsd-virtualization@freebsd.org" Subject: RE: bhyve networking Thread-Topic: bhyve networking Thread-Index: AdPcwrhbbh0fzB+SRv2CNcsNb6p8OQABdzsAAAJH2bQAAMHYgAABdrkYABXFYAAABzXgsAAE0VUw Date: Thu, 26 Apr 2018 13:32:20 +0000 Message-ID: References: , <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> <5AE18222.6000900@omnilan.de> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-originating-ip: [81.136.184.115] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0102MB3321; 7:cNsKcYQhLVtSI93upkwel96gKdtwGowGJwJvofCAnUy5vgKnvdLhUyyRLxvR8jc12b04GokGvGUOvCkp/HkSpTvgatsxWbdumn3TR5HEhqkHHYSdm+itWYbosvOTjzRP9KRPjsJl/A93fTon33SIw9AE7yG4sbKL60MCY8SW4E3hW/P//zCyjDlrSVn1huEMzoWO7JztgGNgSzXm1n75mB5vKWz7jj6Dpja0N1Q2NuvPVYMgdUhaVjXsHrG04m/0 x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0102MB3321; x-ms-traffictypediagnostic: HE1PR0102MB3321: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(788757137089)(75325880899374)(127952516941037)(21532816269658)(17755550239193); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(3231232)(944501410)(52105095)(93006095)(93001095)(10201501046)(3002001)(6041310)(20161123564045)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(6072148)(201708071742011); SRVR:HE1PR0102MB3321; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0102MB3321; x-forefront-prvs: 0654257CF5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(376002)(39380400002)(39850400004)(346002)(396003)(199004)(189003)(13464003)(229853002)(4326008)(486006)(7736002)(53546011)(7696005)(966005)(59450400001)(76176011)(33656002)(8676002)(186003)(99286004)(6506007)(6116002)(3480700004)(3846002)(53936002)(74316002)(11346002)(68736007)(305945005)(6916009)(81156014)(81166006)(221733001)(6246003)(6436002)(8936002)(53946003)(3280700002)(478600001)(446003)(6306002)(7116003)(5660300001)(97736004)(55016002)(2906002)(106356001)(44832011)(26005)(86362001)(14454004)(2940100002)(5250100002)(476003)(316002)(66066001)(2900100001)(3660700001)(15974865002)(102836004)(93886005)(105586002)(25786009)(9686003)(111123002)(21314002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0102MB3321; H:HE1PR0102MB2588.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: KQjOg67hEbA3jrs7wisvJ8OOa3a8DQlXq+3m6pYGBFBeM/Xr1x6jpffehBZGY7eBIbJlEdLPF+WG2d2wA173gOFKNYf09Geuj/g0pUwgr140HNI9itf71wR5MCUidRVV9dmYMS+23vazwaGlZhUh8ciZBGi4pCNB5xaqnuX+EKoa0FBNAVY6nXuXK2KcIYZA spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 5fa5e8db-87f7-45a4-117a-08d5ab7a22fc X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5fa5e8db-87f7-45a4-117a-08d5ab7a22fc X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Apr 2018 13:32:20.3629 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0102MB3321 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 13:32:25 -0000 Hi Folks, I've just realised that the igb1 interface is not up in any of the output I= shared. So I took the switch out of the equation and created tap and brid= ge interfaces manually, then added igb1 and tap0 to bridge0 and brought the= bridge up. Finally, I brought igb1 and tap0 up. Once all the interfaces = were up I amended the guest configuration to replace network0_switch=3D"pub= lic" with network0_device=3D"tap0". Now when I start my guest I have netwo= rk connectivity on the guest VLAN. I'd really like to try and use the switch approach if possible and had thou= ght that creating the switch and adding the igb1 interface would have broug= ht igb1 up automatically. Is that the expected behaviour? Regards, Paul Esson=A0=A0|=A0=A0Redstor Limited t=A0=A0+44 (0)118 951 5235=A0=A0|=A0=A0=A0m=A0=A0+44 (0)776 690 6514 e=A0=A0paul.esson@redstor.com www.redstor.com -----Original Message----- From: Paul Esson=20 Sent: 26 April 2018 13:41 To: Harry Schmalzbauer Cc: freebsd-virtualization@freebsd.org Subject: RE: bhyve networking Hi Folks, Apologies for the lack of detail on my first post. To recap, I am attempti= ng to set-up a guest using vm-bhyve. I have a Dell PER730xd server with qu= ad-port INTEL 350 NIC. The first two ports have been configured on a) a ma= nagement LAN for the host and b) an application LAN for the guests. FreeBSD 11.1-RELEASE-p9 Dell PowerEdge R730xd - INTEL i350 NICs NIC-1 igb0 24:6E:96:B4:61:CC VLAN92 ge-6/0/11 (Host) NIC-2 igb1 24:6E:96:B4:61:CD VLAN101 ge-6/0/18 (Guests) - not a trunk Both interfaces are active as viewed from the host, but I have only assigne= d an ipv4 address to igb0 for management of the host igb0: flags=3D8843 metric 0 mtu 150= 0 options=3D6403bb ether 24:6e:96:b4:61:cc hwaddr 24:6e:96:b4:61:cc inet 172.16.92.20 netmask 0xffffff00 broadcast 172.16.92.255 nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active igb1: flags=3D8c02 metric 0 mtu 1500 = options=3D6403bb ether 24:6e:96:b4:61:cd hwaddr 24:6e:96:b4:61:cd nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active =20 If I assign a temporary address to igb1 I can then ping other computers on = the guests subnet - I've had to hide the address as the network is restrict= ed. # ifconfig igb1 inet xx.xxx.xxx.xx/25 up # ping xx.xxx.xxx.xx PING xx.xxx.x= xx.xx (xx.xxx.xxx.xx): 56 data bytes 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D0 ttl=3D64 time=3D0.145 ms 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D1 ttl=3D64 time=3D0.080 ms 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D2 ttl=3D64 time=3D0.078 ms 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D3 ttl=3D64 time=3D0.077 ms 64 bytes from xx.xxx.xxx.xx: icmp_seq=3D4 ttl=3D64 time=3D0.076 ms I then used the "vm" command to create a virtual switch and add interface i= gb1 to it. This automatically created the bridge interface. root@dc1-olbp-hn-01:~ # vm switch create public root@dc1-olbp-hn-01:~ # vm = switch add public igb1 root@dc1-olbp-hn-01:~ # vm switch info public ------------------------ Virtual Switch: public ------------------------ type: auto ident: bridge0 vlan: - nat: - physical-ports: igb1 bytes-in: 0 (0.000B) bytes-out: 0 (0.000B) Finally, I created a guest VM and gave its NIC the same ipv4 address detail= s I used previously to test igb1 from the host. This automatically created= the tap interface. igb0: flags=3D8843 metric 0 mtu 150= 0 options=3D6403bb ether 24:6e:96:b4:61:cc hwaddr 24:6e:96:b4:61:cc inet 172.16.92.20 netmask 0xffffff00 broadcast 172.16.92.255 nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active igb1: flags=3D8d02 metric 0 mt= u 1500 options=3D6403bb ether 24:6e:96:b4:61:cd hwaddr 24:6e:96:b4:61:cd nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo bridge0: flags=3D8843 metric 0 mtu = 1500 description: vm-public ether 02:ee:ce:b0:6a:00 nd6 options=3D1 groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap0 flags=3D143 ifmaxaddr 0 port 7 priority 128 path cost 2000000 member: igb1 flags=3D143 ifmaxaddr 0 port 2 priority 128 path cost 20000 tap0: flags=3D8943 metric 0= mtu 1500 description: vmnet-testvm-0-public options=3D80000 ether 00:bd:dd:51:0a:00 hwaddr 00:bd:dd:51:0a:00 nd6 options=3D29 media: Ethernet autoselect status: active groups: tap Opened by PID 1791 >From the guest VM I can see that the interface vtnet0 is up and has the rel= evant ipv4 address information. However, I cannot communicate with any oth= er computer on the guest subnet or beyond. vtnet0: flags=3D8943 metric= 0 mtu 1500 options=3D80028 ether 58:9c:fc:08:4a:20 hwaddr 58:9c:fc:08:4a:20 inet xx.xxx.xxx.xx netmask 0xffffff80 broadcast xx.xxx.xxx.xx nd6 options=3D29 media: Ethernet 10Gbase-T status: active lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo The guest configuration file has the following network details network0_typ= e=3D"virtio-net" network0_switch=3D"public" >From the vm-bhyve.log I see the following Apr 26 07:59:23: initialising Apr= 26 07:59:23: [loader: bhyveload] Apr 26 07:59:23: [uefi: no] Apr 26 07:5= 9:23: [cpu: 1] Apr 26 07:59:23: [memory: 256M] Apr 26 07:59:23: [hostbri= dge: standard] Apr 26 07:59:23: [com ports: com1] Apr 26 07:59:23: [uuid:= 417cfb63-491f-11e8-949b-246e96b461cc] Apr 26 07:59:23: [utctime: no] Apr 26 07:59:23: [debug mode: no] Apr 26 07:59:23: [primary disk: disk0] Apr 26 07:59:23: [primary disk dev: sparse-zvol] Apr 26 07:59:23: generate= d static mac 58:9c:fc:08:4a:20 (based on 'testvm:0:1524725963:0') Apr 26 07= :59:23: initialising network device tap0 Apr 26 07:59:23: adding tap0 -> br= idge0 (public) Apr 26 07:59:23: booting Should I have to supply ipv4 details anywhere other than the guest's own vt= net0 interface? If I re-configure the switch to remove the igb1 interface = and add igb0 instead, then change the guest ipv4 address details to the man= agement network (172.16.92.0/24), I can connect to other computers on that = subnet and beyond. =20 vtnet0: flags=3D8943 metric= 0 mtu 1500 options=3D80028 ether 58:9c:fc:08:4a:20 hwaddr 58:9c:fc:08:4a:20 inet 172.16.92.21 netmask 0xffffff80 broadcast 172.16.92.127 nd6 options=3D29 media: Ethernet 10Gbase-T status: active lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo $ ping 172.16.92.11 PING 172.16.92.11 (172.16.92.11): 56 data bytes 64 bytes from 172.16.92.11: icmp_seq=3D0 ttl=3D64 time=3D0.416 ms 64 bytes from 172.16.92.11: icmp_seq=3D1 ttl=3D64 time=3D0.371 ms 64 bytes from 172.16.92.11: icmp_seq=3D2 ttl=3D64 time=3D0.369 ms --- 172.16.92.11 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/= avg/max/stddev =3D 0.369/0.385/0.416/0.022 ms $ telnet bbc.co.uk 443 Trying 151.101.192.81... Connected to bbc.co.uk. Escape character is '^]'. Regards, Paul Esson | Redstor Limited t +44 (0)118 951 5235 | m +44 (0)776 690 6514 e paul.esson@redstor.com www.redstor.com -----Original Message----- From: Rodney W. Grimes Sent: 25 April 2018 22:31 To: Paul Esson Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve networking > Hi Rod, > Can you share a command line for that?=20 Its not just a single command, but you want these in /etc/rc.conf of the GU= EST: network_interfaces=3D"lo0" cloned_interfaces=3D"vlan48" ifconfig_lo0=3D" inet 127.0.0.1 netmask 0xff000000" ifconfig_vtnet0=3D" up" ifconfig_vlan48=3D" inet 192.168.48.38 netmask 0xffffff00 vlan 4= 8 vlandev vtnet0" That may be your issue... is your vtnet0 "up" in the guest. It would help a whole lot to share more of the info about your system, from= commands, not from "vm-bhyve" settings. Like ifconfig -a on the host and the guest would be a starting point. > I also tried presenting an access port from my switch on a specific VLAN = - not trimmed. Trimmed? You mean you set the switch port to untagged mode, and had the sw= itch tag/untag the packets to a specific vlan. Be sure you also set the default incoming tag at the switch if you did this= , some switches do not follow the vlan setting. > Would I still have to tag the interface on the guest in that scenario? No. If I understand what I think you meant by trimmed. > Regards, >=20 >=20 > Paul Esson | Redstor Limited > t +44 (0)118 951 5235 > m +44 (0)776 690 6514 > e paul.esson@redstor.com >=20 > ________________________________ > From: Rodney W. Grimes > Sent: Wednesday, April 25, 2018 9:33:57 PM > To: Paul Esson > Cc: Harry Schmalzbauer; freebsd-virtualization@freebsd.org > Subject: Re: bhyve networking >=20 > [ Charset windows-1252 unsupported, converting... ] > > Hi Harry, > > I?m simply using the ?vm? utility as in > > > > vm switch create public > > vm switch add public igb1 > > > > That must make underlying calls to if config or equivalent as the bridg= e and tap interfaces are created automatically. > > > > The vm template file has these relevant parameters > > > > network0_type=3D?virtio-net? > > network0_switch=3D?public? > > > > I?ve done nothing to the igb1 interface other than connect it to a phys= ical switch on the appropriate VLAN. >=20 > How have you configured your vtnet devices inside the guest? If your=20 > pass a "trunked" ethernet device to a guest the guest is going to need to= run vlan decapuslation. I do this here, and it works fine. >=20 > vtnet0: flags=3D8943 metr= ic 0 mtu 1500 > options=3D80028 > ether 58:9c:fc:0e:8b:ec > nd6 options=3D29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=3D8049 metric 0 mtu 16384 > options=3D600003 > inet 127.0.0.1 netmask 0xff000000 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > nd6 options=3D21 > groups: lo > vlan48: flags=3D8843 metric 0 mtu= 1500 > ether 58:9c:fc:0e:8b:ec > inet 192.168.48.38 netmask 0xffffff00 broadcast 192.168.48.255 > nd6 options=3D29 > media: Ethernet 10Gbase-T > status: active > vlan: 48 vlanpcp: 0 parent interface: vtnet0 > groups: vlan >=20 > ... > > > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > > > Hi Folks, > > > > > > I'm struggling with networking when using vm-bhyve on FreeBSD=20 > > > 11.1-RELEASE. I have two NICs and have configured the first (igb0)=20 > > > on a management network and want to use the second (igb1) for VMs. > > > However, I can't get any VM to communicate through the virtual=20 > > > switch if I have igb1 added to it. If I take the NIC out of the=20 > > > switch and configure an ipv4 address on it I can reach other hosts=20 > > > on the relevant subnet so I believe the poet set-up is valid. If=20 > > > I replace igb1 in the switch with igb0, I can then configure VMs=20 > > > on my management network and they have network connectivity. Can=20 > > > I only use > > > > Hello, > > > > a example of the command you used was nice. > > I guess you're using if_bridge(4) ? the example would clarify. > > But there's ng_bridge(4) and vale(4) also, and others are using=20 > > OpenVSwitch... > > > > > an interface that has an IP address configured at the host level bef= ore adding it to the switch? I've used other > > > > No. > > But the interface has to be in promisc mode. And some offloading=20 > > functions must be disabled, but in case of if_bridge(4), it's done=20 > > automagically (and reverted if you remove the interface again). > > > > Hth, > > > > -harry > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list=20 > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" > > > > >=20 > -- > Rod Grimes rgrimes@freebs= d.org > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" >=20 --=20 Rod Grimes rgrimes@freebsd.= org Paul Esson=A0=A0|=A0=A0Redstor Limited t=A0=A0+44 (0)118 951 5235=A0=A0|=A0=A0=A0m=A0=A0+44 (0)776 690 6514 e=A0= =A0paul.esson@redstor.com www.redstor.com -----Original Message----- From: Harry Schmalzbauer Sent: 26 April 2018 08:39 To: Paul Esson Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve networking Bez=FCglich Paul Esson's Nachricht vom 25.04.2018 23:15 (localtime): > Hi Rod, > Can you share a command line for that? I also tried presenting an=20 > access port from my switch on a specific VLAN - not trimmed. Would I=20 > still have to tag the interface on the guest in that scenario? Hmm, I lost the overview - I'm not familar with 'vm'. To filter a specific id (tag/untag frames) inside the guest: 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnn' 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnm' At boot time by rc(8): vlans_vtnet0=3D"vtnet_dmz vtnet_dmz2" create_args_vtnet_dmz=3D"vlan nnnn" create_args_vtnet_dmz2=3D"vlan nnnm" [To optionally also rename the vlan interfaces after manually creating clon= ed vlan interfaces, which is what the rc.conf(5) example does: ifconfig rename vlan0 vtnet_dmz; ifconfig rename vlan0 vtnet_dmz2; ] Hth, -harry From owner-freebsd-virtualization@freebsd.org Thu Apr 26 13:51:18 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AFA85FAD924 for ; Thu, 26 Apr 2018 13:51:18 +0000 (UTC) (envelope-from luciano@vespaperitivo.it) Received: from baobab.bilink.net (baobab.bilink.net [212.45.144.44]) by mx1.freebsd.org (Postfix) with ESMTP id 47AE268F21 for ; Thu, 26 Apr 2018 13:51:17 +0000 (UTC) (envelope-from luciano@vespaperitivo.it) Received: from baobab.bilink.net (localhost [127.0.0.1]) by baobab.bilink.it (Postfix) with ESMTP id 40Wyxs5V5kzD3sS for ; Thu, 26 Apr 2018 15:45:37 +0200 (CEST) Received: from hermes.mcs.it (hermes.mcs.it [192.168.132.21]) by baobab.bilink.it (Postfix) with ESMTP id 40Wyxs4bXyzD3sP for ; Thu, 26 Apr 2018 15:45:37 +0200 (CEST) Received: from mordeus (unknown [192.168.45.6]) by hermes.mcs.it (Postfix) with ESMTP id 8736B9A804 for ; Thu, 26 Apr 2018 15:45:37 +0200 (CEST) Date: Thu, 26 Apr 2018 15:45:37 +0200 From: Luciano Mannucci To: freebsd-virtualization@freebsd.org Subject: Re: bhyve networking In-Reply-To: References: <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> <5AE18222.6000900@omnilan.de> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.32; amd64-portbld-freebsd10.3) X-Face: 4qPv4GNcD; h<7Q/sK>+GqF4=CR@KmnPkSmwd+#%\F`4yjKO3"C]p'z=(oWRnsYBQGM\5g:4skqQY0NnV'dM:Mm:^/_+I@a"; [-s=ogufdF"9ggQ'=y MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-Id: <40Wyxs4bXyzD3sP@baobab.bilink.it> X-Virus-Scanned: PippoLillo, ClamAV using ClamSMTP X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 13:51:18 -0000 On Thu, 26 Apr 2018 13:32:20 +0000 Paul Esson wrote: > I'd really like to try and use the switch approach if possible and had > thought that creating the switch and adding the igb1 interface would have > brought igb1 up automatically. I had to put ifconfig_igb0="UP" in order to have vm and bhyve working. I think this is not documented. Cheers, Luciano. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL / E-MAIL: posthamster@sublink.sublink.ORG / \ AND POSTINGS / WWW: http://www.lesassaie.IT/ From owner-freebsd-virtualization@freebsd.org Thu Apr 26 14:04:20 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CF78FAE567 for ; Thu, 26 Apr 2018 14:04:20 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F8356B780 for ; Thu, 26 Apr 2018 14:04:19 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id w3QDenP1070773; Thu, 26 Apr 2018 15:40:49 +0200 (CEST) Received: from [217.29.44.241] ([217.29.44.241]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id w3QDenxA005565; Thu, 26 Apr 2018 15:40:49 +0200 (CEST) (envelope-from hausen@punkt.de) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: bhyve networking From: "Patrick M. Hausen" In-Reply-To: Date: Thu, 26 Apr 2018 15:40:49 +0200 Cc: Harry Schmalzbauer , "freebsd-virtualization@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <777E7D86-7847-48CB-AE1E-09C0B323037E@punkt.de> References: <201804252033.w3PKXv8M097827@pdx.rh.CN85.dnsmgr.net> <5AE18222.6000900@omnilan.de> To: Paul Esson X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 14:04:20 -0000 Hi! > Am 26.04.2018 um 15:32 schrieb Paul Esson : > I'd really like to try and use the switch approach if possible and had = thought that creating the switch and adding the igb1 interface would = have brought igb1 up automatically. Is that the expected behaviour? You have to "ifconfig igb1 up" manually for any of the bridging = technologies in FreeBSD as far as I know. Definitely with if_bridge. It is not sufficient to = "ifconfig addm" the physical interface. But of course one just puts ifconfig_igb1=3D"up" into rc.conf and forgets about it on a production system ... HTH, Patrick --=20 punkt.de GmbH Internet - Dienstleistungen - Beratung Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100 76133 Karlsruhe info@punkt.de http://punkt.de AG Mannheim 108285 Gf: Juergen Egeling From owner-freebsd-virtualization@freebsd.org Thu Apr 26 14:31:26 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F139FAEC5F for ; Thu, 26 Apr 2018 14:31:26 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 36442719FC for ; Thu, 26 Apr 2018 14:31:24 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w3QEVM4J001779; Thu, 26 Apr 2018 07:31:22 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w3QEVMRa001778; Thu, 26 Apr 2018 07:31:22 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201804261431.w3QEVMRa001778@pdx.rh.CN85.dnsmgr.net> Subject: Re: bhyve networking In-Reply-To: To: Paul Esson Date: Thu, 26 Apr 2018 07:31:22 -0700 (PDT) CC: Harry Schmalzbauer , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 14:31:26 -0000 [ Charset ISO-8859-1 unsupported, converting... ] > Hi Folks, > > Apologies for the lack of detail on my first post. To recap, I am attempting to set-up a guest using vm-bhyve. I have a Dell PER730xd server with quad-port INTEL 350 NIC. The first two ports have been configured on a) a management LAN for the host and b) an application LAN for the guests. > > FreeBSD 11.1-RELEASE-p9 > Dell PowerEdge R730xd - INTEL i350 NICs > > NIC-1 igb0 24:6E:96:B4:61:CC VLAN92 ge-6/0/11 (Host) > NIC-2 igb1 24:6E:96:B4:61:CD VLAN101 ge-6/0/18 (Guests) - not a trunk > > Both interfaces are active as viewed from the host, but I have only assigned an ipv4 address to igb0 for management of the host > > igb0: flags=8843 metric 0 mtu 1500 options=6403bb > ether 24:6e:96:b4:61:cc > hwaddr 24:6e:96:b4:61:cc > inet 172.16.92.20 netmask 0xffffff00 broadcast 172.16.92.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > igb1: flags=8c02 metric 0 mtu 1500 options=6403bb ^^ MIssing UP, interface is down > ether 24:6e:96:b4:61:cd > hwaddr 24:6e:96:b4:61:cd > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > If I assign a temporary address to igb1 I can then ping other computers on the guests subnet - I've had to hide the address as the network is restricted. > > # ifconfig igb1 inet xx.xxx.xxx.xx/25 up > # ping xx.xxx.xxx.xx > PING xx.xxx.xxx.xx (xx.xxx.xxx.xx): 56 data bytes > 64 bytes from xx.xxx.xxx.xx: icmp_seq=0 ttl=64 time=0.145 ms > 64 bytes from xx.xxx.xxx.xx: icmp_seq=1 ttl=64 time=0.080 ms > 64 bytes from xx.xxx.xxx.xx: icmp_seq=2 ttl=64 time=0.078 ms > 64 bytes from xx.xxx.xxx.xx: icmp_seq=3 ttl=64 time=0.077 ms > 64 bytes from xx.xxx.xxx.xx: icmp_seq=4 ttl=64 time=0.076 ms > > I then used the "vm" command to create a virtual switch and add interface igb1 to it. This automatically created the bridge interface. > > root@dc1-olbp-hn-01:~ # vm switch create public > root@dc1-olbp-hn-01:~ # vm switch add public igb1 > root@dc1-olbp-hn-01:~ # vm switch info public > ------------------------ > Virtual Switch: public > ------------------------ > type: auto > ident: bridge0 > vlan: - > nat: - > physical-ports: igb1 > bytes-in: 0 (0.000B) > bytes-out: 0 (0.000B) > > Finally, I created a guest VM and gave its NIC the same ipv4 address details I used previously to test igb1 from the host. This automatically created the tap interface. > > igb0: flags=8843 metric 0 mtu 1500 options=6403bb > ether 24:6e:96:b4:61:cc > hwaddr 24:6e:96:b4:61:cc > inet 172.16.92.20 netmask 0xffffff00 broadcast 172.16.92.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > igb1: flags=8d02 metric 0 mtu 1500 options=6403bb ^^ mising up, interface is down > ether 24:6e:96:b4:61:cd > hwaddr 24:6e:96:b4:61:cd > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > groups: lo > > bridge0: flags=8843 metric 0 mtu 1500 > description: vm-public > ether 02:ee:ce:b0:6a:00 > nd6 options=1 > groups: bridge > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap0 flags=143 > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > member: igb1 flags=143 > ifmaxaddr 0 port 2 priority 128 path cost 20000 > > tap0: flags=8943 metric 0 mtu 1500 > description: vmnet-testvm-0-public > options=80000 > ether 00:bd:dd:51:0a:00 > hwaddr 00:bd:dd:51:0a:00 > nd6 options=29 > media: Ethernet autoselect > status: active > groups: tap > Opened by PID 1791 > > >From the guest VM I can see that the interface vtnet0 is up and has the relevant ipv4 address information. However, I cannot communicate with any other computer on the guest subnet or beyond. Guest may be up, but the host interface is in state down. > vtnet0: flags=8943 metric 0 mtu 1500 > options=80028 > ether 58:9c:fc:08:4a:20 > hwaddr 58:9c:fc:08:4a:20 > inet xx.xxx.xxx.xx netmask 0xffffff80 broadcast xx.xxx.xxx.xx > nd6 options=29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > groups: lo > > The guest configuration file has the following network details > network0_type="virtio-net" > network0_switch="public" > > >From the vm-bhyve.log I see the following > Apr 26 07:59:23: initialising > Apr 26 07:59:23: [loader: bhyveload] > Apr 26 07:59:23: [uefi: no] > Apr 26 07:59:23: [cpu: 1] > Apr 26 07:59:23: [memory: 256M] > Apr 26 07:59:23: [hostbridge: standard] > Apr 26 07:59:23: [com ports: com1] > Apr 26 07:59:23: [uuid: 417cfb63-491f-11e8-949b-246e96b461cc] > Apr 26 07:59:23: [utctime: no] > Apr 26 07:59:23: [debug mode: no] > Apr 26 07:59:23: [primary disk: disk0] > Apr 26 07:59:23: [primary disk dev: sparse-zvol] > Apr 26 07:59:23: generated static mac 58:9c:fc:08:4a:20 (based on 'testvm:0:1524725963:0') > Apr 26 07:59:23: initialising network device tap0 > Apr 26 07:59:23: adding tap0 -> bridge0 (public) > Apr 26 07:59:23: booting > > Should I have to supply ipv4 details anywhere other than the guest's own vtnet0 interface? If I re-configure the switch to remove the igb1 interface and add igb0 instead, then change the guest ipv4 address details to the management network (172.16.92.0/24), I can connect to other computers on that subnet and beyond. > > vtnet0: flags=8943 metric 0 mtu 1500 > options=80028 > ether 58:9c:fc:08:4a:20 > hwaddr 58:9c:fc:08:4a:20 > inet 172.16.92.21 netmask 0xffffff80 broadcast 172.16.92.127 > nd6 options=29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > groups: lo > $ ping 172.16.92.11 > PING 172.16.92.11 (172.16.92.11): 56 data bytes > 64 bytes from 172.16.92.11: icmp_seq=0 ttl=64 time=0.416 ms > 64 bytes from 172.16.92.11: icmp_seq=1 ttl=64 time=0.371 ms > 64 bytes from 172.16.92.11: icmp_seq=2 ttl=64 time=0.369 ms > --- 172.16.92.11 ping statistics --- > 3 packets transmitted, 3 packets received, 0.0% packet loss > round-trip min/avg/max/stddev = 0.369/0.385/0.416/0.022 ms > > $ telnet bbc.co.uk 443 > Trying 151.101.192.81... > Connected to bbc.co.uk. > Escape character is '^]'. > > > Regards, > > > Paul Esson | Redstor Limited > t +44 (0)118 951 5235 | m +44 (0)776 690 6514 > e paul.esson@redstor.com > www.redstor.com > > > > > > -----Original Message----- > From: Rodney W. Grimes > Sent: 25 April 2018 22:31 > To: Paul Esson > Cc: freebsd-virtualization@freebsd.org > Subject: Re: bhyve networking > > > Hi Rod, > > Can you share a command line for that? > Its not just a single command, but you want these in /etc/rc.conf of the GUEST: > network_interfaces="lo0" > cloned_interfaces="vlan48" > ifconfig_lo0=" inet 127.0.0.1 netmask 0xff000000" > ifconfig_vtnet0=" up" > ifconfig_vlan48=" inet 192.168.48.38 netmask 0xffffff00 vlan 48 vlandev vtnet0" > > That may be your issue... is your vtnet0 "up" in the guest. Or better yet, is your igb1 interface "up" on the host? Add ifconfig_igb1="up" to your hosts /etc/rc.conf file. And type: ifconfig igb1 up and your probelm should resolve. > It would help a whole lot to share more of the info about your system, from commands, not from "vm-bhyve" settings. > Like > ifconfig -a > on the host and the guest would be a starting point. > > > I also tried presenting an access port from my switch on a specific VLAN - not trimmed. > Trimmed? You mean you set the switch port to untagged mode, and had the switch tag/untag the packets to a specific vlan. > Be sure you also set the default incoming tag at the switch if you did this, some switches do not follow the vlan setting. > > > Would I still have to tag the interface on the guest in that scenario? > No. If I understand what I think you meant by trimmed. > > > Regards, > > > > > > Paul Esson | Redstor Limited > > t +44 (0)118 951 5235 > > m +44 (0)776 690 6514 > > e paul.esson@redstor.com > > > > ________________________________ > > From: Rodney W. Grimes > > Sent: Wednesday, April 25, 2018 9:33:57 PM > > To: Paul Esson > > Cc: Harry Schmalzbauer; freebsd-virtualization@freebsd.org > > Subject: Re: bhyve networking > > > > [ Charset windows-1252 unsupported, converting... ] > > > Hi Harry, > > > I?m simply using the ?vm? utility as in > > > > > > vm switch create public > > > vm switch add public igb1 > > > > > > That must make underlying calls to if config or equivalent as the bridge and tap interfaces are created automatically. > > > > > > The vm template file has these relevant parameters > > > > > > network0_type=?virtio-net? > > > network0_switch=?public? > > > > > > I?ve done nothing to the igb1 interface other than connect it to a physical switch on the appropriate VLAN. > > > > How have you configured your vtnet devices inside the guest? If your > > pass a "trunked" ethernet device to a guest the guest is going to need to run vlan decapuslation. I do this here, and it works fine. > > > > vtnet0: flags=8943 metric 0 mtu 1500 > > options=80028 > > ether 58:9c:fc:0e:8b:ec > > nd6 options=29 > > media: Ethernet 10Gbase-T > > status: active > > lo0: flags=8049 metric 0 mtu 16384 > > options=600003 > > inet 127.0.0.1 netmask 0xff000000 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > > nd6 options=21 > > groups: lo > > vlan48: flags=8843 metric 0 mtu 1500 > > ether 58:9c:fc:0e:8b:ec > > inet 192.168.48.38 netmask 0xffffff00 broadcast 192.168.48.255 > > nd6 options=29 > > media: Ethernet 10Gbase-T > > status: active > > vlan: 48 vlanpcp: 0 parent interface: vtnet0 > > groups: vlan > > > > ... > > > > > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > > > > Hi Folks, > > > > > > > > I'm struggling with networking when using vm-bhyve on FreeBSD > > > > 11.1-RELEASE. I have two NICs and have configured the first (igb0) > > > > on a management network and want to use the second (igb1) for VMs. > > > > However, I can't get any VM to communicate through the virtual > > > > switch if I have igb1 added to it. If I take the NIC out of the > > > > switch and configure an ipv4 address on it I can reach other hosts > > > > on the relevant subnet so I believe the poet set-up is valid. If > > > > I replace igb1 in the switch with igb0, I can then configure VMs > > > > on my management network and they have network connectivity. Can > > > > I only use > > > > > > Hello, > > > > > > a example of the command you used was nice. > > > I guess you're using if_bridge(4) ? the example would clarify. > > > But there's ng_bridge(4) and vale(4) also, and others are using > > > OpenVSwitch... > > > > > > > an interface that has an IP address configured at the host level before adding it to the switch? I've used other > > > > > > No. > > > But the interface has to be in promisc mode. And some offloading > > > functions must be disabled, but in case of if_bridge(4), it's done > > > automagically (and reverted if you remove the interface again). > > > > > > Hth, > > > > > > -harry > > > _______________________________________________ > > > freebsd-virtualization@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > > > > > > > > > -- > > Rod Grimes rgrimes@freebsd.org > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > > > -- > Rod Grimes rgrimes@freebsd.org > > Paul Esson??|??Redstor Limited > t??+44 (0)118 951 5235??|???m??+44 (0)776 690 6514 > e??paul.esson@redstor.com > www.redstor.com > > > > > > -----Original Message----- > From: Harry Schmalzbauer > Sent: 26 April 2018 08:39 > To: Paul Esson > Cc: freebsd-virtualization@freebsd.org > Subject: Re: bhyve networking > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 23:15 (localtime): > > Hi Rod, > > Can you share a command line for that? I also tried presenting an > > access port from my switch on a specific VLAN - not trimmed. Would I > > still have to tag the interface on the guest in that scenario? > > Hmm, I lost the overview - I'm not familar with 'vm'. > To filter a specific id (tag/untag frames) inside the guest: > 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnn' > 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnm' > > At boot time by rc(8): > vlans_vtnet0="vtnet_dmz vtnet_dmz2" > create_args_vtnet_dmz="vlan nnnn" > create_args_vtnet_dmz2="vlan nnnm" > > [To optionally also rename the vlan interfaces after manually creating cloned vlan interfaces, which is what the rc.conf(5) example does: > ifconfig rename vlan0 vtnet_dmz; ifconfig rename vlan0 vtnet_dmz2; ] > > Hth, > > -harry > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Thu Apr 26 14:33:10 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A525FAECD9 for ; Thu, 26 Apr 2018 14:33:10 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 793B171C38 for ; Thu, 26 Apr 2018 14:33:09 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w3QEX7cx001812; Thu, 26 Apr 2018 07:33:07 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w3QEX77v001811; Thu, 26 Apr 2018 07:33:07 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201804261433.w3QEX77v001811@pdx.rh.CN85.dnsmgr.net> Subject: Re: bhyve networking In-Reply-To: To: Paul Esson Date: Thu, 26 Apr 2018 07:33:07 -0700 (PDT) CC: Harry Schmalzbauer , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 14:33:10 -0000 [ Charset ISO-8859-1 unsupported, converting... ] > Hi Folks, > > I've just realised that the igb1 interface is not up in any of the output I shared. So I took the switch out of the equation and created tap and bridge interfaces manually, then added igb1 and tap0 to bridge0 and brought the bridge up. Finally, I brought igb1 and tap0 up. Once all the interfaces were up I amended the guest configuration to replace network0_switch="public" with network0_device="tap0". Now when I start my guest I have network connectivity on the guest VLAN. > > I'd really like to try and use the switch approach if possible and had thought that creating the switch and adding the igb1 interface would have brought igb1 up automatically. Is that the expected behaviour? No, the expected behavior is to not alter the state of igb1, that would be doing automagic stuff behind your back, you should add ifconfig_igb1="up" to the hosts /etc/rc.conf file. And I think all your issues well resolve and things shall work as you wanted. > Regards, > > Paul Esson??|??Redstor Limited > t??+44 (0)118 951 5235??|???m??+44 (0)776 690 6514 > e??paul.esson@redstor.com > www.redstor.com > > > > > > -----Original Message----- > From: Paul Esson > Sent: 26 April 2018 13:41 > To: Harry Schmalzbauer > Cc: freebsd-virtualization@freebsd.org > Subject: RE: bhyve networking > > Hi Folks, > > Apologies for the lack of detail on my first post. To recap, I am attempting to set-up a guest using vm-bhyve. I have a Dell PER730xd server with quad-port INTEL 350 NIC. The first two ports have been configured on a) a management LAN for the host and b) an application LAN for the guests. > > FreeBSD 11.1-RELEASE-p9 > Dell PowerEdge R730xd - INTEL i350 NICs > > NIC-1 igb0 24:6E:96:B4:61:CC VLAN92 ge-6/0/11 (Host) > NIC-2 igb1 24:6E:96:B4:61:CD VLAN101 ge-6/0/18 (Guests) - not a trunk > > Both interfaces are active as viewed from the host, but I have only assigned an ipv4 address to igb0 for management of the host > > igb0: flags=8843 metric 0 mtu 1500 options=6403bb > ether 24:6e:96:b4:61:cc > hwaddr 24:6e:96:b4:61:cc > inet 172.16.92.20 netmask 0xffffff00 broadcast 172.16.92.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > igb1: flags=8c02 metric 0 mtu 1500 options=6403bb > ether 24:6e:96:b4:61:cd > hwaddr 24:6e:96:b4:61:cd > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > If I assign a temporary address to igb1 I can then ping other computers on the guests subnet - I've had to hide the address as the network is restricted. > > # ifconfig igb1 inet xx.xxx.xxx.xx/25 up # ping xx.xxx.xxx.xx PING xx.xxx.xxx.xx (xx.xxx.xxx.xx): 56 data bytes > 64 bytes from xx.xxx.xxx.xx: icmp_seq=0 ttl=64 time=0.145 ms > 64 bytes from xx.xxx.xxx.xx: icmp_seq=1 ttl=64 time=0.080 ms > 64 bytes from xx.xxx.xxx.xx: icmp_seq=2 ttl=64 time=0.078 ms > 64 bytes from xx.xxx.xxx.xx: icmp_seq=3 ttl=64 time=0.077 ms > 64 bytes from xx.xxx.xxx.xx: icmp_seq=4 ttl=64 time=0.076 ms > > I then used the "vm" command to create a virtual switch and add interface igb1 to it. This automatically created the bridge interface. > > root@dc1-olbp-hn-01:~ # vm switch create public root@dc1-olbp-hn-01:~ # vm switch add public igb1 root@dc1-olbp-hn-01:~ # vm switch info public > ------------------------ > Virtual Switch: public > ------------------------ > type: auto > ident: bridge0 > vlan: - > nat: - > physical-ports: igb1 > bytes-in: 0 (0.000B) > bytes-out: 0 (0.000B) > > Finally, I created a guest VM and gave its NIC the same ipv4 address details I used previously to test igb1 from the host. This automatically created the tap interface. > > igb0: flags=8843 metric 0 mtu 1500 options=6403bb > ether 24:6e:96:b4:61:cc > hwaddr 24:6e:96:b4:61:cc > inet 172.16.92.20 netmask 0xffffff00 broadcast 172.16.92.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > igb1: flags=8d02 metric 0 mtu 1500 options=6403bb > ether 24:6e:96:b4:61:cd > hwaddr 24:6e:96:b4:61:cd > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > groups: lo > > bridge0: flags=8843 metric 0 mtu 1500 > description: vm-public > ether 02:ee:ce:b0:6a:00 > nd6 options=1 > groups: bridge > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap0 flags=143 > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > member: igb1 flags=143 > ifmaxaddr 0 port 2 priority 128 path cost 20000 > > tap0: flags=8943 metric 0 mtu 1500 > description: vmnet-testvm-0-public > options=80000 > ether 00:bd:dd:51:0a:00 > hwaddr 00:bd:dd:51:0a:00 > nd6 options=29 > media: Ethernet autoselect > status: active > groups: tap > Opened by PID 1791 > > >From the guest VM I can see that the interface vtnet0 is up and has the relevant ipv4 address information. However, I cannot communicate with any other computer on the guest subnet or beyond. > > vtnet0: flags=8943 metric 0 mtu 1500 > options=80028 > ether 58:9c:fc:08:4a:20 > hwaddr 58:9c:fc:08:4a:20 > inet xx.xxx.xxx.xx netmask 0xffffff80 broadcast xx.xxx.xxx.xx > nd6 options=29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > groups: lo > > The guest configuration file has the following network details network0_type="virtio-net" > network0_switch="public" > > >From the vm-bhyve.log I see the following Apr 26 07:59:23: initialising Apr 26 07:59:23: [loader: bhyveload] Apr 26 07:59:23: [uefi: no] Apr 26 07:59:23: [cpu: 1] Apr 26 07:59:23: [memory: 256M] Apr 26 07:59:23: [hostbridge: standard] Apr 26 07:59:23: [com ports: com1] Apr 26 07:59:23: [uuid: 417cfb63-491f-11e8-949b-246e96b461cc] > Apr 26 07:59:23: [utctime: no] > Apr 26 07:59:23: [debug mode: no] > Apr 26 07:59:23: [primary disk: disk0] > Apr 26 07:59:23: [primary disk dev: sparse-zvol] Apr 26 07:59:23: generated static mac 58:9c:fc:08:4a:20 (based on 'testvm:0:1524725963:0') Apr 26 07:59:23: initialising network device tap0 Apr 26 07:59:23: adding tap0 -> bridge0 (public) Apr 26 07:59:23: booting > > Should I have to supply ipv4 details anywhere other than the guest's own vtnet0 interface? If I re-configure the switch to remove the igb1 interface and add igb0 instead, then change the guest ipv4 address details to the management network (172.16.92.0/24), I can connect to other computers on that subnet and beyond. > > vtnet0: flags=8943 metric 0 mtu 1500 > options=80028 > ether 58:9c:fc:08:4a:20 > hwaddr 58:9c:fc:08:4a:20 > inet 172.16.92.21 netmask 0xffffff80 broadcast 172.16.92.127 > nd6 options=29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > groups: lo > $ ping 172.16.92.11 > PING 172.16.92.11 (172.16.92.11): 56 data bytes > 64 bytes from 172.16.92.11: icmp_seq=0 ttl=64 time=0.416 ms > 64 bytes from 172.16.92.11: icmp_seq=1 ttl=64 time=0.371 ms > 64 bytes from 172.16.92.11: icmp_seq=2 ttl=64 time=0.369 ms > --- 172.16.92.11 ping statistics --- > 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.369/0.385/0.416/0.022 ms > > $ telnet bbc.co.uk 443 > Trying 151.101.192.81... > Connected to bbc.co.uk. > Escape character is '^]'. > > > Regards, > > > Paul Esson | Redstor Limited > t +44 (0)118 951 5235 | m +44 (0)776 690 6514 > e paul.esson@redstor.com > www.redstor.com > > > > > > -----Original Message----- > From: Rodney W. Grimes > Sent: 25 April 2018 22:31 > To: Paul Esson > Cc: freebsd-virtualization@freebsd.org > Subject: Re: bhyve networking > > > Hi Rod, > > Can you share a command line for that? > Its not just a single command, but you want these in /etc/rc.conf of the GUEST: > network_interfaces="lo0" > cloned_interfaces="vlan48" > ifconfig_lo0=" inet 127.0.0.1 netmask 0xff000000" > ifconfig_vtnet0=" up" > ifconfig_vlan48=" inet 192.168.48.38 netmask 0xffffff00 vlan 48 vlandev vtnet0" > > That may be your issue... is your vtnet0 "up" in the guest. > > It would help a whole lot to share more of the info about your system, from commands, not from "vm-bhyve" settings. > Like > ifconfig -a > on the host and the guest would be a starting point. > > > I also tried presenting an access port from my switch on a specific VLAN - not trimmed. > Trimmed? You mean you set the switch port to untagged mode, and had the switch tag/untag the packets to a specific vlan. > Be sure you also set the default incoming tag at the switch if you did this, some switches do not follow the vlan setting. > > > Would I still have to tag the interface on the guest in that scenario? > No. If I understand what I think you meant by trimmed. > > > Regards, > > > > > > Paul Esson | Redstor Limited > > t +44 (0)118 951 5235 > > m +44 (0)776 690 6514 > > e paul.esson@redstor.com > > > > ________________________________ > > From: Rodney W. Grimes > > Sent: Wednesday, April 25, 2018 9:33:57 PM > > To: Paul Esson > > Cc: Harry Schmalzbauer; freebsd-virtualization@freebsd.org > > Subject: Re: bhyve networking > > > > [ Charset windows-1252 unsupported, converting... ] > > > Hi Harry, > > > I?m simply using the ?vm? utility as in > > > > > > vm switch create public > > > vm switch add public igb1 > > > > > > That must make underlying calls to if config or equivalent as the bridge and tap interfaces are created automatically. > > > > > > The vm template file has these relevant parameters > > > > > > network0_type=?virtio-net? > > > network0_switch=?public? > > > > > > I?ve done nothing to the igb1 interface other than connect it to a physical switch on the appropriate VLAN. > > > > How have you configured your vtnet devices inside the guest? If your > > pass a "trunked" ethernet device to a guest the guest is going to need to run vlan decapuslation. I do this here, and it works fine. > > > > vtnet0: flags=8943 metric 0 mtu 1500 > > options=80028 > > ether 58:9c:fc:0e:8b:ec > > nd6 options=29 > > media: Ethernet 10Gbase-T > > status: active > > lo0: flags=8049 metric 0 mtu 16384 > > options=600003 > > inet 127.0.0.1 netmask 0xff000000 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > > nd6 options=21 > > groups: lo > > vlan48: flags=8843 metric 0 mtu 1500 > > ether 58:9c:fc:0e:8b:ec > > inet 192.168.48.38 netmask 0xffffff00 broadcast 192.168.48.255 > > nd6 options=29 > > media: Ethernet 10Gbase-T > > status: active > > vlan: 48 vlanpcp: 0 parent interface: vtnet0 > > groups: vlan > > > > ... > > > > > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 20:44 (localtime): > > > > Hi Folks, > > > > > > > > I'm struggling with networking when using vm-bhyve on FreeBSD > > > > 11.1-RELEASE. I have two NICs and have configured the first (igb0) > > > > on a management network and want to use the second (igb1) for VMs. > > > > However, I can't get any VM to communicate through the virtual > > > > switch if I have igb1 added to it. If I take the NIC out of the > > > > switch and configure an ipv4 address on it I can reach other hosts > > > > on the relevant subnet so I believe the poet set-up is valid. If > > > > I replace igb1 in the switch with igb0, I can then configure VMs > > > > on my management network and they have network connectivity. Can > > > > I only use > > > > > > Hello, > > > > > > a example of the command you used was nice. > > > I guess you're using if_bridge(4) ? the example would clarify. > > > But there's ng_bridge(4) and vale(4) also, and others are using > > > OpenVSwitch... > > > > > > > an interface that has an IP address configured at the host level before adding it to the switch? I've used other > > > > > > No. > > > But the interface has to be in promisc mode. And some offloading > > > functions must be disabled, but in case of if_bridge(4), it's done > > > automagically (and reverted if you remove the interface again). > > > > > > Hth, > > > > > > -harry > > > _______________________________________________ > > > freebsd-virtualization@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > > > > > > > > > -- > > Rod Grimes rgrimes@freebsd.org > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > > > -- > Rod Grimes rgrimes@freebsd.org > > Paul Esson??|??Redstor Limited > t??+44 (0)118 951 5235??|???m??+44 (0)776 690 6514 e??paul.esson@redstor.com www.redstor.com > > > > > > -----Original Message----- > From: Harry Schmalzbauer > Sent: 26 April 2018 08:39 > To: Paul Esson > Cc: freebsd-virtualization@freebsd.org > Subject: Re: bhyve networking > > Bez?glich Paul Esson's Nachricht vom 25.04.2018 23:15 (localtime): > > Hi Rod, > > Can you share a command line for that? I also tried presenting an > > access port from my switch on a specific VLAN - not trimmed. Would I > > still have to tag the interface on the guest in that scenario? > > Hmm, I lost the overview - I'm not familar with 'vm'. > To filter a specific id (tag/untag frames) inside the guest: > 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnn' > 'ifconfig vlan[N] create vlandev vtnet0 vlan nnnm' > > At boot time by rc(8): > vlans_vtnet0="vtnet_dmz vtnet_dmz2" > create_args_vtnet_dmz="vlan nnnn" > create_args_vtnet_dmz2="vlan nnnm" > > [To optionally also rename the vlan interfaces after manually creating cloned vlan interfaces, which is what the rc.conf(5) example does: > ifconfig rename vlan0 vtnet_dmz; ifconfig rename vlan0 vtnet_dmz2; ] > > Hth, > > -harry > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Fri Apr 27 07:41:14 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0C39FC35A7 for ; Fri, 27 Apr 2018 07:41:13 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 6A4B76E3B8 for ; Fri, 27 Apr 2018 07:41:13 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id E7A2820B4B61 for ; Fri, 27 Apr 2018 17:41:10 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id E3A5628190A for ; Fri, 27 Apr 2018 17:41:10 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ggnm6kS2Y0Ak for ; Fri, 27 Apr 2018 17:41:10 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 972C628099E; Fri, 27 Apr 2018 17:41:08 +1000 (AEST) Subject: Re: Greetings from SmartOS To: Patrick Mooney References: Cc: freebsd-virtualization@freebsd.org From: Peter Grehan Message-ID: <4ab0e2a4-00ec-4688-96ca-c02a0aa5de3e@freebsd.org> Date: Fri, 27 Apr 2018 00:41:07 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=dNCIZtRb c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=5eVCmCvhg37cu/pjidAGzw==:17 a=IkcTkHD0fZMA:10 a=Kd1tUaAdevIA:10 a=EvLsFUAmAAAA:8 a=6I5d2MoRAAAA:8 a=NEAV23lmAAAA:8 a=eKxBt7Tq1UK3aBsOQJ0A:9 a=QEXdDO2ut3YA:10 a=O3GpQchJntsA:10 a=e19TqW951GkA:10 a=VlZU0XKO32wA:10 a=ihFelORD2_JXWTZZVsTd:22 a=IjZwj45LgO3ly-622nXo:22 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Apr 2018 07:41:14 -0000 Hi Patrick, Good to see you on the list :) > Now that the project is stabilizing on our end, we would like to reach > out about upstreaming fixes or feature work back into FreeBSD. One > small fix to posted interrupt handling on Intel > (https://smartos.org/bugview/OS-6829) might be a good starting point > for us to walk through the process of review and upstreaming. For that one and changes to the base system, I think you've already discovered Phabricator (https://wiki.freebsd.org/Phabricator) I should also mention, for bhyve/UEFI, a git pull request against https://github.com/freebsd/uefi-edk2/tree/bhyve/UDK2014.SP1 is the best way to upstream changes. later, Peter. From owner-freebsd-virtualization@freebsd.org Fri Apr 27 16:43:43 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 835CFFACCE0 for ; Fri, 27 Apr 2018 16:43:43 +0000 (UTC) (envelope-from list+org.freebsd.virtualization@io7m.com) Received: from mail.io7m.com (mail.io7m.com [IPv6:2001:19f0:5:752:f000::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.io7m.com", Issuer "arc7 CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E3D78327B for ; Fri, 27 Apr 2018 16:43:43 +0000 (UTC) (envelope-from list+org.freebsd.virtualization@io7m.com) Received: from almond.int.arc7.info (unknown [IPv6:2a02:390:7502:2:0:2:4:0]) by mail.io7m.com (Postfix) with ESMTPSA id 3248B3E46 for ; Fri, 27 Apr 2018 16:43:42 +0000 (UTC) Date: Fri, 27 Apr 2018 17:43:41 +0100 From: Mark Raynsford To: freebsd-virtualization@freebsd.org Subject: Read-only view of a ZFS filesystem inside a bhyve guest? Message-ID: <20180427174341.03373bc8@almond.int.arc7.info> Organization: io7m.com OpenPGP: id=B84E17747616C6174C68D5E55C1A7B712812CC05; url=http://io7m.com/pgp/B84E_1774_7616_C617_4C68_D5E5_5C1A_7B71_2812_CC05.key MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/so_TmJYKX8rR4Cl84I3fwjK"; protocol="application/pgp-signature" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Apr 2018 16:43:43 -0000 --Sig_/so_TmJYKX8rR4Cl84I3fwjK Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hello. I'm looking to do what the subject says: I have an existing ZFS filesystem (/storage/xyz) and I'd like to provide a read-only view of the filesystem to a set of bhyve guests. The guests in this case could be solely FreeBSD guests, but if there's a pleasant way to allow for OpenBSD or Linux guests, I'd like that. I'm essentially looking to move some jail-based infrastructure to bhyve guests. With the jails, I have a ZFS filesystem on the host that's mounted read-only inside some of the jails using nullfs. I'm not sure if there's something analogous for bhyve guests. I've looked at NFS, but this seems like overkill and possibly hard to secure. Same applies to Samba. sshfs might be an option, but I'd really prefer to have as few daemons listening on the host machine as possible for security reasons. --=20 Mark Raynsford | http://www.io7m.com --Sig_/so_TmJYKX8rR4Cl84I3fwjK Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQS4Thd0dhbGF0xo1eVcGntxKBLMBQUCWuNTPQAKCRBcGntxKBLM BZ1EAQDnum/5Xn00yG0a9usy04cr7wsajDQEVmrzUZicp1KcbQEA+A6b67B4FBdR wetLzwoqp1N11Adr9A/blIaCHqOePQQ= =rZZT -----END PGP SIGNATURE----- --Sig_/so_TmJYKX8rR4Cl84I3fwjK-- From owner-freebsd-virtualization@freebsd.org Fri Apr 27 17:22:43 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5C5FFAD967 for ; Fri, 27 Apr 2018 17:22:42 +0000 (UTC) (envelope-from eborisch@alumni.stanford.edu) Received: from mail-lf0-x244.google.com (mail-lf0-x244.google.com [IPv6:2a00:1450:4010:c07::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45C266BFC3 for ; Fri, 27 Apr 2018 17:22:41 +0000 (UTC) (envelope-from eborisch@alumni.stanford.edu) Received: by mail-lf0-x244.google.com with SMTP id j16-v6so3737108lfb.7 for ; Fri, 27 Apr 2018 10:22:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=1a6oBPyXRVOQ1HqB5hhOECIfR34FL98m6y2DxsBKJ6k=; b=fApa/VzeS5D1w0lg064Ihxnn0dIT88LuBMiNSoW2w7PaU7AbbFMXii2BS3s9csfilA HDXqtnREGFpfPz4ByGMDmMD4BsGf4sEZ/A754EL7uGrIDXjxWBtcWK6AGMhuWcox6SLH lYVTPVJmac/gy5bzYiapq3oXqQQXBa0sKz4SKy7BQd3NanfhIbWBDHSLqKLbRkkgUb1p 76MQ/PA7xFY97IjbcNleUcQcsk0odfPfQWPEfNKNMK23abCoTYyVmXtLB+R+Y53TyhU7 YVn0PHYkEWbwcVU6EZIz1JwbpJpg3E0DFcWk8Xok/PdpBy6fHa1eVh9tdyrx1kFCSzFp xqsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=1a6oBPyXRVOQ1HqB5hhOECIfR34FL98m6y2DxsBKJ6k=; b=b+lZmcUwDZKc1gsgWLJ+SVw3fib73tqFw6oDRUz9zTsWCkj5mkMMRdbC/VdPdy4RHg XejrcXQ1lrrEcn8RXmIL2MCLHfvKsxPQSQlEpkJNe/cTtj/yO6EwM9d5dHC9w73oHpLK bHtj7AGVd9KnkMHNcHDW1ZZCFHUtUWaGwQ6Htu12VwYA8jCDFf1V8UTekGZHAHNg3+0O OQnE46ECkc8DzWxtKOtJZhPKUOjM56tyq+e9Wqhb+nau+n+CmWYy/0N2zr8xz5d38rwg u0rFiDmqIg7OlbU0Aq4vhPCMd7MkjXRPXnBMUFl+2w0YpGMmkkQ2WRUyniGtLjuHPgik zodg== X-Gm-Message-State: ALQs6tDCPfsupnAptOwDAYk2KjfUrwOE+BG4Evlvyv52bQ3HfLDTQ7HD L/NxtyJlm2myRECCrVfTvhNnjiKTiw04rZ22eOWCEg== X-Google-Smtp-Source: AB8JxZqBB3Dzl1q3ZjfXLnNZQoiMPMHgqM5y59Ha4oskEdYvNIVLcxduTg8/BlPGsgFGuhWxfngEmZEivv5AY14WS6U= X-Received: by 2002:a19:7906:: with SMTP id u6-v6mr1972671lfc.34.1524849760527; Fri, 27 Apr 2018 10:22:40 -0700 (PDT) MIME-Version: 1.0 Sender: eborisch@alumni.stanford.edu Received: by 10.46.132.204 with HTTP; Fri, 27 Apr 2018 10:22:39 -0700 (PDT) In-Reply-To: <20180427174341.03373bc8@almond.int.arc7.info> References: <20180427174341.03373bc8@almond.int.arc7.info> From: Eric Borisch Date: Fri, 27 Apr 2018 12:22:39 -0500 X-Google-Sender-Auth: vosLQlLD5oGM-Y4q3uQrQatMj6I Message-ID: Subject: Re: Read-only view of a ZFS filesystem inside a bhyve guest? To: Mark Raynsford Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Apr 2018 17:22:43 -0000 I use NFS for (*nix) guests, and SMB for Windows guests; both work well, and can be restricted to specific IPs / ranges to help minimize security concerns. On Fri, Apr 27, 2018 at 11:43 AM, Mark Raynsford via freebsd-virtualization wrote: > Hello. > > I'm looking to do what the subject says: I have an existing ZFS > filesystem (/storage/xyz) and I'd like to provide a read-only view of > the filesystem to a set of bhyve guests. The guests in this case could > be solely FreeBSD guests, but if there's a pleasant way to allow for > OpenBSD or Linux guests, I'd like that. > > I'm essentially looking to move some jail-based infrastructure to bhyve > guests. With the jails, I have a ZFS filesystem on the host that's > mounted read-only inside some of the jails using nullfs. I'm not sure > if there's something analogous for bhyve guests. > > I've looked at NFS, but this seems like overkill and possibly hard to > secure. Same applies to Samba. sshfs might be an option, but I'd really > prefer to have as few daemons listening on the host machine as possible > for security reasons. > > -- > Mark Raynsford | http://www.io7m.com > > From owner-freebsd-virtualization@freebsd.org Fri Apr 27 18:42:23 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CAA9FAF747 for ; Fri, 27 Apr 2018 18:42:23 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A62B47E96F for ; Fri, 27 Apr 2018 18:42:22 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w3RIgImT007832; Fri, 27 Apr 2018 11:42:18 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w3RIgIIo007831; Fri, 27 Apr 2018 11:42:18 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201804271842.w3RIgIIo007831@pdx.rh.CN85.dnsmgr.net> Subject: Re: Read-only view of a ZFS filesystem inside a bhyve guest? In-Reply-To: <20180427174341.03373bc8@almond.int.arc7.info> To: Mark Raynsford Date: Fri, 27 Apr 2018 11:42:18 -0700 (PDT) CC: freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Apr 2018 18:42:23 -0000 > Hello. > > I'm looking to do what the subject says: I have an existing ZFS > filesystem (/storage/xyz) and I'd like to provide a read-only view of > the filesystem to a set of bhyve guests. The guests in this case could > be solely FreeBSD guests, but if there's a pleasant way to allow for > OpenBSD or Linux guests, I'd like that. > > I'm essentially looking to move some jail-based infrastructure to bhyve > guests. With the jails, I have a ZFS filesystem on the host that's > mounted read-only inside some of the jails using nullfs. I'm not sure > if there's something analogous for bhyve guests. > > I've looked at NFS, but this seems like overkill and possibly hard to > secure. Same applies to Samba. sshfs might be an option, but I'd really > prefer to have as few daemons listening on the host machine as possible > for security reasons. You should be able to "plumb" NFS from the host to the guests on a local bridge device, possibly evening playing games with the use of an unroutable network like 127.1.0.0/24. Host would have 127.1.0.1/24, /etc/exports would list 127.1.0.0/24 as who has access to the NFS shares, guests would be on 127.1.0.2 and up. Create the host bridge and assign it IP 127.1.0.1, create guests taps, inside gueests assign them 127.1.0.2 and up. Untested, but should work. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Sat Apr 28 06:17:38 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2B00FC256B for ; Sat, 28 Apr 2018 06:17:37 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 63D2B6AA8D for ; Sat, 28 Apr 2018 06:17:36 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from imac.bk.cs.huji.ac.il ([132.65.179.42]) by kabab.cs.huji.ac.il with esmtp id 1fCJ2I-000DDU-W2; Sat, 28 Apr 2018 09:08:43 +0300 From: Daniel Braniss Message-Id: Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: Re: Read-only view of a ZFS filesystem inside a bhyve guest? Date: Sat, 28 Apr 2018 09:08:42 +0300 In-Reply-To: <20180427174341.03373bc8@almond.int.arc7.info> Cc: freebsd-virtualization@freebsd.org To: Mark Raynsford References: <20180427174341.03373bc8@almond.int.arc7.info> X-Mailer: Apple Mail (2.3445.6.18) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 06:17:38 -0000 > On 27 Apr 2018, at 19:43, Mark Raynsford via freebsd-virtualization = wrote: >=20 > Hello. >=20 > I'm looking to do what the subject says: I have an existing ZFS > filesystem (/storage/xyz) and I'd like to provide a read-only view of > the filesystem to a set of bhyve guests. The guests in this case could > be solely FreeBSD guests, but if there's a pleasant way to allow for > OpenBSD or Linux guests, I'd like that. >=20 > I'm essentially looking to move some jail-based infrastructure to = bhyve > guests. With the jails, I have a ZFS filesystem on the host that's > mounted read-only inside some of the jails using nullfs. I'm not sure > if there's something analogous for bhyve guests. >=20 > I've looked at NFS, but this seems like overkill and possibly hard to > secure. Same applies to Samba. sshfs might be an option, but I'd = really > prefer to have as few daemons listening on the host machine as = possible > for security reasons. >=20 > --=20 > Mark Raynsford | http://www.io7m.com >=20 since the clients and the server are sharing the zfs volume, I=E2=80=99m doing the following: on the server I did: zfs create -sV 4G h/root.ro newfs /dev/zvol/h/root.ro mount /dev/zol/h/root.ro /mnt copy a working root image to it. umount /mnt the clients then mount it as ro, the vm conflg file has: disk0_type=3Dvirtio-blk=E2=80=9D disk0_name=3D=E2=80=9C/dev/zvol/h/root.ro = =E2=80=9D disk0_dev=3D=E2=80=9Ccustom=E2=80=9D one solution to the fact that the root is read-only is to use unionfs = (probably nullfs will do too) the only problem I have is updating the image. hope this helps danny From owner-freebsd-virtualization@freebsd.org Sat Apr 28 10:37:51 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13F80FAA3A5 for ; Sat, 28 Apr 2018 10:37:51 +0000 (UTC) (envelope-from list+org.freebsd.virtualization@io7m.com) Received: from mail.io7m.com (mail.io7m.com [IPv6:2001:19f0:5:752:f000::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.io7m.com", Issuer "arc7 CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B342E801B8 for ; Sat, 28 Apr 2018 10:37:50 +0000 (UTC) (envelope-from list+org.freebsd.virtualization@io7m.com) Received: from almond.int.arc7.info (unknown [IPv6:2a02:390:7502:2:0:2:4:0]) by mail.io7m.com (Postfix) with ESMTPSA id 9F75331C9; Sat, 28 Apr 2018 10:37:49 +0000 (UTC) Date: Sat, 28 Apr 2018 11:37:48 +0100 From: Mark Raynsford To: Daniel Braniss Cc: freebsd-virtualization@freebsd.org Subject: Re: Read-only view of a ZFS filesystem inside a bhyve guest? Message-ID: <20180428113748.72891422@almond.int.arc7.info> In-Reply-To: References: <20180427174341.03373bc8@almond.int.arc7.info> Organization: io7m.com OpenPGP: id=B84E17747616C6174C68D5E55C1A7B712812CC05; url=http://io7m.com/pgp/B84E_1774_7616_C617_4C68_D5E5_5C1A_7B71_2812_CC05.key MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/NaDD0LvfDH.anCt.7Kvhdwx"; protocol="application/pgp-signature" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 10:37:51 -0000 --Sig_/NaDD0LvfDH.anCt.7Kvhdwx Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 2018-04-28T09:08:42 +0300 Daniel Braniss wrote: > since the clients and the server are sharing the zfs volume, > I=E2=80=99m doing the following: > on the server I did: > zfs create -sV 4G h/root.ro > newfs /dev/zvol/h/root.ro > mount /dev/zol/h/root.ro /mnt > copy a working root image to it. > umount /mnt > the clients then mount it as ro, > the vm conflg file has: > disk0_type=3Dvirtio-blk=E2=80=9D > disk0_name=3D=E2=80=9C/dev/zvol/h/root.ro =E2=80=9D > disk0_dev=3D=E2=80=9Ccustom=E2=80=9D >=20 > one solution to the fact that the root is read-only is to use unionfs (pr= obably nullfs will do too) >=20 > the only problem I have is updating the image. Wow, didn't know this was possible. Is this safe? Two essentially independent operating system instances being able to write to the same zvol? --=20 Mark Raynsford | http://www.io7m.com --Sig_/NaDD0LvfDH.anCt.7Kvhdwx Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQS4Thd0dhbGF0xo1eVcGntxKBLMBQUCWuRO/AAKCRBcGntxKBLM BVWOAP9saddfTN55XrBWM2Oonynt8y77rLrik6uqS7GxSKo3ZgD/e8JN6zcEPWW6 zFGAmQUgUVejemre3j4+Bb6N/Lw2kgk= =NSN+ -----END PGP SIGNATURE----- --Sig_/NaDD0LvfDH.anCt.7Kvhdwx-- From owner-freebsd-virtualization@freebsd.org Sat Apr 28 10:36:47 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D442FFAA29C for ; Sat, 28 Apr 2018 10:36:47 +0000 (UTC) (envelope-from list+org.freebsd.virtualization@io7m.com) Received: from mail.io7m.com (mail.io7m.com [IPv6:2001:19f0:5:752:f000::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.io7m.com", Issuer "arc7 CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 827A880127 for ; Sat, 28 Apr 2018 10:36:47 +0000 (UTC) (envelope-from list+org.freebsd.virtualization@io7m.com) Received: from almond.int.arc7.info (unknown [IPv6:2a02:390:7502:2:0:2:4:0]) by mail.io7m.com (Postfix) with ESMTPSA id 1EFDF31C8; Sat, 28 Apr 2018 10:36:46 +0000 (UTC) Date: Sat, 28 Apr 2018 11:36:35 +0100 From: Mark Raynsford To: "Rodney W. Grimes" Cc: freebsd-virtualization@freebsd.org Subject: Re: Read-only view of a ZFS filesystem inside a bhyve guest? Message-ID: <20180428113635.4b6b567a@almond.int.arc7.info> In-Reply-To: <201804271842.w3RIgIIo007831@pdx.rh.CN85.dnsmgr.net> References: <20180427174341.03373bc8@almond.int.arc7.info> <201804271842.w3RIgIIo007831@pdx.rh.CN85.dnsmgr.net> Organization: io7m.com OpenPGP: id=B84E17747616C6174C68D5E55C1A7B712812CC05; url=http://io7m.com/pgp/B84E_1774_7616_C617_4C68_D5E5_5C1A_7B71_2812_CC05.key MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/MPVsoO6FN4EuNKbsDVX6YYD"; protocol="application/pgp-signature" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 10:36:48 -0000 --Sig_/MPVsoO6FN4EuNKbsDVX6YYD Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On 2018-04-27T11:42:18 -0700 "Rodney W. Grimes" wrote: > > You should be able to "plumb" NFS from the host to the guests on a > local bridge device, possibly evening playing games with the use > of an unroutable network like 127.1.0.0/24. On 2018-04-27T12:22:39 -0500 Eric Borisch wrote: > I use NFS for (*nix) guests, and SMB for Windows guests; both work well, > and can be restricted to specific IPs / ranges to help minimize security > concerns. >=20 Well that's two in favour of NFS. I have one small question: Is there any way to stop nfsd from forking into the background? I run everything on the system under process supervision (runit), and this is the first time I've ever seen a daemon program with no option to stay in the foreground. I can write a wrapper in C that starts up rpcbind and nfsd (the former *does* have an option to stay in the foreground), but I'd rather not if I don't have to. --=20 Mark Raynsford | http://www.io7m.com --Sig_/MPVsoO6FN4EuNKbsDVX6YYD Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQS4Thd0dhbGF0xo1eVcGntxKBLMBQUCWuROswAKCRBcGntxKBLM BUDkAQCkM3VyhnKHoTtEhVunmcBbIjxyqVa3girg2k7fIzxcVwD/UaJPuKYvCNcL sLp+wq4YlTBzqWb19xDy2ynVIlpIhQ0= =mHnS -----END PGP SIGNATURE----- --Sig_/MPVsoO6FN4EuNKbsDVX6YYD-- From owner-freebsd-virtualization@freebsd.org Sat Apr 28 11:26:37 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8635FABA60 for ; Sat, 28 Apr 2018 11:26:37 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3DA5D68C0C for ; Sat, 28 Apr 2018 11:26:36 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from imac.bk.cs.huji.ac.il ([132.65.179.42]) by kabab.cs.huji.ac.il with esmtp id 1fCNzq-000ONR-TH; Sat, 28 Apr 2018 14:26:30 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: Re: Read-only view of a ZFS filesystem inside a bhyve guest? From: Daniel Braniss In-Reply-To: <20180428113748.72891422@almond.int.arc7.info> Date: Sat, 28 Apr 2018 14:26:30 +0300 Cc: freebsd-virtualization@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20180427174341.03373bc8@almond.int.arc7.info> <20180428113748.72891422@almond.int.arc7.info> To: Mark Raynsford X-Mailer: Apple Mail (2.3445.6.18) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 11:26:37 -0000 > On 28 Apr 2018, at 13:37, Mark Raynsford = wrote: >=20 > On 2018-04-28T09:08:42 +0300 > Daniel Braniss wrote: >=20 >> since the clients and the server are sharing the zfs volume, >> I=E2=80=99m doing the following: >> on the server I did: >> zfs create -sV 4G h/root.ro >> newfs /dev/zvol/h/root.ro >> mount /dev/zol/h/root.ro /mnt >> copy a working root image to it. >> umount /mnt >> the clients then mount it as ro, >> the vm conflg file has: >> disk0_type=3Dvirtio-blk=E2=80=9D >> disk0_name=3D=E2=80=9C/dev/zvol/h/root.ro = =E2=80=9D >> disk0_dev=3D=E2=80=9Ccustom=E2=80=9D >>=20 >> one solution to the fact that the root is read-only is to use unionfs = (probably nullfs will do too) >>=20 >> the only problem I have is updating the image. >=20 > Wow, didn't know this was possible. Is this safe? Two essentially > independent operating system instances being able to write to the same > zvol? that=E2=80=99s why it=E2=80=99s mounted rear-only in the client! each client can get another vol for writing, ie /var if you want to have =E2=80=98permanent=E2=80=99 changes that will = survive reboots. updating on the server is possible, but 1- the changes might not be seen by the client 2- opened files will have issues btw, point 2 is also true for NFS. danny >=20 > --=20 > Mark Raynsford | http://www.io7m.com >=20 From owner-freebsd-virtualization@freebsd.org Sat Apr 28 11:28:24 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0DB6FABB5F for ; Sat, 28 Apr 2018 11:28:24 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5279369C57 for ; Sat, 28 Apr 2018 11:28:24 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from imac.bk.cs.huji.ac.il ([132.65.179.42]) by kabab.cs.huji.ac.il with esmtp id 1fCO1e-000OTX-BQ; Sat, 28 Apr 2018 14:28:22 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: Re: Read-only view of a ZFS filesystem inside a bhyve guest? From: Daniel Braniss In-Reply-To: Date: Sat, 28 Apr 2018 14:28:22 +0300 Cc: freebsd-virtualization@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20180427174341.03373bc8@almond.int.arc7.info> <20180428113748.72891422@almond.int.arc7.info> To: Mark Raynsford X-Mailer: Apple Mail (2.3445.6.18) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 11:28:24 -0000 > On 28 Apr 2018, at 14:26, Daniel Braniss wrote: >=20 >=20 >=20 >> On 28 Apr 2018, at 13:37, Mark Raynsford = wrote: >>=20 >> On 2018-04-28T09:08:42 +0300 >> Daniel Braniss wrote: >>=20 >>> since the clients and the server are sharing the zfs volume, >>> I=E2=80=99m doing the following: >>> on the server I did: >>> zfs create -sV 4G h/root.ro >>> newfs /dev/zvol/h/root.ro >>> mount /dev/zol/h/root.ro /mnt >>> copy a working root image to it. >>> umount /mnt >>> the clients then mount it as ro, >>> the vm conflg file has: >>> disk0_type=3Dvirtio-blk=E2=80=9D >>> disk0_name=3D=E2=80=9C/dev/zvol/h/root.ro = =E2=80=9D >>> disk0_dev=3D=E2=80=9Ccustom=E2=80=9D >>>=20 >>> one solution to the fact that the root is read-only is to use = unionfs (probably nullfs will do too) >>>=20 >>> the only problem I have is updating the image. >>=20 >> Wow, didn't know this was possible. Is this safe? Two essentially >> independent operating system instances being able to write to the = same >> zvol? >=20 > that=E2=80=99s why it=E2=80=99s mounted rear-only in the client! grr, hate spell checkers, s/rear/read/ :-) > each client can get another vol for writing, ie /var > if you want to have =E2=80=98permanent=E2=80=99 changes that will = survive reboots. >=20 > updating on the server is possible, but > 1- the changes might not be seen by the client > 2- opened files will have issues >=20 > btw, point 2 is also true for NFS. >=20 > danny >=20 >>=20 >> --=20 >> Mark Raynsford | http://www.io7m.com >>=20 >=20