From nobody Mon May 24 20:43:44 2021 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 276509F0EC3 for ; Mon, 24 May 2021 20:50:04 +0000 (UTC) (envelope-from Eich@academiaingenierosyasociados.es) Received: from server.hostisa.net (server.hostisa.net [51.254.16.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fpq8q3yd9z3Ltd for ; Mon, 24 May 2021 20:50:03 +0000 (UTC) (envelope-from Eich@academiaingenierosyasociados.es) Received: by server.hostisa.net (Postfix, from userid 1025) id C073B4B45EB2; Mon, 24 May 2021 22:43:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=academiaingenierosyasociados.es; s=default; t=1621889047; bh=3svb/Iv4pflC8Acq7XWIrNAZMX2zSvcr4VEcyOQA9E0=; h=To:Subject:Date:From:Reply-To; b=DMbuh90ZbfTCeLBVkEJVioWNob3Kq2RPiAJxe6uMxU8VpFdOG61r7sA38ErLBdFfS gj07744bjrO33oeYAV+lxEE1O1dLGTwuSmQwdE69ZYSGy2R0K8olp7jUiYEXdohTKc aPtbR8vkLA0a/lqd9eZXOlXsh/E6CdJxYstDUdDE= To: freebsd-jail@freebsd.org Subject: Voll engagiert X-PHP-Originating-Script: 1025:scache-16.php Date: Mon, 24 May 2021 20:43:44 +0000 From: Mgm Reply-To: Olsens@europe.com Message-ID: <759a238970cd0a747a757738ed067525@academiaingenierosyasociados.es> List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4Fpq8q3yd9z3Ltd X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none (invalid DKIM record) header.d=academiaingenierosyasociados.es header.s=default header.b=DMbuh90Z; dmarc=none; spf=pass (mx1.freebsd.org: domain of Eich@academiaingenierosyasociados.es designates 51.254.16.47 as permitted sender) smtp.mailfrom=Eich@academiaingenierosyasociados.es X-Spamd-Result: default: False [2.30 / 15.00]; HAS_REPLYTO(0.00)[Olsens@europe.com]; R_SPF_ALLOW(0.00)[+a:c]; HAS_X_POS(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[academiaingenierosyasociados.es:~]; RBL_SEM(1.00)[51.254.16.47:from]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[51.254.16.47:from]; ASN(0.00)[asn:16276, ipnet:51.254.0.0/15, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.70)[-0.697]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[europe.com]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[51.254.16.47:from:127.0.2.255]; BAD_REP_POLICIES(0.10)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; DMARC_NA(0.00)[academiaingenierosyasociados.es]; R_DKIM_PERMFAIL(0.00)[academiaingenierosyasociados.es:s=default]; MAILMAN_DEST(0.00)[freebsd-jail] Angesichts der unvermeidlichen Umstände war ich gezwungen, mich an Sie zu wenden, um zu sehen, ob Sie mit mir zusammenarbeiten können, um ein Projekt von hohem Wert in Höhe von Zweiund vierzig Millionen Dollar durchzuführen. Wenden Sie sich an mich, um weitere Informationen zu dieser äußerst wichtigen Angelegenheit zu erhalten, da Zeit von entscheidender Bedeutung ist. Bitte behandeln Sie diese Nachricht als klassifiziert. Mit bestem Gruß, Sharon Persönliche E-Mail: shannon.olsens@aol.com From nobody Sat May 29 15:59:49 2021 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 72992BFA83F; Sat, 29 May 2021 15:59:52 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FsmTg4KHrz3kDy; Sat, 29 May 2021 15:59:51 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1622303989; bh=FIG5t7pq+3T9XrI2mfFjEWOJEGQJxc1t3FPm+QW1ZHs=; h=X-UI-Sender-Class:From:To:Subject:Date; b=ZtkT6Q+rn1M7jTqvMyu4ndxq3/SGeIA3AE9AjaWN0/ec0dMZSZT07pF06sqVaFfGY RqT/YcLSbLkemqnT1sjnUAC8Ds3OaSrc6qtsC2cVNpHBfgZ/MCS119CZg9OeFt5q5M eJ5KcKmSrm3/ZcUs2L+AhtRguq1BW3ZwvmF9oAEk= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [89.245.246.156] ([89.245.246.156]) by web-mail.gmx.net (3c-app-gmx-bap59.server.lan [172.19.172.129]) (via HTTP); Sat, 29 May 2021 17:59:49 +0200 List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 Message-ID: From: mj-mailinglist@gmx.de To: freebsd-jail@freebsd.org, freebsd-current@freebsd.org Subject: Network in VNET jail does not work on my FreeBSD current bhyve vm Content-Type: text/plain; charset=UTF-8 Date: Sat, 29 May 2021 17:59:49 +0200 Importance: normal Sensitivity: Normal X-Priority: 3 X-Provags-ID: V03:K1:bJOKm1w6Ra+jn5x/7CelHq7lVaNl3Z/2CmukUgLWNKv88o34CBL51VDkf/wa0exc72nOV 7k7eF5yAG/4TAIlkA9rHxyqxLdZW+LOM+TjV3TBBMLJOEqaitxieIFUwDuGBNFWRXPvP1KhtFhhy BfSRUM0p5cOSmBLn79QrlA32iW0nIje1N+gxVrAVqrCb5FeA5oKk22Oast6+pk5fe/RGBRw3PvfI c29PlM3zjGcKzTRuxsQkBqERcRqzwS22T33J5Ep5gwflk13Gcg9NhMywpWJkTXAUy5t4awh/wJJB sw= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:BMr66DBcBak=:g9pzPIjHmd0pPpBStdgPoV /4LvXG7NPmfVx9DoDX7WT0oHb6i2xgAS/PclXSEhG+6KqYCsTzP9tqDUcsEm3wGN0YtLAZZbd L25CUijAN/pqTdpb+zbIOI6K00ukv+bVWEAVq3NTu510KQmM/QNHB2jFOiq9P6Is4AG6ybf16 gdyUAPg+BmM3Ec1oPRBQ/8chzvL1b961kl8AvmPA7bvxr+UEx6U81L0TFCEKifNVyxgMEiXHV 6xCrhG7AKOo/zUE3AOzGdA8f2XBxQ+7mVXSr1rMMPF/dy+j4nmd2sVCMMFyuLzSG3bAmaypL+ VrA9ed0fPWPjtxlnN6zeaPMj+YAtSNoa/efGISCu5NVsQ3a5VLmhJ5sjsibEDN1ORNRH5xQSW TTJ7DuOMGVjtdOzT9nsjJEWVBkKijn3FQ7e3SG7nn67tvkJ+43TMA2tgjP9zVCWJP/Z4R3JOp 0P53+j97C6afu/e3oYHH8CYEY5wi8KOB7Dj1ewVj+8VgTAiOv821rMbjK9nXBB4Ed+p05AeQZ ZlkACKKQCxIECUmwIe3ppfrzKWBWofseL24Jg+/fzWhSR6tjS13XIz4qlh77C8NVrl/ex1mXH hcFfwNN0yBV2qWJ+MCD1PEzI/QYyR0Uy86MIRy57QdcWjwWaVQsDzEPv3sf987xwQEi3Csijj zCAuJHYKp1zltoH1TodMVgshaGha6FqOy39GOT5NIOQbXftmoXEafmn6lkvTdTq670R57wsUZ GImK+ZKXD1vDccdnsF30dbpEyvAXDW/28Woj9REJsEPpDjo0WbpiTDsXL5gpS8+KGCUYNGeRC CqdhYule42q/1iSVjd3qrxDLtLeUw67LiqCoAoGnQd+cxOQAPs/+peNtvH6iC3h5cEgnmRTZz j9tgBq5L2tTZaKRjrLmxE3nMXBBwB6vsJu8uMSNkFL7ilXm3BlbYimo7pMpXzWn+jhrlb6Usa HseLyqgrw5Q== X-Rspamd-Queue-Id: 4FsmTg4KHrz3kDy X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=ZtkT6Q+r; dmarc=pass (policy=none) header.from=gmx.de; spf=pass (mx1.freebsd.org: domain of mj-mailinglist@gmx.de designates 212.227.17.21 as permitted sender) smtp.mailfrom=mj-mailinglist@gmx.de X-Spamd-Result: default: False [-3.60 / 15.00]; FREEMAIL_FROM(0.00)[gmx.de]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; RCPT_COUNT_TWO(0.00)[2]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmx.de,none]; RECEIVED_SPAMHAUS_PBL(0.00)[89.245.246.156:received]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.17.21:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FROM_NO_DN(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[212.227.17.21:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-jail] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N Hello everybody, since a few weeks, my jails on a bhyve-vm, running current are not reachable via network, when configured with VNET. They can't even access the gateway. I don't remember when this problem started, but it's a few weeks. The same jail.conf works on a 13.0 host, on a current system the network does not work. A configuration without VNET on the same jail works. Are there any changes, that i missed? Here is the configuration, maybe someone spots an error, or has an idea what's going on: -- Martin uname on bhyve vm: ------------------ root@fbsd14:~ # uname -a FreeBSD fbsd14.fritz.box 14.0-CURRENT FreeBSD 14.0-CURRENT main-n247020-e0fa04e257c GENERIC-NODEBUG amd64 root@fbsd14:~ # freebsd-version -kru 14.0-CURRENT 14.0-CURRENT 14.0-CURRENT jail.conf on bhyve vm: ---------------------- # set default configuration values mount.devfs = true; exec.clean = true; allow.chflags = 1; allow.raw_sockets = 1; devfs_ruleset = 5; exec.system_user = "root"; exec.jail_user = "root"; exec.timeout = 30; stop.timeout = 30; ######### # Jails # ######### j1 { # Hostname host.hostname = "j1.fritz.box"; host.domainname = "fritz.box"; host.hostuuid = "68c2ad9b-b582-11eb-a925-589cfc0ac350"; osrelease = "14.0-CURRENT"; osreldate = "1400013"; # Network vnet = 1; vnet.interface = "epair2b"; exec.prestart += "ifconfig epair2 create up"; exec.prestart += "ifconfig epair2a description 'IFID=2 JAIL=j1'"; exec.prestart += "ifconfig bridge0 addm epair2a"; command = "ifconfig epair2b inet 192.168.1.101/22"; command += "route -n add -inet default 192.168.0.1"; exec.prestop = "ifconfig epair2b -vnet j1"; exec.poststop += "ifconfig bridge0 deletem epair2a"; exec.poststop += "ifconfig epair2a destroy"; sysvmsg = new; sysvsem = new; sysvshm = new; path = "/jails/j1"; allow.mount.zfs = 1; ## Script execution exec.timeout = 90; # Pre-/Post-Scripts exec.prestart += "logger trying to start jail j1 ..."; exec.poststart += "logger jail j1 has started"; exec.prestop += "logger shutting down jail j1"; exec.poststop += "logger jail j1 has shut down"; # Start Script exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; } ----------------------------------- /etc/rc.conf on bhyve vm: ------------------------- syslogd_flags="-ss" sendmail_enable="NONE" hostname="fbsd14.fritz.box" ifconfig_vtnet0="inet 192.168.1.100 netmask 255.255.252.0" defaultrouter="192.168.0.1" local_unbound_enable="YES" sshd_enable="YES" ntpd_enable="YES" # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable dumpdev="AUTO" zfs_enable="YES" jail_enable="YES" keymap="de" cloned_interfaces="bridge0" ifconfig_bridge0="addm vtnet0 up" # NFS rpc_lockd_enable="YES" rpc_statd_enable="YES" nfs_client_enable="YES" nfsuserd_enable="YES" ------------------------------------- ifconfig on bhyve vm: --------------------- root@fbsd14:~ # ifconfig -f inet:cidr vtnet0: flags=8863 metric 0 mtu 1500 options=80028 ether 58:9c:fc:0a:c3:50 inet 192.168.1.100/22 broadcast 192.168.3.255 media: Ethernet autoselect (10Gbase-T ) status: active nd6 options=29 lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1/8 groups: lo nd6 options=21 bridge0: flags=8843 metric 0 mtu 1500 ether 58:9c:fc:10:ff:bf id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: epair2a flags=143 ifmaxaddr 0 port 4 priority 128 path cost 2000 groups: bridge nd6 options=9 epair2a: flags=8943 metric 0 mtu 1500 description: IFID=2 JAIL=j1 options=8 ether 02:b4:ee:59:b3:0a groups: epair media: Ethernet 10Gbase-T (10Gbase-T ) status: active nd6 options=29 ------------------------------- /etc/rc.conf in jail: --------------------- syslogd_flags="-ss" sendmail_enable="NO" sendmail_submit_enable="NO" sendmail_outbound_enable="NO" sendmail_msp_queue_enable="NO" sshd_enable="YES" --------------------------- ifconfig in jail: ----------------- root@j1:~ # ifconfig -f inet:cidr lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1/8 groups: lo nd6 options=21 epair2b: flags=8843 metric 0 mtu 1500 options=8 ether 02:b4:ee:59:b3:0b inet 192.168.1.101/22 broadcast 192.168.3.255 groups: epair media: Ethernet 10Gbase-T (10Gbase-T ) status: active nd6 options=29 ------------------------------------ uname in jail: -------------- root@j1:~ # uname -a FreeBSD j1.fritz.box 14.0-CURRENT FreeBSD 14.0-CURRENT main-n247020-e0fa04e257c GENERIC-NODEBUG amd64 root@j1:~ # freebsd-version -ru 14.0-CURRENT 14.0-CURRENT From nobody Sun May 30 12:38:53 2021 X-Original-To: jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AAD50C7A2EF for ; Sun, 30 May 2021 12:38:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FtHzL48DWz4hNf for ; Sun, 30 May 2021 12:38:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 775111D484 for ; Sun, 30 May 2021 12:38:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14UCcsBR078456 for ; Sun, 30 May 2021 12:38:54 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14UCcsWZ078455 for jail@FreeBSD.org; Sun, 30 May 2021 12:38:54 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: jail@FreeBSD.org Subject: [Bug 251046] bhyve PCI passthrough does not work inside jail Date: Sun, 30 May 2021 12:38:53 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: me@anatoli.ws X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D251046 Anatoli changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |me@anatoli.ws --- Comment #7 from Anatoli --- Hi All, > Even then I'm not sure why it's useful to jail the bhyve process - what d= oes it buy you? The idea to run bhyve inside jail is to provide an additional layer of secu= rity for potential vm-escape vulnerabilities in bhyve. This is the way VMs are executed on Linux (protected by AppArmor and SEL) a= nd Illumos. Currently it's possible to run bhyve in jail, but not with PCI passthrough. > A better solution would be to extend pci(4) so that bhyve can use it to d= o everything required for PCI passthrough. Mark, could you please give us a hint on what should be done to extend pci(= 4) so jail changes are not needed? We are willing to implement this, but need = some guidance. One more security improvement that bhyve needs is to run it without root, b= ut that's another story for another report. --=20 You are receiving this mail because: You are on the CC list for the bug.=