From owner-freebsd-virtualization@freebsd.org Sun Dec 4 21:00:41 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CB0BC67F1F for ; Sun, 4 Dec 2016 21:00:41 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DB4241A3E for ; Sun, 4 Dec 2016 21:00:40 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uB4L01Cu021875 for ; Sun, 4 Dec 2016 21:00:40 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201612042100.uB4L01Cu021875@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-virtualization@FreeBSD.org Subject: Problem reports for freebsd-virtualization@FreeBSD.org that need special attention Date: Sun, 04 Dec 2016 21:00:40 +0000 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Dec 2016 21:00:41 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 212711 | [typo] bhyve: virtio-rnd PCI ID doesn't match vio New | 212820 | FreeBSD 10-STABLE from latest HEAD and 11-RELEASE Open | 212721 | FreeBSD 11.0-RC2/RC3/RELEASE fails on Hyper-V 201 3 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Sun Dec 4 22:35:29 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38252C67298 for ; Sun, 4 Dec 2016 22:35:29 +0000 (UTC) (envelope-from jjuanino@gmail.com) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 061241BD3 for ; Sun, 4 Dec 2016 22:35:29 +0000 (UTC) (envelope-from jjuanino@gmail.com) Received: by mail-oi0-x231.google.com with SMTP id v84so320881694oie.3 for ; Sun, 04 Dec 2016 14:35:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=X2oYxmDRFSzkKwrH73ehf3YYO8Q9MNku4Qlz3c6KI3k=; b=oZf5lu+wG6apgUIEePwKx9kP/KRdslwXnXSo4XhoCFEpBtde6OlwxIuGDoKNr5Oxt6 PHFfxaxE2iDlvH5LjtHZcHuXAkGx0r2PdLQ0t42exP2DtBJv/U/oWFvaBh7ewW48lTWc EzNXB9My3rGDB1d+GzHWnm1Gy+ZceGy23PtpmqJJk+nc6x5w9sS3abnMegO0y2RpQtLl vnr53gfTDfC2dXrpdqQgG8iIqbdSFFmfDGiTHszgNBDZWY6IB51q2djWSzmgIGtGsQ40 pWyaqxzB1F9vQQdDCgW+FMUZ4NWHnp8Gg9GB2sEvpYTlyeomV8grVf3f1wqtt8hZKVRJ 5xQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=X2oYxmDRFSzkKwrH73ehf3YYO8Q9MNku4Qlz3c6KI3k=; b=RkX/hpwQcFwbKlcbVUG6inhbCoRG21a29u3n0PDmfCTWksqiqO3jmmUQcrvQYOizrq i0sq7tp4Xcd1yFmsaZPfxLVxZVYgh0Vdn3jdAUez72lKymkorCmSCRbEeZ/1QhuiXVdN riFarvGDWHe6Gho+5tKKQ9eTRlgOrmYyiCY590aQC//+u+ejU6MXS/Y9/qj4mmJFjS0c LXasufnFdFL82IRsWRnKcYdJ0iJ6X2Jk3nBdFNKOivuRRJLU7hR4QB/NsnUyoRszA+RH 5tdNG+zSfye1ObHxnpZ9Odvs1lMXozsNx7xRlBHIAvqdR7oLDqSnnOkaxXsH8itvXJHl rMcw== X-Gm-Message-State: AKaTC02psS6NbN8DPV4q5FX2Pd1h8lSTYshauySilhrMjFavgmEiMZMOuRdqxKZaUtdVs38DjdNP6A14GNsSlw== X-Received: by 10.202.225.85 with SMTP id y82mr28616463oig.209.1480890927842; Sun, 04 Dec 2016 14:35:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.202.49.132 with HTTP; Sun, 4 Dec 2016 14:35:27 -0800 (PST) From: =?UTF-8?B?Sm9zw6kgR2FyY8OtYSBKdWFuaW5v?= Date: Sun, 4 Dec 2016 23:35:27 +0100 Message-ID: Subject: bhyve: cannot send jumbo frames from linux or freebsd guest To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Dec 2016 22:35:29 -0000 Hi everybody, On my FreeBSD 11.0-RELEASE I am running three bhyve virtual machines: 1- Windows Server 2012 2- Centos 6 3- FreeBSD 11.0-RELEASE An iSCSI target is configured in the host, and the three above guests runs a iSCSI initiator. I use iSCSI to present zvols to guests, and so be able to increase the guests storage without rebooting. This setup works as expected: initiators connect to target, and everything run smoothly. In order to boost performance, I want to enable jumbo frames in the private, non routing, iSCSI subnet. The subnet is 192.168.253/24, and is configured as follows: # ifconfig bridge6 bridge6: flags=8843 metric 0 mtu 9000 description: vm-iscsi0 ether 02:10:ef:9e:c0:06 inet 192.168.253.1 netmask 0xffffff00 broadcast 192.168.253.255 nd6 options=1 groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap9 flags=943 ifmaxaddr 0 port 22 priority 128 path cost 2000000 member: tap7 flags=943 ifmaxaddr 0 port 20 priority 128 path cost 2000000 member: tap6 flags=943 ifmaxaddr 0 port 19 priority 128 path cost 2000000 192.168.253.1 is the ip address of the FreeBSD bhyve host (the iSCSI target) in this private subnet. Notice that I have set mtu=9000 in the bridge interface. Automatically, the taps interfaces inherit that setting when the guests starts up. Inside each guest, I have configured the mtu setting to 9000. In the FreeBSD and Linux, I get, respectively: # ifconfig vtnet1 vtnet1: flags=8943 metric 0 mtu 9000 options=80028 ether 58:9c:fc:05:55:66 inet 192.168.253.4 netmask 0xffffff00 broadcast 192.168.253.255 nd6 options=29 media: Ethernet 10Gbase-T status: active # ifconfig eth2 eth2 Link encap:Ethernet HWaddr 58:9C:FC:0E:9D:01 inet addr:192.168.253.3 Bcast:192.168.253.255 Mask:255.255.255.0 inet6 addr: fe80::5a9c:fcff:fe0e:9d01/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:19 errors:0 dropped:0 overruns:0 frame:0 TX packets:26 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:65470 (63.9 KiB) TX bytes:108972 (106.4 KiB) In order to test if jumbo frames are set properly, I run the following from the Windows guest: C:\Users\Administrator>ping -n 1 -f -l 8972 192.168.253.1 Pinging 192.168.253.1 with 8972 bytes of data: Reply from 192.168.253.1: bytes=8972 time<1ms TTL=64 Ping statistics for 192.168.253.1: Packets: Sent = 1, Received = 1, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms As you can see, jumbo frames seems to work properly from Windows guest, as I can trasmit a 8972 bytes frame with no fragmentation. A tcpdump in the host side shows the following: 23:22:42.111043 IP (tos 0x0, ttl 128, id 20704, offset 0, flags [DF], proto ICMP (1), length 9000) 192.168.253.2 > 192.168.253.1: ICMP echo request, id 1, seq 301, length 8980 23:22:42.111106 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 9000) 192.168.253.1 > 192.168.253.2: ICMP echo reply, id 1, seq 301, length 8980 But, If I repeat the analogous test from FreeBSD or Linux guest, I get wrong results, as I will explain. >From Linux guest I run: # ping -c 4 -s 8972 -M do 192.168.253.1 PING 192.168.253.1 (192.168.253.1) 8972(9000) bytes of data. --- 192.168.253.1 ping statistics --- 4 packets transmitted, 0 received, 100% packet loss, time 13000ms If I decrease the packet size to 4042, I get a succesful answer from the iSCSI target: # ping -c 4 -s 4042 -M do 192.168.253.1 PING 192.168.253.1 (192.168.253.1) 4042(4070) bytes of data. 4050 bytes from 192.168.253.1: icmp_seq=1 ttl=64 time=0.191 ms 4050 bytes from 192.168.253.1: icmp_seq=2 ttl=64 time=0.203 ms 4050 bytes from 192.168.253.1: icmp_seq=3 ttl=64 time=0.159 ms 4050 bytes from 192.168.253.1: icmp_seq=4 ttl=64 time=0.191 ms --- 192.168.253.1 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3000ms rtt min/avg/max/mdev = 0.159/0.186/0.203/0.016 ms With a 4043 packet size, the ping fails again. >From FreeBSD guest, the test would be: # ping -c 4 -D -s 8972 192.168.253.1 PING 192.168.253.1 (192.168.253.1): 8972 data bytes --- 192.168.253.1 ping statistics --- 4 packets transmitted, 0 packets received, 100.0% packet loss After some trial and error, I get 1994 as the upper bound: # ping -c 4 -D -s 1994 192.168.253.1 PING 192.168.253.1 (192.168.253.1): 1994 data bytes 2002 bytes from 192.168.253.1: icmp_seq=0 ttl=64 time=0.203 ms 2002 bytes from 192.168.253.1: icmp_seq=1 ttl=64 time=0.239 ms 2002 bytes from 192.168.253.1: icmp_seq=2 ttl=64 time=0.224 ms 2002 bytes from 192.168.253.1: icmp_seq=3 ttl=64 time=0.214 ms --- 192.168.253.1 ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.203/0.220/0.239/0.013 ms I have run tcpdump on both FreeBSD guest and host to debug the issue and I get the following: * From FreeBSD guest: # tcpdump -vvn -i vtnet1 & tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes # ping -c 1 -D -s 1995 192.168.253.1 PING 192.168.253.1 (192.168.253.1): 1995 data bytes 23:08:46.808187 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 2023) 192.168.253.4 > 192.168.253.1: ICMP echo request, id 58116, seq 0, length 2003 23:08:46.808430 IP truncated-ip - 1 bytes missing! (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 2023) 192.168.253.1 > 192.168.253.4: ICMP echo reply, id 58116, seq 0, length 2003 --- 192.168.253.1 ping statistics --- 1 packets transmitted, 0 packets received, 100.0% packet loss * From FreeBSD host: # tcpdump -vvn -i bridge6 tcpdump: listening on bridge6, link-type EN10MB (Ethernet), capture size 262144 bytes 23:09:58.315600 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 2023) 192.168.253.4 > 192.168.253.1: ICMP echo request, id 59140, seq 0, length 2003 23:09:58.315663 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 2023) 192.168.253.1 > 192.168.253.4: ICMP echo reply, id 59140, seq 0, length 2003 Notice the "truncated-ip - 1 bytes missing!" from the guest. Indeed, is clear that the issue is related to the Linux or FreeBSD guest, as from Windows guest I cant transmit jumbo frames. Could anyone explain why I cannot send jumbo frames from Linux and FreeBSD guests, but I *can* send them from Windows guest? Am I missing o missunderstanding something? Thanks in advanced, any comment o suggestion will be wellcome. Best regards. From owner-freebsd-virtualization@freebsd.org Mon Dec 5 03:51:32 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A48CC67400 for ; Mon, 5 Dec 2016 03:51:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0FAF920D for ; Mon, 5 Dec 2016 03:51:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uB53pVqR029603 for ; Mon, 5 Dec 2016 03:51:31 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 213396] official FreeBSD vm image not runing on openstack Date: Mon, 05 Dec 2016 03:51:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: diego.casati@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 03:51:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213396 Diego Casati changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |diego.casati@gmail.com --- Comment #1 from Diego Casati --- Created attachment 177673 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D177673&action= =3Dedit update the OpenStack with configuration with cloudware common options. Hi -- This patch adds the following functionality to the FreeBSD OpenStack image: - resize FreeBSD to the size of the OpenStack flavor (growfs). - speeds up the boot process by disabling sendmail. - allows an user to ssh as root with a public key. - make ssh respond faster by disabling DNS lookups (can be reverted). - enable DHCP on the vtnet interface. This will generate a qcow2 image with 1.4G in size. The entire process uses= the release(7) toolchain to generate the image and should work with newer versi= ons of FreeBSD. Tested with 11.0-RELEASE-p1 and also tested on OpenStack Mitaka. To generate your own image: 1. Follow the regular process of build(7) - buildworld buildkernel 2. cd /usr/src/release 3. make cloudware-release WITH_CLOUDWARE=3Dyes CLOUDWARE=3DOPENSTACK I still need to test the full functionality of cloud-init but overall it lo= oks ok. comments? ok? -dcasati --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Dec 5 03:52:36 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAF8DC675CB for ; Mon, 5 Dec 2016 03:52:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A4736A2 for ; Mon, 5 Dec 2016 03:52:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uB53qae8055649 for ; Mon, 5 Dec 2016 03:52:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 213396] official FreeBSD vm image not runing on openstack Date: Mon, 05 Dec 2016 03:52:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: diego.casati@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 03:52:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213396 --- Comment #2 from Diego Casati --- Created attachment 177674 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D177674&action= =3Dedit add OPENSTACK as an option to Makefile.vm Adds an option for building OpenStack specific images. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Dec 5 10:22:14 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75679C670F5 for ; Mon, 5 Dec 2016 10:22:14 +0000 (UTC) (envelope-from mbraig@gmail.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A0287ED for ; Mon, 5 Dec 2016 10:22:14 +0000 (UTC) (envelope-from mbraig@gmail.com) Received: by mail-oi0-x22a.google.com with SMTP id b126so333688287oia.2 for ; Mon, 05 Dec 2016 02:22:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=0LVVNNaFLobhcTeAj0TOF3NXZ7WMAz6qbaKh3ssgGHg=; b=Jj/Ys98md2TUQxekMzeP7ty9aALQ9IaqGSVY2rlnigTgfai6ysu6kGolzK1nQmK+zw lic5wdv8n1JP/MP4fY5dXjw+K4LnuS6gRADCtsWX3ZL3DpS8CPZjJ9HDpYVg1L7ee5ox vxR4/nRNLu0mACmRlYHG/+Rvd/VP6OjGZ5b5+R6DUqD1pnI7WCHlyoRhAln+rJqf0v4J L8QZ6IU4ju/Tq+hHmFZwuw1s+xd9Xtos6Wu9JNWHNqlUzSuhw5ACrAy8upqgJyv4YFhk l8+XAslttZrlKUtKNWN7acMsfBKVlw7adJgbGstFd/RSDfrEH6oreDBf//kInHVmkXvN YYRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=0LVVNNaFLobhcTeAj0TOF3NXZ7WMAz6qbaKh3ssgGHg=; b=dwlmpugG0jmfxbLKeafVcKs6c/gQ3Ia5bLSHI+0VJqaXOnbjFiKYdyM+qOoSbDhppA aFt9IJzsVPwwVmvL5bJyhfM9FPw/cFH7Xe6qH+IjKVcVGd7QDaJDqP51cuUgjZblMZcd JztUxoL+x24fPb9TxZxtOs/kDUoa7+6C5EqorcCkg1jKPimD3PKNr5gAy5JeenXg++lm OpULGM/zLQu1hMk+WnEGgCxpcGEZxNacQAN65RlNvMMbJAU4s0E6yThDzUuA7wrX0epW MfR4UfdTuN81IHaSN36aVQmUS+zwdwBzMykElNxRt/Y5zk4sdxtYh/rnNPCSmHCIPZJS 6tEw== X-Gm-Message-State: AKaTC0363s0v/G2uHKUaIkmMWBnckW4rtIgAz3xgtDbYNCax54X/HUKQlcdPyGTYGzsEBuPQQfS1LtZ3jeLFSQ== X-Received: by 10.157.45.100 with SMTP id v91mr30309835ota.167.1480933333330; Mon, 05 Dec 2016 02:22:13 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.222.198 with HTTP; Mon, 5 Dec 2016 02:22:13 -0800 (PST) From: Michael Braig Date: Mon, 5 Dec 2016 11:22:13 +0100 Message-ID: Subject: Windows Desktop OS only show one CPU on bhyve even though 2-4 are assigned To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 10:22:14 -0000 Dear All, I have now setup Window Desktop OS based on 8.1 and 10, but also Windows Server 2012 R2. The latter correctly shows the amount of assigned CPUs. The Desktop OSs always only show one, but also have high CPU load with especially the Windows network service. Did anybody experience this as well, anybody who can advise me on how Windows will recognize the additional CPUs? I am using an HP ML110 G7 with Xeon CPUs. Regards Michael From owner-freebsd-virtualization@freebsd.org Mon Dec 5 16:41:53 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3369AC68A26 for ; Mon, 5 Dec 2016 16:41:53 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 175B11A60 for ; Mon, 5 Dec 2016 16:41:52 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [10.1.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 32036D310 for ; Mon, 5 Dec 2016 16:41:51 +0000 (UTC) Subject: Re: Windows Desktop OS only show one CPU on bhyve even though 2-4 are assigned To: freebsd-virtualization@freebsd.org References: From: Allan Jude Message-ID: <8a8684d9-1a98-380d-6c11-26590b0a9eaf@freebsd.org> Date: Mon, 5 Dec 2016 11:41:50 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 16:41:53 -0000 On 2016-12-05 05:22, Michael Braig wrote: > Dear All, > > I have now setup Window Desktop OS based on 8.1 and 10, but also Windows > Server 2012 R2. > > The latter correctly shows the amount of assigned CPUs. The Desktop OSs > always only show one, but also have high CPU load with especially the > Windows network service. > > Did anybody experience this as well, anybody who can advise me on how > Windows will recognize the additional CPUs? > > I am using an HP ML110 G7 with Xeon CPUs. > > Regards > Michael > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > This question has been answered before, in the last 2-3 months, if you search the archive. The short answer is that Windows Desktop OS only supports a single CPU socket, optionally with many cores. The default in bhyve is to expose each virtual CPU as a separate socket. There are a set of sysctls that let you control this, so you can instead expose the 4 cpus as cores of a single socket, and it will then work with Windows 8.1/10, but I don't recall what they are off the top of my head, but they are in the archive which you can browse here: https://lists.freebsd.org/pipermail/freebsd-virtualization/ -- Allan Jude From owner-freebsd-virtualization@freebsd.org Mon Dec 5 17:13:31 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16843C6867B for ; Mon, 5 Dec 2016 17:13:31 +0000 (UTC) (envelope-from richard@bader-muenchen.de) Received: from gate1.bader-muenchen.de (gate1.bader-muenchen.de [213.179.151.243]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8CECFEFE for ; Mon, 5 Dec 2016 17:13:30 +0000 (UTC) (envelope-from richard@bader-muenchen.de) Received: from rserver.bader.loc (rserver.bader.loc [192.168.16.72]) by gate1.bader-muenchen.de (8.15.2/8.15.2) with ESMTP id uB5Gtmmx026004; Mon, 5 Dec 2016 17:55:49 +0100 (CET) (envelope-from richard@bader-muenchen.de) Subject: Re: Windows Desktop OS only show one CPU on bhyve even though 2-4 are assigned References: <8a8684d9-1a98-380d-6c11-26590b0a9eaf@freebsd.org> To: freebsd-virtualization@freebsd.org From: richard Message-ID: <46a99768-aedf-69fd-1164-78bdf694c030@bader-muenchen.de> Date: Mon, 5 Dec 2016 17:55:48 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <8a8684d9-1a98-380d-6c11-26590b0a9eaf@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 17:13:31 -0000 On 12/05/16 17:41, Allan Jude wrote: > On 2016-12-05 05:22, Michael Braig wrote: >> Dear All, >> >> I have now setup Window Desktop OS based on 8.1 and 10, but also Windows >> Server 2012 R2. >> >> The latter correctly shows the amount of assigned CPUs. The Desktop OSs >> always only show one, but also have high CPU load with especially the >> Windows network service. >> >> Did anybody experience this as well, anybody who can advise me on how >> Windows will recognize the additional CPUs? >> >> I am using an HP ML110 G7 with Xeon CPUs. >> >> Regards >> Michael >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" >> > This question has been answered before, in the last 2-3 months, if you > search the archive. > > The short answer is that Windows Desktop OS only supports a single CPU > socket, optionally with many cores. The default in bhyve is to expose > each virtual CPU as a separate socket. There are a set of sysctls that > let you control this, so you can instead expose the 4 cpus as cores of a > single socket, and it will then work with Windows 8.1/10, but I don't > recall what they are off the top of my head, but they are in the archive > which you can browse here: > > https://lists.freebsd.org/pipermail/freebsd-virtualization/ > i've put hw.vmm.topology.cores_per_package=4 in /boot/loader.conf richard From owner-freebsd-virtualization@freebsd.org Mon Dec 5 17:58:01 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9CB60C686A8 for ; Mon, 5 Dec 2016 17:58:01 +0000 (UTC) (envelope-from nicolas.gilles@gmail.com) Received: from mail-lf0-x243.google.com (mail-lf0-x243.google.com [IPv6:2a00:1450:4010:c07::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23C32DA6 for ; Mon, 5 Dec 2016 17:58:01 +0000 (UTC) (envelope-from nicolas.gilles@gmail.com) Received: by mail-lf0-x243.google.com with SMTP id o20so26514425lfg.3 for ; Mon, 05 Dec 2016 09:58:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=6WboNQUx1VcFQN4I/nnstV3PJz24JwrHQdcRqoUHsmw=; b=eXKvDE5HyHDa9whN8/ttmHn8Zb+dHEtlMnjnDWL8D/wkUwX2kCE6mY6n9E0WhXDlXJ JJcNmWVVj/jR8tWcB65GxZlOjcSnRZF2zZpy6bnMyjLK1wwdwsejcnN+7v7jTDcWcQX6 dDTXHs9I/wFs4FJmQY31LVOKSWpVLXM67hSBk/fuLH/1ojOVZ968jUfw1rx9NPm5maV7 lnnlxXqi0syzPC8HGE0YdfSQ2k7QObwVmxn7ZhsLbFrucBcdbCAcS023jXyagRgynwmN GAEyAkvBgq2uChzSTnD/G3iU65TPfDgVnBIYTq3sLaCH3gf7ToN+HPM2b7woCjX/eiRE IwRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=6WboNQUx1VcFQN4I/nnstV3PJz24JwrHQdcRqoUHsmw=; b=FjRDhgV6yzPtJcLPdyB3WkTyd5ZHn7STS+p/kawlQ4rKHHlGo97kmeRPIasHmfhpMy chDFDUTd9oCcDMZvZBrd978N1u9IX4XZhpR2WWHz3pQJWD5GKYk6dR25MRQ500VVl5J0 /Fb62lcanawUY+C32WUcKZWpACclzwE4nHylrgaHf83LRNNokMBxGmz1mgrtpDapKSr+ xkbkxUp7A7MndPeT1WCoJ1iuulGfzV8t5D6+578RAB6L5vx0gAb977ICdT4bQDVcaFn2 1DkhyyHTryki1o9hrtzqPyROuYTTYWIJAYl7A+VIr69l6uC5j4xcX6mTQNeKExdmng0f tjkA== X-Gm-Message-State: AKaTC02x7sg1S5/rDtVsaaRVlqV19eHZbBsU8xJg88zJMsA4Xv69c0mJnewDfL0Qvt4iTaPBttPz36z02MDC2A== X-Received: by 10.25.154.142 with SMTP id c136mr21234303lfe.71.1480960678625; Mon, 05 Dec 2016 09:57:58 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.5.10 with HTTP; Mon, 5 Dec 2016 09:57:58 -0800 (PST) In-Reply-To: <46a99768-aedf-69fd-1164-78bdf694c030@bader-muenchen.de> References: <8a8684d9-1a98-380d-6c11-26590b0a9eaf@freebsd.org> <46a99768-aedf-69fd-1164-78bdf694c030@bader-muenchen.de> From: Nicolas Gilles Date: Mon, 5 Dec 2016 18:57:58 +0100 Message-ID: Subject: Re: Windows Desktop OS only show one CPU on bhyve even though 2-4 are assigned To: richard Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 17:58:01 -0000 On Mon, Dec 5, 2016 at 5:55 PM, richard wrote: > On 12/05/16 17:41, Allan Jude wrote: >> >> On 2016-12-05 05:22, Michael Braig wrote: >>> >>> Dear All, >>> >>> I have now setup Window Desktop OS based on 8.1 and 10, but also Windows >>> Server 2012 R2. >>> >>> The latter correctly shows the amount of assigned CPUs. The Desktop OSs >>> always only show one, but also have high CPU load with especially the >>> Windows network service. >>> >>> Did anybody experience this as well, anybody who can advise me on how >>> Windows will recognize the additional CPUs? >>> >>> I am using an HP ML110 G7 with Xeon CPUs. >>> >>> Regards >>> Michael >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to >>> "freebsd-virtualization-unsubscribe@freebsd.org" >>> >> This question has been answered before, in the last 2-3 months, if you >> search the archive. >> >> The short answer is that Windows Desktop OS only supports a single CPU >> socket, optionally with many cores. The default in bhyve is to expose >> each virtual CPU as a separate socket. There are a set of sysctls that >> let you control this, so you can instead expose the 4 cpus as cores of a >> single socket, and it will then work with Windows 8.1/10, but I don't >> recall what they are off the top of my head, but they are in the archive >> which you can browse here: >> >> https://lists.freebsd.org/pipermail/freebsd-virtualization/ >> > i've put > > hw.vmm.topology.cores_per_package=4 > > in /boot/loader.conf > > richard > > And did that fix your issue? It's a little bit annoying is that this setting is "systemwide" but I believe there was talk to make this per-vm, though I don't know how that is coming along... > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@freebsd.org Mon Dec 5 18:14:07 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88ADAC68C71 for ; Mon, 5 Dec 2016 18:14:07 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: from mail-wm0-x233.google.com (mail-wm0-x233.google.com [IPv6:2a00:1450:400c:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E8F318B9 for ; Mon, 5 Dec 2016 18:14:07 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: by mail-wm0-x233.google.com with SMTP id a197so105797195wmd.0 for ; Mon, 05 Dec 2016 10:14:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=E0uff4LxgEijYTi8fb53UthytIfYf3BATlI+AFb0UEw=; b=nwXZFwK7d14YTBELSv1lJE2hQ+PzYftWmyW+xfdDwe1+vMKU9R7ZvcqXmLlFztlbxG rnMYDi/zKxA+jecVm49Ax8MoveFpi9rMIE6qTfnsCW/OvZJnXTI4IXsAjsx+vKkNhJxN tWkfm/6VKG32O2W6EtoLLNd1ZSGKi7OcCI935fV9ZD+62PEdH/VJc7XpkAD9U6eA3Kdf qLeG+zzwqS2LUPgNxLdzw3fKsYMtzEz+hsekzME5KDwxP8VOQHqE/5gVTsba9BjkFaAe q1OvM24+trsOLZK1b+o3Q2/KT6E35MR/SmJDguEZ3xO9ZLSXm7VypIBkodduelCbLamH Vsdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=E0uff4LxgEijYTi8fb53UthytIfYf3BATlI+AFb0UEw=; b=I5T0wa0Wn6xDDcycclsV8YTvS7qxcoDCl7tKIuL3HnI0I4FgLYcCpB6OF/wY1Ucy7G NZ3ygSJGiuIf3/F4cZIs1sOwrhx2uDV4a7TE83HUTpNmrANKH6hcjmOQvz5cRg4cwEms jAXcdJKicB22pymFQdrVoc7Yq5i+z8iDYaKrZXYc8DgFCcbi4Cpb+RjGXmpJqpQU8Hi4 hdsdgaRfHfN40W0CpuSs1BQ5+hLBaO1jOzy2u4hGWPdZotVbnm+30YOPSODMI4YAM96n cSPcGmEXsrFNoZiWlQ0/69XlT0uNz3yL4IIyj5h2yQJucwK8/2A9vwVOR3TT2dmTXOdC JI6w== X-Gm-Message-State: AKaTC00K2RJw8NuS1A76QJM4Pyfn3cqczbEwOAsc5S4gOV1oxFLKl6Hp7QUcs4fsPWAhdQ== X-Received: by 10.28.23.16 with SMTP id 16mr11141767wmx.134.1480961645134; Mon, 05 Dec 2016 10:14:05 -0800 (PST) Received: from [192.168.0.16] (dh207-7-169.xnet.hr. [88.207.7.169]) by smtp.gmail.com with ESMTPSA id i10sm21647640wjd.15.2016.12.05.10.14.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Dec 2016 10:14:04 -0800 (PST) To: freebsd-virtualization@freebsd.org From: Domagoj Stolfa Subject: Automatic provisioning via bhyve Message-ID: <12ca0f5e-9fba-3acd-4da7-e43a40997e69@gmail.com> Date: Mon, 5 Dec 2016 19:14:03 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 18:14:07 -0000 Hello, I was wondering if anyone has experimented or worked on with a way, given the ISO, automatically create any number of bhyve guests with preset installer options, an option to run them and a way to destroy all the virtual machines as well as their images/zvols. Best regards, Domagoj Stolfa. From owner-freebsd-virtualization@freebsd.org Mon Dec 5 19:01:55 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABB23C67922 for ; Mon, 5 Dec 2016 19:01:55 +0000 (UTC) (envelope-from peterg@ptree32.com.au) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 6D95C1234 for ; Mon, 5 Dec 2016 19:01:54 +0000 (UTC) (envelope-from peterg@ptree32.com.au) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id EA4B8212665A for ; Tue, 6 Dec 2016 05:01:29 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id E191B28095F for ; Tue, 6 Dec 2016 05:01:29 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id mZUUnr2ZWSfY for ; Tue, 6 Dec 2016 05:01:29 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 38AA328095D; Tue, 6 Dec 2016 05:01:27 +1000 (AEST) Subject: Re: Windows Desktop OS only show one CPU on bhyve even though 2-4 are assigned To: Allan Jude , freebsd-virtualization@freebsd.org References: <8a8684d9-1a98-380d-6c11-26590b0a9eaf@freebsd.org> From: Peter Grehan Message-ID: Date: Mon, 5 Dec 2016 11:01:42 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <8a8684d9-1a98-380d-6c11-26590b0a9eaf@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=YJDv8VOx c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17 a=N659UExz7-8A:10 a=n5n_aSjo0skA:10 a=6I5d2MoRAAAA:8 a=WAkjZaPtGMlADHqEU0sA:9 a=pILNOxqGKmIA:10 a=IjZwj45LgO3ly-622nXo:22 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 19:01:55 -0000 > The short answer is that Windows Desktop OS only supports a single CPU > socket, optionally with many cores. The default in bhyve is to expose > each virtual CPU as a separate socket. There are a set of sysctls that > let you control this, so you can instead expose the 4 cpus as cores of a > single socket, and it will then work with Windows 8.1/10, but I don't > recall what they are off the top of my head, but they are in the archive > which you can browse here: > > https://lists.freebsd.org/pipermail/freebsd-virtualization/ The post in question is http://docs.FreeBSD.org/cgi/mid.cgi?b26b6124-7ac6-0408-3016-f5678ad144d0 later, Peter. From owner-freebsd-virtualization@freebsd.org Mon Dec 5 19:20:38 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDB2FC68142 for ; Mon, 5 Dec 2016 19:20:38 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 74F041FF1 for ; Mon, 5 Dec 2016 19:20:38 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 692B32013A33 for ; Tue, 6 Dec 2016 05:20:19 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 6034328095F for ; Tue, 6 Dec 2016 05:20:19 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ezYh5s1zYkus for ; Tue, 6 Dec 2016 05:20:19 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 0C22A28095D; Tue, 6 Dec 2016 05:20:15 +1000 (AEST) Subject: Re: Windows Desktop OS only show one CPU on bhyve even though 2-4 are assigned To: Nicolas Gilles , richard References: <8a8684d9-1a98-380d-6c11-26590b0a9eaf@freebsd.org> <46a99768-aedf-69fd-1164-78bdf694c030@bader-muenchen.de> Cc: freebsd-virtualization@freebsd.org From: Peter Grehan Message-ID: Date: Mon, 5 Dec 2016 11:20:30 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=YJDv8VOx c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17 a=N659UExz7-8A:10 a=n5n_aSjo0skA:10 a=6I5d2MoRAAAA:8 a=FV4QrHuGbHCOm3EB_y0A:9 a=pILNOxqGKmIA:10 a=RjIzD_f69BQA:10 a=IjZwj45LgO3ly-622nXo:22 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 19:20:38 -0000 > It's a little bit annoying is that this setting is "systemwide" I agree. > but I believe there was talk to make this per-vm, though I don't know > how that is coming along... Not quickly. I added a bugzilla note to track this: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215077 later, Peter. From owner-freebsd-virtualization@freebsd.org Tue Dec 6 18:46:34 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF3C7C6A3BE for ; Tue, 6 Dec 2016 18:46:34 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 67D58F53 for ; Tue, 6 Dec 2016 18:46:34 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 73D0320AE98B for ; Wed, 7 Dec 2016 04:46:14 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 64B69280996 for ; Wed, 7 Dec 2016 04:46:14 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id tPqMjQTkpDWC for ; Wed, 7 Dec 2016 04:46:14 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 52EA928095F; Wed, 7 Dec 2016 04:46:11 +1000 (AEST) Subject: Re: bhyve: cannot send jumbo frames from linux or freebsd guest To: =?UTF-8?B?Sm9zw6kgR2FyY8OtYSBKdWFuaW5v?= References: From: Peter Grehan Cc: freebsd-virtualization@freebsd.org Message-ID: <414db7f0-ea86-2080-4179-eb77a18a67a5@freebsd.org> Date: Tue, 6 Dec 2016 10:46:25 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=YJDv8VOx c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17 a=N659UExz7-8A:10 a=n5n_aSjo0skA:10 a=syzSIFLCVYJZK2B39fUA:9 a=ncJBWkJFt_E_2-gn:21 a=uSEb3iXFj9whLTb5:21 a=pILNOxqGKmIA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2016 18:46:34 -0000 Hi, > * From FreeBSD guest: > # tcpdump -vvn -i vtnet1 & > tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture > size 262144 bytes > # ping -c 1 -D -s 1995 192.168.253.1 > PING 192.168.253.1 (192.168.253.1): 1995 data bytes > 23:08:46.808187 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto > ICMP (1), length 2023) > 192.168.253.4 > 192.168.253.1: ICMP echo request, id 58116, seq 0, > length 2003 > 23:08:46.808430 IP truncated-ip - 1 bytes missing! (tos 0x0, ttl 64, > id 0, offset 0, flags [DF], proto ICMP (1), length 2023) > 192.168.253.1 > 192.168.253.4: ICMP echo reply, id 58116, seq 0, length 2003 This looks like a bug in the FreeBSD virtio-net driver bpf code, since the host dump shows those sized packets going out correctly. > Could anyone explain why I cannot send jumbo frames from Linux and FreeBSD > guests, but I *can* send them from Windows guest? Am I missing o > missunderstanding something? Thanks in advanced, any comment o suggestion will > be wellcome. There is currently a bug in bhyve with handling of received frames. The virtio implementation advertises 'rx-merge' capability, which allows frames to span buffers that have been queued by the guest. However, bhyve currently doesn't copy the buffer across multiple buffers. I suspect that the Windows virtio guest driver doesn't use rx-merge but instead uses 'indirect descriptors', which is why it works. A short-term workaround for FreeBSD and Linux guests is to modify bhyve to not advertise the rx-merge capability: usr.sbin/bhyve/pci_virtio_net.c #define VTNET_S_HOSTCAPS \ - ( VIRTIO_NET_F_MAC | VIRTIO_NET_F_MRG_RXBUF | VIRTIO_NET_F_STATUS | \ + ( VIRTIO_NET_F_MAC | VIRTIO_NET_F_STATUS | \ (Note that this doesn't work with Net/OpenBSD, where jumbo frames are only supported via rx-merge.) I hope to have a fix for the rx-merge issue in the near future - it's a more efficient way of handling jumbo frames than indirect descriptors. later, Peter. From owner-freebsd-virtualization@freebsd.org Wed Dec 7 03:26:00 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DFC3C674FF for ; Wed, 7 Dec 2016 03:26:00 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: from mail-pg0-x229.google.com (mail-pg0-x229.google.com [IPv6:2607:f8b0:400e:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 504B6B06 for ; Wed, 7 Dec 2016 03:26:00 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: by mail-pg0-x229.google.com with SMTP id f188so156010979pgc.3 for ; Tue, 06 Dec 2016 19:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=sS08kBU3zVtQffAbWE+Q2gMxn5n8wBmfB2lROaq1V9I=; b=lg8EWPJJ1/HgY8Dmv7UaIoovg/utxNuj4Gmvz/u7SOUQDwrOxA0/bbv+HIfVhJPq2y xp3FwndzVTJqKrmxkgMhlhLAao1wsenQ3fBIwm2YbSiaOhFWu+QfV/F+bchzdTvKidFs xnTirJRvxbhcowkAGZlaUg1ZSQbPY+dZDJd2V3kKgxG27DSL+KWKONiEUAqev7PYCw83 Pr15SIys/y3m03QlClLUpyUTUzKHW9ucD4rnLSWG2MX9smTDZqVUhJ0jpWV+MKefmcfQ eZQqAOBSKnJYBOIgqubuwcKd5sXV9StWdfollBpvDhvOOeWrueTcujC3oxcgyg0UYwfC s+WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=sS08kBU3zVtQffAbWE+Q2gMxn5n8wBmfB2lROaq1V9I=; b=Z+41kUpxA3Xjigm7DxXD7JEk8uFsLrIbpk9cT1MFwRPPsVhs7WePze4CLTEER8baq7 /qPnRRJ+iWTprf4AJkdirFdysbT4/Kd0q65bxMhthkHHYYdudZLEchGfB1rtKs8sxpDu wU86SDKbWH92PsvhJ1WyAmPhfJAhxatv4jjrpjjLlQoHO34U5Rhaj2Tz9ff+wv/UAF5W eeb20CQGhqtuCxoQS0Dlxb4KZ1IkilhfUEARDobAC/y5TM11H2JGQ57jji5xRc6UFuOw WWphWFauk1OX+KNQf2YupxBEq6uNpNUlT4cuSiwM3s22puYA26aW4nIK6IBUCNYBup3G efRg== X-Gm-Message-State: AKaTC03cSdwycBE5JGqqB3PTYQzccletvUkBsPCaNV/CzFBskhcMfDDVr5Nrm2fObn1U5A== X-Received: by 10.84.216.26 with SMTP id m26mr143447582pli.22.1481081159800; Tue, 06 Dec 2016 19:25:59 -0800 (PST) Received: from [172.16.1.28] (c-73-155-87-64.hsd1.tx.comcast.net. [73.155.87.64]) by smtp.gmail.com with ESMTPSA id i11sm38064909pgn.17.2016.12.06.19.25.58 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Dec 2016 19:25:59 -0800 (PST) To: freebsd-virtualization@FreeBSD.org From: Rajil Saraswat Subject: Debian 8 CPU stall Message-ID: Date: Tue, 6 Dec 2016 21:25:26 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2016 03:26:00 -0000 Hello, I get these messages in Debian 8 VM running in bhyve FreeBSD-11 release. Any idea what could be the issue: INFO: rcu_sched detected stalls on CPUs/tasks: {} (detected by 0, t=11047 jiffies, g=1038939, c=1038938, q=77) INFO: Stall ended before state dump start Thanks, Rajil From owner-freebsd-virtualization@freebsd.org Wed Dec 7 04:51:00 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4283C6BD00 for ; Wed, 7 Dec 2016 04:51:00 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 9807612FB for ; Wed, 7 Dec 2016 04:51:00 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 82A5F20195CA for ; Wed, 7 Dec 2016 14:50:34 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 7E4D82809CC for ; Wed, 7 Dec 2016 14:50:34 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 7CRUUgsERoOt for ; Wed, 7 Dec 2016 14:50:34 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id BF279280999; Wed, 7 Dec 2016 14:50:32 +1000 (AEST) Subject: Re: Debian 8 CPU stall To: Rajil Saraswat References: Cc: freebsd-virtualization@FreeBSD.org From: Peter Grehan Message-ID: <9c9e83a5-16c6-0ab5-0ac4-af0a54430706@freebsd.org> Date: Tue, 6 Dec 2016 20:50:46 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=YJDv8VOx c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=5eVCmCvhg37cu/pjidAGzw==:17 a=N659UExz7-8A:10 a=n5n_aSjo0skA:10 a=j2x9U6APwcifdffEom8A:9 a=pILNOxqGKmIA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2016 04:51:00 -0000 Hi Rajil, > I get these messages in Debian 8 VM running in bhyve FreeBSD-11 > release. Any idea what could be the issue: > > INFO: rcu_sched detected stalls on CPUs/tasks: {} (detected by 0, > t=11047 jiffies, g=1038939, c=1038938, q=77) > INFO: Stall ended before state dump start That's a sign that a vCPU wasn't able to run for an amount of time. Is the system oversubscribed ? i.e. more vCPUs than physical CPUs ? Or, is the guest performing a lot of i/o ? later, Peter. From owner-freebsd-virtualization@freebsd.org Thu Dec 8 00:05:41 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85ACCC6C750 for ; Thu, 8 Dec 2016 00:05:41 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: from mail-pg0-x242.google.com (mail-pg0-x242.google.com [IPv6:2607:f8b0:400e:c05::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E4B0366 for ; Thu, 8 Dec 2016 00:05:41 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: by mail-pg0-x242.google.com with SMTP id 3so24931578pgd.0 for ; Wed, 07 Dec 2016 16:05:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:references:from:to:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=cy3a3cJFLCbORx8LwqqMLhc/k0c6AJ1s9PiQqaB0k5E=; b=qoMV1UNWtUCl3x3X/+j9Kz5XEf77IrrNsLNh9OVSOANC+7/o8YQa0wnrrr33VYmArv tNgLxSTIeaNojkjVenrB+kBTJMYtp5uW1qhnJU06Q30gt6na3hQJn3RrU7OzNoNGhATV mWjk4nDejxyT6V9HrGdxvnviByHoPWwi/G62CEP9gK5sDpNKSgoMjDYCFrF7fox6wJVd Ke6JnysmCVEJhX6U2oZdfUQMQprh9I50ljtn5holCKNgx8+vwS1s6y2F6TdgFwUWyntO hvElzpmFrnVgB7Jql8KzOCrJJTn0ymNyphIOn26CWnra65L4Ubh/Mf5/bqBrlkxCWsaL VrOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:references:from:to:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=cy3a3cJFLCbORx8LwqqMLhc/k0c6AJ1s9PiQqaB0k5E=; b=NWXIq+wIIJWhucw2Ug0gZhAHXGxJ4xDV/on5RMyVMncPf7cY6QBavsgr56anUB0QID WrR08hpBHqrxMo4ZUBeWZqCoQof0XMF2pdulKAtV9TsPckvxGYo6bPkNEqyeHmZV9IiZ uru9FS6RHlbe489AKZ9VgGyh0oL0mV3XSyYCe/LiN3ogVc5Nku3rZ0AP5A2eNwFnPyCz YAp+Fx2wVQ2+zBrjHFfWrKf3lAlyryWsDREYzYzIKgJr4NggylzQakXXo0jWCOUFgXy2 0q77GLBxKH/ey5qTGypFq77T8v0fkccRCRC3fGFHHsNev7JjPGOAZWxhR1meGVok20Ry rzhg== X-Gm-Message-State: AKaTC03+gkdWy2VOKE4Y0qaeaWaJxIcm2X6z0Ycv98QYs3apj5fdu7FCuopxWwwuz4KvcQ== X-Received: by 10.84.128.46 with SMTP id 43mr153392750pla.100.1481155540633; Wed, 07 Dec 2016 16:05:40 -0800 (PST) Received: from [172.16.1.28] (c-73-155-87-64.hsd1.tx.comcast.net. [73.155.87.64]) by smtp.gmail.com with ESMTPSA id o126sm45249481pga.34.2016.12.07.16.05.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 Dec 2016 16:05:40 -0800 (PST) Subject: Re: Debian 8 CPU stall References: <9c9e83a5-16c6-0ab5-0ac4-af0a54430706@freebsd.org> From: Rajil Saraswat To: freebsd-virtualization@FreeBSD.org Message-ID: <6cff5bf2-9654-8627-83c4-6ab48ee763b5@gmail.com> Date: Wed, 7 Dec 2016 18:05:06 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <9c9e83a5-16c6-0ab5-0ac4-af0a54430706@freebsd.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2016 00:05:41 -0000 On 12/06/2016 10:50 PM, Peter Grehan wrote: > Hi Rajil, > >> I get these messages in Debian 8 VM running in bhyve FreeBSD-11 >> release. Any idea what could be the issue: >> >> INFO: rcu_sched detected stalls on CPUs/tasks: {} (detected by 0, >> t=11047 jiffies, g=1038939, c=1038938, q=77) >> INFO: Stall ended before state dump start > > That's a sign that a vCPU wasn't able to run for an amount of time. > > Is the system oversubscribed ? i.e. more vCPUs than physical CPUs ? > Or, is the guest performing a lot of i/o ? > > later, > > Peter. > No the system is not oversubscribed. I have a 11 vCPU (1 on debian and 1 on ubuntu) on a 24 core machine. The debian jail is running x2go and an ssh server for remote access, so the I/O shouldnt be an issue. The ubuntu jail doesnt give out any warning messages though. Following is the latest error i received in the debian vm: [152444.353007] INFO: rcu_sched self-detected stall on CPU { 0} (t=6809 jiffies g=2685598 c=2685597 q=6) [152444.354261] sending NMI to all CPUs: [152444.354270] NMI backtrace for cpu 0 [152444.354274] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.16.0-4-amd64 #1 Debian 3.16.36-1+deb8u2 [152444.354275] Hardware name: BHYVE, BIOS 1.00 03/14/2014 [152444.354277] task: ffffffff8181a460 ti: ffffffff81800000 task.ti: ffffffff81800000 [152444.354278] RIP: 0010:[] [] default_send_IPI_mask_sequence_phys+0xad/0xe0 [152444.354293] RSP: 0018:ffff88007fc03e18 EFLAGS: 00000046 [152444.354294] RAX: 0000000000000400 RBX: 000000000000a0ea RCX: 0000000000000000 [152444.354296] RDX: 0000000000000000 RSI: 0000000000000200 RDI: 0000000000000300 [152444.354297] RBP: ffffffff818e29c0 R08: ffffffff818e29c0 R09: 00000000000001bb [152444.354298] R10: 0000000000000000 R11: ffff88007fc03b96 R12: 0000000000000400 [152444.354299] R13: 0000000000000096 R14: 0000000000000002 R15: 0000000000000000 [152444.354301] FS: 0000000000000000(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000 [152444.354303] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [152444.354303] CR2: 00000000019a3240 CR3: 00000000797ca000 CR4: 00000000000406f0 [152444.354305] Stack: [152444.354306] 0000000000000000 ffff88007fc0d6a0 ffffffff81853800 0000000000000000 [152444.354308] ffffffff818e2f20 0000000000000006 ffffffff81853800 ffffffff81047cf3 [152444.354309] ffff88007fc0d6a0 ffffffff810c73ea ffffffff00000007 ffffffff810c8f35 [152444.354311] Call Trace: [152444.354313] [152444.354318] [] ? arch_trigger_all_cpu_backtrace+0xc3/0x140 [152444.354327] [] ? rcu_check_callbacks+0x42a/0x670 [152444.354331] [] ? timekeeping_update.constprop.9+0x35/0x70 [152444.354335] [] ? tick_sched_handle.isra.16+0x60/0x60 [152444.354343] [] ? update_process_times+0x40/0x70 [152444.354345] [] ? tick_sched_handle.isra.16+0x20/0x60 [152444.354347] [] ? tick_sched_timer+0x3c/0x60 [152444.354351] [] ? __run_hrtimer+0x67/0x210 [152444.354353] [] ? hrtimer_interrupt+0xe9/0x220 [152444.354359] [] ? smp_apic_timer_interrupt+0x3b/0x50 [152444.354365] [] ? apic_timer_interrupt+0x6d/0x80 [152444.354366] [152444.354374] [] ? mwait_idle+0xa0/0xa0 [152444.354381] [] ? native_safe_halt+0x2/0x10 [152444.354384] [] ? default_idle+0x19/0xd0 [152444.354389] [] ? cpu_startup_entry+0x374/0x470 [152444.354392] [] ? start_kernel+0x497/0x4a2 [152444.354394] [] ? set_init_arg+0x4e/0x4e [152444.354396] [] ? early_idt_handler_array+0x120/0x120 [152444.354398] [] ? x86_64_start_kernel+0x14d/0x15c [152444.354399] Code: 8b 0c 25 00 53 5f ff 80 e5 10 75 f2 44 89 f8 c1 e0 18 89 04 25 10 53 5f ff 41 83 fe 02 44 89 e0 41 0f 45 c6 89 04 25 00 53 5f ff 91 4c 89 ef 57 9d 0f 1f 44 00 00 48 83 c4 08 5b 5d 41 5c 41 I use vm-bhyve for managing the jails which looks like this for debian8: guest="linux" loader="grub" cpu=1 memory=2048M network0_type="virtio-net" network0_switch="lannetwork" disk0_type="virtio-blk" disk0_name="/dev/zvol/vmpool/os2" disk0_dev="custom" passthru0="2/0/0" passthru1="2/0/1" Thanks, Rajil From owner-freebsd-virtualization@freebsd.org Thu Dec 8 02:53:54 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69082C6ADAB for ; Thu, 8 Dec 2016 02:53:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 57DC4184C for ; Thu, 8 Dec 2016 02:53:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uB82rqpj054030 for ; Thu, 8 Dec 2016 02:53:54 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 212721] FreeBSD 11.0-RC2/RC3/RELEASE fails on Hyper-V 2012r2 Date: Thu, 08 Dec 2016 02:53:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RC1 X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: alessandro.segala@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2016 02:53:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D212721 Alessandro changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alessandro.segala@gmail.com --- Comment #47 from Alessandro --- This issue is causing errors with FreeBSD 11.0-RELEASE (up to p4) running on Azure as well. I've seen multiple instances of VMs failing to boot because = of this bug. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Dec 8 15:27:26 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 520B3C6D1D3 for ; Thu, 8 Dec 2016 15:27:26 +0000 (UTC) (envelope-from jjuanino@gmail.com) Received: from mail-wm0-x242.google.com (mail-wm0-x242.google.com [IPv6:2a00:1450:400c:c09::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D7F68B03 for ; Thu, 8 Dec 2016 15:27:25 +0000 (UTC) (envelope-from jjuanino@gmail.com) Received: by mail-wm0-x242.google.com with SMTP id a20so4240008wme.2 for ; Thu, 08 Dec 2016 07:27:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mail-followup-to:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=IUXAdDZjNElQ5T8geKiFyLde8tMmcAaEXxtzfF6nh/w=; b=HkaOhYyDFi58Nmkvc5QB0WtJ0TIk/hHFvSa2DxmWHuMcDs46hHlKO+jmhIRp6tMRyk Vl/GTJXrnmlCkrQhquohRJmHX7pCqMXATbIfZfXv2m8WNvu2AnI2qhk4ZdKTEz/Sp8iv idPw6FTiAEbV8catc7QKb+kdI+ocBLUKrv4rnrGsU0UYtRDLYLuNpERPe8p79hyvTqDE R5qD9Z+uIhq7GPk98PPHJ2Y/lypesYDd1H9CsRfrq2gxyAGEjSRGZqFX1ovn2zac+97p tlNhDFCB7/1yopX0KYnbD9i1ONpxihLhM5alb+9VCxDBi+HdoRpz2jNHHAZHvJKLN9u4 AqRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:mail-followup-to :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=IUXAdDZjNElQ5T8geKiFyLde8tMmcAaEXxtzfF6nh/w=; b=TsaDCHjLL611pWGHU6EfYe7x4tcyc7Wi86yq8pxvyBqd8Q2luFJ34bxaK8fbp20Zb2 fUj4F3/RucuSnjldaZqkGaeW7OrXEHDzFbD+i7kngYE6UDFP5N0RnOORGNyKUQrcbitU WrbJ8Mbm9x1I6eF5pTGzg1jwOuBBqVsBA5S4dm/25gLujUydFhSmCbOh7x1aM5YWVa9A aV+z3dr5XPtCaqhbBBReePrNZZCZxyhwnEOnP9zlj5ujFMxHoNFzaD8aCuSF62xubqA0 ANHjwgu+KCVkd1gajBMzIRm4rjWejJKCynOYyO1e02DQzktoY2AcvAegW9y1aaMJ3+7j 0IpQ== X-Gm-Message-State: AKaTC00tLKPUhQi9LdhSYD33Vz8dAFZmhOx4FkUh5TkJWHhOrHsWZ8lhcDc/D+i5VPGFJA== X-Received: by 10.28.236.83 with SMTP id k80mr2705963wmh.0.1481210843876; Thu, 08 Dec 2016 07:27:23 -0800 (PST) Received: from abel.home.inet (79.108.83.135.dyn.user.ono.com. [79.108.83.135]) by smtp.gmail.com with ESMTPSA id 6sm37555012wjt.5.2016.12.08.07.27.22 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 08 Dec 2016 07:27:23 -0800 (PST) Date: Thu, 8 Dec 2016 16:27:20 +0100 From: =?iso-8859-1?Q?Jos=E9_G=2E?= Juanino To: freebsd-virtualization@freebsd.org Subject: Re: bhyve: cannot send jumbo frames from linux or freebsd guest Message-ID: <20161208152708.upi3qjmsn7kzfd3l@abel.home.inet> Mail-Followup-To: freebsd-virtualization@freebsd.org References: <414db7f0-ea86-2080-4179-eb77a18a67a5@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <414db7f0-ea86-2080-4179-eb77a18a67a5@freebsd.org> X-Operating-System: FreeBSD 11.0-RELEASE-p2 User-Agent: NeoMutt/20161126 (1.7.1) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2016 15:27:26 -0000 El martes 06 de diciembre a las 19:46:25 CET, Peter Grehan escribió: > I suspect that the Windows virtio guest driver doesn't use rx-merge >but instead uses 'indirect descriptors', which is why it works. > > A short-term workaround for FreeBSD and Linux guests is to modify >bhyve to not advertise the rx-merge capability: > >usr.sbin/bhyve/pci_virtio_net.c >#define VTNET_S_HOSTCAPS \ >- ( VIRTIO_NET_F_MAC | VIRTIO_NET_F_MRG_RXBUF | VIRTIO_NET_F_STATUS | \ >+ ( VIRTIO_NET_F_MAC | VIRTIO_NET_F_STATUS | \ > > > (Note that this doesn't work with Net/OpenBSD, where jumbo frames are >only supported via rx-merge.) > > I hope to have a fix for the rx-merge issue in the near future - it's >a more efficient way of handling jumbo frames than indirect >descriptors. Hello, thanks for your response. I have applied your patch and the test from FreeBSD guest runs successfully: # uname -opr FreeBSD 11.0-RELEASE-p1 amd64 # ping -c 4 -D -s 8972 192.168.253.1 PING 192.168.253.1 (192.168.253.1): 8972 data bytes 8980 bytes from 192.168.253.1: icmp_seq=0 ttl=64 time=0.224 ms 8980 bytes from 192.168.253.1: icmp_seq=1 ttl=64 time=0.197 ms 8980 bytes from 192.168.253.1: icmp_seq=2 ttl=64 time=0.245 ms 8980 bytes from 192.168.253.1: icmp_seq=3 ttl=64 time=0.226 ms --- 192.168.253.1 ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.197/0.223/0.245/0.017 ms But the Linux (Centos 6) test does not work: after apply your patch, I can only send 1504 bytes in one frame: # ping -c 1 -s 1476 -M do 192.168.253.1 PING 192.168.253.1 (192.168.253.1) 1476(1504) bytes of data. 1484 bytes from 192.168.253.1: icmp_seq=1 ttl=64 time=0.155 ms --- 192.168.253.1 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.155/0.155/0.155/0.000 ms # ping -c 1 -s 1477 -M do 192.168.253.1 PING 192.168.253.1 (192.168.253.1) 1477(1505) bytes of data. --- 192.168.253.1 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 2089ms Best regards -- José G. Juanino From owner-freebsd-virtualization@freebsd.org Sat Dec 10 02:59:42 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D97CC6FCE9 for ; Sat, 10 Dec 2016 02:59:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C5D115D9 for ; Sat, 10 Dec 2016 02:59:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBA2xgDP097599 for ; Sat, 10 Dec 2016 02:59:42 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 215171] [Hyper-V]Fail to attach SCSI disk from LUN 8 on Win2008R2/Win2012/Win2012R2 Date: Sat, 10 Dec 2016 02:59:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2016 02:59:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215171 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Dec 10 20:01:30 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2914C70CE5 for ; Sat, 10 Dec 2016 20:01:30 +0000 (UTC) (envelope-from zshen10@cs.rochester.edu) Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0103.outbound.protection.outlook.com [104.47.34.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 642291107 for ; Sat, 10 Dec 2016 20:01:29 +0000 (UTC) (envelope-from zshen10@cs.rochester.edu) Received: from MWHPR07MB3197.namprd07.prod.outlook.com (10.172.96.143) by MWHPR07MB3200.namprd07.prod.outlook.com (10.172.96.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.761.9; Sat, 10 Dec 2016 20:01:20 +0000 Received: from MWHPR07MB3197.namprd07.prod.outlook.com ([10.172.96.143]) by MWHPR07MB3197.namprd07.prod.outlook.com ([10.172.96.143]) with mapi id 15.01.0761.020; Sat, 10 Dec 2016 20:01:21 +0000 From: "Shen, Zhuojia" To: "freebsd-virtualization@freebsd.org" Subject: Execute VMXON again without setting CR4.VMXE to 1 Thread-Topic: Execute VMXON again without setting CR4.VMXE to 1 Thread-Index: AQHSUxsYQ5EC4zJOCE6bHVqnTeXTpw== Date: Sat, 10 Dec 2016 20:01:20 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=zshen10@cs.rochester.edu; x-originating-ip: [25.173.33.132] x-ms-office365-filtering-correlation-id: 9c530daa-e4b2-4d7b-ea38-08d421374fbb x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:MWHPR07MB3200; x-microsoft-exchange-diagnostics: 1; MWHPR07MB3200; 7:667jCuxL9v7lGb9Ij0XtpDOtgkuoBY2teA9gMpyjCV+CdHRyRI0w48ILfrnEMF1u8+FdN6YEh2F6an253aJD6KLuLFpQdV1fjx8j+/bonYtoHGxD2lBGskFKjvROrqtlba3Ml7mpmlZ+ONoGFQrP0OYI2AQKwd7U0CVGIiBQfTViCNKdtxE4BQyYl37IvwAcrapvZGFE04KSfTaxeFKr3UauhuPhrDhkpD2xolK2/WSGPqNirFQLA1mu8Mr90VXf719nitEU/+4nPRsfkr4Id5hxlkjmjiVntKvd450fBKt7WlFy6Wil8rg8+utJMecFF8+jnrkPLJ2BXI7ZOylNuije4S1fI/YKlcdRO3hJEEZnw7wvIsnmmTo2K7pNnRBem5zRhIWPnMUYijVay9lxiPQz6h1Nngza4gF2LDsS7F7hY7AgNVrZjVkE+PX/ueJcNOd76EQkMjYY6Lg0xu0vMw== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6041248)(20161123564025)(20161123562025)(20161123555025)(20161123560025)(6072148); SRVR:MWHPR07MB3200; BCL:0; PCL:0; RULEID:; SRVR:MWHPR07MB3200; x-forefront-prvs: 0152EBA40F x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(39840400002)(39850400002)(39450400003)(39410400002)(53754006)(199003)(189002)(19627405001)(106356001)(8936002)(106116001)(77096006)(122556002)(105586002)(99286002)(86362001)(97736004)(102836003)(3846002)(6116002)(38730400001)(2900100001)(6436002)(2501003)(92566002)(66066001)(6506006)(75432002)(5640700002)(68736007)(189998001)(7736002)(7696004)(54356999)(6916009)(110136003)(42882006)(9686002)(74316002)(107886002)(3280700002)(101416001)(3660700001)(5660300001)(450100001)(6606003)(33656002)(88552002)(8676002)(2906002)(50986999)(81156014)(76576001)(81166006)(89122001)(2351001); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR07MB3200; H:MWHPR07MB3197.namprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: cs.rochester.edu does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: cs.rochester.edu X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Dec 2016 20:01:20.8023 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 374e17f4-cf11-4ce2-b3ef-5de76bf4ce41 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR07MB3200 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2016 20:01:30 -0000 Hi All, I believe I have found a bug in the BHyVe VMX implementation for Intel plat= form. Following is the details. In sys/amd64/vmm/intel/vmx.c, there are three functions, vmx_enable (line 5= 00), vmx_disable (line 462), and vmx_restore (line 522), which respectively= enable, disable, and restore VMX on the current processor. There is also a= file scope integer array vmxon_enabled (line 122) which keeps a record whe= ther VMX on each processor is enabled. I noticed that in function vmx_enable, the kernel sets CR4.VMXE to 1, execu= tes the VMXON instruction, and then sets the corresponding cell in vmxon_en= abled to 1. That is fine. However, in function vmx_disable, the kernel exec= utes VMXOFF, clears CR4.VMXE back to 0, but does not clear the correspondin= g cell in vmxon_enabled back to 0. This will cause a problem when the funct= ion vmx_restore is called, in which the kernel executes VMXON again if that= the corresponding cell in that array is 1. That is, it will execute VMXON = without CR4.VMXE being set. Of course, that is only my understanding of the way how these functions wil= l be called. If I am wrong, could anyone of you please advise me how this w= orks? Thanks. Bests, Zhuojia Shen --- Zhuojia Shen Graduate Student Department of Computer Science University of Rochester From owner-freebsd-virtualization@freebsd.org Sat Dec 10 20:44:45 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 954B8C7189F for ; Sat, 10 Dec 2016 20:44:45 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 110711F25 for ; Sat, 10 Dec 2016 20:44:44 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id uBAKihWN073495 for ; Sat, 10 Dec 2016 21:44:43 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 63B02D50; Sat, 10 Dec 2016 21:44:42 +0100 (CET) Message-ID: <584C6939.9060805@omnilan.de> Date: Sat, 10 Dec 2016 21:44:41 +0100 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: BAR and pci passthrough Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Sat, 10 Dec 2016 21:44:43 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2016 20:44:45 -0000 In reply to https://lists.freebsd.org/pipermail/freebsd-virtualization/2015-December/004000.html >/>> The problem probably that the length of the bar is smaller and not />/>> aligned with the pagesize. Could the length of the BAR be modified in />/>> order to perform the pci passthrough? />/> />/> Yes, that's correct - the size of the BAR is not a multiple of the />/> page size which leads to the error. If this BAR is mapped into the />/> guest's address space then it will "leak" an additional 3K into the />/> guest (since the minimum nested mapping is 4KB in size). /> The proper fix is to modify the ppt(4) driver so that > it asks the PCI bus to allocate a full page for BARs that are smaller than a > page. Unfortunately the PCI bus driver doesn't currently provide a way to do > that. Even if it did it would not necessarily always work since the device > may be behind a PCI-PCI bridge whose windows can't grow due to existing > allocations of adjacent addresses. Alternatively the hypervisor could trap > all accesses to this page and only permit accesses to the range that contains > the BAR, but that would be quite slow. My stupid idea was to set in device.hints: hint.ppt.0.msize="0x1000" Since John (jhb@) already posted that the PCI bus driver deosn't provide the necessary capability to increase allocation, this clearly doesn't work. But since this hint was one year ago, I wanted to ask if there are news regarding PCI passthru (ppt's) page size handling or other workarrounds. My specific problem reads: sh -c 'while true ; do (bhyve -u -A -H -P -s 0,hostbridge -s 6,virtio-net,vmnet0 -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait -s 31,lpc -s 1,passthru,6/0/0 -l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd -S -m 4G -c 4 preed); vmm_exit=$?; if [ ${vmm_exit} -ne 0 ]; then break; fi; done ; /usr/sbin/bhyvectl --destroy --vm=preed' bhyve: passthru device 6/0/0 BAR 5: base 0xc3e10000 or size 0x200 not page aligned Please make sure im on CC, I'm not subscribed. Thanks, -Harry P.S.: Thanks to all byhve hackers, really awsome work! I'm evaluating ESXi reliever... Perhaps this obstacle is interesting for some people here (and one reason I need to passthru SATA controller) https://lists.freebsd.org/pipermail/freebsd-stable/2016-October/086157.html