From owner-freebsd-virtualization@freebsd.org Tue Nov 15 13:04:47 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7A73C43E4F for ; Tue, 15 Nov 2016 13:04:47 +0000 (UTC) (envelope-from freebsd@jonathanprice.org) Received: from wright.jonathanprice.org (unknown [IPv6:2001:41d0:2:eb0d::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 419EFB15 for ; Tue, 15 Nov 2016 13:04:47 +0000 (UTC) (envelope-from freebsd@jonathanprice.org) Received: from mail.jonathanprice.org (unknown [IPv6:2001:41d0:2:eb0d::1]) by wright.jonathanprice.org (Postfix) with ESMTPSA id 8952625181; Tue, 15 Nov 2016 13:04:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=jonathanprice.org; s=default; t=1479215084; bh=qcnD73IB37VO1k8RMRFLrt7mzqkj/0ugz1kXgyfGvjc=; h=Date:From:Subject:To:Cc:In-Reply-To:References; b=jqicvlWb+hO32siywYiuwyLvjVe2MI9tU6is1L5ZjwCCt+IyGe5l/+xfEFLZ78rbk mTDxQnk5Utz0BrWwU8IJvBxltB8IM8e/UG68uZ78CRwkPrz5shCmmJ7RvJH1IG71wt HeO41GPHG098JgEU0Zt7zTtjj1NLJT9eqM7STW9U= Mime-Version: 1.0 Date: Tue, 15 Nov 2016 13:04:44 +0000 Message-ID: <2c71a548612d4e976abe568990f9711a@mail.jonathanprice.org> X-Mailer: RainLoop/1.10.4.183 From: freebsd@jonathanprice.org Subject: Re: bhyve windows guest unable to ping default gateway or external networks To: "Rajil Saraswat" Cc: freebsd-virtualization@freebsd.org In-Reply-To: References: <4cf1a528d9a4e85d5aa351ca2ff417a9@mail.jonathanprice.org> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Nov 2016 13:04:47 -0000 Hello, and thanks for the suggestion.=0A=0AI ran the following:=0Asudo if= config em0 -rxcsum -txcsum -vlanmtu -vlanhwcsum -vlanhwtso -tso4=0A=0Awhi= ch resulted in ifconfig showing:=0Aem0: flags=3D8843 metric 0 mtu 1500=0A= options=3D2098=0A=0AIt looks like it removed rxcsum, txcsum, vlanhwtso a= nd tso, but vlanhwcsum and vlan_mtu were not removed.=0A=0AAt this pointe= d I started up the VM and performed the same tests, but unfortunately the= results were still the same.=0A=0AI then directly applied your config to= my ifconfig_em0 line, resulting in this rc.conf entry:=0Aifconfig_em0=3D= "inet 46.105.xxx.xxx netmask 0xffffff00 -rxcsum -txcsum -vlanmtu -vlanhwt= so -tso4 -tso6 -vlanhwtso -vlanhwcsum"=0A=0AUpon rebooting my ifconfig ou= tput was still the same as before the reboot. Just to be sure it didn't f= ix it, I booted up the VM again and tried to ping the default gateway, an= d it failed.=0A=0ANovember 15, 2016 12:07 PM, "Rajil Saraswat" wrote:=0A= On Nov 15, 2016 5:58 AM, wrote:=0A>=0A> Hello,=0A>=0A> I am trying to g= et a windows server VM running under bhyve. So far I can use VNC to insta= ll Windows, but I've been unable to get the networking working. I will pr= ovide details below, and include a summary up here. On the guest, I have = installed the latest version of winvirtio, and I have configured static I= P addressing. The guest can ping itself, but not the host machine or anyt= hing external. It is worth noting that I am also currently running a Cent= OS guest, and this can connect to both the host and the outside world wit= h no issues.=0A>=0A> Host: FreeBSD 11.0-RELEASE=0A> Guest: Tried both Win= dows Server 2016 Essentials and Windows Server 2012 Standard=0A>=0A> bhyv= e command:=0A> bhyve -c 1 -H -w -m 3G -s 0,hostbridge -s 3,ahci-cd,/iohyv= e/ISO// -s '4,ahci-hd,/dev/zvol/zroot/iohyve/win12/disk0,sectorsize=3D512= ' -s 10,virtio-net,tap2 -s '11,fbuf,tcp=3D0.0.0.0:5902 (http://0.0.0.0:59= 02),w=3D800,h=3D600,wait' -s 12,xhci,tablet -s 31,lpc -l com1,/dev/nmdm3A= -l bootrom,/iohyve/Firmware/BHYVE_UEFI.fd/BHYVE_UEFI.fd ioh-win12=0A>=0A= > ifconfig output:=0A> em0: flags=3D8843 metric 0 mtu 1500=0A> options=3D= 4219b=0A> ether e0:69:95:a2:f1:d9=0A> inet 46.105.xxx.xxx netmask 0xfffff= f00 broadcast 46.105.124.255=0A> inet6 fe80::e269:95ff:xxxx:f1d9%em0 pref= ixlen 64 scopeid 0x1=0A> inet6 2001:41d0:2:xxxx::1 prefixlen 56=0A> nd6 o= ptions=3D21=0A> media: Ethernet autoselect (100baseTX )=0A> status: activ= e=0A> lo0: flags=3D8049 metric 0 mtu 16384=0A> options=3D600003=0A> inet6= ::1 prefixlen 128=0A> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2=0A> ine= t 127.0.0.1 netmask 0xff000000=0A> nd6 options=3D21=0A> groups: lo=0A> br= idge0: flags=3D8843 metric 0 mtu 1500=0A> ether 02:cf:0c:8f:13:00=0A> ine= t6 fd00::ffff prefixlen 112=0A> inet 10.0.0.254 netmask 0xffffff00 broadc= ast 10.0.0.255=0A> nd6 options=3D1=0A> groups: bridge=0A> id 00:00:00:00:= 00:00 priority 32768 hellotime 2 fwddelay 15=0A> maxage 20 holdcnt 6 prot= o rstp maxaddr 2000 timeout 1200=0A> root id 00:00:00:00:00:00 priority 3= 2768 ifcost 0 port 0=0A> member: tap2 flags=3D143=0A> ifmaxaddr 0 port 7 = priority 128 path cost 2000000=0A> member: tap1 flags=3D143=0A> ifmaxaddr= 0 port 5 priority 128 path cost 2000000=0A> member: tap0 flags=3D143=0A>= ifmaxaddr 0 port 4 priority 128 path cost 2000000=0A> pflog0: flags=3D0 = metric 0 mtu 33160=0A> groups: pflog=0A> tap0: flags=3D8943 metric 0 mtu = 1500=0A> description: iohyve-centos=0A> options=3D80000=0A> ether 00:bd:5= d:d3:e8:00=0A> nd6 options=3D21=0A> media: Ethernet autoselect=0A> status= : active=0A> groups: tap=0A> Opened by PID 96206=0A> tap1: flags=3D8902 m= etric 0 mtu 1500=0A> description: iohyve-win16es=0A> options=3D80000=0A> = ether 00:bd:61:ab:eb:01=0A> nd6 options=3D21=0A> media: Ethernet autosele= ct=0A> status: no carrier=0A> groups: tap=0A> tap2: flags=3D8943 metric 0= mtu 1500=0A> description: iohyve-win12=0A> options=3D80000=0A> ether 00:= bd:c9:1a:59:02=0A> nd6 options=3D21=0A> media: Ethernet autoselect=0A> st= atus: active=0A> groups: tap=0A> Opened by PID 1911=0A>=0A> It is using N= AT with PF to reach the internet. It should not rely on NAT to be able to= ping the host machine, but I will post the NAT configuration below anywa= y:=0A>=0A> /etc/pf.conf:=0A> # Automatically NAT any VMs=0A> nat on em0 i= net from bridge0:network to any -> (em0)=0A>=0A>=0A> (On the Guest)=0A> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A>=0A> C:UsersAdministrator>i= pconfig=0A>=0A> Ethernet adapter Ethernet:=0A>=0A> IPv4 Address: 10.0.0.3= =0A> Subnet Mask: 255.255.255.0=0A> Default Gateway: 10.0.0.254=0A>=0A> C= :UsersAdministrator>ping 10.0.0.3=0A>=0A> Pinging 10.0.0.3 with 32 bytes = of data:=0A> Reply from 10.0.0.3 (http://10.0.0.3): bytes=3D32 time Reply= from 10.0.0.3 (http://10.0.0.3): bytes=3D32 time Reply from 10.0.0.3 (ht= tp://10.0.0.3): bytes=3D32 time Reply from 10.0.0.3 (http://10.0.0.3): by= tes=3D32 time=0A> C:UsersAdministrator>ping 10.0.0.254=0A>=0A> Pinging 10= .0.0.254 with 32 bytes of data:=0A> Reply from 10.0.0.3 (http://10.0.0.3)= : Destination host unreachable.=0A> Reply from 10.0.0.3 (http://10.0.0.3)= : Destination host unreachable.=0A> Reply from 10.0.0.3 (http://10.0.0.3)= : Destination host unreachable.=0A> Reply from 10.0.0.3 (http://10.0.0.3)= : Destination host unreachable.=0A>=0A> C:UsersAdministrator>arp -a=0A>= =0A> Interface: 10.0.0.3 --- 0xc=0A> Internet Address Physical Address Ty= pe=0A> 10.0.0.255 ff-ff-ff-ff-ff-ff static=0A> 224.0.0.22 01-00-5e-00-00-= 16 static=0A> 224.0.0.252 01-00-5e-00-00-fc static=0A>=0A> (PF log testin= g)=0A> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A>=0A> I was cur= ious to see if PF was seeing packets on the bridge interface, so I added = the following to /etc/pf.conf:=0A> pass log (all) all=0A>=0A> I then ran = the following command, which had no results when I tried a test ping:=0A>= sudo tcpdump -n -e -ttt -i pflog0 host 10.0.0.3=0A>=0A> (I also watched = live traffic on 10.0.0.254 and saw nothing from this IP)=0A>=0A>=0A> Summ= ary=0A> =3D=3D=3D=3D=3D=3D=3D=3D=0A>=0A> I appear to be having some netwo= rk connectivity issues that either lies in bhyve, virtio-net or the winvi= rtio drivers. If anybody has any suggestions for things I could try I'd l= ove to hear it.=0A>=0A> Thanks,=0A>=0A> Jonathan.=0A> ___________________= ____________________________=0A> freebsd-virtualization@freebsd.org (mail= to:freebsd-virtualization@freebsd.org) mailing list=0A> https://lists.fre= ebsd.org/mailman/listinfo/freebsd-virtualization (https://lists.freebsd.o= rg/mailman/listinfo/freebsd-virtualization)=0A> To unsubscribe, send any = mail to "freebsd-virtualization-unsubscribe@freebsd.org (mailto:freebsd-v= irtualization-unsubscribe@freebsd.org)" =0A=0A I had to disable offloadin= g on the host NIC to get the network working inside bhyve session. This s= eems to be a FreeBSD bug in virtio stack. =0A=0A ifconfig_igb0=3D"-rxcsum= -txcsum -vlanmtu -vlanhwtso -tso4 -tso6 -vlanhwtso -vlanhwcsum up" =0A= =0A -Rajil