Date: Sat, 07 Mar 2015 11:22:24 -0500 From: Allan Jude <allanjude@freebsd.org> To: freebsd-virtualization@freebsd.org Subject: Re: bhyve virtio-net MTU Message-ID: <54FB25C0.7060803@freebsd.org> In-Reply-To: <1425732590516.79490@tum.de> References: <1425732590516.79490@tum.de>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ILslUU8kIojoROHhbjhtiL55rL4UfWAtU Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-03-07 07:49, Noah Bergbauer wrote: > Hi, >=20 > I'm running FreeBSD 10.1 on a dedicated server, with a Linux VM in bhyv= e. The handbook tells you to bridge the tap interface to a real network i= nterface, but that's not an option for me because I only have one IPv4 ad= dress. So instead, I assigned an internal IP address to bridge0 and used = pf(4) to set up NAT routing. > All of this works without any issues, but I would like to increase the = MTU of 1500. It's a virtual interface after all, why should it be so low?= FreeBSD's loopback interface's MTU is 16384 and on Linux, it's even 6553= 6. >=20 > So I used ifconfig(8) to increase the MTU of tap0 and just like the man= page says, bridge0 had the same MTU after I added tap0. On the Linux side= , I did the same with eth0 and then I sent 2000 byte pings to the host ma= chine. >=20 > It seems to work, but let's use tcpdump(8) to make sure: >=20 > 02:02:46.244678 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 6, seq 3, length 2008 > 02:02:46.244983 IP 10.42.42.1 > 10.42.42.100: ICMP echo reply, id 1686,= seq 3, length 1480 > 02:02:46.245061 IP 10.42.42.1 > 10.42.42.100: ip-proto-1 > 02:02:47.244953 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 6, seq 4, length 2008 > 02:02:47.245347 IP 10.42.42.1 > 10.42.42.100: ICMP echo reply, id 1686,= seq 4, length 1480 > 02:02:47.245422 IP 10.42.42.1 > 10.42.42.100: ip-proto-1 >=20 > The entire request goes through the virtio NIC, tap0, bridge0 and final= ly to the host's kernel without any issues. Which then sends a _fragmente= d_ reply because apparently it still thinks the MTU is 1500. >=20 > A quick check with route(8) confirms this: >=20 > # route show 10.42.42.100 > route to: 10.42.42.100 > destination: 10.42.42.0 > mask: 255.255.255.0 > fib: 0 > interface: bridge0 > flags: <UP,DONE,STATIC,PINNED> > recvpipe sendpipe ssthresh rtt,msec mtu weight expire > 0 0 0 0 1500 1 0=20 >=20 > So I manually forced a bigger MTU: >=20 > # route change -net 10.42.42.0 -mtu 15000 > change net 10.42.42.0 >=20 > But now the reply packets get truncated instead of fragmented: >=20 > 02:07:36.921165 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 3, length 2008 > 02:07:36.921624 IP truncated-ip - 6 bytes missing! 10.42.42.1 > 10.42.4= 2.100: ICMP echo reply, id 1689, seq 3, length 2008 > 02:07:37.921042 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 4, length 2008 > 02:07:37.921499 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 4, length 2008 > 02:07:38.921522 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 5, length 2008 > 02:07:38.922253 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 5, length 2008 > 02:07:39.921432 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 6, length 2008 > 02:07:39.922165 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 6, length 2008 > 02:07:40.921513 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 7, length 2008 > 02:07:40.922245 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 7, length 2008 > 02:07:41.921393 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 8, length 2008 > 02:07:41.922160 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 8, length 2008 > 02:07:42.921504 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 9, length 2008 > 02:07:42.922348 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 9, length 2008 > 02:07:43.923031 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 10, length 2008 > 02:07:43.924904 IP truncated-ip - 6 bytes missing! 10.42.42.1 > 10.42.4= 2.100: ICMP echo reply, id 1689, seq 10, length 2008 > 02:07:44.926832 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 11, length 2008 > 02:07:44.928511 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 11, length 2008 > 02:07:45.936968 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 12, length 2008 > 02:07:45.937722 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 12, length 2008 > 02:07:46.937453 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 168= 9, seq 13, length 2008 > 02:07:46.938161 IP truncated-ip - 518 bytes missing! 10.42.42.1 > 10.42= =2E42.100: ICMP echo reply, id 1689, seq 13, length 2008 > ^C > 22 packets captured > 24 packets received by filter > 0 packets dropped by kernel >=20 > This last dump was on the Linux side. Dumping at tap0 shows that the re= ply packets are still okay when they reach bhyve. Apparently they get tru= ncated by bhyve's virtio-net as increasing the MTU works just fine with V= irtualBox's virtio-net (at least on my Linux machine). >=20 > Any ideas on how I can fix this? I had a quick look at the code and whi= le a comment indicates that Ethernet-sized packets are assumed (https://s= vnweb.freebsd.org/base/release/10.1.0/usr.sbin/bhyve/pci_virtio_net.c?rev= ision=3D274417&view=3Dmarkup#l257), I was unable to find code that confir= ms this. >=20 > Noah > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" >=20 There is a 'do not fragment' flag you can sent with ping to help debug th= is. --=20 Allan Jude --ILslUU8kIojoROHhbjhtiL55rL4UfWAtU Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJU+yXDAAoJEJrBFpNRJZKf/h0QAKaPsajrY+rRPV48CXU+CSqP ktIxTACtXFKfYCxqEHRyspLCBC3qVOJ05G5VkW1cl0wOuGD1p28FtEOtb9xjypeS 8/boChdQuU9H1pkjWdVtkc71CbMKJ0KNvbuU5C3fw8sn73u6TS7D9wQ8Nd0dI2Yp XWRNWRa9P+uQaJ0hy2lyToPGgvbjIQYvZnfFOdn3M2BMsz11DZQ2758U1+zvazJ4 rkKCBNInCUZzb+tWBaXOeAi8qaBPRAaqV9FUvuo/pf14QhmzAtJPxXfe3hTZInoX +f77M4dY5Q1dXtONcDwXpauinTZhso77vN6FwOweG/cV6AU39uEZVELRUehLBTgT 1rKU5robneZh8vE37jWF/mE2faTAv2VqmTt7Zcvvg7fbhjKgJjjVKYAWVPvGOREZ WiiWP7mHGoqlsKY3G0cZi0yU7ZUY5GpcqCrerxkxLC4+Z3Y6zvG21+B/28fdTHYt gjptkA337TfxwzqGVAKeABxcYLevfWEM0omUTV/ZnrAbDxJu9Te2iGHn4NRhIAgP WoEgOJj2tII9P4f57ZbVRjSe0nPiS2NV7UBFOzLQZjGcfXQ/9Kwd7KAYh354Xj0h yLxQSUU45l8Uhtyers9/a0rBxLDz97tOEm94vTT2Nn9Vf0raWYXh7IkHq6Wrl2jI L+Sh8BEF8rUfo4PocJTW =l9n5 -----END PGP SIGNATURE----- --ILslUU8kIojoROHhbjhtiL55rL4UfWAtU--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54FB25C0.7060803>