Date: Wed, 27 Aug 2014 23:35:17 -0500 From: Bryan Venteicher <bryanv@daemoninthecloset.org> To: Brian Rak <brak@gameservers.com> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: Bug in FreeBSD VirtIO network driver (only with pf enabled) Message-ID: <CAMo0n6RkCB0aCx_G6ZMmJNXopfuAbyTw1=gnuUkTcpQxH3N%2BhQ@mail.gmail.com> In-Reply-To: <53FE2B37.7050309@gameservers.com> References: <53FE2B37.7050309@gameservers.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Aug 27, 2014 at 2:02 PM, Brian Rak <brak@gameservers.com> wrote: > I have a FreeBSD 10 x64 guest installed inside a KVM instance on Linux. > When pf is active, and the server is sending data it causes the Linux hos= t > to report warnings related to GSO. > > I've talked to some Linux developers, and they believe it to be a bug > inside the FreeBSD VirtIO drivers. Based on what I'm seeing, I'm incline= d > to agree with them. > > Reproduction is mildly annoying, you need: > 1) A KVM guest setup with bridged networking, with FreeBSD running side > using the VirtIO network interface. (We tested with CentOS, but the exac= t > distribution should not matter) > 2) pf enabled (I'm using a single rule: "scrub in all") > 3) Some method of sending a bit of traffic from the guest (I use netcat) > > So, when I do this: > > # service pf start > # cat /root/test | nc vultr.com 80 > > The Linux kernel on the host will report: > > kernel: WARNING: CPU: 7 PID: 7772 at net/core/dev.c:2246 > skb_warn_bad_offload+0xc3/0xd0() > kernel: igb: caps=3D(0x0000000640114bb3, 0x0000000000000000) len=3D1498 > data_len=3D0 gso_size=3D1380 gso_type=3D5 ip_summed=3D0 > > If I do: > > # service pf stop > # cat /root/test | nc vultr.com 80 > > No such warning is reported. I can only reproduce this with pf enabled. > The contents of the /root/test don't matter, I'm using 4k of data from > /dev/urandom. The test file just needs to be bigger then the MTU of the > host's network interface. > > I was able to track this down to the virtio_net_hdr being sent by the > FreeBSD guest. With pf enabled, this outbound traffic has the following > header: > > flags =3D 0 > gso_type =3D VIRTIO_NET_HDR_GSO_TCPV4 > hdr_len =3D 66 > gso_size =3D 1440 > csum_start =3D 0 > csum_offset =3D 0 > > But, this is not a valid configuration. With VIRTIO_NET_HDR_GSO_TCPV4 > enabled, you should also be setting VIRTIO_NET_HDR_F_NEEDS_CSUM and > populating the csum_start and csum_offset fields. > > =E2=80=8BIt would appear that somewhere in pf the CSUM_TCP flag is getting = clear for CSUM_TSO packets. I don't believe the stack otherwise sets CSUM_TSO without CSUM_TCP. Now, it is probably safe to assume CSUM_TCP when CSUM_TSO is set: the mbuf's m_pkthdr.csum_data better be valid in such cases though. > http://www.spinics.net/lists/netdev/msg293976.html gives more detail on > this. I don't fully understand this, so I'd probably mangle the > explanation if I tried to give more detail. I can reproduce this at will > now, but fixing it is beyond my abilities. Is there a better place to > report this? I'm not entirely sure who is responsible for maintaining th= e > virtio driver. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMo0n6RkCB0aCx_G6ZMmJNXopfuAbyTw1=gnuUkTcpQxH3N%2BhQ>