Date: Thu, 12 Jan 2012 20:42:49 +0200 From: Ivan Ivanyuk <ivan.ivanyuk@gmail.com> To: freebsd-xen@freebsd.org Subject: Re: [FreeBSD 8.2 amd64 XENHVM] Incorrect behavior of xen networking drivers: unexpected tcp reassembling Message-ID: <CAD_Den-Pd%2Bd7hNGFeuqmuFG6Ut7vWZZoUSNf2DB7A%2BjMFLBNcg@mail.gmail.com> In-Reply-To: <CAD_Den-o6rN4uarRrRpPrdzjX30kSJjrO%2BquTmZQ6cqokTrYFw@mail.gmail.com> References: <CAD_Den-o6rN4uarRrRpPrdzjX30kSJjrO%2BquTmZQ6cqokTrYFw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12 January 2012 00:35, Ivan Ivanyuk <ivan.ivanyuk@gmail.com> wrote: > Hi All, > I can get about 500Mb/s from virtual machine to external host, but > only 200Kb/s from any internal PC to the same external host through > virtual machine router. > > Closest description I found in archives is this: > http://lists.freebsd.org/pipermail/freebsd-xen/2011-May/000902.html. > > My setup is like this: > Dom0 is Debian with =A0xen-hypervisor-4.0-amd64 4.0.1-2. > DomU is FreeBSD 8.2-RELEASE amd64 XENHVM kernel with "device xenpci" opti= on > > > --------- =A0 =A0 =A0 --------------- =A0 =A0 =A0 -------------- =A0 =A0 = =A0 --------------- > |Internet|<--->|eth3 =A0 =A0vif14.0|<--->|xn0 =A0=A0 =A0 =A0 xn1|<--->|vi= f14.1 > eth2|<--->|some PC > ----------=A0 =A0 =A0| =A0 inetbr0 =A0 =A0 | =A0 =A0 | =A0 Freebsd =A0 = =A0| =A0=A0 =A0| =A0 localbr0 =A0 =A0| > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 Dom0 =A0 =A0=A0 | =A0 =A0 | =A0 = =A0 DomU =A0=A0 =A0| =A0 =A0 |=A0 =A0 =A0Dom0 =A0 =A0 =A0| > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0--------------- =A0 =A0 =A0 --------------= =A0 =A0 =A0 --------------- > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 bridge =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0bridge > > So I can see couple of packets with TCP data from Internets coming to > eth3, then the same packets are seen on vif14.0. And then on xn0 I see > only one packet with reassembled TCP payload. > While these big (2976 bytes, 4464 bytes, 8928 bytes, etc) packets are > addressed to DomU - all works. When we try to route them elsewhere - > we get ICMP need fragmentation message sent to origin of these > packets. That's because original (small) TCP packets have DF flag set. > And resulting big TCP packet has DF flag as well. > > So it seems to me that something in the chain "vif14.0<->xn0" is > reassembling TCP packets. > > Is there some sysctl or other settings to control this behavior? (I > tried turning off all offloads on vif14.0 in Dom0, tried to change > fragmentation settings in FreeBSD, nothing changed) Somewhat successful follow-up: There is still some occurrences of reordered packets but original issue was resolved. Turned out this mechanism can be turned off by "ifconfig -lro" in FreeBSD itself. I'm not sure if turning LRO on virtual driver by default is good idea at all, but obviously LRO implementation for xn device doesn't consider DF flag implication on resulting packets. I'm not familiar with other LRO algorithms which used in physical NIC's. Can someone comment if such problem is normal for it? And what is your opinion on using tso, rxcsum and txcsum offloads in virtual envirnoment? Regards, Ivan Ivanyuk
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAD_Den-Pd%2Bd7hNGFeuqmuFG6Ut7vWZZoUSNf2DB7A%2BjMFLBNcg>