Date: Wed, 03 Dec 2014 15:59:14 +0000 From: Steven Hartland <killing@multiplay.co.uk> To: freebsd-net@freebsd.org Subject: Re: Horrendous upload network performance with VLAN (download seems OK) Message-ID: <547F3352.3040101@multiplay.co.uk> In-Reply-To: <727AB395-CF83-4D13-A2C3-50969C2969B0@patpro.net> References: <727AB395-CF83-4D13-A2C3-50969C2969B0@patpro.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Have you tried disabling all the various flags VLAN flags on the NIC? On 03/12/2014 15:46, Patrick Proniewski wrote: > Hello, > > I'm running FreeBSD 9.3-RELEASE up-to-date, on two HP Proliant G6 server blades in the same enclosure. One with VLANs in the uplink, the other without VLANs. They use bxe driver. > > bxe0: <QLogic NetXtreme II BCM57711E 10GbE (A0) BXE v:1.78.78 > > mem 0xfb000000-0xfb7fffff,0xfa800000-0xfaffffff irq 28 at device 0.0 on pci2 > bxe0: PCI BAR0 [10] memory allocated: 0xfb000000-0xfb7fffff (8388608) -> 0xfffffe00fb000000 > bxe0: PCI BAR2 [18] memory allocated: 0xfa800000-0xfaffffff (8388608) -> 0xfffffe00fa800000 > bxe0: Found 10GBase-CX4 media. > bxe0: Ethernet address: 00:17:a4:77:04:10 > bxe0: MSI-X vectors Requested 5 and Allocated 5 > ../.. > (and so on up to bxe7) > > Blade A is configured to access the network through a connection without VLANs (the link provided by Network team comes with no VLAN tagging). Its transfer rate is perfect, both up and down. > > ifconfig_bxe0="inet x.y.z.141/24" > defaultrouter="x.y.z.1" > > # ifconfig bxe0 > bxe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO> > ether 00:17:a4:77:04:00 > inet x.y.z.141 netmask 0xffffff00 broadcast x.y.z.255 > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect (10Gbase-CX4 <full-duplex>) > status: active > > > Blade B is configured to access the network through a link sporting multiple VLANs, so I've created a network interface that uses one of these VLANs. Ping is OK, I can ssh to this server, transfer rate to the server (down) is not fantastic but OK, enough to perform pkg installation or FreeBSD update. Transfer rate from the server to the rest of the world is abysmal, often stalling after few 100's KB. > > ifconfig_bxe0="UP" > vlans_bxe0="161" > ifconfig_bxe0_161="inet x.y.z.142/24" > defaultrouter="x.y.z.1" > > # ifconfig bxe0 > bxe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO> > ether 00:17:a4:77:04:10 > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect (10Gbase-CX4 <full-duplex>) > status: active > > # ifconfig bxe0.161 > bxe0.161: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=303<RXCSUM,TXCSUM,TSO4,TSO6> > ether 00:17:a4:77:04:10 > inet x.y.z.142 netmask 0xffffff00 broadcast x.y.z.255 > inet6 fe80::217:a4ff:fe77:410%bxe0.161 prefixlen 64 scopeid 0x10 > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect (10Gbase-CX4 <full-duplex>) > status: active > vlan: 161 parent interface: bxe0 > > > The same switch is used by those two blade servers running FreeBSD, and by about 14 other blade servers running VMware ESXi 5.x. > ESXi blades use multiple VLANs and work perfectly. A blade running FreeBSD with no VLAN in its uplink (i.e. not sharing the same uplink as ESXi blades) has very good network performances. The only blade with problem is the one running FreeBSD and sharing the uplink of ESXi blades. > > scp of a 347MB file from an ESXi blade server to FreeBSD blade (same switch, same blade chassis, same uplink): > > # scp esxi11.domain.tld:/tardisks/FILE /dev/null > Password: > FILE 100% 347MB 38.6MB/s 00:09 > > scp of a 347MB file from an FreeBSD blade to ESXi blade (same switch, same blade chassis, same uplink): > > # scp FILE esxi11.domain.tld:/dev/null > Password: > FILE 0% 400KB 172.2KB/s - stalled - > > traceroute from FreeBSD blade to ESXi blade: > > # traceroute esxi11.domain.tld > traceroute to esxi11.domain.tld (x.y.z.151), 64 hops max, 52 byte packets > 1 * * * > 2 * * * > ^C > > traceroute from ESXi blade to FreeBSD blade: > > # traceroute freebsdB.domain.tld > traceroute to freebsdB.domain.tld (x.y.z.142), 30 hops max, 40 byte packets > 1 freebsdB (x.y.z.142) 0.123 ms 0.090 ms 0.078 ms > > I'm quite lost here. > Any idea that would explain the problem, and help me resolve it? > > Original thread posted on <https://forums.freebsd.org/threads/horrendous-network-performance-with-vlan.49211/>; > > Patrick > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?547F3352.3040101>