Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 20 Apr 2020 15:00:51 -0500
From:      Tim Daneliuk <tundra@tundraware.com>
To:        FreeBSD Mailing List <freebsd-questions@freebsd.org>
Subject:   More On FreeBSD Network Speed (Was: Changes To nat-ing Behaviour?)
Message-ID:  <a0ae910d-a1ea-a2bd-a180-3efb2c274d56@tundraware.com>

Next in thread | Raw E-Mail | Index | Archive | Help
I have a FreeBSD 11-STABLE firewall/router/NATing host that has an re0 NIC
pointing at the internet and an older PCI Intel em0 device supporting a
nonroutable IP space on an internal LAN.

I recently discovered that my throughput from the LAN to the internet
suddenly lost 50% of the available bandwidth.  Going directly from the FreeBSD
firewall host to internet was fine.

After some hours of fiddling with cables and switches, everything pointed to
the old PCI em0 card in the firewall box.  I had another one here, but it too
showed half speed. Both of these cards have a LOT of hours on them, so we
got newer Intel PCIe em0 card and, voila', problem fixed.  (So, either these
cards aged to point where they cannot run at full speed (which I have only
very rarely ever seen before), or them most recent em0 driver is misconfigured
for these older cards somehow.)

But there continues to be a performance puzzle I cannot quite figure out.
For our purposes, there are three machines on the LAN: The FreeBSD NATing
firewall, a Mac laptop, and a Linux workstation.

Using iPerf3, I tested network throughput pairwise between these machines.
I got some very strange results (results are the same regardless of which
machine served as client or server):

MacOS <-> Linux    ~933 Mbits/sec
MacOS <-> FreeBSD  ~933 Mbits/sec
FreeBSD <-> Linux  ~533 Mbits/sec

Linux and FreeBSD have superframe MTUs of 9000 set MacOS MTU is set to 1500.

The FreeBSD machine does also serve Samba and NFS mounts into the LAN but disabling
this had no effect on the results.  Neither did setting the Linux MTU to 1500.

In short, the network switches and wiring can sustain nearly the full 1G speed
as expected, but - when FreeBSD and Linux try talk - the speed is seriously
compromised.  Clearly, FreeBSD can hit that speed since it did so with the Mac.

I am scratching my head on this one.  Any ideas from the Geniuses Present
would be appreciated.





----------------------------------------------------------------------------
Tim Daneliuk     tundra@tundraware.com
PGP Key:         http://www.tundraware.com/PGP/




Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?a0ae910d-a1ea-a2bd-a180-3efb2c274d56>