From owner-freebsd-questions@freebsd.org Mon Apr 20 20:01:12 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B884B2ADEE2 for ; Mon, 20 Apr 2020 20:01:12 +0000 (UTC) (envelope-from tundra@tundraware.com) Received: from oceanview.tundraware.com (oceanview.tundraware.com [45.55.60.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailman.tundraware.com", Issuer "mailman.tundraware.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 495cyb4LNTz41Lf for ; Mon, 20 Apr 2020 20:01:11 +0000 (UTC) (envelope-from tundra@tundraware.com) Received: from [192.168.0.2] (ozzie.tundraware.com [75.145.138.73]) (authenticated bits=0) by oceanview.tundraware.com (8.15.2/8.15.2) with ESMTPSA id 03KK0uo8038998 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 20 Apr 2020 15:00:57 -0500 (CDT) (envelope-from tundra@tundraware.com) To: FreeBSD Mailing List From: Tim Daneliuk Subject: More On FreeBSD Network Speed (Was: Changes To nat-ing Behaviour?) Message-ID: Date: Mon, 20 Apr 2020 15:00:51 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (oceanview.tundraware.com [45.55.60.57]); Mon, 20 Apr 2020 15:00:57 -0500 (CDT) X-TundraWare-MailScanner-Information: Please contact the ISP for more information X-TundraWare-MailScanner-ID: 03KK0uo8038998 X-TundraWare-MailScanner: Found to be clean X-TundraWare-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached, score=-2.077, required 1, autolearn=not spam, ALL_TRUSTED -1.00, AWL 0.82, BAYES_00 -1.90) X-TundraWare-MailScanner-From: tundra@tundraware.com X-Spam-Status: No X-Rspamd-Queue-Id: 495cyb4LNTz41Lf X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of tundra@tundraware.com designates 45.55.60.57 as permitted sender) smtp.mailfrom=tundra@tundraware.com X-Spamd-Result: default: False [-1.35 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; DMARC_NA(0.00)[tundraware.com]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.92)[-0.920,0]; IP_SCORE(0.56)[ip: (-1.32), ipnet: 45.55.32.0/19(2.92), asn: 14061(1.25), country: US(-0.05)]; TO_DN_ALL(0.00)[]; NEURAL_HAM_MEDIUM(-0.69)[-0.691,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14061, ipnet:45.55.32.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Apr 2020 20:01:12 -0000 I have a FreeBSD 11-STABLE firewall/router/NATing host that has an re0 NIC pointing at the internet and an older PCI Intel em0 device supporting a nonroutable IP space on an internal LAN. I recently discovered that my throughput from the LAN to the internet suddenly lost 50% of the available bandwidth. Going directly from the FreeBSD firewall host to internet was fine. After some hours of fiddling with cables and switches, everything pointed to the old PCI em0 card in the firewall box. I had another one here, but it too showed half speed. Both of these cards have a LOT of hours on them, so we got newer Intel PCIe em0 card and, voila', problem fixed. (So, either these cards aged to point where they cannot run at full speed (which I have only very rarely ever seen before), or them most recent em0 driver is misconfigured for these older cards somehow.) But there continues to be a performance puzzle I cannot quite figure out. For our purposes, there are three machines on the LAN: The FreeBSD NATing firewall, a Mac laptop, and a Linux workstation. Using iPerf3, I tested network throughput pairwise between these machines. I got some very strange results (results are the same regardless of which machine served as client or server): MacOS <-> Linux ~933 Mbits/sec MacOS <-> FreeBSD ~933 Mbits/sec FreeBSD <-> Linux ~533 Mbits/sec Linux and FreeBSD have superframe MTUs of 9000 set MacOS MTU is set to 1500. The FreeBSD machine does also serve Samba and NFS mounts into the LAN but disabling this had no effect on the results. Neither did setting the Linux MTU to 1500. In short, the network switches and wiring can sustain nearly the full 1G speed as expected, but - when FreeBSD and Linux try talk - the speed is seriously compromised. Clearly, FreeBSD can hit that speed since it did so with the Mac. I am scratching my head on this one. Any ideas from the Geniuses Present would be appreciated. ---------------------------------------------------------------------------- Tim Daneliuk tundra@tundraware.com PGP Key: http://www.tundraware.com/PGP/