From owner-freebsd-net@FreeBSD.ORG Sat Apr 14 18:20:54 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2DCFE1065672 for ; Sat, 14 Apr 2012 18:20:54 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45]) by mx1.freebsd.org (Postfix) with ESMTP id 193E18FC08 for ; Sat, 14 Apr 2012 18:20:54 +0000 (UTC) Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1]) (authenticated bits=0) by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id q3EIKrNr026687 for ; Sat, 14 Apr 2012 11:20:53 -0700 (PDT) (envelope-from yuri@rawbw.com) Message-ID: <4F89C005.2020304@rawbw.com> Date: Sat, 14 Apr 2012 11:20:53 -0700 From: Yuri User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.3) Gecko/20120316 Thunderbird/10.0.3 MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Why host transmit rate on 1Gb ethernet is only ~750Mbps ? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Apr 2012 18:20:54 -0000 I am running some tests with gigabit switch between two 9.0 hosts using iperf. The best UDP transmit rate I am getting is 753 Mbits/sec with ~3-6% packet loss @1500 MTU @ 2.5GHz CPU, even though command 'iperf -c X.X.X.X -u -b 1000m' requests the full gigabit. I am trying to understand what exactly usually limits the speed with one way ethernet traffic and also what causes the packet loss. Is it the host's CPU or rather the need to make ~83K system calls in order to send 1Gb of UDP data at 1500 MTU? Is it the NIC or it's driver ('re' on source and 'em' on destination in my case)? Is it the router which is maybe too old? (Linksys EG008W, with tcpdump I made sure there is no ICMP traffic that is coming back from the router during the test). Why can't NIC send say 900 Mbps with MTU 1500 since the only overhead is ethernet+UDP headers which are ~120bytes out of 1500? Why packet loss occurs in such a simple network with just one switch and speeds of 75% of the maximum? How can I troubleshoot such situation and understand the reasons? Yuri