From owner-freebsd-current@FreeBSD.ORG Tue Jan 12 17:29:44 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6027A1065672 for ; Tue, 12 Jan 2010 17:29:44 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-qy0-f174.google.com (mail-qy0-f174.google.com [209.85.221.174]) by mx1.freebsd.org (Postfix) with ESMTP id 0D4A28FC15 for ; Tue, 12 Jan 2010 17:29:43 +0000 (UTC) Received: by qyk4 with SMTP id 4so10207636qyk.7 for ; Tue, 12 Jan 2010 09:29:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=TIOahzTBxsc4m+L7eMqAsZu+oN0WZswbmkLNNRwm9DA=; b=IoJcepTApk/tI/MK3nv8V9zwd06FcN3WG1On60YbrmpCTSqwEhISQLSOXYi7Pb7dhn NENsIbl/XfUvZNAHLkEdvUliBHlrZt2Ln+NbqT6jTNLm7oNfes/IIzXILB+QG90vrFlz JxtQD1lpuDRn3chKiuGK+gRbw4sccAaYtVqeI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=Zr3YEynmtljtQluiqyWcZ44gB6fkW5+rfkpqbwVDr/cQwv9+fFxH4Rv6h8WJ2OAC5C H1f6Vdr/wiTTVbj6mhIFnJXhoiO1azaIXW2SuHyll1RopFjKcQU9Of1JtQxge5QnRLjs 2dYcijUaV5xvbcyHH//ksqV0j2TFf2NFyC73M= Received: by 10.224.24.204 with SMTP id w12mr17676327qab.366.1263317377457; Tue, 12 Jan 2010 09:29:37 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 21sm864358qyk.4.2010.01.12.09.29.34 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 12 Jan 2010 09:29:35 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 12 Jan 2010 09:28:54 -0800 From: Pyun YongHyeon Date: Tue, 12 Jan 2010 09:28:54 -0800 To: David Ehrmann Message-ID: <20100112172854.GI1228@michelle.cdnetworks.com> References: <20100103221630.GV1166@michelle.cdnetworks.com> <4B47B4F6.8030106@gmail.com> <20100109013145.GG18529@michelle.cdnetworks.com> <4B4ACD68.5030907@gmail.com> <20100111203557.GB1228@michelle.cdnetworks.com> <4B4BB679.2060500@gmail.com> <20100112000859.GD1228@michelle.cdnetworks.com> <4B4BCEE9.1060600@gmail.com> <20100112024900.GG1228@michelle.cdnetworks.com> <4B4BFAA7.8030103@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B4BFAA7.8030103@gmail.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-current@freebsd.org Subject: Re: vge traffic problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jan 2010 17:29:44 -0000 On Mon, Jan 11, 2010 at 08:29:27PM -0800, David Ehrmann wrote: > Pyun YongHyeon wrote: > >It seems iperf on FreeBSD was broken. It incorrectly generates > >huge-packet with IP length 0 so other host disconnected the > >TCP connection. Not sure it could be related with threading though. > >Use netperf instead, it would be more reliable than iperf. > > > I saw a lot of warnings when I opened the cap file in Wireshark about > the length in the IP header being wrong. I'll start looking into netperf > Yeah, there must be a bug in iperf thread handling. Maybe default configuration of iperf should be changed until iperf bug is fixed. > >It's normal see some dropped frames under high network load. And > >you can't compare gigabit controller to fast ethernet controller. > > > Very true, and that's why I tried a lower load. I was a little > surprised to see it choking at just 1 Mb/s (that's bits, not bytes), though. Even though vge(4) is not one of the best controller you can still get more than 650Mbps TX, 920Mbps RX for bulk TCP transfers. For smaller sized TCP segments the number would be much lower than that but that's normal for virtually all controllers. I have a local patch which pushes TX performance numbers up to 800Mbps for vge(4) but it requires fast CPU to do that so I'm not sure whether I want to put it to tree or not. Since I didn't ever see this low TX performance numbers on PCIe based controllers there could be incorrectly programmed registers but datasheet said nothing about this issue. > >I have exact the same revision of the hardware and I don't have > >encountered your issue here. Instead of measuring performance > >number with broken iperf, check whether you still get > >"Connection reset by peer" message with csup(1) when you use vge(4) > >interface. If you still see the message, please send me tcpdump > >capture in private. > > > csup is still working. > > I actually think I *might* have the problem solved. Switching the mount > from UDP (why it was the default for NFS in this Linux distro, I don't > know) to TCP seems to have fixed it. My guess is that some sort of race > condition occurred or there's a bug in someone's NFS flow control > mechanism. A 10x CPU and network performance difference must be more > than is usually tested. I hope. > This could be related with NFS. If NFS over TCP works without problems on vge(4) it's more likely NFS issue. > I'll keep testing NFS over TCP and see if it fixed my problem.