From owner-freebsd-current@FreeBSD.ORG Tue Jan 12 02:49:48 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA1F2106566B for ; Tue, 12 Jan 2010 02:49:48 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-qy0-f203.google.com (mail-qy0-f203.google.com [209.85.221.203]) by mx1.freebsd.org (Postfix) with ESMTP id 58B7C8FC0C for ; Tue, 12 Jan 2010 02:49:48 +0000 (UTC) Received: by qyk41 with SMTP id 41so10710786qyk.29 for ; Mon, 11 Jan 2010 18:49:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=TVqIkAfaLqdkAxoGHVIdaXGEO6lCOSPrLX1ig7x0cj0=; b=HyrXkyLc7HLt8ia/Nw6Fv54xqpozpFFtj66ZLZltEn0qFfJIoQ7LSv8iX+JaNXsYr4 UQRpe5DkmB8b1UPt5cu1kxE0P7xwSRIP2ClREanK5+U3a+LAVMDQ76NB0X71/fEm8UJ2 RWgrYZ1vOApILH6WpYOyiwrbDdszZgGuhegFQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=Xm0dpdo2RyAn8i4fldd8xWhuPW1kC5IATCIXi3P2/us9TW8y5vMDdi675O1RRbqhbm +bb2chg+IxummGTWy1x7FVYaf7MBQDxo+pFhEGVxQCY2HpPgNki+RBBXMAzYpTWY2G4X eZrq4fi+HyZbG6G1NitOQugZu2Wt+EppW917o= Received: by 10.224.55.145 with SMTP id u17mr2589867qag.370.1263264582972; Mon, 11 Jan 2010 18:49:42 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 6sm19970726qwd.36.2010.01.11.18.49.41 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 11 Jan 2010 18:49:42 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 11 Jan 2010 18:49:00 -0800 From: Pyun YongHyeon Date: Mon, 11 Jan 2010 18:49:00 -0800 To: David Ehrmann Message-ID: <20100112024900.GG1228@michelle.cdnetworks.com> References: <4B40AFFA.6090706@gmail.com> <20100103221630.GV1166@michelle.cdnetworks.com> <4B47B4F6.8030106@gmail.com> <20100109013145.GG18529@michelle.cdnetworks.com> <4B4ACD68.5030907@gmail.com> <20100111203557.GB1228@michelle.cdnetworks.com> <4B4BB679.2060500@gmail.com> <20100112000859.GD1228@michelle.cdnetworks.com> <4B4BCEE9.1060600@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B4BCEE9.1060600@gmail.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-current@freebsd.org Subject: Re: vge traffic problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jan 2010 02:49:48 -0000 On Mon, Jan 11, 2010 at 05:22:49PM -0800, David Ehrmann wrote: > Pyun YongHyeon wrote: > >>I ran the test, again, and had tcpdump capture the packets (on the host > >>with the vge interface). > >> > >>tcpdump -i vge0 -w dump.cap -K -s 0 host 10.0.1.2 > >> > >>When I opened it up in Wireshark, it's reporting that the outgoing UDP > >>checksums are incorrect; they're always 0x1ae3. That said, maybe the > >> > > > >Because bpf(4) see the frame before controller inserts checksum > >value it always reports incorrect checksum. It's normal for TX > >checksum offload capable controller. > >In order to verify the hardware assisted checksum you should check > >that on receiver side. If the checksum was invalid, receiver > >dropped them. See "netstat -s | grep bad" on receiver host. > > > > > I did this one between two similar FreeBSD systems (8.0 and 8.0 RC1, I > think). There wasn't an error in netstat -s on the receiver, but I ran > tcpdump, anyway. The checksums were good, but iperf ended with "read > failed: connection refused." On the non-vge box, once the vge box is > done sending, it sends four identical UDP packets to the vge host, I > think to ask it for sending statistics. Instead of statistics, it got > back an ICMP packet saying "port unreachable." This happens with a > freebsd client, but not a linux client (I didn't confirm the ICMP > message with tcpdump, though). All iperf versions are the same (2.0.4), > but the linux one doesn't have any threading (which shouldn't be an > issue when I'm not using that option). It seems iperf on FreeBSD was broken. It incorrectly generates huge-packet with IP length 0 so other host disconnected the TCP connection. Not sure it could be related with threading though. Use netperf instead, it would be more reliable than iperf. > >>checksums are done in hardware AFTER tcpdump sees them. > >> > >>I set net.inet.udp.checksum to 0. The bad checksums are gone, but I > >>still see dropped packets. > >> > >> > > > >This doesn't disable TX checksum offload of driver. Instead use > >#ifconfig vge0 -txcsum > > > > > I'm still losing packets, but now it's interesting. The other FreeBSD > machine (a VMware VM on a fairly fast machine with another gigabit > connection) loses just a few, now, and sometimes none. The linux box > (200mhz, runs OpenWrt, 100mbps ethernet) still loses more than 1% of > packets. I cut iperf back to 100kbps. The results were much better (0 > packets lost), but this did happen one time: > It's normal see some dropped frames under high network load. And you can't compare gigabit controller to fast ethernet controller. > [ 3] WARNING: did not receive ack of last datagram after 10 tries. > > Maybe this is related to "port unreachable," and maybe to my nfs > problem. The drops seem to be related to iperf not being able to do > much processing at 200 mhz (which is a little surprising; I was only > asking for 1Mbps). > >>It's on the motherboard, probably wired directly to a PCI-E connection, > >>but yes, it is a VT6130. > >> > > > >Show me "pciconf -lcv" output. > > > vge0@pci0:3:0:0: class=0x020000 card=0x01101106 chip=0x31191106 > rev=0x82 hdr=0x00 > vendor = 'VIA Technologies Inc' > device = ''Velocity' Gigabit Ethernet Controllers > (VT6120/VT6121/VT6122)' > class = network > subclass = ethernet > cap 01[50] = powerspec 3 supports D0 D1 D2 D3 current D0 > cap 10[90] = PCI-Express 1 endpoint max data 128(128) link x1(x1) > cap 05[c0] = MSI supports 1 message, 64 bit, vector masks > > It might not be vge, after all, and might just be its speed...but that > shouldn't cause a tcp nfs issue. I have exact the same revision of the hardware and I don't have encountered your issue here. Instead of measuring performance number with broken iperf, check whether you still get "Connection reset by peer" message with csup(1) when you use vge(4) interface. If you still see the message, please send me tcpdump capture in private.