From owner-freebsd-current@FreeBSD.ORG Tue Jun 16 10:14:03 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 756DE106566B for ; Tue, 16 Jun 2009 10:14:03 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.177]) by mx1.freebsd.org (Postfix) with ESMTP id 42D3D8FC1E for ; Tue, 16 Jun 2009 10:14:03 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by wa-out-1112.google.com with SMTP id m38so1419238waf.27 for ; Tue, 16 Jun 2009 03:14:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:subject :message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=xo3MXb6D/AytLsdDckULtZMIQplnFA5Wdjtpck2PLkg=; b=AkL8acWw2pldOg6JLH6QY8Gl35DtTsaO6/q2G1jSdyk7njVkLgVaY1w9Tu8hWYuM4g yU1vko3j5Sn0hP3InFgDRnOzEjLIDPHLoHybm4hxNUPzICIdrCuUVKsVoX5GWYV/GHnQ w2UMiByN9t7hYD98ADDeqaKCbNz/HniROvZjk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=lYjMw/QdH7pZhlpRxEXJ5smmeCfd2TYnm0b2be3tfbyuO0+OaniLImSW0vpoiSrn2+ ueUUn9Jd8lqrbWwzGtbf6Rwayg6gcFhLUTkBb5ub84U8Reu7Ugiy/qX1dpJItagZYWBZ oFUSeXUq/r5G1TuQ6d2GooCqy487Q45RrrjRw= Received: by 10.115.60.2 with SMTP id n2mr13401678wak.36.1245147242835; Tue, 16 Jun 2009 03:14:02 -0700 (PDT) Received: from michelle.cdnetworks.co.kr ([114.111.62.249]) by mx.google.com with ESMTPS id g25sm2522167wag.43.2009.06.16.03.14.01 (version=SSLv3 cipher=RC4-MD5); Tue, 16 Jun 2009 03:14:01 -0700 (PDT) Received: by michelle.cdnetworks.co.kr (sSMTP sendmail emulation); Tue, 16 Jun 2009 19:17:40 +0900 From: Pyun YongHyeon Date: Tue, 16 Jun 2009 19:17:40 +0900 To: current@freebsd.org Message-ID: <20090616101740.GI78415@michelle.cdnetworks.co.kr> References: <20090615121623.GA1479@roadrunner.spoerlein.net> <20090615125154.GG78415@michelle.cdnetworks.co.kr> <20090616093334.GB31709@acme.spoerlein.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090616093334.GB31709@acme.spoerlein.net> User-Agent: Mutt/1.4.2.3i Cc: Subject: Re: ale(4): Problems with tso, rxcsum and/or txcsum X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2009 10:14:03 -0000 On Tue, Jun 16, 2009 at 11:33:34AM +0200, Ulrich Sp??rlein wrote: > On Mon, 15.06.2009 at 21:51:54 +0900, Pyun YongHyeon wrote: > > On Mon, Jun 15, 2009 at 02:16:23PM +0200, Ulrich Sp??rlein wrote: > > > Hello Pyun, > > > > > > I have connection problems with the onboard GigE of an Asus P5Q board, using a recent 8-CURRENT > > > > > > ale0: port 0xdc00-0xdc7f mem 0xfe9c0000-0xfe9fffff irq 17 at device 0.0 on pci2 > > > ale0: 960 Tx FIFO, 1024 Rx FIFO > > > ale0: Using 1 MSI messages. > > > ale0: 4GB boundary crossed, switching to 32bit DMA addressing mode. > > > miibus0: on ale0 > > > ale0: Ethernet address: 00:24:8c:36:3e:10 > > > ale0: [FILTER] > > > ale0: link state changed to UP > > > > > > ale0@pci0:2:0:0: class=0x020000 card=0x82261043 chip=0x10261969 rev=0xb0 hdr=0x00 > > > vendor = 'Attansic (Now owned by Atheros)' > > > device = 'PCI-E ETHERNET CONTROLLER (AR8121/AR8113 )' > > > class = network > > > subclass = ethernet > > > > > > ale0: flags=8843 metric 0 mtu 1500 > > > options=311b > > > ether 00:24:8c:36:3e:10 > > > inet 192.168.0.146 netmask 0xffffff00 broadcast 192.168.0.255 > > > media: Ethernet autoselect (100baseTX ) > > > status: active > > > > > > > > > When transferring data to the machine at ~10MB/s (100Mbit network only) the ssh > > > connection will die after a couple of minutes with > > > > > > Disconnecting: Bad packet length 1592360521. > > > > > > After disabling tso, txcsum and rxcsum the connection seems to be > > > stable, though. I fail to figure out a pattern, though. Do I need to > > > > Hmm, I think this is the second report that could be related with > > Rx checksum offloading. If disabling Rx checksum fix the issue, I > > have to disable it by default until I understand what's going on. > > I really need to disable tso, rxcsum *and* txcsum to make this card work > stable. :/ > Hmm, let's see which offload was broken. Disabling all offloads make it hard to find broken one. > There is one other weirdness, though, regarding tso. I have been using a > netcat-blast test, where I "upload" /dev/zero to another machine, and > "download" it from the same machine. > > When tso is enabled, upload is seriously impacted, download is fine > though, observe systat output: > > ale0 in 10.805 MB/s 11.101 MB/s 7.739 GB > out 2.574 MB/s 8.740 MB/s 5.891 GB > > When disabling tso, while that test is running, it will immediately become this: > > ale0 in 7.498 MB/s 11.101 MB/s 8.270 GB > out 7.560 MB/s 8.740 MB/s 6.209 GB > > Which looks more normal. Re-activating tso now has no further consequences to > the stream (it only works for new TCP sessions, right?) > I was able to saturate gigabit link with AR8121. Tx performance is about 930Mbps or higher. Since ale(4) does not support ethernet flow-control it could be caused by dropped frames. Check hardware MAC counters, you can get it via "sysctl dev.ale.0.stats". Receiver also should be fast enough to get frames without loss. The above does not explain OpenSSH's output of "Bad packet length". It really means incoming packets were corrupted. So I'd like to know if you disable Rx checksum offloading you can still see the horrible message from OpenSSH. It does not necessarily mean TSO/ Tx checksum offloading works without problems but I'd like to narrow down issues instead of blindly disabling all offloads.