From owner-freebsd-stable@FreeBSD.ORG Tue Aug 17 18:52:13 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 094401065693 for ; Tue, 17 Aug 2010 18:52:13 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id CA7CF8FC20 for ; Tue, 17 Aug 2010 18:52:12 +0000 (UTC) Received: by pwj4 with SMTP id 4so3067899pwj.13 for ; Tue, 17 Aug 2010 11:52:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=EyAooA8wp8pqD/IBdVC+YT0dbxNHY3zWHKnHoiSJLuQ=; b=yHGi3nvmM9uAhqP12Ja4pN6JYGBCcFrwUPIeU4tDC83ZcMUGvbf9ROz0yWQ3WB6aGY V/Tt0pR5S9ES7faUkGp3f83/xD0uw0VFFMO4cGo5I0CE/XtZEjonVrtc1XAFST5QkVEi pc+3vy/c196UHUlxWIxIcM7JspPNBOGcVSsTM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=rbyQefnx6eWi3heUu2qXR499qetO13v5ju94HBZZSOi6o6UqilxkQl8mV/J2Vh8H8L YZO5r7OH3NOW8qDPIWOjGOYCjHDhaRStjeCkDM4Qoylp+VCLzFeQu3JPrIsDrUyaewsS ROtnTbEczZQcYYU3o3+0RNLOexKbvDu4eNyrY= Received: by 10.114.25.5 with SMTP id 5mr8357293way.78.1282071132283; Tue, 17 Aug 2010 11:52:12 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id d35sm14853458waa.9.2010.08.17.11.52.08 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 17 Aug 2010 11:52:09 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 17 Aug 2010 11:52:08 -0700 From: Pyun YongHyeon Date: Tue, 17 Aug 2010 11:52:08 -0700 To: Mike Tancsa Message-ID: <20100817185208.GA6482@michelle.cdnetworks.com> References: <201006102031.o5AKVCH2016467@lava.sentex.ca> <201007021739.o62HdMOU092319@lava.sentex.ca> <20100702193654.GD10862@michelle.cdnetworks.com> <201008162107.o7GL76pA080191@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201008162107.o7GL76pA080191@lava.sentex.ca> User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org, jfvogel@gmail.com Subject: Re: RELENG_7 em problems (and RELENG_8) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Aug 2010 18:52:13 -0000 On Mon, Aug 16, 2010 at 05:07:11PM -0400, Mike Tancsa wrote: > Hi Jack, > FYI, I am still seeing this same problem on RELENG_8 (code > as of today). Unfortunately I cant try Pyun's patch since the > underlying code has changed since then. > > em4@pci0:3:0:0: class=0x020000 card=0x34ec8086 chip=0x10d38086 > rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = 'Intel 82574L Gigabit Ethernet Controller (82574L)' > class = network > subclass = ethernet > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message > cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > cap 11[a0] = MSI-X supports 5 messages in map 0x1c > > pci3: on pcib3 > em4: port 0x1000-0x101f > mem 0xb1900000-0xb191ffff,0xb1920000-0xb1923fff irq 16 at device 0.0 on pci3 > em4: Using MSI interrupt > em4: [FILTER] > em4: Ethernet address: 00:15:17:ed:3e:c4 > Here is updated patch for HEAD and stable/8. http://people.freebsd.org/~yongari/em.csum_tso.20100817.patch It seems to work as expected under my limited environments. If you're using multiple Tx queues with em(4) it would be better to disable Tx checksum offloading as driver always have to create a new checksum context for each frame. This will effectively disable pipelined Tx data DMA which in turn greatly slows down Tx performance for small sized frames. The reason driver have to create a new checksum context when it uses multiple Tx queues comes from hardware limitation. The controller tracks only for the last context descriptor that was written such that driver does not know the state of checksum context configured in other Tx queue. Hope this helps. > > > ---Mike > > > At 03:36 PM 7/2/2010, Pyun YongHyeon wrote: > >On Fri, Jul 02, 2010 at 01:39:22PM -0400, Mike Tancsa wrote: > >> Hi Jack, > >> Just a followup to the email below. I now saw what appears > >> to be the same problem on RELENG_8, but on a different nic and with > >> VLANs. So not sure if this is a general em problem, a problem > >> specific to some em NICs, or a TSO problem in general. The issue > >> seemed to be triggered when I added a new vlan based on > >> > >> em3@pci0:14:0:0: class=0x020000 card=0x109a15d9 > >> chip=0x109a8086 rev=0x00 hdr=0x00 > >> vendor = 'Intel Corporation' > >> device = 'Intel PRO/1000 PL Network Adaptor (82573L)' > >> class = network > >> subclass = ethernet > >> cap 01[c8] = powerspec 2 supports D0 D3 current D0 > >> cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message > >> cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1) > >> > >> pci14: on pcib5 > >> em3: port 0x6000-0x601f > >> mem 0xe8300000-0xe831ffff irq 17 at device 0.0 on pci14 > >> em3: Using MSI interrupt > >> em3: [FILTER] > >> em3: Ethernet address: 00:30:48:9f:eb:81 > >> > >> em3: flags=8943 > >> metric 0 mtu 1500 > >> options=2098 > >> ether 00:30:48:9f:eb:81 > >> inet 10.255.255.254 netmask 0xfffffffc broadcast 10.255.255.255 > >> media: Ethernet autoselect (1000baseT ) > >> status: active > >> > >> I had to disable tso, rxcsum and txsum in order to see the devices on > >> the other side of the two vlans trunked off em3. Unfortunately, the > >> other sides were switches 100km and 500km away so I didnt have any > >> tcpdump capabilities to diagnose the issue. I had already created > >> one vlan off this NIC and all was fine. A few weeks later, I added a > >> new one and I could no longer telnet into the remote switches from > >> the local machine.... But, I could telnet into the switches from > >> machines not on the problem box. Hence, it would appear to be a > >> general TSO issue no ? I disabled tso on the nic (I didnt disable > >> net.inet.tcp.tso as I forgot about that).. Still nothing. I could > >> always ping the remote devices, but no tcp services. I then > >> remembered this issue from before, so I tried disabling tso on the > >> NIC. Still nothing. Then I disabled rxcsum and txcsum and I could > >> then telnet into the remote devices. > >> > >> This newly observed issue was from a buildworld on Mon Jun 14 > >> 11:29:12 EDT 2010. > >> > >> I will try and recreate the issue locally again to see if I can > >> trigger the problem on demand. Any thoughts on what it might be ? > >> Perhaps an issue specific to certain em nics ? > >> > > > >http://www.freebsd.org/cgi/query-pr.cgi?pr=141843 > >I'm not sure whether you're seeing the same issue though. > >I didn't have chance to try latest em(4) on stable/7. > > -------------------------------------------------------------------- > Mike Tancsa, tel +1 519 651 3400 > Sentex Communications, mike@sentex.net > Providing Internet since 1994 www.sentex.net > Cambridge, Ontario Canada www.sentex.net/mike >