From owner-freebsd-current@FreeBSD.ORG Fri Sep 24 18:02:45 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F06916A4CE for ; Fri, 24 Sep 2004 18:02:45 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A101343D46 for ; Fri, 24 Sep 2004 18:02:42 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id i8OI1qFA098661; Fri, 24 Sep 2004 14:01:52 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i8OI1pjk098606; Fri, 24 Sep 2004 14:01:52 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 24 Sep 2004 14:01:51 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Shunsuke SHINOMIYA In-Reply-To: <20040925011147.1388.SHINO@fornext.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-current@freebsd.org Subject: Re: High rate traffic silence an em interface. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 18:02:45 -0000 On Sat, 25 Sep 2004, Shunsuke SHINOMIYA wrote: > traffic over output interface's transmission rate silence an em > interface with 5.3-BETA5. > > I configured a P4 with HTT box with two em interfaces for a router, one > interface is set to 100BaseTX, the other is set to 10BaseT. > > And I sent the IPv4 11Mbps(only 1Mbps exceed 10Mbps) traffic for 10 > seconds from 100BaseTX side to 10BaseT side by smartbits, most of > packets dropped and this measurement terminated with failure. > Then I did ping to a host over 10baseT side at the box, ping outputted > with "No buffers space avilable". I'm seeing this also. For me, the threshold to trigger the problem is somewhere between 50Kpps and 250Kpps, depending on factors I can't identify. However, the box I'm running on also has some interesting interrupt configuration issues, so I'm wondering if what I'm looking at is a driver race condition. It seems not to be related to Giant over the network stack -- I see the weding with and without debug.mpsafenet=1. I saw Sam Leffler also report a similar problem today. Bosko and I have been seeing this if_em problem on some test hardware we have been using for several months. Bosko tried updating to the latest version of the driver from Intel's web site, and it appeared to make the problem go away. However, the version on Intel's web site doesn't have busdma support or fine-grained locking, so if it is a race condition, maybe their version of the driver doesn't trigger it because it's doing these things differently (i.e., might still exist there). You might try using the netrate tool I committed to src/tools/tools/netrate to try and figure out the threshold transmission level necessary to trigger the problem. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research > > >> ping 10.1.1.3 > >PING 10.1.1.3 (10.1.1.3): 56 data bytes > >ping: sendto: No buffer space available > >^C > >--- 10.1.1.3 ping statistics --- > >1 packets transmitted, 0 packets received, 100% packet loss > > I found two methods for recovery. One is up & down the interface of > 10BaseT side, the other (strange?) one is ping6 to the host over 10baseT > side like "ping6 ff02::1%em1". > > "Tx Descriptors not avail1" counter by hw.em1.debug_info increased. > Another counters kept zero. > > > em1: Adapter hardware address = 0xc3dc6b34 > > em1:CTRL = 0x40f01849 > > em1:RCTL = 0x8002 PS=(0x8402) > > em1:tx_int_delay = 0, tx_abs_int_delay = 0 > > em1:rx_int_delay = 0, rx_abs_int_delay = 0 > > em1: fifo workaround = 0, fifo_reset = 0 > > em1: hw tdh = 132, hw tdt = 132 > > em1: Num Tx descriptors avail = 256 > > em1: Tx Descriptors not avail1 = 6430 > > em1: Tx Descriptors not avail2 = 0 > > em1: Std mbuf failed = 0 > > em1: Std mbuf cluster failed = 0 > > em1: Driver dropped packets = 0 > > > Is this a peculiar problem just with me? > > -- > Shunsuke SHINOMIYA > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >