From owner-freebsd-net@FreeBSD.ORG Wed Aug 23 19:54:08 2006 Return-Path: X-Original-To: net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9BEBA16A4DA; Wed, 23 Aug 2006 19:54:08 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from MMS3.broadcom.com (mms3.broadcom.com [216.31.210.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3073B43D5C; Wed, 23 Aug 2006 19:53:55 +0000 (GMT) (envelope-from davidch@broadcom.com) Received: from 10.10.64.154 by MMS3.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.2.2)); Wed, 23 Aug 2006 12:53:50 -0700 X-Server-Uuid: 450F6D01-B290-425C-84F8-E170B39A25C9 Received: by mail-irva-10.broadcom.com (Postfix, from userid 47) id 71BC12AF; Wed, 23 Aug 2006 12:53:50 -0700 (PDT) Received: from mail-irva-8.broadcom.com (mail-irva-8 [10.10.64.221]) by mail-irva-10.broadcom.com (Postfix) with ESMTP id 4D3AC2AE; Wed, 23 Aug 2006 12:53:50 -0700 (PDT) Received: from mail-irva-12.broadcom.com (mail-irva-12.broadcom.com [10.10.64.146]) by mail-irva-8.broadcom.com (MOS 3.7.5a-GA) with ESMTP id EDL92680; Wed, 23 Aug 2006 12:53:50 -0700 (PDT) Received: from NT-IRVA-0750.brcm.ad.broadcom.com (nt-irva-0750 [10.8.194.64]) by mail-irva-12.broadcom.com (Postfix) with ESMTP id E201369CA3; Wed, 23 Aug 2006 12:53:49 -0700 (PDT) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Wed, 23 Aug 2006 12:53:49 -0700 Message-ID: <09BFF2FA5EAB4A45B6655E151BBDD90301D43002@NT-IRVA-0750.brcm.ad.broadcom.com> In-Reply-To: <20060823161649.GE76666@cell.sick.ru> Thread-Topic: bge(4) one packet wedge Thread-Index: AcbGz44wS3LCzDADS6Wj0cKiLu257wAGyTrg From: "David Christensen" To: "Gleb Smirnoff" X-TMWD-Spam-Summary: TS=20060823195352; SEV=2.0.2; DFV=A2006082307; IFV=2.0.4,4.0-8; RPD=4.00.0004; ENG=IBF; RPDID=303030312E30413031303230312E34344543423133432E303031372D412D; CAT=NONE; CON=NONE X-MMS-Spam-Filter-ID: A2006082307_4.00.0004_4.0-8 X-WSS-ID: 68F26DC422G1511765-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: brad@openbsd.org, oleg@FreeBSD.org, net@FreeBSD.org Subject: RE: bge(4) one packet wedge X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Aug 2006 19:54:08 -0000 This "lost interrupt" type of problem is addressed by the use of the status_tag=20 field in the status block. (Listed as bge_rsvd0 in the bge_status_block structure).=20 Everytime the status block is updated a new tag value is written to the status block. =20 When the ISR starts the driver should record the status_tag value. At the end of the ISR, the driver should compare the current status_tag value is the status block with the value recorded on entry to the ISR. If the values are the same then no additional status block updates have occurred so there shouldn't be any packets hanging around. If the values are different then additional packets or completions are waiting around so the ISR should loop around again. At the=20 end of the ISR the driver will write the status_tag value it last handled to a mailbox register, letting the hardware know the last status block update handled. If necessary the hardware will generate a new interrupt and start the process over again. This entire process should be included in the Linux driver, I don't see it being used in the bge driver (bge_intr()). Dave > -----Original Message----- > From: Gleb Smirnoff [mailto:glebius@FreeBSD.org]=20 > Sent: Wednesday, August 23, 2006 9:17 AM > To: David Christensen > Cc: brad@openbsd.org; oleg@FreeBSD.org; net@FreeBSD.org > Subject: bge(4) one packet wedge >=20 > Colleagues, >=20 > I've faced a problem in bge(4) when a single packet is in the RX > ring, but it isn't noticed by the driver. A reception of a packet > triggers interrupt and both packets are processed - an old one > and the new one. >=20 > To reproduce the problem you need to run netperf (from ports > collection): netserver on another host (10.0.0.1) and netperf on > the host, where tested bge(4) is installed - 10.0.0.2. No traffic > except netperf's should flow through this NIC, or the problem won't > be reproduced! >=20 > So, I run netperf client and simultaneously tcpdump on the > another host. After few seconds there is a wedge. The last packet > seen on 10.0.0.1 is the packet sent by 10.0.0.1 to 10.0.0.2. However > it isn't seen on 10.0.0.2. >=20 > Ok, let's look at the receive ring: >=20 > (kgdb) p $sc->bge_rx_saved_considx > $14 =3D 51 > (kgdb) p $sc->bge_ldata.bge_status_block->bge_idx[0].bge_rx_prod_idx > $15 =3D 51 >=20 > Looks like there is nothing to process. >=20 > However, if I run 'ping -c 1 10.0.0.2' I will get an=20 > interrupt and read > two packets: first the old packet, and then recently sent ping. >=20 > --=20 > Totus tuus, Glebius. > GLEBIUS-RIPN GLEB-RIPE >=20 >=20