From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 23:11:34 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A6DE106564A for ; Mon, 13 Sep 2010 23:11:34 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 658848FC16 for ; Mon, 13 Sep 2010 23:11:34 +0000 (UTC) Received: by pzk7 with SMTP id 7so2689635pzk.13 for ; Mon, 13 Sep 2010 16:11:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=1NURhOgLSituW53Sx58LW8FI2MY2iWK6mQYbOi9wj5A=; b=rXsAc3EVM+xqs5bSixfk2zUCfn8BDxF2SdTyb9JiI/l94rXLT7nmiAprp9LeXNYTQC hGB0K9U+5WchuXgrUAPhEz3/6uPSedP2yA/1PwBS5hPUabITOguU/OgXd/TNST2FVvJ0 vHUtLQ74cxpWzjmkqIn/YiahrO0791cQmo754= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=mXefm22HmSkRUCN8tclzXf/iASoFZHn8i0+vLJuDUCSEi4KZnz4Aqwx9imxRP6kygl +Do2XrQ5psPvuFsjxMYy0z4pKfr0JkWAbPMbyzKA5GSVcCmr6xB92smyBGrIaJjbem48 wVnKr597erTLIkvWRCupy9h+BeAjGpi6cPYKQ= Received: by 10.142.200.2 with SMTP id x2mr361408wff.326.1284419493716; Mon, 13 Sep 2010 16:11:33 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 9sm9325231wfd.0.2010.09.13.16.11.30 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Sep 2010 16:11:31 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 13 Sep 2010 16:10:42 -0700 From: Pyun YongHyeon Date: Mon, 13 Sep 2010 16:10:42 -0700 To: Vlad Galu Message-ID: <20100913231042.GL1229@michelle.cdnetworks.com> References: <20100909102826.GB53812@rambler-co.ru> <20100909201050.GG7203@michelle.cdnetworks.com> <20100909211808.GJ7203@michelle.cdnetworks.com> <20100913142707.GL10050@rambler-co.ru> <20100913180447.GA1229@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, Igor Sysoev Subject: Re: bge hangs on recent 7.3-STABLE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 23:11:34 -0000 On Tue, Sep 14, 2010 at 01:08:08AM +0300, Vlad Galu wrote: > On Mon, Sep 13, 2010 at 9:04 PM, Pyun YongHyeon wrote: > > On Mon, Sep 13, 2010 at 06:27:08PM +0400, Igor Sysoev wrote: > >> On Thu, Sep 09, 2010 at 02:18:08PM -0700, Pyun YongHyeon wrote: > >> > >> > On Thu, Sep 09, 2010 at 01:10:50PM -0700, Pyun YongHyeon wrote: > >> > > On Thu, Sep 09, 2010 at 02:28:26PM +0400, Igor Sysoev wrote: > >> > > > Hi, > >> > > > > >> > > > I have several hosts running FreeBSD/amd64 7.2-STABLE updated on 11.01.2010 > >> > > > and 25.02.2010. Hosts process about 10K input and 10K output packets/s > >> > > > without issues. One of them, however, is loaded more than others, so it > >> > > > processes 20K/20K packets/s. > >> > > > > >> > > > Recently, I have upgraded one host to 7.3-STABLE, 24.08.2010. > >> > > > Then bge on this host hung two times. I was able to restart it from > >> > > > console using: > >> > > > ? /etc/rc.d/netif restart bge0 > >> > > > > >> > > > Then I have upgraded the most loaded (20K/20K) host to 7.3-STABLE, 07.09.2010. > >> > > > After reboot bge hung every several seconds. I was able to restart it, > >> > > > but bge hung again after several seconds. > >> > > > > >> > > > Then I have downgraded this host to 7.3-STABLE, 14.08.2010, since there > >> > > > were several if_bge.c commits on 15.08.2010. The same hangs. > >> > > > Then I have downgraded this host to 7.3-STABLE, 17.03.2010, before > >> > > > the first if_bge.c commit after 25.02.2010. Now it runs without hangs. > >> > > > > >> > > > The hosts are amd64 dual core SMP with 4G machines. bge information: > >> > > > > >> > > > bge0@pci0:4:0:0: ? ? ? ?class=0x020000 card=0x165914e4 chip=0x165914e4 rev=0x11 hdr=0x00 > >> > > > ? ? vendor ? ? = 'Broadcom Corporation' > >> > > > ? ? device ? ? = 'NetXtreme Gigabit Ethernet PCI Express (BCM5721)' > >> > > > > >> > > > bge0: mem 0xfe5f0000-0xfe5fffff irq 19 at device 0.0 on pci4 > >> > > > miibus1: on bge0 > >> > > > brgphy0: PHY 1 on miibus1 > >> > > > brgphy0: ?10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto > >> > > > bge0: Ethernet address: 00:e0:81:5f:6e:8a > >> > > > > >> > > > >> > > Could you show me verbose boot message(bge part only)? > >> > > Also show me the output of "pciconf -lcbv". > >> > > > >> > > >> > Forgot to send a patch. Let me know whether attached patch fixes > >> > the issue or not. > >> > >> > Index: sys/dev/bge/if_bge.c > >> > =================================================================== > >> > --- sys/dev/bge/if_bge.c ? ?(revision 212341) > >> > +++ sys/dev/bge/if_bge.c ? ?(working copy) > >> > @@ -3386,9 +3386,11 @@ > >> > ? ? sc->bge_rx_saved_considx = rx_cons; > >> > ? ? bge_writembx(sc, BGE_MBX_RX_CONS0_LO, sc->bge_rx_saved_considx); > >> > ? ? if (stdcnt) > >> > - ? ? ? ? ? bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, sc->bge_std); > >> > + ? ? ? ? ? bge_writembx(sc, BGE_MBX_RX_STD_PROD_LO, (sc->bge_std + > >> > + ? ? ? ? ? ? ? BGE_STD_RX_RING_CNT - 1) % BGE_STD_RX_RING_CNT); > >> > ? ? if (jumbocnt) > >> > - ? ? ? ? ? bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, sc->bge_jumbo); > >> > + ? ? ? ? ? bge_writembx(sc, BGE_MBX_RX_JUMBO_PROD_LO, (sc->bge_jumbo + > >> > + ? ? ? ? ? ? ? BGE_JUMBO_RX_RING_CNT - 1) % BGE_JUMBO_RX_RING_CNT); > >> > ?#ifdef notyet > >> > ? ? /* > >> > ? ? ?* This register wraps very quickly under heavy packet drops. > >> > >> Thank you, it seems the patch has fixed the bug. > >> BTW, I noticed the same hungs on FreeBSD 8.1, date=2010.09.06.23.59.59 > >> I will apply the patch on all my updated hosts. > >> > > > > Thanks for testing. I'm afraid bge(4) in HEAD, stable/8 and > > stable/7(including 8.1-RELEASE and 7.3-RELEASE) may suffer from > > this issue. Let me know what other hosts work with the patch. > > Hi Pyun, > > Thanks for the patch. It seems to have fixed the symptom in my case, > on a card identical to Igor's, but on board of an IBM eServer 306m. > Thanks for reporting and testing! I really appreciate it.