From owner-freebsd-stable@FreeBSD.ORG Tue Apr 24 19:07:20 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 560E5106566C for ; Tue, 24 Apr 2012 19:07:20 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id F1D8B8FC0A for ; Tue, 24 Apr 2012 19:07:19 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7271DB911; Tue, 24 Apr 2012 15:07:19 -0400 (EDT) From: John Baldwin To: pyunyh@gmail.com Date: Tue, 24 Apr 2012 15:07:14 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p13; KDE/4.5.5; amd64; ; ) References: <20120228210329.GA2741@localhost> <201204231024.42237.jhb@freebsd.org> <20120424200719.GB6932@michelle.cdnetworks.com> In-Reply-To: <20120424200719.GB6932@michelle.cdnetworks.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201204241507.14699.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 24 Apr 2012 15:07:19 -0400 (EDT) Cc: Pavel Gorshkov , freebsd-stable@freebsd.org Subject: Re: msk0: interrupt storm X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 19:07:20 -0000 On Tuesday, April 24, 2012 4:07:19 pm YongHyeon PYUN wrote: > On Mon, Apr 23, 2012 at 10:24:41AM -0400, John Baldwin wrote: > > On Wednesday, March 07, 2012 3:40:53 pm YongHyeon PYUN wrote: > > > On Tue, Mar 06, 2012 at 10:36:05AM -0500, John Baldwin wrote: > > > > On Thursday, March 01, 2012 8:29:55 pm YongHyeon PYUN wrote: > > > > > On Wed, Feb 29, 2012 at 01:03:29AM +0400, Pavel Gorshkov wrote: > > > > > > My laptop running 9.0-RELEASE/amd64/GENERIC freezes and > > > > > > (sometimes) unfreezes intermittently, logging the following: > > > > > > > > > > > > Feb 28 23:07:36 lifebook kernel: interrupt storm detected on "irq259:"; > > > > throttling interrupt source > > > > > > > > > > > > $ vmstat -i > > > > > > ... > > > > > > irq259: mskc0 11669511 3456 > > > > > > > > > > > > > > > > > > Looks very similar to this: > > > > > > http://www.freebsd.org/cgi/query-pr.cgi?pr=164569 > > > > > > > > > > > > Any suggestions? > > > > > > > > > > Try disabling MSI and see whether that makes any difference. > > > > > > > > I also get interrupt storms with msk. They do fix themselves when they > > > > happen, and I've seen it happen with the machine is idle. This is on my > > > > little netbook where msk had several problems initially that have since been > > > > fixed. > > > > > > > > mskc0: port 0x2000-0x20ff mem > > > > 0xe0000000-0xe0003fff irq 19 at device 0.0 on pci32 > > > > msk0: on mskc0 > > > > msk0: Ethernet address: 00:24:81:40:e3:ef > > > > miibus0: on msk0 > > > > e1000phy0: PHY 0 on miibus0 > > > > e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > > > > 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow > > > > > > > > mskc0@pci0:32:0:0: class=0x020000 card=0x3056103c chip=0x436c11ab > > > > rev=0x10 hdr=0x00 > > > > vendor = 'Marvell Technology Group Ltd.' > > > > device = '88E8072 PCI-E Gigabit Ethernet Controller' > > > > class = network > > > > subclass = ethernet > > > > > > > > > > John, can you let me know the value of B0_Y2_SP_ISRC2 register in > > > interrupt handler when you see the interrupt storm? > > > > I finally tested this. I added some KTR traces to dump ISRC2 on each > > call to msk_intr() and hacked the interrupt thread code to turn KTR tracing > > off when a storm occurred. The traces look like this: > > > > index cpu timestamp trace > > ------ --- ---------------- ----- > > 148 0 111662766108828 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 147 0 111662765994576 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 146 0 111662765380260 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 145 0 111662765257308 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 144 0 111662765134356 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 143 0 111662765011560 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 142 0 111662764888656 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 141 0 111662764773924 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 140 0 111662764659360 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 139 0 111662764528140 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 138 0 111662764413576 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > 137 0 111662764287852 msk_intr: B0_Y2_SP_ISRC2 = 0x44000000 > > ... > > > > (All traces have the same register value.) The TSC on this netbook runs > > at machdep.tsc_freq: 1596035244 > > > > (The timestamps above are TSC values.) > > > > Let me know if you'd like me to log more stuff in the driver. Thanks! > > wonder why the deivce gets TWSI completion interrupt since the > driver does not monitor temperature sensor. In addition, the > interrupt was already disabled so have no idea how this can happen. > Here, I assume your controller implemented optional temperature > sensor and it is monitored by H/W. > Anyway, try attached patch and let me know whether it makes any > difference. It does fix the interrupt storms. I added a debugging printf to fire each time msk_intr() sees this bit to see if it storms, etc. What I see is that each time I would previously get a single printf reporting an interrupt storm, I now get a single printf reporting that the TWSI_RDY bit was set. -- John Baldwin