From owner-svn-src-stable@FreeBSD.ORG Tue Nov 17 12:28:21 2009 Return-Path: Delivered-To: svn-src-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48C251065679; Tue, 17 Nov 2009 12:28:21 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C3C2E8FC1F; Tue, 17 Nov 2009 12:28:19 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA11610; Tue, 17 Nov 2009 14:28:13 +0200 (EET) (envelope-from avg@freebsd.org) Message-ID: <4B0296DC.6080505@freebsd.org> Date: Tue, 17 Nov 2009 14:28:12 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.23 (X11/20090825) MIME-Version: 1.0 To: Florian Smeets References: <200910292325.n9TNPrGq015678@svn.freebsd.org> <4B0291AB.8090300@smeets.im> In-Reply-To: <4B0291AB.8090300@smeets.im> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: svn-src-stable@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, svn-src-stable-8@freebsd.org, Andrew Thompson Subject: Re: svn commit: r198659 - in stable/8/sys: . amd64/include/xen cddl/contrib/opensolaris contrib/dev/acpica contrib/pf dev/pci dev/usb/controller dev/xen/xenpci X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 12:28:21 -0000 on 17/11/2009 14:06 Florian Smeets said the following: > On 10/30/09 12:25 AM, Andrew Thompson wrote: >> Author: thompsa >> Date: Thu Oct 29 23:25:52 2009 >> New Revision: 198659 >> URL: http://svn.freebsd.org/changeset/base/198659 >> >> Log: >> MFC r198151 >> >> Workaround buggy BIOS code in USB regard. By doing the BIOS to OS >> handover for >> all host controllers at the same time, we avoid problems where the >> BIOS will >> actually write to the USB registers of all the USB host >> controllers every time >> we handover one of them, and consequently reset the OS programmed >> values. >> > > Hi, > > this commit causes some of my servers (all of the same type), to detect > interrupt storms on irq11: > > interrupt storm detected on "irq11:"; throttling interrupt source > > and it does not stop logging these. > > flo@relay3:~ 6 > vmstat -i > interrupt total rate > irq0: clk 229846 990 > irq4: uart0 596 2 > irq5: sis0 2850 12 > irq8: rtc 29595 127 > irq11: sis3 115799 499 > irq14: ata0 2071 8 > Total 380757 1641 > > sis3 is not used and not even up. > > Setting sysctl hw.intr_storm_threshold to high values e.g. 10000 does > not help. > > The kernels on these servers do not have usb compiled in at all, but > it's enabled in the BIOS. > > Setting hw.pci.usb_early_takeover=0 "fixes" the problem. > > Is this expected? This is not expected in a sense that nobody planned to introduce interrupt storms. But it is possible, because this is one of the edge cases introduced by the commit in question, and the tunable is there exactly to deal with the edge cases. What might be happening here is that your USB controller(s) is configured with interrupt 11, but you don't see it in vmstat -i output, because there is no USB driver to discover the hardware and its configuration. If we take over the controllers, BIOS no longer controls the devices and it is expected that the OS would do it, but it doesn't. So if interrupts are generated by USB hardware they remain unhandled and cause the storm. Thus, there are 3 choices: 1. what you have already done (and it does fix the problem, no quote-unquote needed); 2. add USB drivers, so that USB interrupts are handled; 3. disable USB in BIOS. Depending on your goals one choice may suite better than the others. Does this make sense? -- Andriy Gapon