From owner-freebsd-current@FreeBSD.ORG Mon Apr 11 06:54:30 2005 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 365C116A4CE for ; Mon, 11 Apr 2005 06:54:30 +0000 (GMT) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7334D43D31 for ; Mon, 11 Apr 2005 06:54:27 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.11] (junior-wifi.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.1/8.13.1) with ESMTP id j3B6vwci028857; Mon, 11 Apr 2005 00:57:59 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <425A1E67.1050501@samsco.org> Date: Mon, 11 Apr 2005 00:51:19 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.5) Gecko/20050218 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Matthew Dillon References: <20050406233405.O47071@carver.gumbysoft.com> <200504081656.51917.jhb@FreeBSD.org> <20050410152946.W82708@carver.gumbysoft.com> <20050410172818.D82708@carver.gumbysoft.com> <200504110231.j3B2VOYr047361@apollo.backplane.com> <425A10DD.70500@samsco.org> <200504110628.j3B6Sw3m048327@apollo.backplane.com> In-Reply-To: <200504110628.j3B6Sw3m048327@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.8 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.0.2 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on pooker.samsco.org cc: freebsd-current@freebsd.org Subject: Re: Potential source of interrupt aliasing X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Apr 2005 06:54:30 -0000 Matthew Dillon wrote: > Sheesh Scott, you don't have to be nasty about it. I'm just trying to > help. I've seen billions of interrupt routing related problems but > not one interrupt aliasing issue. Yes, uncalled for, please accept my apologies. Definitely a case of hitting the Send button too soon. > > What I/O APIC chipset and stepping does Doug have on that motherboard? > Intel has a ton of errata for their I/O APICs. I see a mention of > having to turn off earlier revs of the chip and revert to legacy > 'boot interrupt' operation, but that's the only mention I see, > and it's only for old versions (< B-0 stepping) of the 80332. > > Still, perhaps the BIOS is doing this even with later revs of the chip > in the MB. This is the motherboard chipset, mainly i7501, 7520, and 440GX (not so sure on the last one since that doesn't involve hardware I own). The 7520 documents a register to turn off the 'boot interrupt' routing, but the 7501 doesn't. Since it might be a more general problem, we want to figure out a general way to handle it. It only happens on FreeBSD 5.x/6.x with the full IOAPIC handling (there are 3 IOAPICs on the 7501 and 7520). 4.x (and presumably DFly) doesn't see it because it doesn't enable more than the first IOAPIC. In fact, what allows routing to work at all on 4.x is that the interrupts on the second and third APICs are masked by default in the IOAPIC and magically re-routed to irq16 on the first via this 'boot interrupt' feature. I have no idea how this applies to non-Intel chipsets with more than one IOAPIC, like the ServerWorks GC series. It doesn't appear to be a problem on Opteron chipsets, but that might be a whole different ball of wax. An interesting datapoint is that Linux RH9 and FC2 don't seem to have any aliasing issues on the same exact hardware. Or, maybe they lie very well about it. So either they know the secret handshake with the chipset, or they don't mask at the APIC while the interrupt is being serviced (which is very hard for FreeBSD to emulate because interrupts are handled in ithreads). Just to review, the sequence of events: - ahd fires irq50 - low-level handler runs - vector decoded - ahd ithread scheduled - EOI - write the mask to the APIC for irq50 - source is masked, boot interrupt feature activated in chipset - irq50 remains asserted, so is aliased to irq16 (uhci) - low-level handler runs - vector decoded - uhci ithread scheduled - EOI - write the mask to the APIC for irq16 -- time passes -- - ahd ithread runs - quench ahd hardware - irq50 de-asserted - irq16 also de-asserted by association - ithread completes - irq50 unmasked in the APIC -- time passes -- - uhci ithread runs - nothing to quench - ithread completes - irq16 unmasked in the APIC - repeat > > There's a reference to the '80332 I/O Processor' developers manual which > implies a more detailed explanation of 'boot interrupt' operation. Since the 80331 and 80332 include a PCI bridge, it wouldn't surprise me that there is a similar mechanism at work in them. But again, we are only talking about Intel motherboard chipsets here, not IOPs. Scott