Date: Wed, 22 Jun 2005 15:50:24 +0100 (BST) From: Robert Watson <rwatson@FreeBSD.org> To: Emanuel Strobl <Emanuel.strobl@gmx.net> Cc: Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?= <des@des.no>, freebsd-current@freebsd.org Subject: Re: lapic@2k interrukts eating CPU cycles Message-ID: <20050622154538.H26664@fledge.watson.org> In-Reply-To: <200506221606.42862@harrymail> References: <200506091423.39940@harrymail> <200506221554.41750@harrymail> <42B96E66.4040503@samsco.org> <200506221606.42862@harrymail>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 22 Jun 2005, Emanuel Strobl wrote: >> Because you have 'options APIC' in your kernel, and your running a post >> 5.2 version of FreeBSD that has a massively improved interrupt routing >> system in it. > > I've been running 5-current on that machine with "options APIC" for a > long time, back to when I had to include NO-MIXEDMODE to get it working > until I think Robeert Watson gave me some patches. But I never had lapic > so I was wondering... Might have been John Baldwin. In 5.x, timer interrupts are generated by a general purpose programmable timer and delivered as interrupts to one of your CPUs on an MP system, or your only CPU on a UP system. Depending on your hardware, where the timer went varied a bit -- typically, round robin, or maybe your first CPU or third CPU. The timer tick would then be broadcasted to the other CPUs using an inter-processor interrupt (IPI), so that all CPUs would see it for the purposes of timer-based preemption, profiling, statistics, and so on. In 6.x, we use the timer in the CPU-local apic (LAPIC) to interrupt each CPU, rather than delivering all the interrupts to a particular CPU and forwarding them. This lowers the IPI traffic overhead, which can be substantial due to synchronizing and preventing IPI deadlock. In particular, spin locks are used around IPI interactions to prevent two or more CPUs from deadlocking by sending each other IPIs simultaneously, and therefore being unable to acknowledge each other (there's a whole literature on why and how to do this, if you google or check a decent MP OS text). So in 5.x, you saw the primary timer interrupt traffic explicitly, and most interrupt monitoring tools didn't monitor IPI load. In 6.x, you see the timer interrupt traffic on each CPU as lapic load for the CPU. In 6.x, we currently don't monitor IPI interrupt load, but if we did in 5.x and 6.x, you'd see that the IPI traffic level is much lower, making up for the increased LAPIC interrupt traffic. I have a feature request in to John to add statistics gathering on IPIs, since he's currently reworking the interrupt paths. Robert N M Watson
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050622154538.H26664>