Date: Sun, 18 Mar 2001 22:13:32 -0800 (PST) From: Matt Dillon <dillon@earth.backplane.com> To: Bruce Evans <bde@zeta.org.au> Cc: John Baldwin <jhb@FreeBSD.ORG>, Matthew Jacob <mjacob@feral.com>, arch@FreeBSD.ORG Subject: Re: 'final' man pages Message-ID: <200103190613.f2J6DWH04251@earth.backplane.com> References: <Pine.BSF.4.21.0103182337250.28547-100000@besplex.bde.org>
next in thread | previous in thread | raw e-mail | index | archive | help
I should make an addendum here, since BDE indicated 'by masks'. What
I forgot to say was that, of course, for a mask-based spl mechanism
to work without a giant lock, the mask needs to be per-cpu and the
interrupt code either needs to check all the cpu's masks, or the aggregate
needs to be cached. Caching the aggregate is actually fairly easy to
do. You can simply OR-in to a single common global when adding mask
elements. But you take a hit when removing mask elements (splx()
equivalent). You have to iterate through all the cpu's or have some
sort of master lock, which can be expensive.
There is a way to do it using only bus locking, and that is to aggregate
the multiple cpu's masks for any given interrupt number together into a
single word. For example, if you had 4 cpu's each interrupt would eat
4 bits. Each cpu would have to use several locked OR instructions to
set their individual bits and AND to clear them (because the 'mask' now
spans multiple words), but the interrupt can use a single memory access
to test all the per-cpu masks for a particular irq all at once, since
they all reside in the same word.
Another requirement when using a software interrupt mask is that you
need an interrupt-in-progress mask as well as an interrupt-disable-mask.
The spl*() function equivalent would have to set the
interrupt-disable-mask and then spin until any running interrupt (on
another cpu) completes by waiting for the equivalent interrupt-in-progress
mask to clear. Once clear, the disable mask prevents further interrupts
from occuring. This was the direction I proposed for SMPng as an
intermediate step before going to mutexes for the interrupt subsystem
but SMPng decided to go directly to the MUTEXes.
Theoretically the MUTEX system is supposed to do away with this sort
of masking requirement, but it requires serious work on the interrupt
subsystem and mainline kernel code to operate as expected since most
mainline code still assumes a mask/spl*() API. This is why current
is still disabling interrupts in SMP unsafe code. If people recall,
this led to a number of minor bugs rearing their heads like the FSYNC
code getting into a cpu-bound loop waiting for async I/O ops to finish,
etc.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200103190613.f2J6DWH04251>