Date: Wed, 02 Oct 1996 22:33:06 +0200 From: Tor Egge <Tor.Egge@idt.ntnu.no> To: bde@zeta.org.au Cc: freebsd-hackers@freebsd.org Subject: Re: Interrupt lossage in FreeBSD-current. Message-ID: <199610022033.WAA23298@pat.idt.unit.no> In-Reply-To: Your message of "Wed, 2 Oct 1996 17:31:49 %2B1000" References: <199610020731.RAA28429@godzilla.zeta.org.au>
next in thread | previous in thread | raw e-mail | index | archive | help
> Perhaps it should run at spl < splsoftclock. (Loss of ordinary clock
> interrupts is worse than loss of RTC interrupts, so perhaps it should
> also run at spl < splclock. This is now automatic because
> splsoftclock < splclock, but splsoftclock should probably be changed
> so that it is >= splclock.)
If you lose one RTC interrupt, you lose all RTC interrupts
thereafter, since the interrupt handler must enable further
interrupts.
If you lose one clock interrupt, the clock is one tick wrong. This error
can later be corrected by use of xntpd or timed.
Thus I disagree: loss of an RTC interrupt is far worse.
About running at spl < splsoftclock , see the end of this message.
> >In an attempt to avoid losing an RTC interrupt, I've recently added
> >some code to the interrupt handler in the device driver to check for a
> >pending timer interrupt and a repeatedly pending RTC interrupt. If
> >such an interrupt is pending, the restart of the device is performed
> >in a timeout routine called by softclock instead of in the interrupt
> >handler, causing all pending hardware interrupts to be processed.
>
> It would be better to avoid using timeouts. The timeout may be further
> delayed, and many real interrupts may occur before it is called. This
> can probably be fixed by adding another software interrupt handler like
> the sio one but running at splhigh().
The device in question is automatically suspended when it has
generated an interrupt, thus it is not critical for the correctness
that it is restarted at once. A timeout of 1 tick will be processed
very soon when an unhandled timer interrupt is pending. If the device
is always restarted from the hardware interrupt context, it may
generate a continuous stream of hardware interrupts causing a delay of
e.g. 0.3 seconds inside another interrupt handler.
> >This seems to work, and prohibit the loss of RTC interrupts, but
> >I found no good method of checking for pending/blocked interrupts:
> >
> > ((imen & 1) || (inb(IO_ICU1) & 1))
> >
> >and
> >
> > ((imen & 256) || (inb(IO_ICU2) & 1))
> >
> >are not what I would consider easily readable code to check for
> >pending IRQ0 and IRQ8.
>
> It would be more correct to use `ipending' instead of `imen'. `ipending'
> gives pending interrupts that the system already knows about. `imen' is
> in implementation detail. However, I think there are more races for
> `ipending' (some btrl's of it in icu.s are done with interrupts enabled).
>
The bit flag in ipending is only set when the interrupt is blocked by
cpl, and it is always cleared before the interrupt handler is called.
Thus ipending is not usable with regards to hardware interrupts.
I have worked around this problem by letting all other hardware
interrupts block the interrupt for the device. I now use ipending to
check for a pending SWI_CLOCK.
> It may be unnecesssary to check ipending or imen, since pending interrupts
> will be restarted automatically and you don't want to restart them twice.
> OTOH, it may be necessary to change `ipending' to get multiple restarts
> to work right. Checking it is the least of your problems.
But something was causing an RTC interrupt to be lost. I've only
experienced it while profiling a program (while the RTC interrupt rate
is 1024 Hz), thus I can only assume that the RTC does not like a
latency longer than the interval between two RTC interrupts.
> >A macro as e.g.:
> >
> >#define INTRBLOCKED(i) ((imen & (1<<(i))) || (inb((i)>=8?IO_ICU2:IO_ICU1)&(1<<((i)&7))))
> >
> >can cause the code to be more readable, but may also add some overhead
> >unless the compiler manages to reduce it to the variants shown earlier.
>
> gcc will optimize this nicely provided `i' is constant.
>
> >Maybe a better way is to rewrite the fastintr handler, to enable the
>
> Do you really need to use a fastintr handler? The fastness of a fastintr
> handler isn't very important, at least on current hardware. The important
> thing is to take complete control of the machine so that timing constraints
> can be satisfied. This currently only works right if there is only one
> fairly short fastintr handler or several very short ones. The sum of
> the interrupt latencies must be <= the minimum permissible latency.
> The minimum permissible latency is 87us if there is an 8250 UART running
> at 115200 bps, 2*87us if there is a 16550 UART running at 115200 bps
> (since the fifo size less the fifo trigger level is 2). The constraints
> for clock latencies are much weaker (the normal worst case is 2*
> 1000000/1024 us for the RTC in profiling mode).
No. I don't need to use a fastintr handler, and I've now reverted to
using slow interrupts (maximum rate: 61500 interrupts/s, i.e. 16 us/interrupt.
> >ICUs before calling the interrupt handler? Then the interrupt handler
> >could enable interrupts, disable interrupts, and just check the imen
> >variable instead of polling the interrupt controller. Then a macro for
>
> Enabling interrupts would defeat the point of it being a fastintr handler.
> Other interrupts could occur and take a long time to handle. However,
> clock and other fastintr interrupts might not take too long.
It would eliminate the low latency for handling this device. That is
no problem, as it does not depend on low latency handling.
> >the check would be trivial, e.g.:
> >
> >#define INTRBLOCKED(i) (imen & (1<<(i)))
> >
> >I cannot immediately see any reasons not to reenable the ICUs before
> >calling the interrupt handler from the fast interrupt vector code in
> >vector.s, since interrupts are disabled by default when the interrupt
> >handler is called.
>
> Yes, this makes no difference. Also, the ICUs get reenabled immediately
> if the AUTO_EOI_* options are used. The problems start with temporarily
> reemabling interrupts in a fastintr handler. Another problem is that
> here's nothing to stop infinite nesting of the fastintr handler.
That depends on the device in question. Nesting
should be no problem as long as interrupts are disabled again
before telling the device that it can generate further interrupts.
I have now reverted to using slow interrupts. What I do in addition is:
1. loop through intr_mptr[], blocking the interrupt for the device
during any hardware interrupt.
2. loop through the imasks array, blocking the interrupt for the
device during any software interrupt.
3. In the interrupt handler for the device, check ipending for
a pending SWI_CLOCK, and if any so, perform the restart of the
device in the timeout routine instead of in the interupt
handler.
4. In hardclock(), softclock() is no longer called directly,
since splsoftclock() does not block the device.
1. is done to avoid starvation of other hardware interrupt handlers.
2., 3. and 4. is done to avoid starvation of the timeout() handling. (e.g.
avoid ncr dead? messages).
- Tor Egge
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199610022033.WAA23298>