From owner-freebsd-hackers  Wed Oct  2 00:41:35 1996
Return-Path: owner-hackers
Received: (from root@localhost)
          by freefall.freebsd.org (8.7.5/8.7.3) id AAA17321
          for hackers-outgoing; Wed, 2 Oct 1996 00:41:35 -0700 (PDT)
Received: from godzilla.zeta.org.au (godzilla.zeta.org.au [203.2.228.19])
          by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id AAA17315
          for <freebsd-hackers@FreeBSD.ORG>; Wed, 2 Oct 1996 00:41:26 -0700 (PDT)
Received: (from bde@localhost) by godzilla.zeta.org.au (8.7.6/8.6.9) id RAA28429; Wed, 2 Oct 1996 17:31:49 +1000
Date: Wed, 2 Oct 1996 17:31:49 +1000
From: Bruce Evans <bde@zeta.org.au>
Message-Id: <199610020731.RAA28429@godzilla.zeta.org.au>
To: freebsd-hackers@FreeBSD.ORG, Tor.Egge@idt.ntnu.no
Subject: Re: Interrupt lossage in FreeBSD-current.
Sender: owner-hackers@FreeBSD.ORG
X-Loop: FreeBSD.org
Precedence: bulk

>To: freebsd-hackers@freebsd.org
>Subject: Interrupt lossage in FreeBSD-current.

This should probably be To: freebsd-current :-).

>I've written a device driver for an ISA card that has the potential of
>generating interrupts as fast as the machine is able to handle them.
>It is not a common situation, but it may happens for short periods 
>at a time (e.g. 0.3 seconds).  
>
>One problem though, is that this may cause a high interrupt latency
>(0.3 seconds), with the net effect that one RTC interrupt is
>lost. Profiling and CPU time accountings stops working.

Perhaps it should run at spl < splsoftclock.  (Loss of ordinary clock
interrupts is worse than loss of RTC interrupts, so perhaps it should
also run at spl < splclock.  This is now automatic because
splsoftclock < splclock, but splsoftclock should probably be changed
so that it is >= splclock.)

>In an attempt to avoid losing an RTC interrupt, I've recently added
>some code to the interrupt handler in the device driver to check for a
>pending timer interrupt and a repeatedly pending RTC interrupt.  If
>such an interrupt is pending, the restart of the device is performed
>in a timeout routine called by softclock instead of in the interrupt
>handler, causing all pending hardware interrupts to be processed.

It would be better to avoid using timeouts.  The timeout may be further
delayed, and many real interrupts may occur before it is called.  This
can probably be fixed by adding another software interrupt handler like
the sio one but running at splhigh().

>This seems to work, and prohibit the loss of RTC interrupts, but
>I found no good method of checking for pending/blocked interrupts:
>
>   ((imen & 1) || (inb(IO_ICU1) & 1))
>
>and
>
>   ((imen & 256) || (inb(IO_ICU2) & 1))
>
>are not what I would consider easily readable code to check for
>pending IRQ0 and IRQ8.

It would be more correct to use `ipending' instead of `imen'.  `ipending'
gives pending interrupts that the system already knows about.  `imen' is
in implementation detail.  However, I think there are more races for
`ipending' (some btrl's of it in icu.s are done with interrupts enabled).

It may be unnecesssary to check ipending or imen, since pending interrupts
will be restarted automatically and you don't want to restart them twice.
OTOH, it may be necessary to change `ipending' to get multiple restarts
to work right.  Checking it is the least of your problems.

>A macro as e.g.:
>
>#define INTRBLOCKED(i) ((imen & (1<<(i))) || (inb((i)>=8?IO_ICU2:IO_ICU1)&(1<<((i)&7))))
>
>can cause the code to be more readable, but may also add some overhead
>unless the compiler manages to reduce it to the variants shown earlier.

gcc will optimize this nicely provided `i' is constant.

>Maybe a better way is to rewrite the fastintr handler, to enable the

Do you really need to use a fastintr handler?  The fastness of a fastintr
handler isn't very important, at least on current hardware.  The important
thing is to take complete control of the machine so that timing constraints
can be satisfied.  This currently only works right if there is only one
fairly short fastintr handler or several very short ones.  The sum of
the interrupt latencies must be <= the minimum permissible latency.
The minimum permissible latency is 87us if there is an 8250 UART running
at 115200 bps, 2*87us if there is a 16550 UART running at 115200 bps
(since the fifo size less the fifo trigger level is 2).  The constraints
for clock latencies are much weaker (the normal worst case is 2*
1000000/1024 us for the RTC in profiling mode).

>ICUs before calling the interrupt handler? Then the interrupt handler
>could enable interrupts, disable interrupts, and just check the imen
>variable instead of polling the interrupt controller. Then a macro for

Enabling interrupts would defeat the point of it being a fastintr handler.
Other interrupts could occur and take a long time to handle.  However,
clock and other fastintr interrupts might not take too long.

>the check would be trivial, e.g.:
>
>#define INTRBLOCKED(i)   (imen & (1<<(i)))
>
>I cannot immediately see any reasons not to reenable the ICUs before
>calling the interrupt handler from the fast interrupt vector code in
>vector.s, since interrupts are disabled by default when the interrupt
>handler is called.

Yes, this makes no difference.  Also, the ICUs get reenabled immediately
if the AUTO_EOI_* options are used.  The problems start with temporarily
reemabling interrupts in a fastintr handler.  Another problem is that
here's nothing to stop infinite nesting of the fastintr handler.

Bruce