From owner-freebsd-hackers  Tue Jul 15 05:38:56 1997
Return-Path: <owner-hackers>
Received: (from root@localhost)
          by hub.freebsd.org (8.8.5/8.8.5) id FAA24473
          for hackers-outgoing; Tue, 15 Jul 1997 05:38:56 -0700 (PDT)
Received: from godzilla.zeta.org.au (godzilla.zeta.org.au [203.2.228.19])
          by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA24379;
          Tue, 15 Jul 1997 05:37:52 -0700 (PDT)
Received: (from bde@localhost) by godzilla.zeta.org.au (8.8.5/8.6.9) id WAA21429; Tue, 15 Jul 1997 22:34:34 +1000
Date: Tue, 15 Jul 1997 22:34:34 +1000
From: Bruce Evans <bde@zeta.org.au>
Message-Id: <199707151234.WAA21429@godzilla.zeta.org.au>
To: luigi@labinfo.iet.unipi.it, smp@csn.net
Subject: Re: interrupt latency
Cc: hackers@FreeBSD.ORG, smp@FreeBSD.ORG
Sender: owner-hackers@FreeBSD.ORG
X-Loop: FreeBSD.org
Precedence: bulk

>the problem:
>
>  in our SMP system an INT may be sent to a CPU that can't enter the kernel
>because another CPU is currently inside the kernel.  this causes a long delay
>between the time the INT tickels the hardware and the time the ISR actually
>runs.
>
>---
>So I guess the 2 points in time are the actual instance of hardware
>INT and the time the ISR completes.  The closest the software can get to
>determining the start point would be on entry to _XintrNN (isa/apic_vector.s)
>A convenient stop point would be immediately after the lock is acquired
>(ie, after GET_MPLOCK in _XintrNN), the completion of the ISR code should be
>constant.

High-resolution kernel profiling (option GUPROF + kgmon -B) would show
the time spent by interrupt handlers (if it worked on SMP systems :-]).
The results wouldn't be very interesting.  Some ISRs take several msec.
You don't need hardware to measure such gross latency.

I think a latency of several msec is actually only bad (relative to the
uniprocessor case) for clock and fast interrupts.  Network interrupts
probably should have higher priority than disk interrupts, but disk
interrupts can currently preempt network interrupts, and vice versa.
Handling pcaudio (clock) interrupts at 16KHz is a good test.  The clock
interrupts should not be affected by disk and network ISRs.

>But this hides the time between the actual firing of the hardware INT and
>the time a CPU accepts it.  This could be fairly long if the "accepting CPU"
>is stuck behing the "giant lock", as hardware INTs are blocked during this
>time.

You'll have to arrange for INTs sent to CPUs not in the kernel to be sent
to the one in the kernel just to match uniprocessor interrupt latency.

Bruce