Date: Sat, 6 Aug 2016 12:51:48 -0700 From: Nathan Whitehorn <nwhitehorn@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: Michal Meloun <mmel@freebsd.org>, "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org>, Svatopluk Kraus <skra@freebsd.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: INTRNG (Was: svn commit: r301453....) Message-ID: <2e638a0a-0d99-1aa7-4912-1015c8e2e947@freebsd.org> In-Reply-To: <CANCZdfreALoF1UnqY7h4BXCbRLZ_7z6FNL6Gmvh4hrVKNZ-g4w@mail.gmail.com> References: <201606051620.u55GKD5S066398@repo.freebsd.org> <57976867.6080705@FreeBSD.org> <f2edac8f-2859-cd98-754e-881e2b2d1e63@freebsd.org> <5798E104.5020104@FreeBSD.org> <a5d43044-1733-6cc7-2e99-e85b60b0fcf3@freebsd.org> <579A25BB.8070206@FreeBSD.org> <30790e40-58b4-3371-c0f0-b7545571f389@freebsd.org> <579AFFC5.1040005@FreeBSD.org> <eb603349-eb88-866d-7a26-9e026518fd39@freebsd.org> <579CD355.1050203@FreeBSD.org> <460fa0b3-ddb7-6247-2412-3d75a589d5e7@freebsd.org> <579CF7C8.1040302@FreeBSD.org> <24107713-6d50-c21d-ccf1-7dbdb36cc484@freebsd.org> <579E1BE2.7020500@FreeBSD.org> <7f053bb8-ab03-e46c-1c72-d757348e4e54@freebsd.org> <cefdfaab-a95f-2a92-89bd-3d0cef2a75ab@freebsd.org> <57A09F34.4050400@FreeBSD.org> <ad1e6337-468e-f35d-7454-444a561cb103@freebsd.org> <57A30B72.7070809@FreeBSD.org> <1946069a-d0f9-2c19-80a5-0b490682574b@freebsd.org> <57A5F480.20309@FreeBSD.org> <1d63e3aa-1a2f-992f-ae83-656eb185d386@freebsd.org> <CANCZdfreALoF1UnqY7h4BXCbRLZ_7z6FNL6Gmvh4hrVKNZ-g4w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 08/06/16 09:58, Warner Losh wrote: > On Sat, Aug 6, 2016 at 10:44 AM, Nathan Whitehorn > <nwhitehorn@freebsd.org> wrote: > >> Fair enough! I don't think we need that for, e.g., GPIOs (see cases 1-2 >> above), just for bus enumeration schemes (ACPI, OFW are probably the only >> ones) that usually require a ton of this kind of thing anyway. But, >> fundamentally, it doesn't matter. There are three important things from my >> end: >> 1. That it is possible to, at bus enumeration time, permanently assign an >> IRQ to an interrupt specifier from OFW/ACPI. >> 2. That that assignment not depend on having the PIC attached yet. >> 3. That the implementation details of that mechanism be reasonably >> abstracted so that they can change later or vary platform to platform. >> >> Whether mapping tables are in some central place (subr_intr.c) or in the >> parent bus, how the PIC API works, whether they are stored in that table in >> the form of a union or in different tables, doesn't matter for those three >> at all. And, with a constant API (3) we can even change our minds later >> without a lot of hassle. > First, I hate mapping tables at the nexus, unless they are created > dynamically at run time. There's too much variation between boards, > SoCs, etc to have that code live in the nexus otherwise. They simply > don't scale. This board has interrupts 1-16 wired this way, but that > board didn't do that and has an external PIC. This SoC based on > Cortext A<whatever> uses the GPIC, while that one based on the > same Cortext A<whatever> chose to use Atmel's PIC. Perhaps I'm > misunderstanding something here as to what is meant by a table > though. The table in question is just a mapping of abstract IRQ numbers to the corresponding interrupt specifier and parent. For example, 5: <&gicX, 7 2> Where 5 is the (potentially arbitrary) assigned number for the system corresponding to whatever <7 2> means on gic0. The table is built on the fly so that OFW bus nodes can specify interrupts as something <&gicX, 7 2> and get back scalars that work with rman and the PCI APIs and all the other places that want definitions of interrupts as scalars. It naturally handles all of your examples. The discussion here is whether to: a) keep an OFW/FDT-specific table like this in ofwbus.c (and analogs for ACPI, etc. in their respective places) b) keep a table that maps numbers to some union of OFW/ACPI/whatever data in nexus or intr.c or some global place It doesn't matter much from a functionality perspective (or an API perspective) either way. The question is more about which is easier to maintain and extend long-term. Since the API doesn't change either way, we're free to decide incorrectly and revisit it at will later with little consequence. > Next, In your list there's another dependency that's implicit > but maybe not called out. You can have PICs that cascade into > other PICs, or GPIO controllers that need to enable external > PIC-like things before they can route interrupts from things > that are downstream (interrupt wise) from them. Maybe I'm > just hung up on the phrase "the PIC" and it really means > "whatever complex thing or things handles getting the > interrupt routed to the CPU." I don't see this design so much > on basic eval boards, but do see it in more complex boards > that control complicated things. Yes. We've supported this forever on PPC (where it is very common) and it works here too. The implementation is really simple. First, here's an explanation of the general interrupt mapping system, for context: Let's suppose you have some interrupt hierarchy like this: CPU <- pic0 <- some device PIC0 has a bunch of interrupt pins. As you enumerate the system, the code encounters interrupt specifiers like <&pic0, 3 1>, <&pic0, 4, 1>, <&pic1, 3, 2>. Through OFW_BUS_MAP_INTR(), the bus enumeration code turns these into some arbitrary scalar IRQs assigned by OFW_BUS_MAP_INTR(). The mapping from the (interrupt parent, specifier) tuple to that assigned scalar IRQ is stored somewhere (see above) for later. When the bus devices that use those IRQs attach, they call bus_activate_resource() and bus_setup_intr(). At this point, some code close to the root of the hierarchy (again, see above for exactly where) looks up the corresponding vector specifier in the table, looks up the registered PIC corresponding to the interrupt parent key, and tells the driver attached to the interrupt parent to think of the scalar IRQ as meaning whatever string of numbers it saw earlier. It also tells the machine-dependent interrupt layer that the given scalar IRQ is handled by the device_t for the PIC so that it can ask the PIC to mask/unmask/bind/etc. the interrupt. When PICs attach, they register themselves with whatever bus layer is doing this mapping to say that a given interrupt parent key (the phandle for OFW/FDT) corresponds to the device_t for the PIC. The PIC attached directly to the CPU's interrupt pin[s] (pic0 here) also registers itself somehow with the MD interrupt system. On PPC, there is a global device_t called "root_pic" that the driver sets. When "some device" signals an interrupt, pic0 (the hardware) signals the CPU. The CPU signals the kernel. The MD interrupt layer notices and calls the driver for pic0 (the root PIC). That code inspects whatever registers on PIC0 give it the interrupt line and then signals the MD interrupt layer (let's say by calling a function called arm_dispatch_irq(int irq)) with the corresponding scalar IRQ. That then invokes the corresponding filters and/or ithreads. So how does this change with cascaded PICs? Very little. Let's suppose you have some interrupt hierarchy like this: CPU <- pic0 <- pic1 <- some device PIC0 has a bunch of interrupt pins, one of which is connected to pic1, which provides a bunch more interrupt pins. Both pic0 and pic1 register themselves with the bus layer by the appropriate handles and get their mask/unmask/bind functions called by the interrupt layer during interrupt setup for interrupts they handle. PIC0 registers itself as the root during attachment because it notices that it does not have any interrupts in its resource list. Child PICs, on the other hand, have interrupt properties in the FDT corresponding to the pin on pic0 that is attached to pic1. These get assigned as normal through newbus and the child PIC (pic1) calls bus_setup_intr and registers a filter handler. When "some device" signals an interrupt, pic1 drives a line on pic0, which drives a line on the CPU. The driver for PIC0 calls arm_dispatch_irq() with the IRQ corresponding to pic1, which calls pic1's interrupt handler. Since filter handlers run in primary interrupt context, pic1's interrupt handler can look exactly like what pic0 does when signalled by the interrupt layer: it runs through its registers, finds any lines with active interrupts, then calls arm_dispatch_irq() with the appropriate corresponding scalar IRQ. The nice thing here is that cascaded PICs require zero special handling by the system. You can just treat them as normal interrupts, with no special methods required in the bus layer, the interrupt system, or the PIC driver. -Nathan > > Generally, though, I like the direction things are going. > > Warner >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2e638a0a-0d99-1aa7-4912-1015c8e2e947>