Date: Mon, 15 Nov 2010 00:08:08 GMT From: Peter Jeremy <peter.jeremy@alcatel-lucent.com> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/152253: [patch] Enhancements to digi(4) to prevent interrupt storms Message-ID: <201011150008.oAF088g2064159@www.freebsd.org> Resent-Message-ID: <201011150010.oAF0A9uh064511@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 152253 >Category: kern >Synopsis: [patch] Enhancements to digi(4) to prevent interrupt storms >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Nov 15 00:10:08 UTC 2010 >Closed-Date: >Last-Modified: >Originator: Peter Jeremy >Release: 7.3-STABLE >Organization: Alcatel-Lucent Australia >Environment: FreeBSD aalp01.au.alcatel-lucent.com 7.3-STABLE FreeBSD 7.3-STABLE #1: Fri Jun 18 14:22:50 EST 2010 root@aalp01.au.alcatel-lucent.com:/var/obj/usr/src/sys/DL360 i386 >Description: I have experienced interrupt storms associated with digi(4) and Accelport Xem cards on various versions of FreeBSD. The problem appears to be that the card generates an interrupt for no obvious reason and, in the absence of driver support to acknowledge the interrupt, the card generates an interrupt storm. I found that ensuring that the card did not share an interrupt helped but was not a complete cure on all systems (due to the difficulty of finding interrupt numbers that were not used by anything else). Whilst Digi do not support FreeBSD, they do support Linux and the latest version of the driver source can be found at http://ftp1.digi.com/support/driver/40002347_c.tgz I initially tried updating the Xem firmware included in digi(4), using sxfep.bin and sxbios.bin from the above file - to no avail. (The patch for this is not included below due to size but is just a matter of turning the contents of those two binary files into C arrays in Xem.fepos.h and Xem.bios.h respectively - further details upon request). I then worked through the Linux source code comparing the behaviour. The attached patch comprises: 1) Implementation of a sysctl to control digi_debug 2) Liberal application of 'volatile' modifiers to memory references on the card. 3) Various changes to PCI access width and magic numbers to match Linux 4) PCI interrupt acknowledgement write (copied from Linux) 5) Enable interrupt handling (even though the actual I/O remains polled). Note that these changes have only been tested on Xem because I don't have access to any other cards. Note that this patch is for 7-STABLE because digi(4) has not been converted for the TTYng changes in 8.x. The digi(4) souce code has not been touched for many years and is the same in 7.x, 8.x and 9.x. I intend to adapt digi(4) for the TTYng changes but the resultant patches will be relative to this source code, rather than the existing source code. >How-To-Repeat: Install Digi Accelport Xem card on a shared interrupt. >Fix: Patch attached with submission follows: Index: digi.c =================================================================== RCS file: /usr/ncvs/src/sys/dev/digi/digi.c,v retrieving revision 1.63 diff -u -r1.63 digi.c --- digi.c 27 Sep 2006 19:56:58 -0000 1.63 +++ digi.c 18 Jun 2010 01:58:03 -0000 @@ -46,8 +46,9 @@ #include <sys/mbuf.h> #include <sys/malloc.h> #include <sys/module.h> -#include <sys/tty.h> +#include <sys/sysctl.h> #include <sys/syslog.h> +#include <sys/tty.h> #include <sys/fcntl.h> #include <sys/serial.h> #include <sys/bus.h> @@ -75,7 +76,6 @@ static void digistart(struct tty *tp); static int digiparam(struct tty *tp, struct termios *t); static void digiclose(struct tty *tp); -static void digi_intr(void *); static int digi_init(struct digi_softc *_sc); static int digi_loadmoduledata(struct digi_softc *); static int digi_inuse(struct digi_softc *); @@ -94,7 +94,14 @@ static struct con_bios *con_bios_list; devclass_t digi_devclass; static char driver_name[] = "digi"; -unsigned digi_debug = 0; + +#ifdef DEBUG +unsigned long digi_debug = 0; + +SYSCTL_ULONG(_debug, OID_AUTO, digi_debug, CTLFLAG_RW, &digi_debug, 0, + "digi(4) debug flags"); +TUNABLE_ULONG("debug.digi_debug", &digi_debug); +#endif static struct speedtab digispeedtab[] = { { 0, 0}, /* old (sysV-like) Bx codes */ @@ -159,7 +166,7 @@ /* interrupt OK! */ return; } - log(LOG_ERR, "digi%d: Interrupt didn't work, use polled mode\n", unit); + log(LOG_ERR, "digi%d: Interrupt didn't work, use polled mode\n", sc->res.unit); #endif sc->callout = timeout(digi_poll, sc, (hz >= 200) ? hz / 100 : 1); } @@ -182,7 +189,7 @@ } static int -digi_bcopy(const void *vfrom, void *vto, size_t sz) +digi_bcopy(const void *vfrom, void volatile *vto, size_t sz) { volatile const char *from = (volatile const char *)vfrom; volatile char *to = (volatile char *)vto; @@ -212,7 +219,8 @@ digi_init(struct digi_softc *sc) { int i, cnt, resp; - u_char *ptr; + u_char volatile *ptr; + u_char *cptr; int lowwater; struct digi_p *port; volatile struct board_chan *bc; @@ -302,6 +310,11 @@ } DLOG(DIGIDB_INIT, (sc->dev, "Got init reset after %d us\n", i)); + /* Clear POST area */ + ptr = sc->setwin(sc, MISCGLOBAL); + for (i = 0; i < 16; i++) + *(uint8_t volatile *)(ptr + i) = 0; + /* Now upload the BIOS */ cnt = (sc->bios.size < sc->win_size - BIOSOFFSET) ? sc->bios.size : sc->win_size - BIOSOFFSET; @@ -323,10 +336,8 @@ } ptr = sc->setwin(sc, 0); - vW(ptr + 0) = 0x0401; - vW(ptr + 2) = 0x0bf0; - vW(ptr + 4) = 0x0000; - vW(ptr + 6) = 0x0000; + vD(ptr + 0) = 0x0bf00401; + vD(ptr + 4) = 0x00000000; break; } @@ -334,7 +345,7 @@ DLOG(DIGIDB_INIT, (sc->dev, "BIOS uploaded\n")); ptr = sc->setwin(sc, MISCGLOBAL); - W(ptr) = 0; + vW(ptr) = 0; if (sc->pcibus) { PCIPORT = FEPCLR; @@ -399,7 +410,7 @@ outb(sc->port, FEPREQ | FEPMEM); outb(sc->port, FEPCLR | FEPMEM); - for (i = 0; W(ptr); i++) { + for (i = 0; vW(ptr); i++) { if (i > hz) { log(LOG_ERR, "digi%d: FEP/OS move failed\n", sc->res.unit); @@ -449,14 +460,12 @@ DLOG(DIGIDB_INIT, (sc->dev, "FEP/OS loaded\n")); ptr = sc->setwin(sc, 0xc30); - W(ptr + 4) = 0x1004; - W(ptr + 6) = 0xbfc0; - W(ptr + 0) = 0x03; - W(ptr + 2) = 0x00; + vD(ptr + 4) = 0xbfc01004; + vD(ptr + 0) = 0x00000003; /* Clear the confirm word */ ptr = sc->setwin(sc, FEPSTAT); - W(ptr + 0) = 0; + vW(ptr + 0) = 0; if (sc->port) outb(sc->port, 0); /* XXX necessary ? */ @@ -469,13 +478,13 @@ /* A BIOS request to execute the FEP/OS */ ptr = sc->setwin(sc, 0xc40); - W(ptr + 0) = 1; - W(ptr + 2) = FEPCODE >> 4; - W(ptr + 4) = 4; + vW(ptr + 0) = 1; + vW(ptr + 2) = FEPCODE >> 4; + vW(ptr + 4) = 4; /* Clear the confirm word */ ptr = sc->setwin(sc, FEPSTAT); - W(ptr + 0) = 0; + vW(ptr + 0) = 0; /* Run the BIOS request */ outb(sc->port, FEPREQ | FEPMEM); /* send interrupt to BIOS */ @@ -498,8 +507,12 @@ DLOG(DIGIDB_INIT, (sc->dev, "FEP/OS started after %d iterations\n", i)); if (sc->model >= PCXEM) { +#if 0 + /* Interrupt configuration */ ptr = sc->setwin(sc, 0xe04); vW(ptr) = 2; +#endif + /* Read number of ports */ ptr = sc->setwin(sc, 0xc02); sc->numports = vW(ptr); } else { @@ -551,22 +564,23 @@ tp->t_open = digiopen; tp->t_close = digiclose; tp->t_sc = port; + cptr = (u_char *)(intptr_t)ptr; if (sc->model == PCXEVE) { - port->txbuf = ptr + + port->txbuf = cptr + (((bc->tseg - sc->mem_seg) << 4) & 0x1fff); - port->rxbuf = ptr + + port->rxbuf = cptr + (((bc->rseg - sc->mem_seg) << 4) & 0x1fff); port->txwin = FEPWIN | ((bc->tseg - sc->mem_seg) >> 9); port->rxwin = FEPWIN | ((bc->rseg - sc->mem_seg) >> 9); } else if (sc->model == PCXI || sc->model == PCXE) { - port->txbuf = ptr + ((bc->tseg - sc->mem_seg) << 4); - port->rxbuf = ptr + ((bc->rseg - sc->mem_seg) << 4); + port->txbuf = cptr + ((bc->tseg - sc->mem_seg) << 4); + port->rxbuf = cptr + ((bc->rseg - sc->mem_seg) << 4); port->txwin = port->rxwin = 0; } else { - port->txbuf = ptr + + port->txbuf = cptr + (((bc->tseg - sc->mem_seg) << 4) % sc->win_size); - port->rxbuf = ptr + + port->rxbuf = cptr + (((bc->rseg - sc->mem_seg) << 4) % sc->win_size); port->txwin = FEPWIN | (((bc->tseg - sc->mem_seg) << 4) / sc->win_size); @@ -585,6 +599,7 @@ fepcmd_w(port, SRXHWATER, (3 * port->rxbufsize) >> 2, 10); bc->edelay = 100; + bc->idata = 1; /*XXXXXX*/ ttyinitmode(tp, 0, 0); port->send_ring = 1; /* Default action on signal RI */ @@ -1080,7 +1095,7 @@ return (0); } -static void +void digi_intr(void *vp) { struct digi_p *port; @@ -1113,46 +1128,46 @@ window = sc->window; sc->setwin(sc, 0); - if (sc->model >= PCXEM && W(sc->vmem + 0xd00)) { + if (sc->model >= PCXEM && vW(sc->vmem + 0xd00)) { struct con_bios *con = con_bios_list; - register u_char *ptr; + register u_char volatile *ptr; - ptr = sc->vmem + W(sc->vmem + 0xd00); + ptr = sc->vmem + vW(sc->vmem + 0xd00); while (con) { - if (ptr[1] && W(ptr + 2) == W(con->bios + 2)) + if (ptr[1] && vW(ptr + 2) == vW(con->bios + 2)) /* Not first block -- exact match */ break; - if (W(ptr + 4) >= W(con->bios + 4) && - W(ptr + 4) <= W(con->bios + 6)) + if (vW(ptr + 4) >= W(con->bios + 4) && + vW(ptr + 4) <= W(con->bios + 6)) /* Initial search concetrator BIOS */ break; } if (con == NULL) { log(LOG_ERR, "digi%d: wanted bios LREV = 0x%04x" - " not found!\n", sc->res.unit, W(ptr + 4)); - W(ptr + 10) = 0; - W(sc->vmem + 0xd00) = 0; + " not found!\n", sc->res.unit, vW(ptr + 4)); + vW(ptr + 10) = 0; + vW(sc->vmem + 0xd00) = 0; goto eoi; } cxcon = con->bios; - W(ptr + 4) = W(cxcon + 4); - W(ptr + 6) = W(cxcon + 6); + vW(ptr + 4) = W(cxcon + 4); + vW(ptr + 6) = W(cxcon + 6); if (ptr[1] == 0) - W(ptr + 2) = W(cxcon + 2); - W(ptr + 8) = (ptr[1] << 6) + W(cxcon + 8); + vW(ptr + 2) = W(cxcon + 2); + vW(ptr + 8) = (ptr[1] << 6) + W(cxcon + 8); size = W(cxcon + 10) - (ptr[1] << 10); if (size <= 0) { - W(ptr + 8) = W(cxcon + 8); - W(ptr + 10) = 0; + vW(ptr + 8) = W(cxcon + 8); + vW(ptr + 10) = 0; } else { if (size > 1024) size = 1024; - W(ptr + 10) = size; - bcopy(cxcon + (ptr[1] << 10), ptr + 12, size); + vW(ptr + 10) = size; + bcopy(cxcon + (ptr[1] << 10), (void *)((intptr_t)ptr + 12), size); } - W(sc->vmem + 0xd00) = 0; + vW(sc->vmem + 0xd00) = 0; goto eoi; } @@ -1299,6 +1314,10 @@ } sc->gdata->eout = etail; eoi: + /* Ack any interrupt */ + if (sc->pcibus) + (void)sc->vmem[0x200002]; + if (sc->window != 0) sc->towin(sc, 0); if (window != 0) @@ -1403,7 +1422,7 @@ static void fepcmd(struct digi_p *port, int cmd, int op1, int ncmds) { - u_char *mem; + u_char volatile *mem; unsigned tail, head; int count, n; @@ -1414,7 +1433,7 @@ head = port->sc->gdata->cin; mem[head + 0] = cmd; mem[head + 1] = port->pnum; - *(u_short *)(mem + head + 2) = op1; + *(u_short volatile *)(mem + head + 2) = op1; head = (head + 4) & port->sc->gdata->cmax; port->sc->gdata->cin = head; @@ -1503,7 +1522,7 @@ bus_teardown_intr(sc->dev, sc->res.irq, sc->res.irqHandler); #ifdef DIGI_INTERRUPT if (sc->res.irq != NULL) { - bus_release_resource(dev, SYS_RES_IRQ, sc->res.irqrid, + bus_release_resource(sc->dev, SYS_RES_IRQ, sc->res.irqrid, sc->res.irq); sc->res.irq = NULL; } Index: digi.h =================================================================== RCS file: /usr/ncvs/src/sys/dev/digi/digi.h,v retrieving revision 1.19 diff -u -r1.19 digi.h --- digi.h 14 Oct 2004 18:37:59 -0000 1.19 +++ digi.h 18 Jun 2010 01:52:53 -0000 @@ -40,14 +40,14 @@ #define CE_NTYPES 3 #define CE_RECORD(com, errnum) (++(com)->delta_error_counts[errnum]) -/*#define DIGI_INTERRUPT*/ +#define DIGI_INTERRUPT #ifndef DEBUG #define DEBUG #endif #ifdef DEBUG -extern unsigned digi_debug; +extern unsigned long digi_debug; #define DLOG(level, args) if (digi_debug & (level)) device_printf args #else #define DLOG(level, args) @@ -148,8 +148,8 @@ struct cdev *ctldev; } res; - u_char *vmem; /* virtual memory address */ - u_char *memcmd; + u_char volatile *vmem; /* virtual memory address */ + u_char volatile *memcmd; volatile u_char *memevent; long pmem; /* physical memory address */ @@ -178,7 +178,7 @@ struct callout_handle inttest; /* int test timeout handle */ const char *module; - u_char *(*setwin)(struct digi_softc *_sc, unsigned _addr); + u_char volatile *(*setwin)(struct digi_softc *_sc, unsigned _addr); void (*hidewin)(struct digi_softc *_sc); void (*towin)(struct digi_softc *_sc, int _win); #ifdef DEBUG @@ -197,3 +197,4 @@ int digi_shutdown(device_t _dev); void digi_delay(struct digi_softc *_sc, const char *_txt, u_long _timo); +void digi_intr(void *); Index: digi_isa.c =================================================================== RCS file: /usr/ncvs/src/sys/dev/digi/digi_isa.c,v retrieving revision 1.13 diff -u -r1.13 digi_isa.c --- digi_isa.c 30 May 2004 20:08:30 -0000 1.13 +++ digi_isa.c 18 Jun 2010 01:27:20 -0000 @@ -70,14 +70,14 @@ }; #define DIGI_NVALIDMEM (sizeof(digi_validmem) / sizeof(digi_validmem[0])) -static u_char * +static u_char volatile * digi_isa_setwin(struct digi_softc *sc, unsigned int addr) { outb(sc->wport, sc->window = FEPWIN | (addr >> sc->win_bits)); return (sc->vmem + (addr % sc->win_size)); } -static u_char * +static u_char volatile * digi_xi_setwin(struct digi_softc *sc, unsigned int addr) { outb(sc->wport, sc->window = FEPMEM); @@ -320,7 +320,7 @@ { struct digi_softc *sc = device_get_softc(dev); int i, t, res; - u_char *ptr; + u_char volatile *ptr; int reset; u_long msize, iosize; long scport; Index: digi_pci.c =================================================================== RCS file: /usr/ncvs/src/sys/dev/digi/digi_pci.c,v retrieving revision 1.12 diff -u -r1.12 digi_pci.c --- digi_pci.c 5 Mar 2005 18:30:10 -0000 1.12 +++ digi_pci.c 18 Jun 2010 01:13:35 -0000 @@ -49,7 +49,7 @@ #include <dev/digi/digi.h> #include <dev/digi/digi_pci.h> -static u_char * +static volatile u_char * digi_pci_setwin(struct digi_softc *sc, unsigned int addr) { return (sc->vmem + addr); @@ -175,9 +175,11 @@ return (ENXIO); } - pci_write_config(dev, 0x40, 0, 4); - pci_write_config(dev, 0x46, 0, 4); + pci_write_config(dev, 0x40, 0, 1); + pci_write_config(dev, 0x46, 0, 1); + /* Limit burst length to 2 double-words */ + pci_write_config(dev, 0x42, 1, 1); sc->res.mem = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &sc->res.mrid, RF_ACTIVE); @@ -190,7 +192,8 @@ return (ENXIO); } retVal = bus_setup_intr(dev, sc->res.irq, INTR_TYPE_TTY, - digiintr, sc, &sc->res.irqHandler); + NULL, digi_intr, sc, &sc->res.irqHandler); + bus_teardown_intr(sc->dev, sc->res.irq, sc->res.irqHandler); #else DLOG(DIGIDB_IRQ, (sc->dev, "Interrupt support compiled out\n")); #endif Index: digi_pci.h =================================================================== RCS file: /usr/ncvs/src/sys/dev/digi/digi_pci.h,v retrieving revision 1.1 diff -u -r1.1 digi_pci.h --- digi_pci.h 2 May 2001 01:08:04 -0000 1.1 +++ digi_pci.h 18 Jun 2010 00:31:53 -0000 @@ -39,4 +39,4 @@ #define PCI_DEVICE_920_8 0x0027 /* XR-Plus 920 K, 8 port */ #define PCI_DEVICE_920_2 0x0034 /* XR-Plus 920 K, 2 port */ -#define PCIPORT sc->vmem[0x200000] +#define PCIPORT (((u_char volatile *)(sc->vmem))[0x200000]) >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201011150008.oAF088g2064159>