Date: Sun, 20 Jun 2010 21:33:29 +0000 (UTC) From: Alexander Motin <mav@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r209371 - in head/sys: amd64/amd64 amd64/include conf dev/acpica i386/i386 i386/include isa kern pc98/cbus sys x86/isa x86/x86 Message-ID: <201006202133.o5KLXTG1023067@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: mav Date: Sun Jun 20 21:33:29 2010 New Revision: 209371 URL: http://svn.freebsd.org/changeset/base/209371 Log: Implement new event timers infrastructure. It provides unified APIs for writing event timer drivers, for choosing best possible drivers by machine independent code and for operating them to supply kernel with hardclock(), statclock() and profclock() events in unified fashion on various hardware. Infrastructure provides support for both per-CPU (independent for every CPU core) and global timers in periodic and one-shot modes. MI management code at this moment uses only periodic mode, but one-shot mode use planned for later, as part of tickless kernel project. For this moment infrastructure used on i386 and amd64 architectures. Other archs are welcome to follow, while their current operation should not be affected. This patch updates existing drivers (i8254, RTC and LAPIC) for the new order, and adds event timers support into the HPET driver. These drivers have different capabilities: LAPIC - per-CPU timer, supports periodic and one-shot operation, may freeze in C3 state, calibrated on first use, so may be not exactly precise. HPET - depending on hardware can work as per-CPU or global, supports periodic and one-shot operation, usually provides several event timers. i8254 - global, limited to periodic mode, because same hardware used also as time counter. RTC - global, supports only periodic mode, set of frequencies in Hz limited by powers of 2. Depending on hardware capabilities, drivers preferred in following orders, either LAPIC, HPETs, i8254, RTC or HPETs, LAPIC, i8254, RTC. User may explicitly specify wanted timers via loader tunables or sysctls: kern.eventtimer.timer1 and kern.eventtimer.timer2. If requested driver is unavailable or unoperational, system will try to replace it. If no more timers available or "NONE" specified for second, system will operate using only one timer, multiplying it's frequency by few times and uing respective dividers to honor hz, stathz and profhz values, set during initial setup. Added: head/sys/kern/kern_et.c (contents, props changed) head/sys/sys/timeet.h (contents, props changed) head/sys/x86/x86/timeevents.c (contents, props changed) Modified: head/sys/amd64/amd64/mp_machdep.c head/sys/amd64/include/apicvar.h head/sys/conf/files head/sys/conf/files.amd64 head/sys/conf/files.i386 head/sys/dev/acpica/acpi_hpet.c head/sys/dev/acpica/acpi_hpet.h head/sys/i386/i386/mp_machdep.c head/sys/i386/include/apicvar.h head/sys/isa/rtc.h head/sys/kern/kern_clock.c head/sys/pc98/cbus/clock.c head/sys/sys/systm.h head/sys/x86/isa/atrtc.c head/sys/x86/isa/clock.c head/sys/x86/x86/local_apic.c Modified: head/sys/amd64/amd64/mp_machdep.c ============================================================================== --- head/sys/amd64/amd64/mp_machdep.c Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/amd64/amd64/mp_machdep.c Sun Jun 20 21:33:29 2010 (r209371) @@ -706,10 +706,11 @@ init_secondary(void) load_es(_udatasel); load_fs(_ufssel); mtx_unlock_spin(&ap_boot_mtx); - /* wait until all the AP's are up */ while (smp_started == 0) ia32_pause(); + /* Start per-CPU event timers. */ + cpu_initclocks_ap(); sched_throw(NULL); Modified: head/sys/amd64/include/apicvar.h ============================================================================== --- head/sys/amd64/include/apicvar.h Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/amd64/include/apicvar.h Sun Jun 20 21:33:29 2010 (r209371) @@ -158,12 +158,6 @@ #define APIC_BUS_PCI 2 #define APIC_BUS_MAX APIC_BUS_PCI -enum lapic_clock { - LAPIC_CLOCK_NONE, - LAPIC_CLOCK_HARDCLOCK, - LAPIC_CLOCK_ALL -}; - /* * An APIC enumerator is a psuedo bus driver that enumerates APIC's including * CPU's and I/O APIC's. @@ -234,7 +228,6 @@ int lapic_set_lvt_triggermode(u_int apic enum intr_trigger trigger); void lapic_set_tpr(u_int vector); void lapic_setup(int boot); -enum lapic_clock lapic_setup_clock(enum lapic_clock srcsdes); #endif /* !LOCORE */ #endif /* _MACHINE_APICVAR_H_ */ Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/conf/files Sun Jun 20 21:33:29 2010 (r209371) @@ -2085,6 +2085,7 @@ kern/kern_context.c standard kern/kern_descrip.c standard kern/kern_dtrace.c optional kdtrace_hooks kern/kern_environment.c standard +kern/kern_et.c standard kern/kern_event.c standard kern/kern_exec.c standard kern/kern_exit.c standard Modified: head/sys/conf/files.amd64 ============================================================================== --- head/sys/conf/files.amd64 Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/conf/files.amd64 Sun Jun 20 21:33:29 2010 (r209371) @@ -297,6 +297,7 @@ x86/cpufreq/p4tcc.c optional cpufreq x86/isa/atpic.c optional atpic isa x86/isa/atrtc.c standard x86/isa/clock.c standard +x86/x86/timeevents.c standard x86/isa/elcr.c standard x86/isa/isa.c standard x86/isa/isa_dma.c standard Modified: head/sys/conf/files.i386 ============================================================================== --- head/sys/conf/files.i386 Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/conf/files.i386 Sun Jun 20 21:33:29 2010 (r209371) @@ -381,6 +381,7 @@ x86/cpufreq/smist.c optional cpufreq x86/isa/atpic.c optional atpic x86/isa/atrtc.c optional atpic x86/isa/clock.c optional native +x86/x86/timeevents.c standard x86/isa/elcr.c standard x86/isa/isa.c optional isa x86/isa/isa_dma.c optional isa Modified: head/sys/dev/acpica/acpi_hpet.c ============================================================================== --- head/sys/dev/acpica/acpi_hpet.c Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/dev/acpica/acpi_hpet.c Sun Jun 20 21:33:29 2010 (r209371) @@ -28,12 +28,21 @@ __FBSDID("$FreeBSD$"); #include "opt_acpi.h" +#ifdef __amd64__ +#define DEV_APIC +#else +#include "opt_apic.h" +#endif #include <sys/param.h> #include <sys/bus.h> #include <sys/kernel.h> #include <sys/module.h> +#include <sys/proc.h> #include <sys/rman.h> #include <sys/time.h> +#include <sys/smp.h> +#include <sys/sysctl.h> +#include <sys/timeet.h> #include <sys/timetc.h> #include <contrib/dev/acpica/include/acpi.h> @@ -42,46 +51,69 @@ __FBSDID("$FreeBSD$"); #include <dev/acpica/acpivar.h> #include <dev/acpica/acpi_hpet.h> +#ifdef DEV_APIC +#include "pcib_if.h" +#endif + #define HPET_VENDID_AMD 0x4353 #define HPET_VENDID_INTEL 0x8086 ACPI_SERIAL_DECL(hpet, "ACPI HPET support"); -static devclass_t acpi_hpet_devclass; +static devclass_t hpet_devclass; /* ACPI CA debugging */ #define _COMPONENT ACPI_TIMER ACPI_MODULE_NAME("HPET") -struct acpi_hpet_softc { +struct hpet_softc { device_t dev; + int mem_rid; + int intr_rid; + int irq; + int useirq; struct resource *mem_res; + struct resource *intr_res; + void *intr_handle; ACPI_HANDLE handle; + uint64_t freq; + struct timecounter tc; + struct hpet_timer { + struct eventtimer et; + struct hpet_softc *sc; + int num; + int mode; + int intr_rid; + int irq; + int pcpu_master; + int pcpu_slaves[MAXCPU]; + struct resource *intr_res; + void *intr_handle; + uint32_t caps; + uint32_t vectors; + uint32_t div; + uint32_t last; + char name[8]; + } t[32]; + int num_timers; }; static u_int hpet_get_timecount(struct timecounter *tc); -static void acpi_hpet_test(struct acpi_hpet_softc *sc); +static void hpet_test(struct hpet_softc *sc); static char *hpet_ids[] = { "PNP0103", NULL }; -struct timecounter hpet_timecounter = { - .tc_get_timecount = hpet_get_timecount, - .tc_counter_mask = ~0u, - .tc_name = "HPET", - .tc_quality = 900, -}; - static u_int hpet_get_timecount(struct timecounter *tc) { - struct acpi_hpet_softc *sc; + struct hpet_softc *sc; sc = tc->tc_priv; return (bus_read_4(sc->mem_res, HPET_MAIN_COUNTER)); } static void -hpet_enable(struct acpi_hpet_softc *sc) +hpet_enable(struct hpet_softc *sc) { uint32_t val; @@ -92,7 +124,7 @@ hpet_enable(struct acpi_hpet_softc *sc) } static void -hpet_disable(struct acpi_hpet_softc *sc) +hpet_disable(struct hpet_softc *sc) { uint32_t val; @@ -101,8 +133,113 @@ hpet_disable(struct acpi_hpet_softc *sc) bus_write_4(sc->mem_res, HPET_CONFIG, val); } +static int +hpet_start(struct eventtimer *et, + struct bintime *first, struct bintime *period) +{ + struct hpet_timer *mt = (struct hpet_timer *)et->et_priv; + struct hpet_timer *t; + struct hpet_softc *sc = mt->sc; + uint32_t fdiv; + + t = (mt->pcpu_master < 0) ? mt : &sc->t[mt->pcpu_slaves[curcpu]]; + if (period != NULL) { + t->mode = 1; + t->div = (sc->freq * (period->frac >> 32)) >> 32; + if (period->sec != 0) + t->div += sc->freq * period->sec; + if (first == NULL) + first = period; + } else { + t->mode = 2; + t->div = 0; + } + fdiv = (sc->freq * (first->frac >> 32)) >> 32; + if (first->sec != 0) + fdiv += sc->freq * first->sec; + t->last = bus_read_4(sc->mem_res, HPET_MAIN_COUNTER); + if (t->mode == 1 && (t->caps & HPET_TCAP_PER_INT)) { + t->caps |= HPET_TCNF_TYPE; + bus_write_4(sc->mem_res, HPET_TIMER_CAP_CNF(t->num), + t->caps | HPET_TCNF_VAL_SET); + bus_write_4(sc->mem_res, HPET_TIMER_COMPARATOR(t->num), + t->last + fdiv); + bus_read_4(sc->mem_res, HPET_TIMER_COMPARATOR(t->num)); + bus_write_4(sc->mem_res, HPET_TIMER_COMPARATOR(t->num), + t->div); + } else { + bus_write_4(sc->mem_res, HPET_TIMER_COMPARATOR(t->num), + t->last + fdiv); + } + t->caps |= HPET_TCNF_INT_ENB; + bus_write_4(sc->mem_res, HPET_ISR, 1 << t->num); + bus_write_4(sc->mem_res, HPET_TIMER_CAP_CNF(t->num), t->caps); + return (0); +} + +static int +hpet_stop(struct eventtimer *et) +{ + struct hpet_timer *mt = (struct hpet_timer *)et->et_priv; + struct hpet_timer *t; + struct hpet_softc *sc = mt->sc; + + t = (mt->pcpu_master < 0) ? mt : &sc->t[mt->pcpu_slaves[curcpu]]; + t->mode = 0; + t->caps &= ~(HPET_TCNF_INT_ENB | HPET_TCNF_TYPE); + bus_write_4(sc->mem_res, HPET_TIMER_CAP_CNF(t->num), t->caps); + return (0); +} + +static int +hpet_intr_single(void *arg) +{ + struct hpet_timer *t = (struct hpet_timer *)arg; + struct hpet_timer *mt; + struct hpet_softc *sc = t->sc; + uint32_t now; + + if (t->mode == 1 && + (t->caps & HPET_TCAP_PER_INT) == 0) { + t->last += t->div; + now = bus_read_4(sc->mem_res, HPET_MAIN_COUNTER); + if ((int32_t)(now - (t->last + t->div / 2)) > 0) + t->last = now - t->div / 2; + bus_write_4(sc->mem_res, + HPET_TIMER_COMPARATOR(t->num), t->last + t->div); + } else if (t->mode == 2) + t->mode = 0; + mt = (t->pcpu_master < 0) ? t : &sc->t[t->pcpu_master]; + if (mt->et.et_active) { + mt->et.et_event_cb(&mt->et, + mt->et.et_arg ? mt->et.et_arg : curthread->td_intr_frame); + } + return (FILTER_HANDLED); +} + +static int +hpet_intr(void *arg) +{ + struct hpet_softc *sc = (struct hpet_softc *)arg; + int i; + uint32_t val; + + val = bus_read_4(sc->mem_res, HPET_ISR); + if (val) { + bus_write_4(sc->mem_res, HPET_ISR, val); + val &= sc->useirq; + for (i = 0; i < sc->num_timers; i++) { + if ((val & (1 << i)) == 0) + continue; + hpet_intr_single(&sc->t[i]); + } + return (FILTER_HANDLED); + } + return (FILTER_STRAY); +} + static ACPI_STATUS -acpi_hpet_find(ACPI_HANDLE handle, UINT32 level, void *context, +hpet_find(ACPI_HANDLE handle, UINT32 level, void *context, void **status) { char **ids; @@ -115,16 +252,15 @@ acpi_hpet_find(ACPI_HANDLE handle, UINT3 } if (*ids == NULL) return (AE_OK); - if (ACPI_FAILURE(acpi_GetInteger(handle, "_UID", &uid))) - uid = 0; - if (id == uid) + if (ACPI_FAILURE(acpi_GetInteger(handle, "_UID", &uid)) || + id == uid) *((int *)status) = 1; return (AE_OK); } /* Discover the HPET via the ACPI table of the same name. */ static void -acpi_hpet_identify(driver_t *driver, device_t parent) +hpet_identify(driver_t *driver, device_t parent) { ACPI_TABLE_HPET *hpet; ACPI_STATUS status; @@ -132,7 +268,7 @@ acpi_hpet_identify(driver_t *driver, dev int i, found; /* Only one HPET device can be added. */ - if (devclass_get_device(acpi_hpet_devclass, 0)) + if (devclass_get_device(hpet_devclass, 0)) return; for (i = 1; ; i++) { /* Search for HPET table. */ @@ -142,12 +278,12 @@ acpi_hpet_identify(driver_t *driver, dev /* Search for HPET device with same ID. */ found = 0; AcpiWalkNamespace(ACPI_TYPE_DEVICE, ACPI_ROOT_OBJECT, - 100, acpi_hpet_find, NULL, (void *)(uintptr_t)hpet->Sequence, (void *)&found); + 100, hpet_find, NULL, (void *)(uintptr_t)hpet->Sequence, (void *)&found); /* If found - let it be probed in normal way. */ if (found) continue; /* If not - create it from table info. */ - child = BUS_ADD_CHILD(parent, ACPI_DEV_BASE_ORDER, "acpi_hpet", 0); + child = BUS_ADD_CHILD(parent, ACPI_DEV_BASE_ORDER, "hpet", 0); if (child == NULL) { printf("%s: can't add child\n", __func__); continue; @@ -158,7 +294,7 @@ acpi_hpet_identify(driver_t *driver, dev } static int -acpi_hpet_probe(device_t dev) +hpet_probe(device_t dev) { ACPI_FUNCTION_TRACE((char *)(uintptr_t) __func__); @@ -173,13 +309,15 @@ acpi_hpet_probe(device_t dev) } static int -acpi_hpet_attach(device_t dev) +hpet_attach(device_t dev) { - struct acpi_hpet_softc *sc; - int rid, num_timers; - uint32_t val, val2; - uintmax_t freq; - uint16_t vendor; + struct hpet_softc *sc; + struct hpet_timer *t; + int i, j, num_msi, num_timers, num_percpu_et, num_percpu_t, cur_cpu; + int pcpu_master; + static int maxhpetet = 0; + uint32_t val, val2, cvectors; + uint16_t vendor, rev; ACPI_FUNCTION_TRACE((char *)(uintptr_t) __func__); @@ -187,8 +325,8 @@ acpi_hpet_attach(device_t dev) sc->dev = dev; sc->handle = acpi_get_handle(dev); - rid = 0; - sc->mem_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, + sc->mem_rid = 0; + sc->mem_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &sc->mem_rid, RF_ACTIVE); if (sc->mem_res == NULL) return (ENOMEM); @@ -213,31 +351,53 @@ acpi_hpet_attach(device_t dev) return (ENXIO); } - freq = (1000000000000000LL + val / 2) / val; + sc->freq = (1000000000000000LL + val / 2) / val; + val = bus_read_4(sc->mem_res, HPET_CAPABILITIES); + vendor = (val & HPET_CAP_VENDOR_ID) >> 16; + rev = val & HPET_CAP_REV_ID; + num_timers = 1 + ((val & HPET_CAP_NUM_TIM) >> 8); + /* + * ATI/AMD violates IA-PC HPET (High Precision Event Timers) + * Specification and provides an off by one number + * of timers/comparators. + * Additionally, they use unregistered value in VENDOR_ID field. + */ + if (vendor == HPET_VENDID_AMD && rev < 0x10 && num_timers > 0) + num_timers--; + sc->num_timers = num_timers; if (bootverbose) { - val = bus_read_4(sc->mem_res, HPET_CAPABILITIES); - - /* - * ATI/AMD violates IA-PC HPET (High Precision Event Timers) - * Specification and provides an off by one number - * of timers/comparators. - * Additionally, they use unregistered value in VENDOR_ID field. - */ - num_timers = 1 + ((val & HPET_CAP_NUM_TIM) >> 8); - vendor = val >> 16; - if (vendor == HPET_VENDID_AMD && num_timers > 0) - num_timers--; device_printf(dev, - "vend: 0x%x rev: 0x%x num: %d hz: %jd opts:%s%s\n", - vendor, val & HPET_CAP_REV_ID, - num_timers, freq, - (val & HPET_CAP_LEG_RT) ? " legacy_route" : "", - (val & HPET_CAP_COUNT_SIZE) ? " 64-bit" : ""); + "vendor 0x%x, rev 0x%x, %jdHz%s, %d timers,%s\n", + vendor, rev, + sc->freq, (val & HPET_CAP_COUNT_SIZE) ? " 64bit" : "", + num_timers, (val & HPET_CAP_LEG_RT) ? " legacy route" : ""); + } + num_msi = 0; + for (i = 0; i < num_timers; i++) { + t = &sc->t[i]; + t->sc = sc; + t->num = i; + t->mode = 0; + t->intr_rid = -1; + t->irq = -1; + t->pcpu_master = -1; + t->caps = bus_read_4(sc->mem_res, HPET_TIMER_CAP_CNF(i)); + t->vectors = bus_read_4(sc->mem_res, HPET_TIMER_CAP_CNF(i) + 4); + if (bootverbose) { + device_printf(dev, + " t%d: irqs 0x%08x (%d)%s%s%s\n", i, + t->vectors, (t->caps & HPET_TCNF_INT_ROUTE) >> 9, + (t->caps & HPET_TCAP_FSB_INT_DEL) ? ", MSI" : "", + (t->caps & HPET_TCAP_SIZE) ? ", 64bit" : "", + (t->caps & HPET_TCAP_PER_INT) ? ", periodic" : ""); + } +#ifdef DEV_APIC + if (t->caps & HPET_TCAP_FSB_INT_DEL) + num_msi++; +#endif } - if (testenv("debug.acpi.hpet_test")) - acpi_hpet_test(sc); - + hpet_test(sc); /* * Don't attach if the timer never increments. Since the spec * requires it to be at least 10 MHz, it has to change in 1 us. @@ -253,15 +413,154 @@ acpi_hpet_attach(device_t dev) } /* Announce first HPET as timecounter. */ if (device_get_unit(dev) == 0) { - hpet_timecounter.tc_frequency = freq; - hpet_timecounter.tc_priv = sc; - tc_init(&hpet_timecounter); + sc->tc.tc_get_timecount = hpet_get_timecount, + sc->tc.tc_counter_mask = ~0u, + sc->tc.tc_name = "HPET", + sc->tc.tc_quality = 900, + sc->tc.tc_frequency = sc->freq; + sc->tc.tc_priv = sc; + tc_init(&sc->tc); + } + /* If not disabled - setup and announce event timers. */ + if (resource_int_value(device_get_name(dev), device_get_unit(dev), + "clock", &i) == 0 && i == 0) + return (0); + num_percpu_et = min(num_msi / mp_ncpus, 2); + num_percpu_t = num_percpu_et * mp_ncpus; + cur_cpu = CPU_FIRST(); + pcpu_master = 0; + /* Find common legacy IRQ vectors for all timers. */ + cvectors = 0xffff0000; + /* + * HPETs in AMD chipsets before SB800 have problems with IRQs >= 16 + * Lower are also not always working for different reasons. + * SB800 fixed it, but seems do not implements level triggering + * properly, that makes it very unreliable - it freezes after any + * interrupt loss. Avoid legacy IRQs for AMD. + */ + if (vendor == HPET_VENDID_AMD) + cvectors = 0x00000000; + sc->useirq = 0; + for (i = 0; i < num_timers; i++) { + t = &sc->t[i]; +#ifdef DEV_APIC + if (t->caps & HPET_TCAP_FSB_INT_DEL) { + if ((j = PCIB_ALLOC_MSIX( + device_get_parent(device_get_parent(dev)), dev, + &t->irq))) { + device_printf(dev, + "Can't allocate interrupt: %d.\n", j); + } else if (!(t->intr_res = + bus_alloc_resource(dev, SYS_RES_IRQ, &t->intr_rid, + t->irq, t->irq, 1, RF_SHAREABLE | RF_ACTIVE))) { + device_printf(dev, "Can't map interrupt.\n"); + } else if ((bus_setup_intr(dev, t->intr_res, + INTR_MPSAFE | INTR_TYPE_CLK, + (driver_filter_t *)hpet_intr_single, NULL, + t, &t->intr_handle))) { + device_printf(dev, "Can't setup interrupt.\n"); + } else { + bus_describe_intr(dev, t->intr_res, + t->intr_handle, "t%d", i); + num_msi++; + if (num_percpu_t > 0) { + if (cur_cpu == CPU_FIRST()) + pcpu_master = i; + t->pcpu_master = pcpu_master; + sc->t[pcpu_master]. + pcpu_slaves[cur_cpu] = i; + bus_bind_intr(dev, t->intr_res, cur_cpu); + cur_cpu = CPU_NEXT(cur_cpu); + num_percpu_t--; + } + } + } else +#endif + if ((cvectors & t->vectors) != 0) { + cvectors &= t->vectors; + sc->useirq |= (1 << i); + } + } + bus_write_4(sc->mem_res, HPET_ISR, 0xffffffff); + sc->irq = -1; + sc->intr_rid = -1; + /* If at least one timer needs legacy IRQ - setup it. */ + if (sc->useirq) { + j = i = fls(cvectors) - 1; + while (j > 0 && (cvectors & (1 << (j - 1))) != 0) + j--; + if (!(sc->intr_res = bus_alloc_resource(dev, SYS_RES_IRQ, + &sc->intr_rid, j, i, 1, RF_SHAREABLE | RF_ACTIVE))) + device_printf(dev,"Can't map interrupt.\n"); + else if ((bus_setup_intr(dev, sc->intr_res, + INTR_MPSAFE | INTR_TYPE_CLK, + (driver_filter_t *)hpet_intr, NULL, + sc, &sc->intr_handle))) { + device_printf(dev, "Can't setup interrupt.\n"); + } else { + sc->irq = rman_get_start(sc->intr_res); + /* Bind IRQ to BSP to avoid live migration. */ + bus_bind_intr(dev, sc->intr_res, CPU_FIRST()); + } + } + /* Program and announce event timers. */ + for (i = 0; i < num_timers; i++) { + t = &sc->t[i]; + t->caps &= ~(HPET_TCNF_FSB_EN | HPET_TCNF_INT_ROUTE); + t->caps &= ~(HPET_TCNF_VAL_SET | HPET_TCNF_INT_ENB); + t->caps |= HPET_TCNF_32MODE; +#ifdef DEV_APIC + if (t->irq >= 0) { + uint64_t addr; + uint32_t data; + + if (PCIB_MAP_MSI( + device_get_parent(device_get_parent(dev)), dev, + t->irq, &addr, &data) == 0) { + bus_write_4(sc->mem_res, + HPET_TIMER_FSB_ADDR(i), addr); + bus_write_4(sc->mem_res, + HPET_TIMER_FSB_VAL(i), data); + t->caps |= HPET_TCNF_FSB_EN; + } else + t->irq = -2; + } else +#endif + t->caps |= (sc->irq << 9) | HPET_TCNF_INT_TYPE; + bus_write_4(sc->mem_res, HPET_TIMER_CAP_CNF(i), t->caps); + /* Skip event timers without set up IRQ. */ + if (t->irq < 0 && + (sc->irq < 0 || (t->vectors & (1 << sc->irq)) == 0)) + continue; + /* Announce the reset. */ + if (maxhpetet == 0) + t->et.et_name = "HPET"; + else { + sprintf(t->name, "HPET%d", maxhpetet); + t->et.et_name = t->name; + } + t->et.et_flags = ET_FLAGS_PERIODIC | ET_FLAGS_ONESHOT; + t->et.et_quality = 450; + if (t->pcpu_master >= 0) { + t->et.et_flags |= ET_FLAGS_PERCPU; + t->et.et_quality += 100; + } + if ((t->caps & HPET_TCAP_PER_INT) == 0) + t->et.et_quality -= 10; + t->et.et_frequency = sc->freq; + t->et.et_start = hpet_start; + t->et.et_stop = hpet_stop; + t->et.et_priv = &sc->t[i]; + if (t->pcpu_master < 0 || t->pcpu_master == i) { + et_register(&t->et); + maxhpetet++; + } } return (0); } static int -acpi_hpet_detach(device_t dev) +hpet_detach(device_t dev) { ACPI_FUNCTION_TRACE((char *)(uintptr_t) __func__); @@ -270,9 +569,9 @@ acpi_hpet_detach(device_t dev) } static int -acpi_hpet_suspend(device_t dev) +hpet_suspend(device_t dev) { - struct acpi_hpet_softc *sc; + struct hpet_softc *sc; /* * Disable the timer during suspend. The timer will not lose @@ -286,20 +585,58 @@ acpi_hpet_suspend(device_t dev) } static int -acpi_hpet_resume(device_t dev) +hpet_resume(device_t dev) { - struct acpi_hpet_softc *sc; + struct hpet_softc *sc; + struct hpet_timer *t; + int i; /* Re-enable the timer after a resume to keep the clock advancing. */ sc = device_get_softc(dev); hpet_enable(sc); - + /* Restart event timers that were running on suspend. */ + for (i = 0; i < sc->num_timers; i++) { + t = &sc->t[i]; +#ifdef DEV_APIC + if (t->irq >= 0) { + uint64_t addr; + uint32_t data; + + if (PCIB_MAP_MSI( + device_get_parent(device_get_parent(dev)), dev, + t->irq, &addr, &data) == 0) { + bus_write_4(sc->mem_res, + HPET_TIMER_FSB_ADDR(i), addr); + bus_write_4(sc->mem_res, + HPET_TIMER_FSB_VAL(i), data); + } + } +#endif + if (t->mode == 0) + continue; + t->last = bus_read_4(sc->mem_res, HPET_MAIN_COUNTER); + if (t->mode == 1 && (t->caps & HPET_TCAP_PER_INT)) { + t->caps |= HPET_TCNF_TYPE; + bus_write_4(sc->mem_res, HPET_TIMER_CAP_CNF(t->num), + t->caps | HPET_TCNF_VAL_SET); + bus_write_4(sc->mem_res, HPET_TIMER_COMPARATOR(t->num), + t->last + t->div); + bus_read_4(sc->mem_res, HPET_TIMER_COMPARATOR(t->num)); + bus_write_4(sc->mem_res, HPET_TIMER_COMPARATOR(t->num), + t->div); + } else { + bus_write_4(sc->mem_res, HPET_TIMER_COMPARATOR(t->num), + t->last + sc->freq / 1024); + } + bus_write_4(sc->mem_res, HPET_ISR, 1 << t->num); + bus_write_4(sc->mem_res, HPET_TIMER_CAP_CNF(t->num), t->caps); + } return (0); } /* Print some basic latency/rate information to assist in debugging. */ static void -acpi_hpet_test(struct acpi_hpet_softc *sc) +hpet_test(struct hpet_softc *sc) { int i; uint32_t u1, u2; @@ -326,24 +663,56 @@ acpi_hpet_test(struct acpi_hpet_softc *s device_printf(sc->dev, "time per call: %ld ns\n", ts.tv_nsec / 1000); } -static device_method_t acpi_hpet_methods[] = { +#ifdef DEV_APIC +static int +hpet_remap_intr(device_t dev, device_t child, u_int irq) +{ + struct hpet_softc *sc = device_get_softc(dev); + struct hpet_timer *t; + uint64_t addr; + uint32_t data; + int error, i; + + for (i = 0; i < sc->num_timers; i++) { + t = &sc->t[i]; + if (t->irq != irq) + continue; + error = PCIB_MAP_MSI( + device_get_parent(device_get_parent(dev)), dev, + irq, &addr, &data); + if (error) + return (error); + hpet_disable(sc); /* Stop timer to avoid interrupt loss. */ + bus_write_4(sc->mem_res, HPET_TIMER_FSB_ADDR(i), addr); + bus_write_4(sc->mem_res, HPET_TIMER_FSB_VAL(i), data); + hpet_enable(sc); + return (0); + } + return (ENOENT); +} +#endif + +static device_method_t hpet_methods[] = { /* Device interface */ - DEVMETHOD(device_identify, acpi_hpet_identify), - DEVMETHOD(device_probe, acpi_hpet_probe), - DEVMETHOD(device_attach, acpi_hpet_attach), - DEVMETHOD(device_detach, acpi_hpet_detach), - DEVMETHOD(device_suspend, acpi_hpet_suspend), - DEVMETHOD(device_resume, acpi_hpet_resume), + DEVMETHOD(device_identify, hpet_identify), + DEVMETHOD(device_probe, hpet_probe), + DEVMETHOD(device_attach, hpet_attach), + DEVMETHOD(device_detach, hpet_detach), + DEVMETHOD(device_suspend, hpet_suspend), + DEVMETHOD(device_resume, hpet_resume), + +#ifdef DEV_APIC + DEVMETHOD(bus_remap_intr, hpet_remap_intr), +#endif {0, 0} }; -static driver_t acpi_hpet_driver = { - "acpi_hpet", - acpi_hpet_methods, - sizeof(struct acpi_hpet_softc), +static driver_t hpet_driver = { + "hpet", + hpet_methods, + sizeof(struct hpet_softc), }; - -DRIVER_MODULE(acpi_hpet, acpi, acpi_hpet_driver, acpi_hpet_devclass, 0, 0); -MODULE_DEPEND(acpi_hpet, acpi, 1, 1, 1); +DRIVER_MODULE(hpet, acpi, hpet_driver, hpet_devclass, 0, 0); +MODULE_DEPEND(hpet, acpi, 1, 1, 1); Modified: head/sys/dev/acpica/acpi_hpet.h ============================================================================== --- head/sys/dev/acpica/acpi_hpet.h Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/dev/acpica/acpi_hpet.h Sun Jun 20 21:33:29 2010 (r209371) @@ -57,7 +57,7 @@ #define HPET_TCAP_PER_INT 0x00000010 /* Supports periodic interrupts */ #define HPET_TCNF_TYPE 0x00000008 /* 1 = periodic, 0 = one-shot */ #define HPET_TCNF_INT_ENB 0x00000004 -#define HPET_TCNT_INT_TYPE 0x00000002 /* 1 = level triggered, 0 = edge */ +#define HPET_TCNF_INT_TYPE 0x00000002 /* 1 = level triggered, 0 = edge */ #define HPET_TIMER_COMPARATOR(x) ((x) * 0x20 + 0x108) #define HPET_TIMER_FSB_VAL(x) ((x) * 0x20 + 0x110) #define HPET_TIMER_FSB_ADDR(x) ((x) * 0x20 + 0x114) Modified: head/sys/i386/i386/mp_machdep.c ============================================================================== --- head/sys/i386/i386/mp_machdep.c Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/i386/i386/mp_machdep.c Sun Jun 20 21:33:29 2010 (r209371) @@ -728,10 +728,11 @@ init_secondary(void) } mtx_unlock_spin(&ap_boot_mtx); - /* wait until all the AP's are up */ while (smp_started == 0) ia32_pause(); + /* Start per-CPU event timers. */ + cpu_initclocks_ap(); /* enter the scheduler */ sched_throw(NULL); Modified: head/sys/i386/include/apicvar.h ============================================================================== --- head/sys/i386/include/apicvar.h Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/i386/include/apicvar.h Sun Jun 20 21:33:29 2010 (r209371) @@ -187,12 +187,6 @@ #define APIC_BUS_PCI 2 #define APIC_BUS_MAX APIC_BUS_PCI -enum lapic_clock { - LAPIC_CLOCK_NONE, - LAPIC_CLOCK_HARDCLOCK, - LAPIC_CLOCK_ALL -}; - /* * An APIC enumerator is a psuedo bus driver that enumerates APIC's including * CPU's and I/O APIC's. @@ -263,7 +257,6 @@ int lapic_set_lvt_triggermode(u_int apic enum intr_trigger trigger); void lapic_set_tpr(u_int vector); void lapic_setup(int boot); -enum lapic_clock lapic_setup_clock(enum lapic_clock srcsdes); #endif /* !LOCORE */ #endif /* _MACHINE_APICVAR_H_ */ Modified: head/sys/isa/rtc.h ============================================================================== --- head/sys/isa/rtc.h Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/isa/rtc.h Sun Jun 20 21:33:29 2010 (r209371) @@ -114,11 +114,7 @@ #ifdef _KERNEL extern struct mtx clock_lock; extern int atrtcclock_disable; -int atrtc_setup_clock(void); int rtcin(int reg); -void atrtc_start(void); -void atrtc_rate(unsigned rate); -void atrtc_enable_intr(void); void atrtc_restore(void); void writertc(int reg, u_char val); #endif Modified: head/sys/kern/kern_clock.c ============================================================================== --- head/sys/kern/kern_clock.c Sun Jun 20 18:26:04 2010 (r209370) +++ head/sys/kern/kern_clock.c Sun Jun 20 21:33:29 2010 (r209371) @@ -392,7 +392,7 @@ initclocks(dummy) * Set divisors to 1 (normal case) and let the machine-specific * code do its bit. */ - mtx_init(&time_lock, "time lock", NULL, MTX_SPIN); + mtx_init(&time_lock, "time lock", NULL, MTX_DEF); cpu_initclocks(); /* @@ -449,7 +449,7 @@ timer2clock(int usermode, uintfptr_t pc) *cnt -= t2hz; if (*cnt >= t2hz) *cnt = 0; - profclock(usermode, pc); + profclock(usermode, pc); } } @@ -599,10 +599,10 @@ startprofclock(p) return; if ((p->p_flag & P_PROFIL) == 0) { p->p_flag |= P_PROFIL; - mtx_lock_spin(&time_lock); + mtx_lock(&time_lock); if (++profprocs == 1) cpu_startprofclock(); - mtx_unlock_spin(&time_lock); + mtx_unlock(&time_lock); } } @@ -626,10 +626,10 @@ stopprofclock(p) if ((p->p_flag & P_PROFIL) == 0) return; p->p_flag &= ~P_PROFIL; - mtx_lock_spin(&time_lock); + mtx_lock(&time_lock); if (--profprocs == 0) cpu_stopprofclock(); - mtx_unlock_spin(&time_lock); + mtx_unlock(&time_lock); } } Added: head/sys/kern/kern_et.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/kern/kern_et.c Sun Jun 20 21:33:29 2010 (r209371) @@ -0,0 +1,232 @@ +/*- + * Copyright (c) 2010 Alexander Motin <mav@FreeBSD.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer, + * without modification, immediately at the beginning of the file. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/param.h> +#include <sys/kernel.h> +#include <sys/sysctl.h> +#include <sys/systm.h> +#include <sys/queue.h> +#include <sys/timeet.h> + +SLIST_HEAD(et_eventtimers_list, eventtimer); +static struct et_eventtimers_list eventtimers = SLIST_HEAD_INITIALIZER(et_eventtimers); + +struct mtx et_eventtimers_mtx; +MTX_SYSINIT(et_eventtimers_init, &et_eventtimers_mtx, "et_mtx", MTX_SPIN); + +SYSCTL_NODE(_kern, OID_AUTO, eventtimer, CTLFLAG_RW, 0, "Event timers"); +SYSCTL_NODE(_kern_eventtimer, OID_AUTO, et, CTLFLAG_RW, 0, ""); + +/* + * Register a new event timer hardware. + */ +int +et_register(struct eventtimer *et) +{ + struct eventtimer *tmp, *next; + + if (et->et_quality >= 0 || bootverbose) { + printf("Event timer \"%s\" frequency %ju Hz quality %d\n", + et->et_name, (uintmax_t)et->et_frequency, + et->et_quality); + } + et->et_sysctl = SYSCTL_ADD_NODE(NULL, + SYSCTL_STATIC_CHILDREN(_kern_eventtimer_et), OID_AUTO, et->et_name, + CTLFLAG_RW, 0, "event timer description"); + SYSCTL_ADD_UINT(NULL, SYSCTL_CHILDREN(et->et_sysctl), OID_AUTO, + "flags", CTLFLAG_RD, &(et->et_flags), 0, + "Event timer capabilities"); + SYSCTL_ADD_INT(NULL, SYSCTL_CHILDREN(et->et_sysctl), OID_AUTO, + "frequency", CTLFLAG_RD, &(et->et_frequency), 0, + "Event timer base frequency"); + SYSCTL_ADD_INT(NULL, SYSCTL_CHILDREN(et->et_sysctl), OID_AUTO, + "quality", CTLFLAG_RD, &(et->et_quality), 0, + "Goodness of event timer"); + ET_LOCK(); + if (SLIST_EMPTY(&eventtimers) || + SLIST_FIRST(&eventtimers)->et_quality < et->et_quality) { + SLIST_INSERT_HEAD(&eventtimers, et, et_all); + } else { + SLIST_FOREACH(tmp, &eventtimers, et_all) { + next = SLIST_NEXT(tmp, et_all); + if (next == NULL || next->et_quality < et->et_quality) { + SLIST_INSERT_AFTER(tmp, et, et_all); + break; + } + } + } + ET_UNLOCK(); + return (0); +} + +/* + * Deregister event timer hardware. + */ +int +et_deregister(struct eventtimer *et) +{ + int err = 0; + + if (et->et_deregister_cb != NULL) { + if ((err = et->et_deregister_cb(et, et->et_arg)) != 0) + return (err); + } + + ET_LOCK(); + SLIST_REMOVE(&eventtimers, et, eventtimer, et_all); + ET_UNLOCK(); + sysctl_remove_oid(et->et_sysctl, 1, 1); + return (0); +} + +/* + * Find free event timer hardware with specified parameters. + */ +struct eventtimer * +et_find(const char *name, int check, int want) +{ + struct eventtimer *et = NULL; + + SLIST_FOREACH(et, &eventtimers, et_all) { + if (et->et_active) + continue; + if (name != NULL && strcasecmp(et->et_name, name) != 0) + continue; + if (name == NULL && et->et_quality < 0) + continue; + if ((et->et_flags & check) != want) + continue; *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201006202133.o5KLXTG1023067>