Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 7 Jun 2010 21:24:29 -0800
From:      Tsuyoshi Ozawa <ozawa@t-oza.net>
To:        Alexander Motin <mav@freebsd.org>
Cc:        FreeBSD-Current <freebsd-current@freebsd.org>, freebsd-arch@freebsd.org
Subject:   Re: RFC: New event timers infrastructure
Message-ID:  <AANLkTikyWXHYezBW1mAlTU0Oi_Z1qlPbAyZllux_wFSB@mail.gmail.com>
In-Reply-To: <AANLkTimfyn0aCs7PFRHL8RONvg03_QVXK_90oy084IIx@mail.gmail.com>
References:  <4C0C1AE4.8050807@FreeBSD.org> <AANLkTimfyn0aCs7PFRHL8RONvg03_QVXK_90oy084IIx@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
2010/6/6, Alexander Motin <mav@freebsd.org>:
> Hi.
>
> Most of x86 systems now has at least 4 types of event timers: i8254,
> RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily
> hardcoded and absolutely not scalable. I have reimplemented it, trying
> to solve these issues.
>
> I did such things:
>  - created unified timer driver's API (sys/timeet.h, kernel/kern_et.c).
> It supports global and per-CPU timers, periodic and one-shot. Provides
> driver and consumer interfaces for choosing timers and operating them;
>  - cleaned existing x86 event timer driver's code and modified it for
> new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC
> timer is now per-CPU and supports both periodic and one-shot modes;
>  - extended HPET driver to support it's event timers in periodic and
> one-shot mode (dev/acpica/acpi_hpet.c). Support for per-CPU operation
> and FSB interrupts planned for later;
>  - written mostly machine-independent mid-layer for managing any present
> timers to provide clocks needed for kernel (x86/x86/timeevents.c). It
> supports both global and per-CPU timers. Now it supports only periodic
> mode, but one-shot mode support planned for later.
>
> All this stuff deeply configurable via both loader tunables on boot and
> sysctls in real time:
>
> %sysctl kern.eventtimer
> kern.eventtimer.choice: LAPIC(500) HPET(400) HPET1(390) HPET2(390)
> i8254(100) RTC(0)
> kern.eventtimer.et.LAPIC.flags: 7
> kern.eventtimer.et.LAPIC.frequency: 99752386
> kern.eventtimer.et.LAPIC.quality: 500
> kern.eventtimer.et.HPET.flags: 3
> kern.eventtimer.et.HPET.frequency: 14318180
> kern.eventtimer.et.HPET.quality: 400
> kern.eventtimer.et.HPET1.flags: 3
> kern.eventtimer.et.HPET1.frequency: 14318180
> kern.eventtimer.et.HPET1.quality: 390
> kern.eventtimer.et.HPET2.flags: 3
> kern.eventtimer.et.HPET2.frequency: 14318180
> kern.eventtimer.et.HPET2.quality: 390
> kern.eventtimer.et.RTC.flags: 1
> kern.eventtimer.et.RTC.frequency: 32768
> kern.eventtimer.et.RTC.quality: 0
> kern.eventtimer.et.i8254.flags: 1
> kern.eventtimer.et.i8254.frequency: 1193182
> kern.eventtimer.et.i8254.quality: 100
> kern.eventtimer.timer2: NONE
> kern.eventtimer.timer1: i8254
> kern.eventtimer.singlemul: 2
>
> By default system chooses two timers with highest "quality" for
> hardclock and statclock/profclock. User may affect that choice via
> disabling unwanted drivers and/or via direct specification of wanted
> ones. It is possible to change timers on-flight via sysctls:
>
> %sysctl kern.eventtimer.timer1=hpet
> kern.eventtimer.timer1: i8254 -> HPET
> %sysctl kern.eventtimer.timer2=hpet1
> kern.eventtimer.timer2: NONE -> HPET1
>
> After every timer change, if two timers available, mid-layer
> cross-checks them, and if one of them is not functional - replaces it.
>
> If there is no second timer available, or user specified to not use it -
> mid-layer automatically increases rate of the first timer and divide
> it's frequency to satisfy system needs as good as possible. User may
> specify how fast he wish to run fist timer relative to hz by setting
> kern.eventtimer.singlemul tunable/sysctl.
>
> When profiling is active, mid-layer automatically rises respective timer
> frequency to about 8KHz (was 1KHz previously) and decreases it back on
> profiling end.
>
> All above was tested on i386 and amd64. XEN was not affected and builds
> fine. pc98 was slightly touched. It wasn't tested, but builds fine. It's
> pc98/cbus/clock.c needs respective rewrite to use new features. Other
> architectures are untouched, but if any of them may benefit from this
> functionality - it should be possible to share most of the code.
>
> Latest patches can be found here:
> http://people.freebsd.org/~mav/et.20100606.patch
>
> Known issues:
>  - i8254 timer generates 18Hz interrupt rate when not used and not
> disabled. I haven't found a way to disable it's interrupt source while
> holding spinlock.
>  - timer drivers code will need some more cleaning after interrupt
> handler will be able to return both argument and frame same time.
>
> Feedback is very appreciated.
>
> --
> Alexander Motin
>

This is excellent!
I'll try this to apply this patch and  rewrite my old dynamic ticks code to
fit this event timer API .

Thank you, Alexander !
-- 
Tsuyoshi Ozawa
<ozawa@t-oza.net>

2010/06/07 7:03 "Alexander Motin" <mav@freebsd.org>:

Hi.

Most of x86 systems now has at least 4 types of event timers: i8254,
RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily
hardcoded and absolutely not scalable. I have reimplemented it, trying
to solve these issues.

I did such things:
 - created unified timer driver's API (sys/timeet.h, kernel/kern_et.c).
It supports global and per-CPU timers, periodic and one-shot. Provides
driver and consumer interfaces for choosing timers and operating them;
 - cleaned existing x86 event timer driver's code and modified it for
new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC
timer is now per-CPU and supports both periodic and one-shot modes;
 - extended HPET driver to support it's event timers in periodic and
one-shot mode (dev/acpica/acpi_hpet.c). Support for per-CPU operation
and FSB interrupts planned for later;
 - written mostly machine-independent mid-layer for managing any present
timers to provide clocks needed for kernel (x86/x86/timeevents.c). It
supports both global and per-CPU timers. Now it supports only periodic
mode, but one-shot mode support planned for later.

All this stuff deeply configurable via both loader tunables on boot and
sysctls in real time:

%sysctl kern.eventtimer
kern.eventtimer.choice: LAPIC(500) HPET(400) HPET1(390) HPET2(390)
i8254(100) RTC(0)
kern.eventtimer.et.LAPIC.flags: 7
kern.eventtimer.et.LAPIC.frequency: 99752386
kern.eventtimer.et.LAPIC.quality: 500
kern.eventtimer.et.HPET.flags: 3
kern.eventtimer.et.HPET.frequency: 14318180
kern.eventtimer.et.HPET.quality: 400
kern.eventtimer.et.HPET1.flags: 3
kern.eventtimer.et.HPET1.frequency: 14318180
kern.eventtimer.et.HPET1.quality: 390
kern.eventtimer.et.HPET2.flags: 3
kern.eventtimer.et.HPET2.frequency: 14318180
kern.eventtimer.et.HPET2.quality: 390
kern.eventtimer.et.RTC.flags: 1
kern.eventtimer.et.RTC.frequency: 32768
kern.eventtimer.et.RTC.quality: 0
kern.eventtimer.et.i8254.flags: 1
kern.eventtimer.et.i8254.frequency: 1193182
kern.eventtimer.et.i8254.quality: 100
kern.eventtimer.timer2: NONE
kern.eventtimer.timer1: i8254
kern.eventtimer.singlemul: 2

By default system chooses two timers with highest "quality" for
hardclock and statclock/profclock. User may affect that choice via
disabling unwanted drivers and/or via direct specification of wanted
ones. It is possible to change timers on-flight via sysctls:

%sysctl kern.eventtimer.timer1=hpet
kern.eventtimer.timer1: i8254 -> HPET
%sysctl kern.eventtimer.timer2=hpet1
kern.eventtimer.timer2: NONE -> HPET1

After every timer change, if two timers available, mid-layer
cross-checks them, and if one of them is not functional - replaces it.

If there is no second timer available, or user specified to not use it -
mid-layer automatically increases rate of the first timer and divide
it's frequency to satisfy system needs as good as possible. User may
specify how fast he wish to run fist timer relative to hz by setting
kern.eventtimer.singlemul tunable/sysctl.

When profiling is active, mid-layer automatically rises respective timer
frequency to about 8KHz (was 1KHz previously) and decreases it back on
profiling end.

All above was tested on i386 and amd64. XEN was not affected and builds
fine. pc98 was slightly touched. It wasn't tested, but builds fine. It's
pc98/cbus/clock.c needs respective rewrite to use new features. Other
architectures are untouched, but if any of them may benefit from this
functionality - it should be possible to share most of the code.

Latest patches can be found here:
http://people.freebsd.org/~mav/et.20100606.patch

Known issues:
 - i8254 timer generates 18Hz interrupt rate when not used and not
disabled. I haven't found a way to disable it's interrupt source while
holding spinlock.
 - timer drivers code will need some more cleaning after interrupt
handler will be able to return both argument and frame same time.

Feedback is very appreciated.

--
Alexander Motin



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTikyWXHYezBW1mAlTU0Oi_Z1qlPbAyZllux_wFSB>