Date: Tue, 8 Jun 2010 09:06:06 -0500 From: Brandon Gooch <jamesbrandongooch@gmail.com> To: Tsuyoshi Ozawa <ozawa@t-oza.net> Cc: Alexander Motin <mav@freebsd.org>, FreeBSD-Current <freebsd-current@freebsd.org>, freebsd-arch@freebsd.org Subject: Re: RFC: New event timers infrastructure Message-ID: <AANLkTimBrBjtoIIp_wUWp-nanTdvCx6ppwxIEGZSKFco@mail.gmail.com> In-Reply-To: <AANLkTikyWXHYezBW1mAlTU0Oi_Z1qlPbAyZllux_wFSB@mail.gmail.com> References: <4C0C1AE4.8050807@FreeBSD.org> <AANLkTimfyn0aCs7PFRHL8RONvg03_QVXK_90oy084IIx@mail.gmail.com> <AANLkTikyWXHYezBW1mAlTU0Oi_Z1qlPbAyZllux_wFSB@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jun 8, 2010 at 12:24 AM, Tsuyoshi Ozawa <ozawa@t-oza.net> wrote: > 2010/6/6, Alexander Motin <mav@freebsd.org>: >> Hi. >> >> Most of x86 systems now has at least 4 types of event timers: i8254, >> RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily >> hardcoded and absolutely not scalable. I have reimplemented it, trying >> to solve these issues. >> >> I did such things: >> =A0- created unified timer driver's API (sys/timeet.h, kernel/kern_et.c)= . >> It supports global and per-CPU timers, periodic and one-shot. Provides >> driver and consumer interfaces for choosing timers and operating them; >> =A0- cleaned existing x86 event timer driver's code and modified it for >> new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC >> timer is now per-CPU and supports both periodic and one-shot modes; >> =A0- extended HPET driver to support it's event timers in periodic and >> one-shot mode (dev/acpica/acpi_hpet.c). Support for per-CPU operation >> and FSB interrupts planned for later; >> =A0- written mostly machine-independent mid-layer for managing any prese= nt >> timers to provide clocks needed for kernel (x86/x86/timeevents.c). It >> supports both global and per-CPU timers. Now it supports only periodic >> mode, but one-shot mode support planned for later. >> >> All this stuff deeply configurable via both loader tunables on boot and >> sysctls in real time: >> >> %sysctl kern.eventtimer >> kern.eventtimer.choice: LAPIC(500) HPET(400) HPET1(390) HPET2(390) >> i8254(100) RTC(0) >> kern.eventtimer.et.LAPIC.flags: 7 >> kern.eventtimer.et.LAPIC.frequency: 99752386 >> kern.eventtimer.et.LAPIC.quality: 500 >> kern.eventtimer.et.HPET.flags: 3 >> kern.eventtimer.et.HPET.frequency: 14318180 >> kern.eventtimer.et.HPET.quality: 400 >> kern.eventtimer.et.HPET1.flags: 3 >> kern.eventtimer.et.HPET1.frequency: 14318180 >> kern.eventtimer.et.HPET1.quality: 390 >> kern.eventtimer.et.HPET2.flags: 3 >> kern.eventtimer.et.HPET2.frequency: 14318180 >> kern.eventtimer.et.HPET2.quality: 390 >> kern.eventtimer.et.RTC.flags: 1 >> kern.eventtimer.et.RTC.frequency: 32768 >> kern.eventtimer.et.RTC.quality: 0 >> kern.eventtimer.et.i8254.flags: 1 >> kern.eventtimer.et.i8254.frequency: 1193182 >> kern.eventtimer.et.i8254.quality: 100 >> kern.eventtimer.timer2: NONE >> kern.eventtimer.timer1: i8254 >> kern.eventtimer.singlemul: 2 >> >> By default system chooses two timers with highest "quality" for >> hardclock and statclock/profclock. User may affect that choice via >> disabling unwanted drivers and/or via direct specification of wanted >> ones. It is possible to change timers on-flight via sysctls: >> >> %sysctl kern.eventtimer.timer1=3Dhpet >> kern.eventtimer.timer1: i8254 -> HPET >> %sysctl kern.eventtimer.timer2=3Dhpet1 >> kern.eventtimer.timer2: NONE -> HPET1 >> >> After every timer change, if two timers available, mid-layer >> cross-checks them, and if one of them is not functional - replaces it. >> >> If there is no second timer available, or user specified to not use it - >> mid-layer automatically increases rate of the first timer and divide >> it's frequency to satisfy system needs as good as possible. User may >> specify how fast he wish to run fist timer relative to hz by setting >> kern.eventtimer.singlemul tunable/sysctl. >> >> When profiling is active, mid-layer automatically rises respective timer >> frequency to about 8KHz (was 1KHz previously) and decreases it back on >> profiling end. >> >> All above was tested on i386 and amd64. XEN was not affected and builds >> fine. pc98 was slightly touched. It wasn't tested, but builds fine. It's >> pc98/cbus/clock.c needs respective rewrite to use new features. Other >> architectures are untouched, but if any of them may benefit from this >> functionality - it should be possible to share most of the code. >> >> Latest patches can be found here: >> http://people.freebsd.org/~mav/et.20100606.patch >> >> Known issues: >> =A0- i8254 timer generates 18Hz interrupt rate when not used and not >> disabled. I haven't found a way to disable it's interrupt source while >> holding spinlock. >> =A0- timer drivers code will need some more cleaning after interrupt >> handler will be able to return both argument and frame same time. >> >> Feedback is very appreciated. >> >> -- >> Alexander Motin >> > > This is excellent! > I'll try this to apply this patch and =A0rewrite my old dynamic ticks cod= e to > fit this event timer API . > > Thank you, Alexander ! > -- > Tsuyoshi Ozawa > <ozawa@t-oza.net> > > 2010/06/07 7:03 "Alexander Motin" <mav@freebsd.org>: > > Hi. > > Most of x86 systems now has at least 4 types of event timers: i8254, > RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily > hardcoded and absolutely not scalable. I have reimplemented it, trying > to solve these issues. > > I did such things: > =A0- created unified timer driver's API (sys/timeet.h, kernel/kern_et.c). > It supports global and per-CPU timers, periodic and one-shot. Provides > driver and consumer interfaces for choosing timers and operating them; > =A0- cleaned existing x86 event timer driver's code and modified it for > new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC > timer is now per-CPU and supports both periodic and one-shot modes; > =A0- extended HPET driver to support it's event timers in periodic and > one-shot mode (dev/acpica/acpi_hpet.c). Support for per-CPU operation > and FSB interrupts planned for later; > =A0- written mostly machine-independent mid-layer for managing any presen= t > timers to provide clocks needed for kernel (x86/x86/timeevents.c). It > supports both global and per-CPU timers. Now it supports only periodic > mode, but one-shot mode support planned for later. > > All this stuff deeply configurable via both loader tunables on boot and > sysctls in real time: > > %sysctl kern.eventtimer > kern.eventtimer.choice: LAPIC(500) HPET(400) HPET1(390) HPET2(390) > i8254(100) RTC(0) > kern.eventtimer.et.LAPIC.flags: 7 > kern.eventtimer.et.LAPIC.frequency: 99752386 > kern.eventtimer.et.LAPIC.quality: 500 > kern.eventtimer.et.HPET.flags: 3 > kern.eventtimer.et.HPET.frequency: 14318180 > kern.eventtimer.et.HPET.quality: 400 > kern.eventtimer.et.HPET1.flags: 3 > kern.eventtimer.et.HPET1.frequency: 14318180 > kern.eventtimer.et.HPET1.quality: 390 > kern.eventtimer.et.HPET2.flags: 3 > kern.eventtimer.et.HPET2.frequency: 14318180 > kern.eventtimer.et.HPET2.quality: 390 > kern.eventtimer.et.RTC.flags: 1 > kern.eventtimer.et.RTC.frequency: 32768 > kern.eventtimer.et.RTC.quality: 0 > kern.eventtimer.et.i8254.flags: 1 > kern.eventtimer.et.i8254.frequency: 1193182 > kern.eventtimer.et.i8254.quality: 100 > kern.eventtimer.timer2: NONE > kern.eventtimer.timer1: i8254 > kern.eventtimer.singlemul: 2 > > By default system chooses two timers with highest "quality" for > hardclock and statclock/profclock. User may affect that choice via > disabling unwanted drivers and/or via direct specification of wanted > ones. It is possible to change timers on-flight via sysctls: > > %sysctl kern.eventtimer.timer1=3Dhpet > kern.eventtimer.timer1: i8254 -> HPET > %sysctl kern.eventtimer.timer2=3Dhpet1 > kern.eventtimer.timer2: NONE -> HPET1 > > After every timer change, if two timers available, mid-layer > cross-checks them, and if one of them is not functional - replaces it. > > If there is no second timer available, or user specified to not use it - > mid-layer automatically increases rate of the first timer and divide > it's frequency to satisfy system needs as good as possible. User may > specify how fast he wish to run fist timer relative to hz by setting > kern.eventtimer.singlemul tunable/sysctl. > > When profiling is active, mid-layer automatically rises respective timer > frequency to about 8KHz (was 1KHz previously) and decreases it back on > profiling end. > > All above was tested on i386 and amd64. XEN was not affected and builds > fine. pc98 was slightly touched. It wasn't tested, but builds fine. It's > pc98/cbus/clock.c needs respective rewrite to use new features. Other > architectures are untouched, but if any of them may benefit from this > functionality - it should be possible to share most of the code. > > Latest patches can be found here: > http://people.freebsd.org/~mav/et.20100606.patch > > Known issues: > =A0- i8254 timer generates 18Hz interrupt rate when not used and not > disabled. I haven't found a way to disable it's interrupt source while > holding spinlock. > =A0- timer drivers code will need some more cleaning after interrupt > handler will be able to return both argument and frame same time. > > Feedback is very appreciated. > > -- > Alexander Motin This is an exciting advance for FreeBSD, and I'm looking forward to testing the new infrastructure code. Alexander, do you feel that the code is at a stage where meaningful user testing can occur? Tsuyoshi, will you be providing code to Alexander to integrate into his patch(es), or will your development continue as a separate effort? Thank you both so much for taking on this task! -Brandon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTimBrBjtoIIp_wUWp-nanTdvCx6ppwxIEGZSKFco>