From owner-freebsd-current@FreeBSD.ORG Tue Jun 8 14:06:07 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE73F106566C; Tue, 8 Jun 2010 14:06:07 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-yw0-f182.google.com (mail-yw0-f182.google.com [209.85.211.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5DB438FC23; Tue, 8 Jun 2010 14:06:07 +0000 (UTC) Received: by ywh12 with SMTP id 12so3337230ywh.14 for ; Tue, 08 Jun 2010 07:06:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=0Lps0RBloQ/dEHpYgMnL5AS2Yzqli+7NPkNobv2w+ak=; b=C0Qx2VihVzcu4Sn9ri/wJZiiw8ef2kpk0i9sTQ5CaHZjbIJbSlTCGPhsRGCxSWbkRO Ib/+vPPUmvrEA8QCM+7bYt50LuIrd4RdqvqhOCs5HwVJ77e7XgntXqFPAxhhgbSVxv/9 dCXJsXziQrEap2+LfjwCfsd2pRaOExP49RP4Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=w/8pN8OxTOvlqPI3M0TPVZycBLsNWbAXeKdB7kw2clJ1vyEjGBAYhxo85Uoty7ls7N dtKty+zJaiMNq+7aNFsyf1qRUozU7jeXf0H3iM1hOwTPDp4UkU/m+rQezpAzDlBWWNId ADtGo2VVzRywe2e9CEv+KWSPg+TJCUlyD29Xg= MIME-Version: 1.0 Received: by 10.101.177.16 with SMTP id e16mr16125636anp.221.1276005966451; Tue, 08 Jun 2010 07:06:06 -0700 (PDT) Received: by 10.231.182.196 with HTTP; Tue, 8 Jun 2010 07:06:06 -0700 (PDT) In-Reply-To: References: <4C0C1AE4.8050807@FreeBSD.org> Date: Tue, 8 Jun 2010 09:06:06 -0500 Message-ID: From: Brandon Gooch To: Tsuyoshi Ozawa Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Alexander Motin , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: RFC: New event timers infrastructure X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2010 14:06:07 -0000 On Tue, Jun 8, 2010 at 12:24 AM, Tsuyoshi Ozawa wrote: > 2010/6/6, Alexander Motin : >> Hi. >> >> Most of x86 systems now has at least 4 types of event timers: i8254, >> RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily >> hardcoded and absolutely not scalable. I have reimplemented it, trying >> to solve these issues. >> >> I did such things: >> =A0- created unified timer driver's API (sys/timeet.h, kernel/kern_et.c)= . >> It supports global and per-CPU timers, periodic and one-shot. Provides >> driver and consumer interfaces for choosing timers and operating them; >> =A0- cleaned existing x86 event timer driver's code and modified it for >> new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC >> timer is now per-CPU and supports both periodic and one-shot modes; >> =A0- extended HPET driver to support it's event timers in periodic and >> one-shot mode (dev/acpica/acpi_hpet.c). Support for per-CPU operation >> and FSB interrupts planned for later; >> =A0- written mostly machine-independent mid-layer for managing any prese= nt >> timers to provide clocks needed for kernel (x86/x86/timeevents.c). It >> supports both global and per-CPU timers. Now it supports only periodic >> mode, but one-shot mode support planned for later. >> >> All this stuff deeply configurable via both loader tunables on boot and >> sysctls in real time: >> >> %sysctl kern.eventtimer >> kern.eventtimer.choice: LAPIC(500) HPET(400) HPET1(390) HPET2(390) >> i8254(100) RTC(0) >> kern.eventtimer.et.LAPIC.flags: 7 >> kern.eventtimer.et.LAPIC.frequency: 99752386 >> kern.eventtimer.et.LAPIC.quality: 500 >> kern.eventtimer.et.HPET.flags: 3 >> kern.eventtimer.et.HPET.frequency: 14318180 >> kern.eventtimer.et.HPET.quality: 400 >> kern.eventtimer.et.HPET1.flags: 3 >> kern.eventtimer.et.HPET1.frequency: 14318180 >> kern.eventtimer.et.HPET1.quality: 390 >> kern.eventtimer.et.HPET2.flags: 3 >> kern.eventtimer.et.HPET2.frequency: 14318180 >> kern.eventtimer.et.HPET2.quality: 390 >> kern.eventtimer.et.RTC.flags: 1 >> kern.eventtimer.et.RTC.frequency: 32768 >> kern.eventtimer.et.RTC.quality: 0 >> kern.eventtimer.et.i8254.flags: 1 >> kern.eventtimer.et.i8254.frequency: 1193182 >> kern.eventtimer.et.i8254.quality: 100 >> kern.eventtimer.timer2: NONE >> kern.eventtimer.timer1: i8254 >> kern.eventtimer.singlemul: 2 >> >> By default system chooses two timers with highest "quality" for >> hardclock and statclock/profclock. User may affect that choice via >> disabling unwanted drivers and/or via direct specification of wanted >> ones. It is possible to change timers on-flight via sysctls: >> >> %sysctl kern.eventtimer.timer1=3Dhpet >> kern.eventtimer.timer1: i8254 -> HPET >> %sysctl kern.eventtimer.timer2=3Dhpet1 >> kern.eventtimer.timer2: NONE -> HPET1 >> >> After every timer change, if two timers available, mid-layer >> cross-checks them, and if one of them is not functional - replaces it. >> >> If there is no second timer available, or user specified to not use it - >> mid-layer automatically increases rate of the first timer and divide >> it's frequency to satisfy system needs as good as possible. User may >> specify how fast he wish to run fist timer relative to hz by setting >> kern.eventtimer.singlemul tunable/sysctl. >> >> When profiling is active, mid-layer automatically rises respective timer >> frequency to about 8KHz (was 1KHz previously) and decreases it back on >> profiling end. >> >> All above was tested on i386 and amd64. XEN was not affected and builds >> fine. pc98 was slightly touched. It wasn't tested, but builds fine. It's >> pc98/cbus/clock.c needs respective rewrite to use new features. Other >> architectures are untouched, but if any of them may benefit from this >> functionality - it should be possible to share most of the code. >> >> Latest patches can be found here: >> http://people.freebsd.org/~mav/et.20100606.patch >> >> Known issues: >> =A0- i8254 timer generates 18Hz interrupt rate when not used and not >> disabled. I haven't found a way to disable it's interrupt source while >> holding spinlock. >> =A0- timer drivers code will need some more cleaning after interrupt >> handler will be able to return both argument and frame same time. >> >> Feedback is very appreciated. >> >> -- >> Alexander Motin >> > > This is excellent! > I'll try this to apply this patch and =A0rewrite my old dynamic ticks cod= e to > fit this event timer API . > > Thank you, Alexander ! > -- > Tsuyoshi Ozawa > > > 2010/06/07 7:03 "Alexander Motin" : > > Hi. > > Most of x86 systems now has at least 4 types of event timers: i8254, > RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily > hardcoded and absolutely not scalable. I have reimplemented it, trying > to solve these issues. > > I did such things: > =A0- created unified timer driver's API (sys/timeet.h, kernel/kern_et.c). > It supports global and per-CPU timers, periodic and one-shot. Provides > driver and consumer interfaces for choosing timers and operating them; > =A0- cleaned existing x86 event timer driver's code and modified it for > new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC > timer is now per-CPU and supports both periodic and one-shot modes; > =A0- extended HPET driver to support it's event timers in periodic and > one-shot mode (dev/acpica/acpi_hpet.c). Support for per-CPU operation > and FSB interrupts planned for later; > =A0- written mostly machine-independent mid-layer for managing any presen= t > timers to provide clocks needed for kernel (x86/x86/timeevents.c). It > supports both global and per-CPU timers. Now it supports only periodic > mode, but one-shot mode support planned for later. > > All this stuff deeply configurable via both loader tunables on boot and > sysctls in real time: > > %sysctl kern.eventtimer > kern.eventtimer.choice: LAPIC(500) HPET(400) HPET1(390) HPET2(390) > i8254(100) RTC(0) > kern.eventtimer.et.LAPIC.flags: 7 > kern.eventtimer.et.LAPIC.frequency: 99752386 > kern.eventtimer.et.LAPIC.quality: 500 > kern.eventtimer.et.HPET.flags: 3 > kern.eventtimer.et.HPET.frequency: 14318180 > kern.eventtimer.et.HPET.quality: 400 > kern.eventtimer.et.HPET1.flags: 3 > kern.eventtimer.et.HPET1.frequency: 14318180 > kern.eventtimer.et.HPET1.quality: 390 > kern.eventtimer.et.HPET2.flags: 3 > kern.eventtimer.et.HPET2.frequency: 14318180 > kern.eventtimer.et.HPET2.quality: 390 > kern.eventtimer.et.RTC.flags: 1 > kern.eventtimer.et.RTC.frequency: 32768 > kern.eventtimer.et.RTC.quality: 0 > kern.eventtimer.et.i8254.flags: 1 > kern.eventtimer.et.i8254.frequency: 1193182 > kern.eventtimer.et.i8254.quality: 100 > kern.eventtimer.timer2: NONE > kern.eventtimer.timer1: i8254 > kern.eventtimer.singlemul: 2 > > By default system chooses two timers with highest "quality" for > hardclock and statclock/profclock. User may affect that choice via > disabling unwanted drivers and/or via direct specification of wanted > ones. It is possible to change timers on-flight via sysctls: > > %sysctl kern.eventtimer.timer1=3Dhpet > kern.eventtimer.timer1: i8254 -> HPET > %sysctl kern.eventtimer.timer2=3Dhpet1 > kern.eventtimer.timer2: NONE -> HPET1 > > After every timer change, if two timers available, mid-layer > cross-checks them, and if one of them is not functional - replaces it. > > If there is no second timer available, or user specified to not use it - > mid-layer automatically increases rate of the first timer and divide > it's frequency to satisfy system needs as good as possible. User may > specify how fast he wish to run fist timer relative to hz by setting > kern.eventtimer.singlemul tunable/sysctl. > > When profiling is active, mid-layer automatically rises respective timer > frequency to about 8KHz (was 1KHz previously) and decreases it back on > profiling end. > > All above was tested on i386 and amd64. XEN was not affected and builds > fine. pc98 was slightly touched. It wasn't tested, but builds fine. It's > pc98/cbus/clock.c needs respective rewrite to use new features. Other > architectures are untouched, but if any of them may benefit from this > functionality - it should be possible to share most of the code. > > Latest patches can be found here: > http://people.freebsd.org/~mav/et.20100606.patch > > Known issues: > =A0- i8254 timer generates 18Hz interrupt rate when not used and not > disabled. I haven't found a way to disable it's interrupt source while > holding spinlock. > =A0- timer drivers code will need some more cleaning after interrupt > handler will be able to return both argument and frame same time. > > Feedback is very appreciated. > > -- > Alexander Motin This is an exciting advance for FreeBSD, and I'm looking forward to testing the new infrastructure code. Alexander, do you feel that the code is at a stage where meaningful user testing can occur? Tsuyoshi, will you be providing code to Alexander to integrate into his patch(es), or will your development continue as a separate effort? Thank you both so much for taking on this task! -Brandon