From owner-freebsd-arch@FreeBSD.ORG Sun Jun 20 14:46:13 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDABA1065670; Sun, 20 Jun 2010 14:46:13 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 775A98FC18; Sun, 20 Jun 2010 14:46:13 +0000 (UTC) Received: by iwn7 with SMTP id 7so3305606iwn.13 for ; Sun, 20 Jun 2010 07:46:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=uuFrrXNXz0/vR6/rgJgwZSuOMqYfBU8dt3xZAmaVr2Q=; b=lcDicXNMLnsbYrD1z/XNAqbZNVRW5HHCTywZHel30mHFEXrkkwXviRq7ucvzN0/jzy soDA++LiPWUHTdZkdfdYZk/ki3hEUdyy+SmqUJCsGkFyxN5LMplpAT6sYh/xHXLqZBx1 1vm3hq3t2oUZ29F/Pijteo/RrMzJEMxuL6NNA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=xqSVJV+dxySSdOqSCccdH8JkJH5M8vZ4Gbu9natewtTI1H/OrnT7uAv0IejxaL2kFb LtqRb5a9FF9W8FGhEBADjKGsUTC3SWOmmJImTl2FgjLCdxCdtm0hpP2u0O/ySUOTOIdI SGrfwoqRuij8yVRWBgeNcLum3w3VGQc4aaA0o= MIME-Version: 1.0 Received: by 10.231.178.162 with SMTP id bm34mr3833857ibb.86.1277045172594; Sun, 20 Jun 2010 07:46:12 -0700 (PDT) Received: by 10.231.182.212 with HTTP; Sun, 20 Jun 2010 07:46:12 -0700 (PDT) In-Reply-To: <4C1DB296.5040605@FreeBSD.org> References: <4C0C1AE4.8050807@FreeBSD.org> <4C1DB296.5040605@FreeBSD.org> Date: Sun, 20 Jun 2010 09:46:12 -0500 Message-ID: From: Brandon Gooch To: Alexander Motin Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: RFC: New event timers infrastructure X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jun 2010 14:46:14 -0000 On Sun, Jun 20, 2010 at 1:17 AM, Alexander Motin wrote: > Brandon Gooch wrote: >> I've been testing these patches since the first iteration >> (et.20100606), and I haven't discovered any related issues. > > Thank you! > >> I am unclear about the number of interrupts I should expect from the >> hpet0 device (compared to the 99 from the rtc at 100Hz), so here is >> the output of vmstat -i with and without the "et" patches: >> >> With "et" patches: >> >> interrupt total rate >> irq1: atkbd0 369 3 >> irq9: acpi0 961 8 >> irq12: psm0 1002 9 >> irq18: uhci5 140 1 >> irq19: uhci2 ehci0* 4823 45 >> irq20: hpet0 23893 223 >> irq23: uhci3 ehci1 11 0 >> irq256: vgapci0 1031 9 >> irq257: hdac0 14 0 >> irq258: iwn0 4258 39 >> irq259: bge0 1 0 >> Total 36503 341 >> >> Without "et" patches: >> >> interrupt total rate >> irq1: atkbd0 449 2 >> irq0: clk 17334 99 >> irq9: acpi0 1701 9 >> irq12: psm0 8784 50 >> irq18: uhci5 188 1 >> irq19: uhci2 ehci0* 5828 33 >> irq23: uhci3 ehci1 11 0 >> irq256: vgapci0 1896 10 >> irq257: hdac0 14 0 >> irq258: iwn0 29571 169 >> irq259: bge0 1 0 >> Total 65777 378 >> >> And lastly, the values of the kern.eventtimer sysctls: >> >> $ sysctl kern.eventtimer >> kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) HPET3(440) i8254(100) >> kern.eventtimer.timer2: HPET1 >> kern.eventtimer.timer1: HPET >> kern.eventtimer.singlemul: 4 > > I don't see there neither LAPIC, nor RTC timer. I suppose you have > disabled them via device hints. I also suppose you've done it to left > with only 100Hz IRQs of i8254 timer. Now, with the patch, these two are > still disabled, but system got 4 more HPET timers. As they have higher > precedence, two of them were taken (HPET and HPET1). Number if > interrupts can be explained by the line: > > Starting kernel event timers: HPET @ 100Hz, HPET1 @ 128Hz Yes, I had the LAPIC and RTC timers disabled via my /boot/loader.conf, as per your suggestions here: http://wiki.freebsd.org/TuningPowerConsumption > As result, you've got 228Hz IRQs (which then redistributed to every > CPU). As your HPET timer doesn't support FSB interrupts, all it's timers > share same IRQ vector, seen as "hpet0". Understood. > If you wish to get back to 100Hz IRQs as before, you may remove your > previous clock-disabling hints and add instead: > > kern.eventtimer.timer1=HPET > kern.eventtimer.timer2=NONE > kern.eventtimer.singlemul=1 In by /boot/loader.conf, I now have: # Power Saving kern.hz="100" #hint.apic.0.clock="0" #hint.atrtc.0.clock="0" hint.p4tcc.0.disabled="1" hint.p4tcc.1.disabled="1" hint.acpi_throttle.0.disabled="1" hint.acpi_throttle.1.disabled="1" hw.pci.do_power_nodriver="3" In /etc/sysctl.conf, I have as suggested: kern.eventtimer.timer1=HPET kern.eventtimer.timer2=NONE kern.eventtimer.singlemul=1 In my /etc/rc.conf, I'm still using: performance_cpu_freq="NONE" economy_cpu_freq="NONE" performance_cx_lowest="C3" economy_cx_lowest="C3" I'm running powerd: powerd_enable="YES" powerd_flags="-a adaptive -b adaptive -n adaptive" But in my dmesg, I see: # dmesg | grep Starting Starting kernel event timers: LAPIC @ 100Hz, HPET @ 128Hz Starting kernel event timers: LAPIC @ 400Hz, NONE @ 0Hz Starting kernel event timers: HPET @ 200Hz, NONE @ 0Hz So it seems as if the HPET doesn't honor kern.hz setting? Maybe I still need to explicitly disable the apic timers... Would you suggest using LAPIC as opposed to HPET? I have seen power savings being able to use C3 state, but if tickless kernel eventually buys those savings back, perhaps LAPIC would be better... > As result, you will have single timer, running at HZ rate. Instead of > HPET there you may choose any timer: > LAPIC - it is per-CPU, so saves on IPI interrupts, supports one-shot > mode and so suitable for further tickless kernel, but it doesn't work in > C3 state; So if LAPIC is disabled in C3 state, is there a possibility that the "on-the-fly" method of changing clocks could kick in when the CPU enters C3 state, switching to the HPET when it happens? > HPET{x} - on this hardware it can't be used as per-CPU, it supports > one-shot mode, but less suitable for further tickless kernel, as CPUs > can't run independently; Since the HPET supports running while in C3, is there a strategy that could be used to provide a tickless mode with both LAPIC and HPET being used dynamically and interchangebly? I feel as though the questions I'm trying to ask don't make much sense, my apologies :) > i8254 - somewhat faster, as it doesn't needs status reading, but due to > being also a timecounter it can't be used in one-shot mode; > RTC - somewhat slower, has limited set of supported frequencies (powers > of 2), only periodic mode. Anyway, here is a glimpse into the VM interrupts stats using current (stated) configuration: $ vmstat -i interrupt total rate irq1: atkbd0 3968 2 irq9: acpi0 11079 7 irq12: psm0 64878 45 irq18: uhci5 1131 0 irq19: uhci2 ehci0* 7149 5 irq20: hpet0 281935 197 irq23: uhci3 ehci1 11 0 cpu0:timer 4417 3 irq256: vgapci0 31240 21 irq257: hdac0 14 0 irq258: iwn0 39537 27 irq259: bge0 1 0 cpu1:timer 4269 2 Total 449629 315 -Brandon