Date: Sat, 17 Jul 2010 19:26:03 +0300 From: Alexander Motin <mav@FreeBSD.org> To: Marius Strobl <marius@alchemy.franken.de> Cc: freebsd-sparc64@FreeBSD.org Subject: Re: [RFC] Event timers on sparc64/sun4v Message-ID: <4C41D99B.10202@FreeBSD.org> In-Reply-To: <20100717152459.GU4706@alchemy.franken.de> References: <4C404018.6040405@FreeBSD.org> <20100716213503.GT4706@alchemy.franken.de> <4C40D6F5.6070208@FreeBSD.org> <20100717152459.GU4706@alchemy.franken.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Marius Strobl wrote: > On Sat, Jul 17, 2010 at 01:02:29AM +0300, Alexander Motin wrote: >> Marius Strobl wrote: >>> - using the stick instead of the tick counter for machines with CPUs >>> and thus tick counters running at different speeds has turned out >>> to be suboptimal, probably due to the fact that the 12.5MHz the >>> stick counters typically are driven by don't provide sufficient >>> granularity. >> On x86 ACPI HPET timers often run about 15MHz, i8254 - about 1.2MHz. >> What's wrong with 12.5MHz here? > > When using the stick counter instead of the tick one on machines > consisting of CPUs running at the same speed everything seems fine > except that top(1) TIME output is implausible. Given that with > this setup the only difference between using the stick and the tick > counter is the frequency at which the counter is driven my best > bet is that the stick counter just doesn't provide sufficient > granularity. If it is granularity, then it is caused not by the base frequency. Here is typical x86 timer frequencies: kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.HPET.frequency: 14318180 kern.timecounter.tc.TSC.frequency: 2000085680 , but TSC is not used on SMP. Others have frequencies not higher then stick and still working fine. IMHO while counter is monotonic and it's frequency is higher then frequency of context switches - it should not be important. I would looked for some different reason. > Using the stick counter on machines consisting of CPUs running at > different speeds (well, actually all the combinations of using > stick/tick for hardclock, timecounter, CPU ticker and cycle > counter I tried as they didn't appear totally wrong) additionally > has the problem of processes getting killed as they are diagnosed > to have exceeded their maximum CPU limit, although with the in-tree > code only the timecounter provided by the host-PCI-bridge should > be used for this calculation as far as the MD initialization is > concerned when the stick counter is used to drive hardclock. On my SB100 I've seen only tick timecounter registered. If there is some other timecounter hardware in a system, why it is not registered? It would be much easier to experiment, having more trusted spare parts. >>> Thus the more desireable variant for these machines >>> probably is to provide the tick counter of the BSP as the only >>> non-per-CPU timer and forward it to the APs via IPIs. >> It would be possible if timer was programmable from any CPU. But as I >> understand - it require thread to be binded, which handled by >> infrastructure only for per-CPU timer. > > Wouldn't it be sufficient to bind curthread to the BSP in > tick_et_start() in that case? For one-shot mode this probably > is to much overhead (assuming a tickless kernel) but for > periodic mode IMO this approach should be sound. tick_et_start() is called under spin lock and sometimes critical section. You can't call CPU binding there. For per-CPU timers reconfiguration there is special logic implemented in MI code using IPIs. By the way I have some doubts about tick_get_timecount_mp() correctness. It tries to bind thread to BSP, but what if it is called inside interrupt handler, or under lock, or some else. I have doubt binding will work in that case. >>> This also >>> leaves the stick counter of all >= US-III machines generally >>> available for driving statclock, which likely is also desirable. >> It would be nice, but I don't know how separate their interrupts. > > I think this should be possible in the soft interrupt dispatch. > However, meanwhile it came to my mind that there was a problem > with using the stick counter on US-IIIi machines (which also > only can consist of CPUs running at the same frequency though). Is this hardware working at all? May be there is something wrong by definition or it is misused? >>> - I don't like wasting CPU cycles for determining whether the >>> workaround for BlackBird CPUs is needed (assuming #1 above is >>> implemented so distinguishing stick/tick is no longer needed) >>> with every (s)tick interrupt which are a lot as this just won't >>> ever change during runtime, i.e. I'd like to keep the different >>> interrupt handlers which are set up as needed. >> Does it worth code duplication? Won't it be always cached/ predicted/ >> prefetched? I have doubt that difference can ever be measured, as this >> function is minor part of things done on interrupt. > > I wouldn't be surprised these branches to actually make a measurable > difference; f.e. moving updating the PIL counter from before calling > the tick interrupt handler to incrementing it afterwards reduced the > delay until it's called by 30% on average on a US-II SMP machine, in > turn resulting in a more steady clock and lesser drift which needs > compensation (see r157825). Besides the code already is there, just > don't nuke it :) As you wish. -- Alexander Motin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C41D99B.10202>