From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 18 20:45:51 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id ACB3C106566B; Fri, 18 Mar 2011 20:45:50 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-hackers@freebsd.org Date: Fri, 18 Mar 2011 16:45:40 -0400 User-Agent: KMail/1.6.2 References: <201103171436.22283.jkim@FreeBSD.org> <201103181410.00726.jkim@FreeBSD.org> <20110318201155.GF78089@deviant.kiev.zoral.com.ua> In-Reply-To: <20110318201155.GF78089@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201103181645.41978.jkim@FreeBSD.org> Cc: Kostik Belousov , Bruce Evans , Maxim Dounin Subject: Re: get_cyclecount(9) deprecation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Mar 2011 20:45:51 -0000 On Friday 18 March 2011 04:11 pm, Kostik Belousov wrote: > On Fri, Mar 18, 2011 at 02:09:58PM -0400, Jung-uk Kim wrote: > > On Friday 18 March 2011 01:05 pm, Bruce Evans wrote: > > > On Sat, 19 Mar 2011, Bruce Evans wrote: > > > > On Fri, 18 Mar 2011, Kostik Belousov wrote: > > > >> We definitely do not support configurations with different > > > >> models of CPUs in SMP, this is what Simmetric is about. > > > >> Different as in frequency or stepping. > > > > > > > > ... > > > > Now there is even more asymmetry > > > > in core frequencies, with the hardware transiently slowing > > > > down or stopping cores independently, at least for cores in > > > > different packages. > > > > > > Also, with virtualization, the virtualizer cannot reasonably > > > even provide an invariant TSC that runs at the same rate on all > > > cores. It should provide an invariant TSC that claims to run at > > > the same rate on all cores, but then the cores cannot run at > > > the same rate except on average, since some of the cores will > > > have to run the virtualizer some of the time, and it is > > > unreasonable to distribute the overhead for this evenly except > > > on average. > > > > Ah, virtualization... It is really painful to make it reasonably > > correct. For example, VMware claims that all timers (including > > TSC and excluding RTC) runs at "apparent time", which is concept > > of constant ticks in virtualized guest. It also means TSCs are > > always "virtually" constant and synchronized across all virtual > > cores in guest environment. If it loses periodic timer ticks, > > lost ticks are "compensated" later. This also means timecounter > > does not exactly scale well based on realtime and its frequency > > fluctuates so much, if my understanding is correct: > > > > http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pd > >f > > > > I am not sure how others handle this but VirtualBox gives me > > really wacky TSC frequency sometimes. When it happens, it > > becomes unusably slow. So, I know something is really wrong > > there, too. However, Xen seems to do much smarter than that > > because it has its own concept of virtual TSC, thanks to its > > para-virtualization architecture. > > Most likely, all 'full-hardware' hypervisors have to disable rdtsc > for guests, intercepting the instruction by trap and emulating it. > This means that most basic assumptions about rdtsc are not held > in virtualized environment, like relative cost or accuracy. That's exactly what's happening with "hardware-assisted" virtualization. If it is pure emulation, it may run on arbitrary host CPU (not-translated x86 on x86) or completely artificial (translated) depending on emulators. Therefore, it has zero advantage over other timers. Sometimes, it may be worse. Good news is I added a tunable "machdep.disable_tsc" exactly for that reason. :-) > Anyway, I was unable to make any reasonable use of virtualization > except kernel debugging, which is more then satisfactory performed > by QEMU. Ah, QEMU is not hypervisor. Yeah, I've used QEMU for years for the same reason but its timing is awfully wrong and SMP is not exactly SMP there. :-( Jung-uk Kim