From owner-freebsd-arch@FreeBSD.ORG Wed Jun 6 17:06:03 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7486A1065686 for ; Wed, 6 Jun 2012 17:06:03 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id E91A28FC14 for ; Wed, 6 Jun 2012 17:06:02 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id F3CA17300A; Wed, 6 Jun 2012 19:24:39 +0200 (CEST) Date: Wed, 6 Jun 2012 19:24:39 +0200 From: Luigi Rizzo To: Konstantin Belousov Message-ID: <20120606172439.GA42362@onelab2.iet.unipi.it> References: <20120606165115.GQ85127@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120606165115.GQ85127@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.4.2.3i Cc: arch@freebsd.org Subject: Re: Fast gettimeofday(2) and clock_gettime(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2012 17:06:03 -0000 On Wed, Jun 06, 2012 at 07:51:15PM +0300, Konstantin Belousov wrote: > A positive result from the recent flame-bait on arch@ is the working > implementation of the fast gettimeofday(2) and clock_gettime(2). The great job! congratulations and thanks for this work! cheers luigi > speedup I see is around 6-7x on the 2600K. I think the speedup could > be even bigger on the previous generation of CPUs, where lock > operations and syscall entry are costlier. A sample test runs of > tools/tools/syscall_timing are presented at the end of message. > > Patch finds yet another use for the shared page, exporting > time-keeping information for the binuptime(9) algorithm and > re-implementing binuptime(9) in userspace. Kernel directs usermode > whether the rdtsc instruction can be used, there is a global override > sysctl kern.timecounter.fast_gettime to turn it off regardless of > hardware capabilities. > > The whole struct vdso_timekeep is versioned, as well as individual > struct vdso_timehands, which should allow to implement future > algorithms without breaking binary compatibility. The code is > structured to eventually move __vdso_* functions out of libc into > VDSO, if it ever materialize. This desire explains vdso prefix and > header file names. > > I implemented and tested the userspace timecounter on amd64, both for > 64 and 32 bit binaries, it would probably work for i386 too. Other > architecture maintainers are welcome to add neccessary support there. > You need to provide machine/vdso.h header with definitions of > VDSO_TIMEHANDS_MD fields for struct vdso_timehands, which should > provide information for userspace to implement fast > tc_get_timecount(). The fields are filled in per-arch > cpu_fill_vdso_timehands(9) function. If your architecture support > 32bit compat, there are cpu_fill_vdso_timehands32(9) and > VDSO_TIMEHANDS_MD32 to code as well. After that, the > lib/libc//sys/__vdso_gettc.c should contain an implemention of > __vdso_gettc() function, exact analogue of tc_get_timecount(). > > Another potential improvement for the patch is to start using rdtscp > instruction on the CPUs which support it. Then we could correct rdtsc > skews between packages, provided kernel starts maintaining this > information, instead of refusing to activate tsc timecounter. In > particular, on one Nehalem box I see the rdtsc SMP test failing, but > Nehalems do have useful rdtsc, so it is could be fixed later. > > Patch is available at http://people.freebsd.org/~kib/misc/moronix.2.patch > It is not a commit candidate yet, since non-x86 architectures are not > handled even at compilation, and i386 is not tested. > > sandy% /usr/home/pooma/build/bsd/DEV/stuff/tests/syscall_timing_32 gettimeofday > Clock resolution: 0.000000076 > test loop time iterations periteration > gettimeofday 0 1.000994225 21623297 0.000000046 > gettimeofday 1 1.000994980 21596492 0.000000046 > gettimeofday 2 1.001070595 21598326 0.000000046 > gettimeofday 3 1.000922308 21581398 0.000000046 > gettimeofday 4 1.000984264 21605539 0.000000046 > gettimeofday 5 1.000989697 21601659 0.000000046 > gettimeofday 6 1.000996261 21598385 0.000000046 > gettimeofday 7 1.001002223 21583933 0.000000046 > gettimeofday 8 1.000985847 21599442 0.000000046 > gettimeofday 9 1.000994977 21600935 0.000000046 > sandy% sudo sysctl kern.timecounter.fast_gettime=0 ~ > kern.timecounter.fast_gettime: 1 -> 0 > sandy% /usr/home/pooma/build/bsd/DEV/stuff/tests/syscall_timing_32 gettimeofday > Clock resolution: 0.000000076 > test loop time iterations periteration > gettimeofday 0 1.001002747 3219274 0.000000310 > gettimeofday 1 1.000971052 3220793 0.000000310 > gettimeofday 2 1.001067494 3220768 0.000000310 > gettimeofday 3 1.000929999 3220812 0.000000310 > gettimeofday 4 1.000996106 3217503 0.000000311 > gettimeofday 5 1.001058438 3220346 0.000000310 > gettimeofday 6 1.000911510 3217308 0.000000311 > gettimeofday 7 1.001085906 3220128 0.000000310 > gettimeofday 8 1.000920338 3216582 0.000000311 > gettimeofday 9 1.000983577 3219559 0.000000310 >