From owner-freebsd-arch@FreeBSD.ORG Wed Jun 6 16:51:30 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A116106564A for ; Wed, 6 Jun 2012 16:51:30 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6E1118FC26 for ; Wed, 6 Jun 2012 16:51:28 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q56GpGbG040880 for ; Wed, 6 Jun 2012 19:51:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q56GpFvi022201 for ; Wed, 6 Jun 2012 19:51:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q56GpFQI022200 for arch@freebsd.org; Wed, 6 Jun 2012 19:51:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 6 Jun 2012 19:51:15 +0300 From: Konstantin Belousov To: arch@freebsd.org Message-ID: <20120606165115.GQ85127@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kZU6r8y0YpRwyDfh" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Subject: Fast gettimeofday(2) and clock_gettime(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2012 16:51:30 -0000 --kZU6r8y0YpRwyDfh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline A positive result from the recent flame-bait on arch@ is the working implementation of the fast gettimeofday(2) and clock_gettime(2). The speedup I see is around 6-7x on the 2600K. I think the speedup could be even bigger on the previous generation of CPUs, where lock operations and syscall entry are costlier. A sample test runs of tools/tools/syscall_timing are presented at the end of message. Patch finds yet another use for the shared page, exporting time-keeping information for the binuptime(9) algorithm and re-implementing binuptime(9) in userspace. Kernel directs usermode whether the rdtsc instruction can be used, there is a global override sysctl kern.timecounter.fast_gettime to turn it off regardless of hardware capabilities. The whole struct vdso_timekeep is versioned, as well as individual struct vdso_timehands, which should allow to implement future algorithms without breaking binary compatibility. The code is structured to eventually move __vdso_* functions out of libc into VDSO, if it ever materialize. This desire explains vdso prefix and header file names. I implemented and tested the userspace timecounter on amd64, both for 64 and 32 bit binaries, it would probably work for i386 too. Other architecture maintainers are welcome to add neccessary support there. You need to provide machine/vdso.h header with definitions of VDSO_TIMEHANDS_MD fields for struct vdso_timehands, which should provide information for userspace to implement fast tc_get_timecount(). The fields are filled in per-arch cpu_fill_vdso_timehands(9) function. If your architecture support 32bit compat, there are cpu_fill_vdso_timehands32(9) and VDSO_TIMEHANDS_MD32 to code as well. After that, the lib/libc//sys/__vdso_gettc.c should contain an implemention of __vdso_gettc() function, exact analogue of tc_get_timecount(). Another potential improvement for the patch is to start using rdtscp instruction on the CPUs which support it. Then we could correct rdtsc skews between packages, provided kernel starts maintaining this information, instead of refusing to activate tsc timecounter. In particular, on one Nehalem box I see the rdtsc SMP test failing, but Nehalems do have useful rdtsc, so it is could be fixed later. Patch is available at http://people.freebsd.org/~kib/misc/moronix.2.patch It is not a commit candidate yet, since non-x86 architectures are not handled even at compilation, and i386 is not tested. sandy% /usr/home/pooma/build/bsd/DEV/stuff/tests/syscall_timing_32 gettimeofday Clock resolution: 0.000000076 test loop time iterations periteration gettimeofday 0 1.000994225 21623297 0.000000046 gettimeofday 1 1.000994980 21596492 0.000000046 gettimeofday 2 1.001070595 21598326 0.000000046 gettimeofday 3 1.000922308 21581398 0.000000046 gettimeofday 4 1.000984264 21605539 0.000000046 gettimeofday 5 1.000989697 21601659 0.000000046 gettimeofday 6 1.000996261 21598385 0.000000046 gettimeofday 7 1.001002223 21583933 0.000000046 gettimeofday 8 1.000985847 21599442 0.000000046 gettimeofday 9 1.000994977 21600935 0.000000046 sandy% sudo sysctl kern.timecounter.fast_gettime=0 ~ kern.timecounter.fast_gettime: 1 -> 0 sandy% /usr/home/pooma/build/bsd/DEV/stuff/tests/syscall_timing_32 gettimeofday Clock resolution: 0.000000076 test loop time iterations periteration gettimeofday 0 1.001002747 3219274 0.000000310 gettimeofday 1 1.000971052 3220793 0.000000310 gettimeofday 2 1.001067494 3220768 0.000000310 gettimeofday 3 1.000929999 3220812 0.000000310 gettimeofday 4 1.000996106 3217503 0.000000311 gettimeofday 5 1.001058438 3220346 0.000000310 gettimeofday 6 1.000911510 3217308 0.000000311 gettimeofday 7 1.001085906 3220128 0.000000310 gettimeofday 8 1.000920338 3216582 0.000000311 gettimeofday 9 1.000983577 3219559 0.000000310 --kZU6r8y0YpRwyDfh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk/PioMACgkQC3+MBN1Mb4jPzwCfS14QKbr3jY5UhMGJDowJalb/ NrAAoNhv10qQJOytIVY46eOp5IZ3Z9s1 =D2Fs -----END PGP SIGNATURE----- --kZU6r8y0YpRwyDfh--