From owner-freebsd-arch@FreeBSD.ORG Wed Jun 6 20:59:46 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56E061065672; Wed, 6 Jun 2012 20:59:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id CB9678FC18; Wed, 6 Jun 2012 20:59:45 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q56Kxct6080208; Wed, 6 Jun 2012 23:59:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q56KxcOL023466; Wed, 6 Jun 2012 23:59:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q56KxckZ023465; Wed, 6 Jun 2012 23:59:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 6 Jun 2012 23:59:38 +0300 From: Konstantin Belousov To: John Baldwin Message-ID: <20120606205938.GS85127@deviant.kiev.zoral.com.ua> References: <20120606165115.GQ85127@deviant.kiev.zoral.com.ua> <201206061423.53179.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qZLIv6EoKi7YuaSc" Content-Disposition: inline In-Reply-To: <201206061423.53179.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-arch@freebsd.org Subject: Re: Fast gettimeofday(2) and clock_gettime(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2012 20:59:46 -0000 --qZLIv6EoKi7YuaSc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 06, 2012 at 02:23:53PM -0400, John Baldwin wrote: > On Wednesday, June 06, 2012 12:51:15 pm Konstantin Belousov wrote: > > A positive result from the recent flame-bait on arch@ is the working > > implementation of the fast gettimeofday(2) and clock_gettime(2). The > > speedup I see is around 6-7x on the 2600K. I think the speedup could > > be even bigger on the previous generation of CPUs, where lock > > operations and syscall entry are costlier. A sample test runs of > > tools/tools/syscall_timing are presented at the end of message. >=20 > In general this looks good but I see a few nits / races: >=20 > 1) You don't follow the model of clearing tk_current to 0 while you > are updating the structure that the in-kernel timecounter code > uses. This also means you have to avoid using a tk_current of 0 > and that userland has to keep spinning as long as tk_current is 0. > Without this I believe userland can read a partially updated > structure. I changed the code to be much more similar to the kern_tc.c. I (re)added the generation field, which is set to 0 upon kernel touching timehands. I think this can only happen if tc_windups occurs quite close in succession, or usermode thread is suspended for long enough. BTW, even generation could loop back to the previous value if thread is stopped. There was apparently another issue with version 2. The bcopy() is not atomic, so potentially libc could read wrong tk_current. I redid the interface to write to the shared page to allow use of real atomics. >=20 > 2) You read tk->tk_boottime without the tk_current protection in your > non-uptime routines. This is racey as the kernel alters the > boottime when it skews time for large adjustments from ntp, etc. > To be really safe you need to read the boottime inside the loop > into a local variable and perhaps use a boolean parameter to decide > if you should add it to the computed uptime. I moved the bootime to timehands from timekeep, thank you for the clarification. >=20 > > sandy% /usr/home/pooma/build/bsd/DEV/stuff/tests/syscall_timing_32=20 > gettimeofday > > Clock resolution: 0.000000076 > > test loop time iterations periteration > > gettimeofday 0 1.000994225 21623297 0.000000046 > > gettimeofday 1 1.000994980 21596492 0.000000046 > > gettimeofday 2 1.001070595 21598326 0.000000046 > > gettimeofday 3 1.000922308 21581398 0.000000046 > > gettimeofday 4 1.000984264 21605539 0.000000046 > > gettimeofday 5 1.000989697 21601659 0.000000046 > > gettimeofday 6 1.000996261 21598385 0.000000046 > > gettimeofday 7 1.001002223 21583933 0.000000046 > > gettimeofday 8 1.000985847 21599442 0.000000046 > > gettimeofday 9 1.000994977 21600935 0.000000046 > > sandy% sudo sysctl kern.timecounter.fast_gettime=3D0 >=20 > I think this means you can call gettimeofday() in about 46 ns now > vs 310 the "old" way? Yes. This is for 32bit, while for 64 bit binaries the numbers are 155->25 ns on the same hw. Updated patch is at=20 http://people.freebsd.org/~kib/misc/moronix.3.patch --qZLIv6EoKi7YuaSc Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk/PxLkACgkQC3+MBN1Mb4jxiwCfcpH7xT549HAK2pcuZFMjR6V7 pjsAoKXKsHQmD+JU5VnKmiUXve1yOlcH =U/tF -----END PGP SIGNATURE----- --qZLIv6EoKi7YuaSc--