From owner-svn-src-all@freebsd.org Fri Feb 15 11:33:49 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3DA214D758B; Fri, 15 Feb 2019 11:33:49 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 432D48CD86; Fri, 15 Feb 2019 11:33:49 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1D2481E080; Fri, 15 Feb 2019 11:33:49 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x1FBXmZN060956; Fri, 15 Feb 2019 11:33:48 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x1FBXmHU060955; Fri, 15 Feb 2019 11:33:48 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201902151133.x1FBXmHU060955@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 15 Feb 2019 11:33:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r344157 - stable/12/lib/libc/x86/sys X-SVN-Group: stable-12 X-SVN-Commit-Author: kib X-SVN-Commit-Paths: stable/12/lib/libc/x86/sys X-SVN-Commit-Revision: 344157 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 432D48CD86 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.966,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2019 11:33:49 -0000 Author: kib Date: Fri Feb 15 11:33:48 2019 New Revision: 344157 URL: https://svnweb.freebsd.org/changeset/base/344157 Log: MFC r343855, r343859: Use ifunc to select the barrier instruction for RDTSC. Modified: stable/12/lib/libc/x86/sys/__vdso_gettc.c Directory Properties: stable/12/ (props changed) Modified: stable/12/lib/libc/x86/sys/__vdso_gettc.c ============================================================================== --- stable/12/lib/libc/x86/sys/__vdso_gettc.c Fri Feb 15 11:28:32 2019 (r344156) +++ stable/12/lib/libc/x86/sys/__vdso_gettc.c Fri Feb 15 11:33:48 2019 (r344157) @@ -1,6 +1,6 @@ /*- * Copyright (c) 2012 Konstantin Belousov - * Copyright (c) 2016, 2017 The FreeBSD Foundation + * Copyright (c) 2016, 2017, 2019 The FreeBSD Foundation * All rights reserved. * * Portions of this software were developed by Konstantin Belousov @@ -50,15 +50,9 @@ __FBSDID("$FreeBSD$"); #ifdef WANT_HYPERV #include #endif +#include #include "libc_private.h" -static enum LMB { - LMB_UNKNOWN, - LMB_NONE, - LMB_MFENCE, - LMB_LFENCE -} lfence_works = LMB_UNKNOWN; - static void cpuidp(u_int leaf, u_int p[4]) { @@ -84,68 +78,36 @@ cpuidp(u_int leaf, u_int p[4]) : "0" (leaf)); } -static enum LMB -select_lmb(void) +static void +rdtsc_mb_lfence(void) { - u_int p[4]; - static const char intel_id[] = "GenuntelineI"; - cpuidp(0, p); - return (memcmp(p + 1, intel_id, sizeof(intel_id) - 1) == 0 ? - LMB_LFENCE : LMB_MFENCE); + lfence(); } static void -init_fence(void) +rdtsc_mb_mfence(void) { -#if defined(__i386__) - u_int cpuid_supported, p[4]; - lfence_works = LMB_NONE; - __asm __volatile( - " pushfl\n" - " popl %%eax\n" - " movl %%eax,%%ecx\n" - " xorl $0x200000,%%eax\n" - " pushl %%eax\n" - " popfl\n" - " pushfl\n" - " popl %%eax\n" - " xorl %%eax,%%ecx\n" - " je 1f\n" - " movl $1,%0\n" - " jmp 2f\n" - "1: movl $0,%0\n" - "2:\n" - : "=r" (cpuid_supported) : : "eax", "ecx", "cc"); - if (cpuid_supported) { - cpuidp(0x1, p); - if ((p[3] & CPUID_SSE2) != 0) - lfence_works = select_lmb(); - } -#elif defined(__amd64__) - lfence_works = select_lmb(); -#else -#error "Arch" -#endif + mfence(); } static void -rdtsc_mb(void) +rdtsc_mb_none(void) { +} -again: - if (__predict_true(lfence_works == LMB_LFENCE)) { - lfence(); - return; - } else if (lfence_works == LMB_MFENCE) { - mfence(); - return; - } else if (lfence_works == LMB_NONE) { - return; - } - init_fence(); - goto again; +DEFINE_UIFUNC(static, void, rdtsc_mb, (void), static) +{ + u_int p[4]; + /* Not a typo, string matches our cpuidp() registers use. */ + static const char intel_id[] = "GenuntelineI"; + + if ((cpu_feature & CPUID_SSE2) == 0) + return (rdtsc_mb_none); + cpuidp(0, p); + return (memcmp(p + 1, intel_id, sizeof(intel_id) - 1) == 0 ? + rdtsc_mb_lfence : rdtsc_mb_mfence); } static u_int