From owner-svn-src-all@freebsd.org Mon Aug 21 17:39:14 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5713DEA383; Mon, 21 Aug 2017 17:39:14 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F94F7D1E5; Mon, 21 Aug 2017 17:39:14 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v7LHdDlG023812; Mon, 21 Aug 2017 17:39:13 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v7LHdD6k023805; Mon, 21 Aug 2017 17:39:13 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201708211739.v7LHdD6k023805@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Mon, 21 Aug 2017 17:39:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r322763 - head/lib/libc/amd64/sys X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/lib/libc/amd64/sys X-SVN-Commit-Revision: 322763 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Aug 2017 17:39:14 -0000 Author: kib Date: Mon Aug 21 17:39:12 2017 New Revision: 322763 URL: https://svnweb.freebsd.org/changeset/base/322763 Log: Optimize libc to get and set TLS using the RDFSBASE and RDGSBASE instructions, if supported both by CPU and kernel. Reviewed by: jhb (previous version) Tested by: pho (previous version) Sponsored by: The FreeBSD Foundation MFC after: 3 weeks Differential revision: https://reviews.freebsd.org/D12023 Added: head/lib/libc/amd64/sys/amd64_detect_rdfsgsbase.c (contents, props changed) head/lib/libc/amd64/sys/amd64_detect_rdfsgsbase.h (contents, props changed) Modified: head/lib/libc/amd64/sys/Makefile.inc head/lib/libc/amd64/sys/amd64_get_fsbase.c head/lib/libc/amd64/sys/amd64_get_gsbase.c head/lib/libc/amd64/sys/amd64_set_fsbase.c head/lib/libc/amd64/sys/amd64_set_gsbase.c Modified: head/lib/libc/amd64/sys/Makefile.inc ============================================================================== --- head/lib/libc/amd64/sys/Makefile.inc Mon Aug 21 17:38:02 2017 (r322762) +++ head/lib/libc/amd64/sys/Makefile.inc Mon Aug 21 17:39:12 2017 (r322763) @@ -1,7 +1,11 @@ # from: Makefile.inc,v 1.1 1993/09/03 19:04:23 jtc Exp # $FreeBSD$ -SRCS+= amd64_get_fsbase.c amd64_get_gsbase.c amd64_set_fsbase.c \ +SRCS+= \ + amd64_detect_rdfsgsbase.c \ + amd64_get_fsbase.c \ + amd64_get_gsbase.c \ + amd64_set_fsbase.c \ amd64_set_gsbase.c MDASM= vfork.S brk.S cerror.S exect.S getcontext.S \ Added: head/lib/libc/amd64/sys/amd64_detect_rdfsgsbase.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lib/libc/amd64/sys/amd64_detect_rdfsgsbase.c Mon Aug 21 17:39:12 2017 (r322763) @@ -0,0 +1,63 @@ +/*- + * Copyright (c) 2017 The FreeBSD Foundation + * All rights reserved. + * + * This software was developed by Konstantin Belousov + * under sponsorship from the FreeBSD Foundation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#define IN_RTLD 1 +#include +#undef IN_RTLD +#include +#include +#include "amd64_detect_rdfsgsbase.h" +#include "libc_private.h" + +static int state = RDFSGS_UNKNOWN; + +int +amd64_detect_rdfsgsbase(void) +{ + u_int p[4]; + + if (__predict_true(state != RDFSGS_UNKNOWN)) + return (state); + + if (__getosreldate() >= P_OSREL_WRFSBASE) { + do_cpuid(0x0, p); + if (p[0] >= 0x7) { + cpuid_count(0x7, 0x0, p); + if ((p[1] & CPUID_STDEXT_FSGSBASE) != 0) { + state = RDFSGS_SUPPORTED; + return (state); + } + } + } + state = RDFSGS_UNSUPPORTED; + return (state); +} Added: head/lib/libc/amd64/sys/amd64_detect_rdfsgsbase.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lib/libc/amd64/sys/amd64_detect_rdfsgsbase.h Mon Aug 21 17:39:12 2017 (r322763) @@ -0,0 +1,43 @@ +/*- + * Copyright (c) 2017 The FreeBSD Foundation + * All rights reserved. + * + * This software was developed by Konstantin Belousov + * under sponsorship from the FreeBSD Foundation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _AMD64_DETECT_RDFSGSBASE_H_ +#define _AMD64_DETECT_RDFSGSBASE_H_ + +enum { + RDFSGS_UNKNOWN, + RDFSGS_SUPPORTED, + RDFSGS_UNSUPPORTED, +}; + +int amd64_detect_rdfsgsbase(void); + +#endif Modified: head/lib/libc/amd64/sys/amd64_get_fsbase.c ============================================================================== --- head/lib/libc/amd64/sys/amd64_get_fsbase.c Mon Aug 21 17:38:02 2017 (r322762) +++ head/lib/libc/amd64/sys/amd64_get_fsbase.c Mon Aug 21 17:39:12 2017 (r322763) @@ -1,7 +1,11 @@ /*- * Copyright (c) 2003 Peter Wemm + * Copyright (c) 2017 The FreeBSD Foundation * All rights reserved. * + * Portions of this software were developed by Konstantin Belousov + * under sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -27,11 +31,18 @@ #include __FBSDID("$FreeBSD$"); +#include +#include #include +#include "amd64_detect_rdfsgsbase.h" int amd64_get_fsbase(void **addr) { + if (amd64_detect_rdfsgsbase() == RDFSGS_SUPPORTED) { + *addr = (void *)rdfsbase(); + return (0); + } return (sysarch(AMD64_GET_FSBASE, addr)); } Modified: head/lib/libc/amd64/sys/amd64_get_gsbase.c ============================================================================== --- head/lib/libc/amd64/sys/amd64_get_gsbase.c Mon Aug 21 17:38:02 2017 (r322762) +++ head/lib/libc/amd64/sys/amd64_get_gsbase.c Mon Aug 21 17:39:12 2017 (r322763) @@ -1,7 +1,11 @@ /*- * Copyright (c) 2003 Peter Wemm + * Copyright (c) 2017 The FreeBSD Foundation * All rights reserved. * + * Portions of this software were developed by Konstantin Belousov + * under sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -27,11 +31,18 @@ #include __FBSDID("$FreeBSD$"); +#include +#include #include +#include "amd64_detect_rdfsgsbase.h" int amd64_get_gsbase(void **addr) { + if (amd64_detect_rdfsgsbase() == RDFSGS_SUPPORTED) { + *addr = (void *)rdgsbase(); + return (0); + } return (sysarch(AMD64_GET_GSBASE, addr)); } Modified: head/lib/libc/amd64/sys/amd64_set_fsbase.c ============================================================================== --- head/lib/libc/amd64/sys/amd64_set_fsbase.c Mon Aug 21 17:38:02 2017 (r322762) +++ head/lib/libc/amd64/sys/amd64_set_fsbase.c Mon Aug 21 17:39:12 2017 (r322763) @@ -1,7 +1,11 @@ /*- * Copyright (c) 2003 Peter Wemm + * Copyright (c) 2017 The FreeBSD Foundation * All rights reserved. * + * Portions of this software were developed by Konstantin Belousov + * under sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -27,11 +31,18 @@ #include __FBSDID("$FreeBSD$"); +#include +#include #include +#include "amd64_detect_rdfsgsbase.h" int amd64_set_fsbase(void *addr) { + if (amd64_detect_rdfsgsbase() == RDFSGS_SUPPORTED) { + wrfsbase((uintptr_t)addr); + return (0); + } return (sysarch(AMD64_SET_FSBASE, &addr)); } Modified: head/lib/libc/amd64/sys/amd64_set_gsbase.c ============================================================================== --- head/lib/libc/amd64/sys/amd64_set_gsbase.c Mon Aug 21 17:38:02 2017 (r322762) +++ head/lib/libc/amd64/sys/amd64_set_gsbase.c Mon Aug 21 17:39:12 2017 (r322763) @@ -1,7 +1,11 @@ /*- * Copyright (c) 2003 Peter Wemm + * Copyright (c) 2017 The FreeBSD Foundation * All rights reserved. * + * Portions of this software were developed by Konstantin Belousov + * under sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -27,11 +31,18 @@ #include __FBSDID("$FreeBSD$"); +#include +#include #include +#include "amd64_detect_rdfsgsbase.h" int amd64_set_gsbase(void *addr) { + if (amd64_detect_rdfsgsbase() == RDFSGS_SUPPORTED) { + wrgsbase((uintptr_t)addr); + return (0); + } return (sysarch(AMD64_SET_GSBASE, &addr)); }