From owner-svn-src-head@freebsd.org Fri Jul 6 19:48:49 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49F5C1024C15; Fri, 6 Jul 2018 19:48:49 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB53E93803; Fri, 6 Jul 2018 19:48:48 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CD41C5399; Fri, 6 Jul 2018 19:48:48 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w66JmmNq081177; Fri, 6 Jul 2018 19:48:48 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w66Jmmw6081174; Fri, 6 Jul 2018 19:48:48 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201807061948.w66Jmmw6081174@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 6 Jul 2018 19:48:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336045 - in head/sys: amd64/include i386/include sys X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head/sys: amd64/include i386/include sys X-SVN-Commit-Revision: 336045 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 19:48:49 -0000 Author: kib Date: Fri Jul 6 19:48:47 2018 New Revision: 336045 URL: https://svnweb.freebsd.org/changeset/base/336045 Log: Save a call to pmap_remove() if entry cannot have any pages mapped. Due to the way rtld creates mappings for the shared objects, each dso causes unmap of at least three guard map entries. For instance, in the buildworld load, this change reduces the amount of pmap_remove() calls by 1/5. Profiled by: alc Reviewed by: alc, markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D16148 Modified: head/sys/amd64/include/counter.h head/sys/amd64/include/pcpu.h head/sys/i386/include/pcpu.h head/sys/sys/pcpu.h Modified: head/sys/amd64/include/counter.h ============================================================================== --- head/sys/amd64/include/counter.h Fri Jul 6 19:47:09 2018 (r336044) +++ head/sys/amd64/include/counter.h Fri Jul 6 19:48:47 2018 (r336045) @@ -83,18 +83,11 @@ counter_u64_zero_inline(counter_u64_t c) static inline void counter_u64_add(counter_u64_t c, int64_t inc) { - int64_t *p; -#ifdef notyet + __asm __volatile("addq\t%1,%%gs:(%0)" : : "r" ((char *)c - (char *)&__pcpu[0]), "ri" (inc) : "memory", "cc"); -#endif - /* temporary */ - critical_enter(); - p = zpcpu_get(c); - *p += inc; - critical_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Modified: head/sys/amd64/include/pcpu.h ============================================================================== --- head/sys/amd64/include/pcpu.h Fri Jul 6 19:47:09 2018 (r336044) +++ head/sys/amd64/include/pcpu.h Fri Jul 6 19:48:47 2018 (r336045) @@ -76,8 +76,7 @@ uint32_t pc_pcid_gen; \ uint32_t pc_smp_tlb_done; /* TLB op acknowledgement */ \ uint32_t pc_ibpb_set; \ - char __pad[216] /* be divisor of PAGE_SIZE \ - after cache alignment */ + char __pad[3288] /* pad to UMA_PCPU_ALLOC_SIZE */ #define PC_DBREG_CMD_NONE 0 #define PC_DBREG_CMD_LOAD 1 Modified: head/sys/i386/include/pcpu.h ============================================================================== --- head/sys/i386/include/pcpu.h Fri Jul 6 19:47:09 2018 (r336044) +++ head/sys/i386/include/pcpu.h Fri Jul 6 19:48:47 2018 (r336045) @@ -80,7 +80,7 @@ caddr_t pc_pmap_eh_ptep; \ uint32_t pc_smp_tlb_done; /* TLB op acknowledgement */ \ uint32_t pc_ibpb_set; \ - char __pad[538] + char __pad[3610] #ifdef _KERNEL Modified: head/sys/sys/pcpu.h ============================================================================== --- head/sys/sys/pcpu.h Fri Jul 6 19:47:09 2018 (r336044) +++ head/sys/sys/pcpu.h Fri Jul 6 19:48:47 2018 (r336045) @@ -185,14 +185,6 @@ struct pcpu { PCPU_MD_FIELDS; } __aligned(CACHE_LINE_SIZE); -#ifdef CTASSERT -/* - * To minimize memory waste in per-cpu UMA zones, size of struct pcpu - * should be denominator of PAGE_SIZE. - */ -CTASSERT((PAGE_SIZE / sizeof(struct pcpu)) * sizeof(struct pcpu) == PAGE_SIZE); -#endif - #ifdef _KERNEL STAILQ_HEAD(cpuhead, pcpu); @@ -208,6 +200,19 @@ extern struct pcpu *cpuid_to_pcpu[]; #define curvidata PCPU_GET(vidata) #define UMA_PCPU_ALLOC_SIZE PAGE_SIZE + +#ifdef CTASSERT +#if defined(__i386__) || defined(__amd64__) +/* Required for counters(9) to work on x86. */ +CTASSERT(sizeof(struct pcpu) == UMA_PCPU_ALLOC_SIZE); +#else +/* + * To minimize memory waste in per-cpu UMA zones, size of struct pcpu + * should be denominator of PAGE_SIZE. + */ +CTASSERT((PAGE_SIZE / sizeof(struct pcpu)) * sizeof(struct pcpu) == PAGE_SIZE); +#endif /* UMA_PCPU_ALLOC_SIZE && x86 */ +#endif /* CTASSERT */ /* Accessor to elements allocated via UMA_ZONE_PCPU zone. */ static inline void *