From owner-svn-src-head@freebsd.org Fri Jul 6 19:50:46 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 026881024F6A; Fri, 6 Jul 2018 19:50:46 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A45C093B08; Fri, 6 Jul 2018 19:50:45 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 86E9453AE; Fri, 6 Jul 2018 19:50:45 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w66Jojmt081397; Fri, 6 Jul 2018 19:50:45 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w66Joibw081390; Fri, 6 Jul 2018 19:50:44 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201807061950.w66Joibw081390@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 6 Jul 2018 19:50:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336047 - in head/sys: amd64/include i386/include sys X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head/sys: amd64/include i386/include sys X-SVN-Commit-Revision: 336047 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 19:50:46 -0000 Author: kib Date: Fri Jul 6 19:50:44 2018 New Revision: 336047 URL: https://svnweb.freebsd.org/changeset/base/336047 Log: Expand x86 struct pcpus to UMA_PCPU_ALLOC_SIZE AKA PAGE_SIZE. This restores counters(9) operation. Revert r336024. Improve assert of pcpu size on x86. Reviewed by: mmacy Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D16163 Modified: head/sys/amd64/include/counter.h head/sys/amd64/include/pcpu.h head/sys/i386/include/pcpu.h head/sys/sys/pcpu.h Modified: head/sys/amd64/include/counter.h ============================================================================== --- head/sys/amd64/include/counter.h Fri Jul 6 19:50:25 2018 (r336046) +++ head/sys/amd64/include/counter.h Fri Jul 6 19:50:44 2018 (r336047) @@ -83,18 +83,11 @@ counter_u64_zero_inline(counter_u64_t c) static inline void counter_u64_add(counter_u64_t c, int64_t inc) { - int64_t *p; -#ifdef notyet + __asm __volatile("addq\t%1,%%gs:(%0)" : : "r" ((char *)c - (char *)&__pcpu[0]), "ri" (inc) : "memory", "cc"); -#endif - /* temporary */ - critical_enter(); - p = zpcpu_get(c); - *p += inc; - critical_exit(); } #endif /* ! __MACHINE_COUNTER_H__ */ Modified: head/sys/amd64/include/pcpu.h ============================================================================== --- head/sys/amd64/include/pcpu.h Fri Jul 6 19:50:25 2018 (r336046) +++ head/sys/amd64/include/pcpu.h Fri Jul 6 19:50:44 2018 (r336047) @@ -76,8 +76,7 @@ uint32_t pc_pcid_gen; \ uint32_t pc_smp_tlb_done; /* TLB op acknowledgement */ \ uint32_t pc_ibpb_set; \ - char __pad[216] /* be divisor of PAGE_SIZE \ - after cache alignment */ + char __pad[3288] /* pad to UMA_PCPU_ALLOC_SIZE */ #define PC_DBREG_CMD_NONE 0 #define PC_DBREG_CMD_LOAD 1 Modified: head/sys/i386/include/pcpu.h ============================================================================== --- head/sys/i386/include/pcpu.h Fri Jul 6 19:50:25 2018 (r336046) +++ head/sys/i386/include/pcpu.h Fri Jul 6 19:50:44 2018 (r336047) @@ -80,7 +80,7 @@ caddr_t pc_pmap_eh_ptep; \ uint32_t pc_smp_tlb_done; /* TLB op acknowledgement */ \ uint32_t pc_ibpb_set; \ - char __pad[538] + char __pad[3610] #ifdef _KERNEL Modified: head/sys/sys/pcpu.h ============================================================================== --- head/sys/sys/pcpu.h Fri Jul 6 19:50:25 2018 (r336046) +++ head/sys/sys/pcpu.h Fri Jul 6 19:50:44 2018 (r336047) @@ -185,14 +185,6 @@ struct pcpu { PCPU_MD_FIELDS; } __aligned(CACHE_LINE_SIZE); -#ifdef CTASSERT -/* - * To minimize memory waste in per-cpu UMA zones, size of struct pcpu - * should be denominator of PAGE_SIZE. - */ -CTASSERT((PAGE_SIZE / sizeof(struct pcpu)) * sizeof(struct pcpu) == PAGE_SIZE); -#endif - #ifdef _KERNEL STAILQ_HEAD(cpuhead, pcpu); @@ -208,6 +200,19 @@ extern struct pcpu *cpuid_to_pcpu[]; #define curvidata PCPU_GET(vidata) #define UMA_PCPU_ALLOC_SIZE PAGE_SIZE + +#ifdef CTASSERT +#if defined(__i386__) || defined(__amd64__) +/* Required for counters(9) to work on x86. */ +CTASSERT(sizeof(struct pcpu) == UMA_PCPU_ALLOC_SIZE); +#else +/* + * To minimize memory waste in per-cpu UMA zones, size of struct pcpu + * should be denominator of PAGE_SIZE. + */ +CTASSERT((PAGE_SIZE / sizeof(struct pcpu)) * sizeof(struct pcpu) == PAGE_SIZE); +#endif /* UMA_PCPU_ALLOC_SIZE && x86 */ +#endif /* CTASSERT */ /* Accessor to elements allocated via UMA_ZONE_PCPU zone. */ static inline void *