From owner-svn-src-head@freebsd.org Sun Sep 10 19:00:39 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C210E217EB; Sun, 10 Sep 2017 19:00:39 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6B398826A8; Sun, 10 Sep 2017 19:00:39 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8AJ0cUE059847; Sun, 10 Sep 2017 19:00:38 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8AJ0c2N059845; Sun, 10 Sep 2017 19:00:38 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201709101900.v8AJ0c2N059845@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Sun, 10 Sep 2017 19:00:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r323393 - in head/sys: sys vm X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: in head/sys: sys vm X-SVN-Commit-Revision: 323393 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Sep 2017 19:00:39 -0000 Author: mjg Date: Sun Sep 10 19:00:38 2017 New Revision: 323393 URL: https://svnweb.freebsd.org/changeset/base/323393 Log: Move vmmeter atomic counters into dedicated cache lines Prior to the change they were subject to extreme false sharing. In particular this change shaves about 3 seconds real time of -j 80 buildkernel. Reviewed by: alc, markj Differential Revision: https://reviews.freebsd.org/D12281 Modified: head/sys/sys/vmmeter.h head/sys/vm/vm_meter.c Modified: head/sys/sys/vmmeter.h ============================================================================== --- head/sys/sys/vmmeter.h Sun Sep 10 18:08:25 2017 (r323392) +++ head/sys/sys/vmmeter.h Sun Sep 10 19:00:38 2017 (r323393) @@ -60,6 +60,12 @@ struct vmtotal { #if defined(_KERNEL) || defined(_WANT_VMMETER) #include +#ifdef _KERNEL +#define VMMETER_ALIGNED __aligned(CACHE_LINE_SIZE) +#else +#define VMMETER_ALIGNED +#endif + /* * System wide statistics counters. * Locking: @@ -126,14 +132,15 @@ struct vmmeter { u_int v_free_target; /* (c) pages desired free */ u_int v_free_min; /* (c) pages desired free */ u_int v_free_count; /* (f) pages free */ - u_int v_wire_count; /* (a) pages wired down */ - u_int v_active_count; /* (q) pages active */ u_int v_inactive_target; /* (c) pages desired inactive */ - u_int v_inactive_count; /* (q) pages inactive */ - u_int v_laundry_count; /* (q) pages eligible for laundering */ u_int v_pageout_free_min; /* (c) min pages reserved for kernel */ u_int v_interrupt_free_min; /* (c) reserved pages for int code */ u_int v_free_severe; /* (c) severe page depletion point */ + u_int v_wire_count VMMETER_ALIGNED; /* (a) pages wired down */ + u_int v_active_count VMMETER_ALIGNED; /* (a) pages active */ + u_int v_inactive_count VMMETER_ALIGNED; /* (a) pages inactive */ + u_int v_laundry_count VMMETER_ALIGNED; /* (a) pages eligible for + laundering */ }; #endif /* _KERNEL || _WANT_VMMETER */ Modified: head/sys/vm/vm_meter.c ============================================================================== --- head/sys/vm/vm_meter.c Sun Sep 10 18:08:25 2017 (r323392) +++ head/sys/vm/vm_meter.c Sun Sep 10 19:00:38 2017 (r323393) @@ -56,7 +56,7 @@ __FBSDID("$FreeBSD$"); #include #include -struct vmmeter vm_cnt = { +struct vmmeter __exclusive_cache_line vm_cnt = { .v_swtch = EARLY_COUNTER, .v_trap = EARLY_COUNTER, .v_syscall = EARLY_COUNTER,