From owner-svn-src-all@FreeBSD.ORG Wed Oct 24 18:36:41 2012 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A24201E3; Wed, 24 Oct 2012 18:36:41 +0000 (UTC) (envelope-from jimharris@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 8ACA98FC12; Wed, 24 Oct 2012 18:36:41 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q9OIafq9073005; Wed, 24 Oct 2012 18:36:41 GMT (envelope-from jimharris@svn.freebsd.org) Received: (from jimharris@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q9OIafqo073002; Wed, 24 Oct 2012 18:36:41 GMT (envelope-from jimharris@svn.freebsd.org) Message-Id: <201210241836.q9OIafqo073002@svn.freebsd.org> From: Jim Harris Date: Wed, 24 Oct 2012 18:36:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r242014 - head/sys/kern X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 18:36:41 -0000 Author: jimharris Date: Wed Oct 24 18:36:41 2012 New Revision: 242014 URL: http://svn.freebsd.org/changeset/base/242014 Log: Pad tdq_lock to avoid false sharing with tdq_load and tdq_cpu_idle. This enables CPU searches (which read tdq_load) to operate independently of any contention on the spinlock. Some scheduler-intensive workloads running on an 8C single-socket SNB Xeon show considerable improvement with this change (2-3% perf improvement, 5-6% decrease in CPU util). Sponsored by: Intel Reviewed by: jeff Modified: head/sys/kern/sched_ule.c Modified: head/sys/kern/sched_ule.c ============================================================================== --- head/sys/kern/sched_ule.c Wed Oct 24 18:33:44 2012 (r242013) +++ head/sys/kern/sched_ule.c Wed Oct 24 18:36:41 2012 (r242014) @@ -223,8 +223,13 @@ static int sched_idlespinthresh = -1; * locking in sched_pickcpu(); */ struct tdq { - /* Ordered to improve efficiency of cpu_search() and switch(). */ + /* + * Ordered to improve efficiency of cpu_search() and switch(). + * tdq_lock is padded to avoid false sharing with tdq_load and + * tdq_cpu_idle. + */ struct mtx tdq_lock; /* run queue lock. */ + char pad[64 - sizeof(struct mtx)]; struct cpu_group *tdq_cg; /* Pointer to cpu topology. */ volatile int tdq_load; /* Aggregate load. */ volatile int tdq_cpu_idle; /* cpu_idle() is active. */