From owner-svn-src-stable-9@FreeBSD.ORG Wed Mar 28 10:15:43 2012 Return-Path: Delivered-To: svn-src-stable-9@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5EEA11065670; Wed, 28 Mar 2012 10:15:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 301AA8FC12; Wed, 28 Mar 2012 10:15:43 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q2SAFhsH029634; Wed, 28 Mar 2012 10:15:43 GMT (envelope-from mav@svn.freebsd.org) Received: (from mav@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q2SAFglt029632; Wed, 28 Mar 2012 10:15:42 GMT (envelope-from mav@svn.freebsd.org) Message-Id: <201203281015.q2SAFglt029632@svn.freebsd.org> From: Alexander Motin Date: Wed, 28 Mar 2012 10:15:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-9@freebsd.org X-SVN-Group: stable-9 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r233598 - stable/9/sys/sys X-BeenThere: svn-src-stable-9@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for only the 9-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Mar 2012 10:15:43 -0000 Author: mav Date: Wed Mar 28 10:15:42 2012 New Revision: 233598 URL: http://svn.freebsd.org/changeset/base/233598 Log: MFC r232852: Tune cpuset macros to optimize cases when CPU_SETSIZE fits into single machine word. For example, it turns CPU_SET() into expected shift and OR, removing extra shift, AND and additional index on memory access. Generated code checked for kernel (optimized) and user-level (unoptimized) cases with GCC and CLANG. Modified: stable/9/sys/sys/cpuset.h Directory Properties: stable/9/sys/ (props changed) Modified: stable/9/sys/sys/cpuset.h ============================================================================== --- stable/9/sys/sys/cpuset.h Wed Mar 28 08:11:46 2012 (r233597) +++ stable/9/sys/sys/cpuset.h Wed Mar 28 10:15:42 2012 (r233598) @@ -36,11 +36,18 @@ #define CPUSETBUFSIZ ((2 + sizeof(long) * 2) * _NCPUWORDS) -#define __cpuset_mask(n) ((long)1 << ((n) % _NCPUBITS)) -#define CPU_CLR(n, p) ((p)->__bits[(n)/_NCPUBITS] &= ~__cpuset_mask(n)) +/* + * Macros addressing word and bit within it, tuned to make compiler + * optimize cases when CPU_SETSIZE fits into single machine word. + */ +#define __cpuset_mask(n) \ + ((long)1 << ((_NCPUWORDS == 1) ? (__size_t)(n) : ((n) % _NCPUBITS))) +#define __cpuset_word(n) ((_NCPUWORDS == 1) ? 0 : ((n) / _NCPUBITS)) + +#define CPU_CLR(n, p) ((p)->__bits[__cpuset_word(n)] &= ~__cpuset_mask(n)) #define CPU_COPY(f, t) (void)(*(t) = *(f)) -#define CPU_ISSET(n, p) (((p)->__bits[(n)/_NCPUBITS] & __cpuset_mask(n)) != 0) -#define CPU_SET(n, p) ((p)->__bits[(n)/_NCPUBITS] |= __cpuset_mask(n)) +#define CPU_ISSET(n, p) (((p)->__bits[__cpuset_word(n)] & __cpuset_mask(n)) != 0) +#define CPU_SET(n, p) ((p)->__bits[__cpuset_word(n)] |= __cpuset_mask(n)) #define CPU_ZERO(p) do { \ __size_t __i; \ for (__i = 0; __i < _NCPUWORDS; __i++) \ @@ -55,7 +62,7 @@ #define CPU_SETOF(n, p) do { \ CPU_ZERO(p); \ - ((p)->__bits[(n)/_NCPUBITS] = __cpuset_mask(n)); \ + ((p)->__bits[__cpuset_word(n)] = __cpuset_mask(n)); \ } while (0) /* Is p empty. */ @@ -126,10 +133,10 @@ } while (0) #define CPU_CLR_ATOMIC(n, p) \ - atomic_clear_long(&(p)->__bits[(n)/_NCPUBITS], __cpuset_mask(n)) + atomic_clear_long(&(p)->__bits[__cpuset_word(n)], __cpuset_mask(n)) #define CPU_SET_ATOMIC(n, p) \ - atomic_set_long(&(p)->__bits[(n)/_NCPUBITS], __cpuset_mask(n)) + atomic_set_long(&(p)->__bits[__cpuset_word(n)], __cpuset_mask(n)) /* Convenience functions catering special cases. */ #define CPU_OR_ATOMIC(d, s) do { \