From owner-freebsd-arch@freebsd.org Wed Mar 23 07:23:21 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2714AD8632 for ; Wed, 23 Mar 2016 07:23:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8DEB41993 for ; Wed, 23 Mar 2016 07:23:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 8D4CDAD8631; Wed, 23 Mar 2016 07:23:21 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CF09AD8630 for ; Wed, 23 Mar 2016 07:23:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 289261990; Wed, 23 Mar 2016 07:23:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u2N7NGdw056754 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 23 Mar 2016 09:23:16 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u2N7NGdw056754 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u2N7NGl3056753; Wed, 23 Mar 2016 09:23:16 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 23 Mar 2016 09:23:16 +0200 From: Konstantin Belousov To: John Baldwin Cc: arch@freebsd.org Subject: Re: More fine-grained NUMA knobs Message-ID: <20160323072315.GW1741@kib.kiev.ua> References: <27035442.YCnG4SQ1mX@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <27035442.YCnG4SQ1mX@ralph.baldwin.cx> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Mar 2016 07:23:21 -0000 On Tue, Mar 22, 2016 at 05:13:05PM -0700, John Baldwin wrote: > Some of the I/O device affinity stuff such as bus_get_domain() (and the > bus_get_cpus() I have in review in D5519) are useful on their own right even > if the VM system is not doing NUMA-aware allocations. I think it would be > useful to be able to enable these two "prongs" of NUMA awareness > independently. To that end, I have a little strawman patch that adds two > new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA. I actually think it is > probably worth enabling DEVICE_NUMA by default on x86 (and bumping the > default MAXMEMDOM to, say, 8 (quad-socket haswell)). > > You can see the simple patch at: > > https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts > > Thoughts? I do not like that you check both defined(xxx_NUMA) and MAXMEMDOM > 1. IMO the > 1 part should be dropped. I do not see why would it be useful even as optimization. Otherwise, this looks good.