From owner-freebsd-current@FreeBSD.ORG Thu May 29 18:04:16 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE1D9BF9; Thu, 29 May 2014 18:04:16 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 931CE24A8; Thu, 29 May 2014 18:04:16 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 87D96B95D; Thu, 29 May 2014 14:04:15 -0400 (EDT) From: John Baldwin To: Adrian Chadd Subject: Re: Processor cores not properly detected/activated? Date: Thu, 29 May 2014 13:18:35 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <20140524014713.GF13462@carrick-users.bishnet.net> <201405281358.35924.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201405291318.36130.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 29 May 2014 14:04:15 -0400 (EDT) Cc: Attilio Rao , freebsd-current , Jia-Shiun Li , Alan Somers , Tim Bishop X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 18:04:16 -0000 On Thursday, May 29, 2014 12:53:54 pm Adrian Chadd wrote: > On 28 May 2014 10:58, John Baldwin wrote: > > On Wednesday, May 28, 2014 1:51:28 pm Adrian Chadd wrote: > >> On 28 May 2014 06:56, John Baldwin wrote: > >> > >> > >> > Userland cpusets only default to 128 (CPU_MAXSIZE in ). > >> > Changing MAXCPU to even 128 is unfortunately a potential KBI change since it > >> > changes the size of 'cpuset_t'. We can certainly bump these in HEAD for 11, > >> > but we might not be able to MFC them without introducing ABI breakage. > >> > (The cpuset APIs do allow the size of cpuset_t to change as the size is > >> > encoded in the API calls, so there is that, it's more that if some public > >> > structure embeds a cpuset_t in the kernel that we would have problems. I > >> > thought 'struct pcpu' did, but it does not.) > >> > > >> > Hmm, smp_rendezvous() accepts a cpuset_t as its first argument (and is a > >> > public symbol used by kernel modules such as dtrace). 'struct rmlock' also > >> > embeds a cpuset_t. So, I think we can't bump cpuset_t without breaking > >> > the KBI. We can bump it in HEAD however. (Note, if re@ signed off, we could > >> > perhaps merge to 10, but we tend to be very hesitant about breaking the KBI.) > >> > One thing we could do safely is bump the userland cpuset size to 256 in 10. > >> > It's really only MAXCPU that is problematic. > >> > > >> > In particular, I propose we bump the userland cpuset_t size to 256 now (and > >> > go ahead and merge that to 10). In HEAD only we can bump MAXCPU for amd64 > >> > to 256. > >> > >> Since 11 is going to be around for a few years, can we experiment > >> bumping it up to something compute-cluster-computer-sized just to get > >> it over with? Something stupid, like 4096 or something? > > > > It costs wired memory to increase it for the kernel. The userland set size > > can be increased rather arbitrarily, so we don't need to make it but so large > > as it is easy to bump later (even with a branch). > > Well, what about making the API/KBI use cpuset_t pointers for things > rather than including it as a bitmask? Do you think there'd be a > noticable performance overhead for the bits where it's indirecting > through a pointer to get to the bitmask data? The wired memory is not due to cpuset_t. The wired memory usage is due to things that do 'struct foo foo_bits[MAXCPU]'. The KBI issues I mentioned above are 'struct rmlock' (so now you want any rmlock users to malloc space, or you want rmlock_init() call malloc? (that seems like a bad idea)). The other one is smp_rendezvous. Plus, it's not just a pointer, you really need a (pointer, size_t) tuple similar to what cpuset_getaffinity(), etc. use. -- John Baldwin