Date: Mon, 8 May 2023 12:01:33 -0700 From: John Baldwin <jhb@FreeBSD.org> To: Ed Maste <emaste@freebsd.org>, freebsd-arch <freebsd-arch@freebsd.org>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Support for more than 256 CPU cores Message-ID: <5e59d762-6f58-46a4-bceb-de7fcb87682d@FreeBSD.org> In-Reply-To: <CAPyFy2DODJVhs5o8xddaj7GD8zZfC3g1zm_guWKeCmeE07wn-w@mail.gmail.com> References: <CAPyFy2DODJVhs5o8xddaj7GD8zZfC3g1zm_guWKeCmeE07wn-w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 5/5/23 6:38 AM, Ed Maste wrote: > FreeBSD supports up to 256 CPU cores in the default kernel configuration > (on Tier-1 architectures). Systems with more than 256 cores are > available now, and will become increasingly common over FreeBSD 14’s > lifetime. The FreeBSD Foundation is supporting the effort to increase > MAXCPU, and PR269572[1] is open to track tasks and changes. > > As a project we have scalability work ahead of us to make best use of > high core count machines, but at a minimum we should be able to boot a > GENERIC kernel on such systems, and have an ABI for the FreeBSD 14 > release that supports such a configuration. > > Some changes have already been committed in support of increased MAXCPU, > including increasing MAX_APIC_ID (commit c8113dad7ed4) and a number of > changes to reduce bloat (such as commits 42f722e721cd, e72f7ed43eef, > 78cfa762ebf2 and 74ac712f72cf). > > The next step is to increase the maximum cpuset size for userland. > I have this change open in review D39941[2] and an exp-run request in > PR271213[3]. Following that the kernel change for increasing MAXCPU is > in D36838[4]. > > Additional work on bloat reduction will continue after this change, and > looking forward FreeBSD is going to need ongoing effort from the > community and the FreeBSD Foundation to continue improving scalability. > > [1] https://bugs.freebsd.org/269572 > [2] https://reviews.freebsd.org/D39941 > [3] https://bugs.freebsd.org/271213 > [4] https://reviews.freebsd.org/D36838 FWIW, I think it will be useful for main to run with a larger userspace MAXCPU than kernel for at least a while so that we have better testing of that configuration and to give headroom for bumping MAXCPU in the kernel during the 14.x branch. The only other viable path I think which would be more work would be to rework cpuset_t in userspace to always use a dynamically sized mask. This could perhaps be done in an API-preserving manner by making cpuset_t an opaque wrapper type in userland and requiring CPU_* to indirect to functions in libc, etc. That's a fair bit more work however. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5e59d762-6f58-46a4-bceb-de7fcb87682d>