Date: Wed, 10 Aug 2005 16:20:06 -0600 (MDT) From: "M. Warner Losh" <imp@bsdimp.com> To: jhb@freebsd.org Cc: andre@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Special schedulers, one CPU only kernel, one only userland Message-ID: <20050810.162006.48492066.imp@bsdimp.com> In-Reply-To: <200508100911.50004.jhb@FreeBSD.org> References: <42F9ECF2.8080809@freebsd.org> <200508100911.50004.jhb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In message: <200508100911.50004.jhb@FreeBSD.org>
John Baldwin <jhb@FreeBSD.ORG> writes:
: First, you might want to go find a copy of "Unix Systems for Modern
: Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers"
: by Curt Schimmel as he describes this model as one of the early SMP models
: for UNIX kernels before fine-grained locking was used. (It's known as a
: master-slave system IIRC). I think this is the model that BSD/OS employed
: for SMP in its 4.x series before they did their version of SMPng.
Solbourne's OS/MP version 4.0[ABC] used this approach. It was called
at the time 'Assymetric Multi Processing' or ASMP. All kernel traffic
went to CPU0, while user jobs were scheduled for the additional CPUs.
Since it was BSD 4.2 based kernel, it used spl locking as well as
having a big kernel lock for CPU0. There was no other locking in the
kernel than at the entry/exit to the kernel. OS/MP 4.1 and later used
true SMP locking. 4.0D, iirc, used a modified version of ASMP locking
that would allow only one CPU into the whole kernel at a time, but
that CPU floated around. The engineer doing the work said it gave
better performance for higher cpu work loads because CPU0 wouldn't
have to be interrupted all the time.
OS/MP 4.1 performed much better than OS/MP 4.0, and 4.0D better than
4.0C for what we would call today a 'world stone' test. Especially on
the 12 CPU machines.
You'd be trading locking overhead for latency if you push the ideas
that you have too far. If only one CPU can be used for the kernel,
then if the other CPU wants to use kernel services, it has to queue
the request to cpu0 and then sit idle. Maybe this isn't a big deal
for your workload, but for many workloads it results in lower
performance.
Warner
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050810.162006.48492066.imp>