Date: Thu, 27 Jun 2002 12:00:33 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: "Gary Thorpe" <gat7634@hotmail.com> Cc: arch@FreeBSD.ORG Subject: Re: Larry McVoy's slides on cache coherent clusters Message-ID: <200206271900.g5RJ0X4w000411@apollo.backplane.com> References: <F115p3MSi6xzmeWgUSp000017fa@hotmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:I think this ties in to Mr. Lambert's question about the future of FreeBSD
:very much. I think the NUMA model will eventually dominate all future large
:systems in the next 10 years (and SMP will come to be standard on small
:systems) and FreBSD will probably have to run efficiently on them to compete
:with Linux etc. Having seemless clusters (by this I mean clusters that work
:as a single system with one system image and identity) would probably be a
:an interesting problem also, since only a few OSes have made any serious
:attempt at implementing them. PVM, MPI, and MOSIX cannot for example migrate
:I/O among machines (network load balancing maybe?).
Well, I'm not so sure. I think partitioning will come to dominate all
future large machines in the future. They may well be NUMA, but NUMA
will be relegated to the role of being simply a faster communications
medium. We will certainly see cache-coherent shared memory across
the network used in major ways (we see primitive versions of this now)
because their relative costs will be cheaper then NUMA (and will always
be cheaper then NUMA).
The distinction is important from the point of view of OS design. Even
in NUMA systems the difference between local and remote memory is too
great for a non-deterministic implementation (which is essentially what
Linux has) and does not mesh well with the uniform cache architecture
implemented by Linux, Solaris, BSD, etc... most modern OSs. The natural
conclusion is to partition instead and develop more formalized,
deterministic, network-transportable mechanisms for sharing data
that can be abstracted out using mmap().
NUMA then becomes just another, faster transport mechanism. That is
the direction I believe the BSDs will take... transparent clustering
with NUMA transport, network transport, or a hybrid of both.
-Matt
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200206271900.g5RJ0X4w000411>