Date: Sun, 20 Jun 1999 14:40:25 -0400 (EDT) From: Richard Cownie <tich@ma.ikos.com> To: freebsd-smp@FreeBSD.ORG, ohartman@ipamzlx.physik.uni-mainz.de Subject: Re: SMP, 4GB RAM, 4x CPU Message-ID: <199906201840.OAA18071@isolation.ma.ikos.com>
next in thread | raw e-mail | index | archive | help
Since I'm the source of recent messages about problems with 4 cpu's
and 4GB dram, I'll respond.
1) To the best of my knowledge, 3.2-STABLE works fine with 4 cpu's,
but does not yet support 4GB DRAM. Not sure if it works with 3GB ?
2) To get the use of 4GB dram, I am running 19990604-CURRENT. This
has problems with 4GB dram (but I've submitted a fix), and also
with 4 cpu's. However, in the nature of software development,
if you're running -current then things will be broken from time
to time. This seems to be particularly true of 4GB/4cpu stuff
because very few people have these kinds of systems. If you can
cope with the dram limits in 3.2-STABLE, then that's probably what
you should plan to use to get real work done.
3) The Linux alternative is interesting and I have myself given this
serious consideration. For my particular application, the show-stopper
is that Linux will only handle 2GB of DRAM; also it's unclear how
big a single process can be - I believe there are problems around 1GB.
With FreeBSD I can use 4GB of DRAM (actually 3998MB for arcane
chipset reasons), and a single process can get >2.5GB - with further
tuning of kernel configuration I believe a single process can be >3.5GB.
So if your applications need large virtual memory OR large physical
memory, Linux just doesn't cut it.
4) The biggest system we have here is an Intel SC450NX from SAG Electronics
(www.sagelec.com) with 4 x Xeon-500MHz, 4GB DRAM, 6 x 9GB disk.
Total cost approx $21K. This machine is very very fast and I'm happy
with it (running FreeBSD).
5) There are two different issues which may cause trouble with performance
scaling to 4 cpus:
a) Memory bandwidth. For the EDA applications we're running, cache
locality even with 512K cache is pretty good, my rough estimate
is that each extra cpu slows down the other's by 5%, so total
throughput for 1,2,3,4 jobs is about 1.0, 1.9, 3.2, 3.4
But the machine so fast that we rarely have 4 jobs running
simultaneously.
b) Contention for kernel resources. I believe the locking in the
FreeBSD kernel is not very fine-grained as yet, so if several
cpu's are trying to make system calls all at once, they'll be
largely serialized ? However, if your application spend 90%
of their time in user space (and if you're doing Fortran
floating-point number crunching then that's probably true),
this is probably not an issue.
6) If you can get your work done with 2way SMP systems with <= 1GB dram,
then these are better value than the 4way. The motherboards are cheaper,
the dram is cheaper, and overclocked Celerons are an amazing bargain,
e.g. see the Power2U Dude at www.computernerd.com. And of course
with 2 2way systems you don't suffer the same SMP inefficiency as with
1 4way system.
Good luck
Richard Cownie
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199906201840.OAA18071>