Date: Mon, 15 Sep 2008 22:48:52 -0500 From: Stephen Montgomery-Smith <stephen@math.missouri.edu> To: Steve Kargl <sgk@troutmask.apl.washington.edu> Cc: Martin Cracauer <cracauer@cons.org>, freebsd-current@freebsd.org Subject: Re: Improved multiprocessor usage on amd64 Message-ID: <48CF2CA4.1000802@math.missouri.edu> In-Reply-To: <48CF2AEF.9070208@math.missouri.edu> References: <48CDBC78.4010409@math.missouri.edu> <20080915195021.GA69528@cons.org> <48CEFF74.8020602@math.missouri.edu> <20080916033459.GA31220@troutmask.apl.washington.edu> <48CF2AEF.9070208@math.missouri.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Stephen Montgomery-Smith wrote:
> Steve Kargl wrote:
>> On Mon, Sep 15, 2008 at 07:36:04PM -0500, Stephen Montgomery-Smith wrote:
>>> ... and each thread is a loop of the form
>>>
>>> while (1) {
>>> wait until told to start;
>>> do massive amounts of floating point arithmetic (only additions and
>>> multiplications) on large arrays;
>>> tell the master process that you are done;
>>> }
>>>
>>>> Do you have about as many threads as processor or more?
>>> Both ways. The time difference between the two approaches is
>>> negligible.
>>>
>>
>> Are you using ULE? With my MPI applications, if the number of
>> launched processes exceeds the number of cpus by 1, ULE falls
>> through the floor. I have a nagging feeling that there is a problem
>> with cpu affinity.
>>
>> http://lists.freebsd.org/pipermail/freebsd-current/2008-July/086917.html
>>
Let me say a little bit more.
I have this gut feeling that the problem has a lot to do with cache
management. My program has each thread doing, in effect, huge matrix
multiplications, each one working on their own little bit. If a CPU
core changes from one thread to another, it then has to flush out the
cache to RAM, and read in a whole bunch of other RAM into cache.
I have this sense that Linux and FreeBSD have something in its internals
where it figures this out, and after a while starts changing the time
between when it changes from one process to another. But Linux has a
faster learning curve than FreeBSD.
But this is all pure speculation on my part, because I have very little
ideas as to how these internals work.
Stephen
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48CF2CA4.1000802>