Date: Thu, 2 Feb 2012 12:10:01 -0800 (PST) From: Dennis Glatting <dg@pki2.com> To: Anton Shterenlikht <mexas@bristol.ac.uk> Cc: Dennis Glatting <freebsd@pki2.com>, freebsd-questions@freebsd.org Subject: Re: OpenMP on FreeBSD Message-ID: <alpine.BSF.2.00.1202021200420.99120@btw.pki2.com> In-Reply-To: <20120202095515.GA78375@mech-cluster241.men.bris.ac.uk> References: <20120202000947.GA71405@mech-cluster241.men.bris.ac.uk> <1328142599.53835.6.camel@btw.pki2.com> <20120202095515.GA78375@mech-cluster241.men.bris.ac.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 2 Feb 2012, Anton Shterenlikht wrote:
> On Wed, Feb 01, 2012 at 04:29:59PM -0800, Dennis Glatting wrote:
>> On Thu, 2012-02-02 at 00:09 +0000, Anton Shterenlikht wrote:
>>> I'm new to OpenMP. I wonder
>>> if there are any special considerations
>>> when running OpenMP on FreeBSD?
>>>
>>
>> I run OpenMP. No special consideration. Here's a chunk from my Makefile:
>>
>> TARG=ecc.enc ecc.dec
>>
>> $TARG: *.cc *.h Makefile
>> g++ -Wall -fopenmp -g -O ${INCL} ${LIBS} -o ecc.enc *.cc
>> g++ -Wall -fopenmp -g -O ${INCL} ${LIBS} -o ecc.dec *.cc
>>
>>
>>> For example, I have this OMP parallelised
>>> fortran program, nested do loops, compiled
>>> with gfortran46. When I run it with 2 threads
>>> on a 2-cpu box, I see in top(1):
>>>
>>> PID UID PRI NICE SIZE RES STATE C TIME CPU COMMAND
>>> 63995 1001 89 0 57048K 34272K CPU1 1 1:06 55.08% dummy.sx{dummy.sx}
>>> 63995 1001 87 0 57048K 34272K RUN 1 1:02 52.39% dummy.sx{dummy.sx}
>>> 11 0 155 ki31 0K 32K RUN 0 376:58 51.46% idle{idle: cpu0}
>>> 11 0 155 ki31 0K 32K RUN 1 368:18 45.36% idle{idle: cpu1}
>>>
>>> I wonder why, even after a minute of run time,
>>> I still have nearly a whole cpu idle?
>>>
>>
>> What is the program doing? I/O can significantly limit OMP value. Also,
>> you need to make sure you code your loops properly or else you have a
>> single-threaded application, without warning.
>
> Here's my parallel bit:
>
> !$OMP PARALLEL DEFAULT(NONE) &
> !$OMP SHARED(AEND,SPACE1,SPACE2,SIZE1,SIZE2,SIZE3) &
> !$OMP PRIVATE(STEP,RANDN,X1,X2,X3)
>
> !$OMP DO SCHEDULE(RUNTIME)
>
> DO X3 = 1,SIZE3
> DO X2 = 1,SIZE2
> DO X1 = 1,SIZE1
>
> IF(SPACE1(X1,X2,X3).EQ.0) THEN
> AEND = .FALSE. ! UPDATE AT LEAST ONE CELL
> CALL RANDOM_NUMBER(RANDN) ! 0 <= RANDN < 1
> STEP = NINT(RANDN*2-1) ! STEP = [-1 0 1]
> SPACE2(X1,X2,X3) = SPACE1(X1+STEP(1),X2+STEP(2),X3+STEP(3))
> END IF
>
> END DO
> END DO
> END DO
>
> !$OMP END DO
> !$OMP END PARALLEL
>
> There's no I/O at all. Not sure what you mean
> by proper looping. The threads are definitely
> created. I use "setenv OMP_NUM_THREADS" to set
> the number of threads. Then I monitor thread
> creation with top -H. The number of threads
> shown there matches what I set. So I'm pretty
> sure the executable is multi-threaded.
>
The issue I found in C++ is a "barrier" had to be defined, which is the
first tagged bracket in my code snippet. Without that barrier the pragma
on the "for" loop was single threaded. Can't say about Fortran, sorry.
I haven't coded Fortran since collage and can't comment on your code.
> Perhaps I should explore various SCHEDULE options?
>
Nope. When it works you will notice. Under FreeBSD I do a "top -P" in a
second window. You will see the cores get busy.
> By the way, what sort of speed-up do you
> see with your loop? And what ratio threads/cores
> is optimal for you?
>
A forty five minute task down to seven minutes on an eight core, AMD 8150
processor running at 4GHz (slightly over clocked). The effort would take
less time if I recoded using a custom thread/code solution but I'm talking
about maybe another minute or two. In the end I felt better was the enemy
of good.
> Many thanks
>
> --
> Anton Shterenlikht
> Room 2.6, Queen's Building
> Mech Eng Dept
> Bristol University
> University Walk, Bristol BS8 1TR, UK
> Tel: +44 (0)117 331 5944
> Fax: +44 (0)117 929 4423
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.1202021200420.99120>