From owner-freebsd-questions@FreeBSD.ORG Thu Feb 2 20:10:22 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C153B1065674 for ; Thu, 2 Feb 2012 20:10:22 +0000 (UTC) (envelope-from dg@pki2.com) Received: from btw.pki2.com (btw.pki2.com [IPv6:2001:470:a:6fd::2]) by mx1.freebsd.org (Postfix) with ESMTP id 845F98FC1F for ; Thu, 2 Feb 2012 20:10:22 +0000 (UTC) Received: from btw.pki2.com (btw.pki2.com [192.168.23.1]) by btw.pki2.com (8.14.5/8.14.5) with ESMTP id q12KA1iA099548 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 2 Feb 2012 12:10:02 -0800 (PST) (envelope-from dg@pki2.com) Date: Thu, 2 Feb 2012 12:10:01 -0800 (PST) From: Dennis Glatting X-X-Sender: dennisg@btw.pki2.com To: Anton Shterenlikht In-Reply-To: <20120202095515.GA78375@mech-cluster241.men.bris.ac.uk> Message-ID: References: <20120202000947.GA71405@mech-cluster241.men.bris.ac.uk> <1328142599.53835.6.camel@btw.pki2.com> <20120202095515.GA78375@mech-cluster241.men.bris.ac.uk> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-yoursite-MailScanner-Information: Dennis Glatting X-yoursite-MailScanner-ID: q12KA1iA099548 X-yoursite-MailScanner: Found to be clean X-MailScanner-From: dg@pki2.com Cc: Dennis Glatting , freebsd-questions@freebsd.org Subject: Re: OpenMP on FreeBSD X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2012 20:10:22 -0000 On Thu, 2 Feb 2012, Anton Shterenlikht wrote: > On Wed, Feb 01, 2012 at 04:29:59PM -0800, Dennis Glatting wrote: >> On Thu, 2012-02-02 at 00:09 +0000, Anton Shterenlikht wrote: >>> I'm new to OpenMP. I wonder >>> if there are any special considerations >>> when running OpenMP on FreeBSD? >>> >> >> I run OpenMP. No special consideration. Here's a chunk from my Makefile: >> >> TARG=ecc.enc ecc.dec >> >> $TARG: *.cc *.h Makefile >> g++ -Wall -fopenmp -g -O ${INCL} ${LIBS} -o ecc.enc *.cc >> g++ -Wall -fopenmp -g -O ${INCL} ${LIBS} -o ecc.dec *.cc >> >> >>> For example, I have this OMP parallelised >>> fortran program, nested do loops, compiled >>> with gfortran46. When I run it with 2 threads >>> on a 2-cpu box, I see in top(1): >>> >>> PID UID PRI NICE SIZE RES STATE C TIME CPU COMMAND >>> 63995 1001 89 0 57048K 34272K CPU1 1 1:06 55.08% dummy.sx{dummy.sx} >>> 63995 1001 87 0 57048K 34272K RUN 1 1:02 52.39% dummy.sx{dummy.sx} >>> 11 0 155 ki31 0K 32K RUN 0 376:58 51.46% idle{idle: cpu0} >>> 11 0 155 ki31 0K 32K RUN 1 368:18 45.36% idle{idle: cpu1} >>> >>> I wonder why, even after a minute of run time, >>> I still have nearly a whole cpu idle? >>> >> >> What is the program doing? I/O can significantly limit OMP value. Also, >> you need to make sure you code your loops properly or else you have a >> single-threaded application, without warning. > > Here's my parallel bit: > > !$OMP PARALLEL DEFAULT(NONE) & > !$OMP SHARED(AEND,SPACE1,SPACE2,SIZE1,SIZE2,SIZE3) & > !$OMP PRIVATE(STEP,RANDN,X1,X2,X3) > > !$OMP DO SCHEDULE(RUNTIME) > > DO X3 = 1,SIZE3 > DO X2 = 1,SIZE2 > DO X1 = 1,SIZE1 > > IF(SPACE1(X1,X2,X3).EQ.0) THEN > AEND = .FALSE. ! UPDATE AT LEAST ONE CELL > CALL RANDOM_NUMBER(RANDN) ! 0 <= RANDN < 1 > STEP = NINT(RANDN*2-1) ! STEP = [-1 0 1] > SPACE2(X1,X2,X3) = SPACE1(X1+STEP(1),X2+STEP(2),X3+STEP(3)) > END IF > > END DO > END DO > END DO > > !$OMP END DO > !$OMP END PARALLEL > > There's no I/O at all. Not sure what you mean > by proper looping. The threads are definitely > created. I use "setenv OMP_NUM_THREADS" to set > the number of threads. Then I monitor thread > creation with top -H. The number of threads > shown there matches what I set. So I'm pretty > sure the executable is multi-threaded. > The issue I found in C++ is a "barrier" had to be defined, which is the first tagged bracket in my code snippet. Without that barrier the pragma on the "for" loop was single threaded. Can't say about Fortran, sorry. I haven't coded Fortran since collage and can't comment on your code. > Perhaps I should explore various SCHEDULE options? > Nope. When it works you will notice. Under FreeBSD I do a "top -P" in a second window. You will see the cores get busy. > By the way, what sort of speed-up do you > see with your loop? And what ratio threads/cores > is optimal for you? > A forty five minute task down to seven minutes on an eight core, AMD 8150 processor running at 4GHz (slightly over clocked). The effort would take less time if I recoded using a custom thread/code solution but I'm talking about maybe another minute or two. In the end I felt better was the enemy of good. > Many thanks > > -- > Anton Shterenlikht > Room 2.6, Queen's Building > Mech Eng Dept > Bristol University > University Walk, Bristol BS8 1TR, UK > Tel: +44 (0)117 331 5944 > Fax: +44 (0)117 929 4423 >