From owner-freebsd-cluster@FreeBSD.ORG Wed Dec 1 13:47:05 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5544716A4CE for ; Wed, 1 Dec 2004 13:47:05 +0000 (GMT) Received: from rose.ift.uni.wroc.pl (rose.ift.uni.wroc.pl [156.17.88.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id 716EA43D54 for ; Wed, 1 Dec 2004 13:47:03 +0000 (GMT) (envelope-from zkoza@ift.uni.wroc.pl) Received: from localhost (zkoza@localhost)iB1DjND10475 for ; Wed, 1 Dec 2004 14:45:23 +0100 (CET) Date: Wed, 1 Dec 2004 14:45:23 +0100 (CET) From: Koza Zbigniew To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: clustering: how to start? X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Dec 2004 13:47:05 -0000 We have just bought 4 dual-processor Xeon boxes for a physics department and want to turn it into a computing cluster. This means we want that the users will communicate with it via ssh/sftp and see all processors as if they were part of a one virtual UNIX machine with 8 processors on board. A typical job we want to run is (a bunch of) numerical simulation programs that run for 2 weeks or so. Thus, the cluster will serve only as a numerical workstation. We have experimented with Debian and OpenMosix. The first outcome of our tests is that Intels' so much advertised Hyperthreading technology is a disaster. Then, OpenMosix seems to have a bug that can bring the whole system to halt if one tries to run simultanieously several memory-consuming processes. We have also problems with compiling custom Linux kernel so that it could see all the swap space and memory it should see. Then I recalled that several years ago I experimented with FreeBSD and I liked it. Now I hear that FreeBSD supports SMP "natively", so no kernel recompilation will be necessary. I thought - maybe we could try and use FreeBSD instead of Debian? Now my question is: where should I start? Which docs should I read? Which program(s) should I use to cluster our 4 FreeBSD boxes? regards, Z. Koza From owner-freebsd-cluster@FreeBSD.ORG Wed Dec 1 13:51:11 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F27A16A4CE for ; Wed, 1 Dec 2004 13:51:11 +0000 (GMT) Received: from smtpq1.home.nl (smtpq1.home.nl [213.51.128.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id E67EE43D31 for ; Wed, 1 Dec 2004 13:51:10 +0000 (GMT) (envelope-from dodell@sitetronics.com) Received: from [213.51.128.132] (port=46752 helo=smtp1.home.nl) by smtpq1.home.nl with esmtp (Exim 4.30) id 1CZUsv-0000Lw-JH; Wed, 01 Dec 2004 14:51:09 +0100 Received: from cc740438-a.deven1.ov.home.nl ([82.75.136.183]:3252 helo=[192.168.1.100]) by smtp1.home.nl with esmtp (Exim 4.30) id 1CZUsu-0007x2-Kd; Wed, 01 Dec 2004 14:51:08 +0100 Message-ID: <41ADCC4C.2050703@sitetronics.com> Date: Wed, 01 Dec 2004 14:51:08 +0100 From: "Devon H. O'Dell" User-Agent: Mozilla Thunderbird 0.9 (Windows/20041103) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Koza Zbigniew References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-AtHome-MailScanner-Information: Please contact support@home.nl for more information X-AtHome-MailScanner: Found to be clean cc: freebsd-cluster@freebsd.org Subject: Re: clustering: how to start? X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Dec 2004 13:51:11 -0000 Koza Zbigniew wrote: [snip] > Then I recalled that several years ago I experimented with FreeBSD and I > liked it. Now I hear that FreeBSD supports SMP "natively", so > no kernel recompilation will be necessary. > I thought - maybe we could try and use FreeBSD instead of Debian? > Now my question is: where should I start? > Which docs should I read? > Which program(s) should I use to cluster our 4 FreeBSD boxes? > > regards, > Z. Koza FreeBSD won't do process distribution natively. Want a project? :) Other than that, there are other programs you can use to distribute applications in userspace (such as any of the MPI packages; I've had good experience with lam-mpi). To monitor the status of your clusters, you can use an application called Ganglia. Good luck! Devon H. O'Dell > _______________________________________________ > freebsd-cluster@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-cluster > To unsubscribe, send any mail to "freebsd-cluster-unsubscribe@freebsd.org" > > From owner-freebsd-cluster@FreeBSD.ORG Wed Dec 1 16:15:22 2004 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5906E16A4CE for ; Wed, 1 Dec 2004 16:15:22 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0762943D41 for ; Wed, 1 Dec 2004 16:15:22 +0000 (GMT) (envelope-from sumirati@gmail.com) Received: by wproxy.gmail.com with SMTP id 57so57635wri for ; Wed, 01 Dec 2004 08:15:21 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=IEByw+pA9plIeslzdjj9v6tyb/bQwHavRP62CMW+xT75xrPhsThcHpuAQIkrsCDoJPnQIligfRrAWK+7/PB8wJxy5+1HDe4AHHmMSCdGmKkwt4Lwn07T/7JSKPszGWU5DoOLVuh83IgAUmqBwLTYwORnxIRDKvs28x8uZjXHB9g= Received: by 10.54.41.71 with SMTP id o71mr737285wro; Wed, 01 Dec 2004 08:15:21 -0800 (PST) Received: by 10.54.36.44 with HTTP; Wed, 1 Dec 2004 08:15:21 -0800 (PST) Message-ID: <4d3144d3041201081522248250@mail.gmail.com> Date: Wed, 1 Dec 2004 17:15:21 +0100 From: "Marc. Perisa" To: "Devon H. O'Dell" In-Reply-To: <41ADCC4C.2050703@sitetronics.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <41ADCC4C.2050703@sitetronics.com> cc: freebsd-cluster@freebsd.org cc: Koza Zbigniew Subject: Re: clustering: how to start? X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: "Marc. Perisa" List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Dec 2004 16:15:22 -0000 On Wed, 01 Dec 2004 14:51:08 +0100, Devon H. O'Dell wrote: > Koza Zbigniew wrote: > [snip] > > > Then I recalled that several years ago I experimented with FreeBSD and I > > liked it. Now I hear that FreeBSD supports SMP "natively", so > > no kernel recompilation will be necessary. > > I thought - maybe we could try and use FreeBSD instead of Debian? > > Now my question is: where should I start? > > Which docs should I read? > > Which program(s) should I use to cluster our 4 FreeBSD boxes? > > > > FreeBSD won't do process distribution natively. Want a project? :) > > Other than that, there are other programs you can use to distribute > applications in userspace (such as any of the MPI packages; I've had > good experience with lam-mpi). To monitor the status of your clusters, > you can use an application called Ganglia. > Hi, the "normal" way to go is to use MPI for number crunching operations on "normal" hardware. Since MPI is quite industrial standard most programs which are spanning multiple machines support this interface. To get a high real-world performance you should use high speed interconnects with a low latency. In general: Are you writing the software yourself? Have you ever developed for MPI? If not, then start reading at http://www-unix.mcs.anl.gov/mpi/ A general tutorial to setup such a system under FreeBSD may be found here: http://www.bsdhound.com/newsread.php?newsid=205 Hope that helps Marc