From owner-freebsd-cluster Thu Dec 12 9:44:35 2002 Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 517D937B401 for ; Thu, 12 Dec 2002 09:44:33 -0800 (PST) Received: from mgr3.xmission.com (mgr3.xmission.com [198.60.22.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id DE41943E4A for ; Thu, 12 Dec 2002 09:44:31 -0800 (PST) (envelope-from glewis@eyesbeyond.com) Received: from mail by mgr3.xmission.com with spam-scanned (Exim 3.35 #1) id 18MXNv-0005FB-03; Thu, 12 Dec 2002 10:44:31 -0700 Received: from [207.135.128.145] (helo=misty.eyesbeyond.com) by mgr3.xmission.com with esmtp (Exim 3.35 #1) id 18MXNt-0005Ef-03; Thu, 12 Dec 2002 10:44:30 -0700 Received: (from glewis@localhost) by misty.eyesbeyond.com (8.11.6/8.11.6) id gBCHiO819072; Fri, 13 Dec 2002 04:14:24 +1030 (CST) (envelope-from glewis@eyesbeyond.com) X-Authentication-Warning: misty.eyesbeyond.com: glewis set sender to glewis@eyesbeyond.com using -f Date: Fri, 13 Dec 2002 04:14:24 +1030 From: Greg Lewis To: Andy Sporner Cc: "Ronald G. Minnich" , freebsd-cluster@FreeBSD.ORG Subject: Re: sharing files within a cluster Message-ID: <20021213041424.A18994@misty.eyesbeyond.com> References: <3DF8B417.9040509@nentec.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3DF8B417.9040509@nentec.de>; from sporner@nentec.de on Thu, Dec 12, 2002 at 05:06:47PM +0100 X-Spam-Status: No, hits=-3.2 required=8.0 tests=IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, SIGNATURE_SHORT_DENSE,SPAM_PHRASE_03_05,USER_AGENT, USER_AGENT_MUTT,X_AUTH_WARNING version=2.43 X-Spam-Level: Sender: owner-freebsd-cluster@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Dec 12, 2002 at 05:06:47PM +0100, Andy Sporner wrote: > Ronald G. Minnich wrote: > >On Thu, 12 Dec 2002, Andy Sporner wrote: > >>I think this is called PXE. Pre-XEcution boot environment from Intel. > >>When you select Network Boot, it hits a DHCP server for an IP address > >>and then a PXE server for a floppy image. Then it kicks off the image > >>just like a FD. > >> > > > >The systems I've used have a 32KB limit. So you use PXE to boot the thing > >that boots the thing. It's stupid. > > > >Also can't boot over things it doesn't know, e.g. myrinet. We boot over > >myrinet. > > Most "real" servers this works perfectly. I know because > I was involved in building them! > > My suggestion to you is that if you can't say something > constructive is that you should say nothing. That's the > problem with academic people they just don't know when > to just keep quiet. Ron is making a real world point though, so his posting is both constructive and relevant. His postings may be brief, but they are to the point and draw on a lot of experience with HPC clusters. In this case he is spot on. PXE is useful, but limited. PXE clients that I've seen put arbitrary limits on both the size of the image and the download time. I've seen at least one PXE client that will time out _in the middle of downloading the image_. Thats right, its getting the image but it will still time out. So, we use PXE to download etherboot and boot into that and then use that to do the actual booting since it doesn't have these limitations. As Ron points out, this is stupid. PXE also has no facilities for multicast that I'm aware of so you have to tftp your image. The etherboot client we have has a multicast client built in, another advantage on big clusters where tftp doesn't scale. As a result we have a preference for NICs with etherboot flashed onto the ROM, not PXE NICs. So, yes PXE is useful. But get a big enough cluster and it falls in a screaming heap. -- Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message