From owner-freebsd-hackers Wed Jan 29 21:30:28 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6500E37B405 for ; Wed, 29 Jan 2003 21:30:26 -0800 (PST) Received: from geekpunk.net (adsl-154-186-63.bna.bellsouth.net [68.154.186.63]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9137D43F93 for ; Wed, 29 Jan 2003 21:30:18 -0800 (PST) (envelope-from bandix@geekpunk.net) Received: from localhost.my.domain (taran [127.0.0.1]) by geekpunk.net (8.12.6/8.12.6) with ESMTP id h0TNojHa008646; Wed, 29 Jan 2003 17:50:45 -0600 (CST) (envelope-from bandix@geekpunk.net) Received: (from bandix@localhost) by localhost.my.domain (8.12.6/8.12.6/Submit) id h0TNoeLH008645; Wed, 29 Jan 2003 17:50:40 -0600 (CST) (envelope-from bandix) Date: Wed, 29 Jan 2003 17:50:40 -0600 From: "Brandon D. Valentine" To: "Matthew N. Dodd" Cc: David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. Message-ID: <20030129235040.GY16038@geekpunk.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> <20030129173416.U8642@sasami.jurai.net> <15928.23728.549120.559276@canoe.velocet.net> <20030129180043.S8642@sasami.jurai.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030129180043.S8642@sasami.jurai.net> User-Agent: Mutt/1.4i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 29, 2003 at 06:06:20PM -0500, Matthew N. Dodd wrote: > > What you really want is SCSI over IP. Anything else is just a hack and > not to be trusted. I think that NFS is less of a hack than NBD though. IMO NBD is less of a hack than you think it is. It is one of the necessary components for creating a single system image from a cluster of commodity hardware and this is something Linux developers are working earnestly on. They're targeting a poor man's NUMA. In this case the idea is that each node [network] boots a very minimal operating system image which acts as a slave to the master node, making its hardware (disks in this case) available via some interconnect using network block devices. Then the operating system on the master node can mount the devices under one unified /. OpenMosix is working on solving the other parts of the puzzle, which are scheduling and cache/memory coherency. Myrinet is rapidly converging on SGI's CrayLink interconnect as a low-latency memory bus, making this possible in the next few years on PC hardware. Of course I think they're using an inferior operating system as the basis for all this, but since I'm not willing to put code where my mouth is in this instance, I'm not going to go around telling them which OS to use. ;-) > Of course if Linux still suffers from poor NFS performance that might > explain why they came up with NBD in the first place. As someone who currently has several terabytes served off of Linux 2.4/XFS servers let me say that Linux *definitely* still suffers from poor NFS performance. Not to mention fragility. The Linux in-kernel NFS client and server are notorious for locking up machines around here when remote NFS mounts go away unexpectedly. There provide little fault tolerance at all, with both soft and hard NFS. Brandon D. Valentine -- brandon@dvalentine.com http://www.geekpunk.net "We've been raised on replicas of fake and winding roads, and day after day up on this beautiful stage we've been playing tambourine for minimum wage, but we are real; I know we are real." -- David Berman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message