Date: Wed, 22 Jun 2005 07:09:34 -0500 From: Eric Anderson <anderson@centtech.com> To: Allan Fields <bsd@afields.ca> Cc: freebsd-fs@freebsd.org Subject: Re: Cluster Filesystem for FreeBSD - any interest? Message-ID: <42B954FE.2070406@centtech.com> In-Reply-To: <20050622023727.GA19408@afields.ca> References: <42B825CC.806@centtech.com> <20050622023727.GA19408@afields.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Allan Fields wrote: > On Tue, Jun 21, 2005 at 09:35:56AM -0500, Eric Anderson wrote: > >>This is something I've brought up before on other lists, but I'm curious >>if anyone is interested in developing a BSD licensed clustered >>filesystem for FreeBSD (and anyone else)? > > > A few questions: > > Could this be done as a stackable file system (vnode layer distributed > file system) or did you have something else in mind (i.e. specifically > a full implementation of a network filesystem including storage > layer)? Hmm. I'm not sure if it can or not. I'll try to explain what I'm dreaming of. I currently have about 1000 clients needing access to the same pools of data (read/write) all the time. The data changes constantly. There is a lot of this data. We use NFS currently. FreeBSD is *very* fast and stable at serving NFS data. The problem is, that even though it is very fast and stable, I still cannot pump out enough bits fast enough with one machine, and if that one machine fails (hardware problems, etc), then all my machines are hung waiting for me to bring it back online. So, what I would love to have, is this kind of setup: shared media storage (fibre channel SAN, iscsi, or something like ggated possibly), connected up to a cluster of hosts running FreeBSD. Each FreeBSD server has access to the logical disks, same partitions, and can mount them all r/w. Now, I can kind of do this now, however there are obviously some issues with this currently. I want all machines in this cluster to be able to serve the data via NFS (or http, or anything else for that matter really - if you can make NFS work, anything will pretty much work) simultaneously from the same partitions, and see writes immediately as the other hosts in the cluster commit them. I currently have a solution just like this for Linux - Polyserve (http://www.polyserve.com) has a clustered filesystem for linux, that works very well. I've even tried to convince them to port it to FreeBSD, but it falls on deaf ears, so it's time to make our own. > Why not a port of an existing network filesystem say from Linux? > (A BSD rewrite could be done, if the code was GPLed.) Would > cross-platform capabilities make sense? That would work fine I'm sure - but I have found some similar threads in the past that claim it would be just as hard and time consuming to port one as it would be to create one from scratch. Cross platform capabilities would be great, but I'm mostly interested in getting FreeBSD into this arena (as it will soon be an extremely important one to be in). > How do you see this comparing to device-level solutions? I know > the argument can be made to implement file systems/storage > abstractions at multiple layers, but I thought I might ask. I'm not sure of a device level solution that does this. I think the OS has to know to commit the meta-data to a journal, or otherwise let the other machines know about locking, etc, in order for this to work. > The other thing is there a wealth of filesystem papers out there, > any in specific caught your eye? No - can you point me to some? I'll be honest here - I'm not a code developer. I would love to learn some C here, and 'just do it', but filesystems aren't exactly simple, so I'm looking for a group of people that would love to code up something amazing like this - I'll support the developers and hopefully learn something in the process. My goal personally would be to do anything I could to make the developers work most productively, and do testing. I can probably provide equipment, and a good testbed for it. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42B954FE.2070406>