Date: Sun, 27 Nov 2005 20:05:03 -0800 From: mike <mike503@gmail.com> To: freebsd-hackers@freebsd.org Subject: Possible way to distribute NFS? Message-ID: <bd9320b30511272005q43cb974dm47d5fbaa1e389f90@mail.gmail.com> In-Reply-To: <bd9320b30511271904l77cf6d3ar8b7eb50425c4cf78@mail.gmail.com> References: <bd9320b30511271904l77cf6d3ar8b7eb50425c4cf78@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
(Note: also sent this to -fs, but it's relatively low traffic, and a lot of people on -hackers have the knowledge of whether or not this is even an idea worth giving a shot) I've been looking for clustered FS or (if I have to) clustered NFS solutions, and there's not many out there. Unless you want to go with an expensive appliance. Currently you can't "roll your own" at all with FreeBSD. However, I think I've figured out a way that might work; basically using the memcached[1] idea of hashing the requests to identify which server is serving the data could alleviate any issues with NFS not being "globally" aware, and allow for multiple NFS servers to read from the same physical shared storage (note that the expectation is the shared storage is being accessed from multiple physical servers, and redundancy is done below the actual FS layer) If someone could write an NFS client using FUSE (perhaps?) and have it integrate with memcached[1], couldn't that basically allow for n+1 scaling out of NFS servers and storage? Basically any time the FS would need to issue or check a file lock, it would hit the memcache API to check if it's been locked or "represented" yet by a specific NFS server. (Note: whether or not it actually needs a "lock" or it's just saying "I am NFS server #1 and /foo/bar/baz.txt will be served from my server" is something I'd let someone smarter figure out - perhaps just taking ownership of files regardless of the need to lock it would work?) I'm just brain dumping here. Note that I am not that advanced of a programmer (I'm a PHP scripter) so filesystem and hardware I/O semantics are not my specialty; this is just what I've got from Googling until I'm blue in the face and why multi-path storage solutions are still not very cheap/open source (only a couple possibilities, each with limitations and caveats...) Perhaps someone could shed some light, tell me it's stupid, or tell me it's worthwhile? NFS is pretty much tried and true, adding a small middle layer in between to allow for distribution of NFS requests and alleviating the lock conflicts might be a much more usable solution? Perhaps this could fill the void of no GFS for FreeBSD? My idea for design would be N number of NFS servers behind a single virtual IP (or this "middleware" layer would be listening on this virtual IP intercepting all the NFS requests and routing them appropriately to the NFS servers behind them - also, for all intents and purposes, this middleware daemon *could* just be on the same physical machine as each NFS server - saving the need for separate middleware servers too) Thanks, mike [1] "memcached" is a distributed memory caching system - http://www.danga.com/memcached/ - it can be scaled up or out, and can handle failover inside of the protocol (if a memcached server disappears, it's info is forgotten and freed up for another active memcached server)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bd9320b30511272005q43cb974dm47d5fbaa1e389f90>