Date: Mon, 13 Jan 2003 10:29:09 +0100 From: Andy Sporner <sporner@nentec.de> To: "Rouzer, Charles A (Chuck)" <car@vitalit.com> Cc: freebsd-cluster@FreeBSD.ORG Subject: Re: freebsd cluster target market Message-ID: <3E2286E5.20609@nentec.de> References: <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> <001701c2ba84$6f307010$0201000a@LAPTOP> <37722441765.20030112230029@buz.ch> <003a01c2ba88$b6f20c70$0201000a@LAPTOP> <31724072031.20030112232740@buz.ch> <004901c2ba8a$cc16ba90$0201000a@LAPTOP>
next in thread | previous in thread | raw e-mail | index | archive | help
This seems to be a good of time as any to post my status on the bproc port. I have been at this since Dec-23 and I can only say that it is not as easy as I had imagined. It has been relegated to a lower priority because there are many things that I garnered from it that make sense in my phase 2 plans. However I have a fundamental problem with some of the architectural choices that were made. If somebody has more cycles, I can make ready my initial work on porting the BPROC stuff in the form of a tar file. What is missing is the actual mechanics of the process extraction and the hooks in the OS. Where I differ with some of the ideas is that the communications should be done in a user process, not in the kernel. Call me a traditionalist, but I don't see that much savings in this particular context to load more stuff there. It is more attractive to make a series of IOCTL functions to insert and remove processes leaving a bare minimum of things in the kernel. I have searched the web and found a project called "Maui" that is BPROC API compatible and also exists for FreeBSD. I have not looked much more into it. But it might be a point on where some urgent needs for this kind of thing can be resolved. As for filesystems and the like. I had also time to make a survey of all of the items in consideration: - DRBD - IScsi - The problems of NFS. I have come to the conclusion that, while a low layer solution is fine, there still is a problem coordinating access to filesystems built on top. Iscsi would make failover of the devices easier, but you still have to syncronize access. As for NFS, problems are noted with regard to changing things under the cover--due largely to the opaque nature of the file handles. I am still studying this, but I think it is possible to convert the filehandles into a physical reference (such as filesystem_id:Inode) so that this reference can float around and be stateless within a cluster. In this way this application does not even know which server he got the information from. This takes care of the top level syncronization. The main point I have not come to a conclusion on yet, is to either patch NFS to handle things this way, or to just do a new FS from scratch. Adding NFS functionality would be great, but there are some new directions I also want to pursue as well. Such as a registration service so that a Filesystem can be found on the network by request. That way when a failover occurs, the I/O can continue. Additionally I would like this to be TCP based so that very large buffers can be transmitted ( > 64K). This has the additional benefit of being ready for WAN. This is really just a small snip from a document I will be publishing on the web (as soon as Deutsche Telekom get re-establish my DSL!!!). What is coming in the same time frame is release 215 of my software suite, which now includes a network NAT to do load balancing as well as regular failover. This would allow for highly available virtual servers. The next step after 215 is to focus on the filesystem that I described above. Hopefully 2nd week of February this is realizable. It would be good if I-scsi is available because that just would make the failover stuff more flexible. I plan initially to use two servers with shared SCSI busses. It would be great to throw this away in favor of I-SCSI. Bye for now (more work to do..... ;-) Andy To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-cluster" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E2286E5.20609>