From owner-freebsd-scsi@FreeBSD.ORG Mon Sep 12 00:11:35 2005 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 870E816A41F; Mon, 12 Sep 2005 00:11:35 +0000 (GMT) (envelope-from jonathan@fosburgh.org) Received: from pop-siberian.atl.sa.earthlink.net (pop-siberian.atl.sa.earthlink.net [207.69.195.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id 32D9A43D45; Mon, 12 Sep 2005 00:11:35 +0000 (GMT) (envelope-from jonathan@fosburgh.org) Received: from user-0cetsk1.cable.mindspring.com ([24.238.242.129] helo=gw.fosburgh.org) by pop-siberian.atl.sa.earthlink.net with esmtp (Exim 3.36 #10) id 1EEbv4-0007g1-00; Sun, 11 Sep 2005 20:11:34 -0400 Received: by gw.fosburgh.org (Postfix, from userid 1001) id E8305B781; Sun, 11 Sep 2005 19:13:54 -0500 (CDT) From: Jonathan Fosburgh To: freebsd-questions@freebsd.org Date: Sun, 11 Sep 2005 19:13:53 -0500 User-Agent: KMail/1.8.2 References: <20050909185935.GI31865@ratchet.nebcorp.com> In-Reply-To: <20050909185935.GI31865@ratchet.nebcorp.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200509111913.54170.jonathan@fosburgh.org> Cc: Danny Howard , freebsd-scsi@freebsd.org Subject: Re: Fibre Channel disks to two Systems? X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Sep 2005 00:11:35 -0000 On Friday 09 September 2005 01:59 pm, Danny Howard wrote: > > >From my research, I am thus far most impressed with the SANbloc 2Gb, > > which holds fourteen FC drives in a 3U rackmount. It can be had with > redundant RAID controllers, or as a JBOD. There are similar products > from other vendors as well. Unfortunately I have no experience doing anything SAN related on FreeBSD, but I think I can say that you would do just fine with any Engenio-based (IBM, StorageTek, Maxxan, etc) product. They seem to manufacture most of the modular fibre channel storage out there, and they also sell FC-attached SATA hardware. Of course if someone actually mentions they support FreeBSD then probably go with that, otherwise you are likely to be on your own. > > I could concievably do the RAID in software by running a gstripe across a > set of gmirrors. > > As I understand it, I can have an FC loop with one or more drives, > connected to two servers, and either server can talk to one or the other > drives exclusively. My QUESTION is: how is the arbitration done in > FreeBSD? You run camcontrol on either server and activate / deactivate > drives in the loop? > > What happens if say, the primary server locks up in some weird manner? > Can it block the backup server from talking to the drives? (We can > always have a NOC tech turn off a badly failed primary database, and > power-cycle the disk array, if needed ...) > > A really far-out idea I had was that with fourteen drive bays I could > have two hot spares, and then set up a stripe across four mirrored pairs > (4x2 = 8-disk RAID10) and then with the remaining four drives assign > each to be a third component of the gmirrored pairs, let the gmirrors > sync up, then detach those drives from the gmirrors, mount them on the > backup database, gstripe those containers together, and have a > point-in-time "snapshot" of the drive array that could be mounted on the > backup server, from which I could run database dumps, or conduct > failover tests, etc. (I could kick this around -geom. :) > > Uhmmm, has anyone done similar? Suggestions? Feedback? Advice? > > Or, should I try to get a NetApp, or similar device, even though FreeBSD > does not support iSCSI, because NFS performance over GigE may still beat > FC? I can't believe that FreeBSD would have such poor support for FC that NFS performs better. :) Remember you dealing with block I/O on the storage array and file I/O on the NetApp (which I think supports either NFS or CIFS). There are (or have been) some proprietary database vendors (Microsoft comes to mind...) who don't allow their databases to run on any kind of NAS setup (which is what the NetApp is). Granted I don't know if Postres gives support for putting databases on raw devices or if it only supports putting the database on a filesystem, so you may not get the full benefits of using block storage. Still, for a database, I think you should use block storage if at all possible. Alas, I can't speak to how you can give multiple FreeBSD systems access to the same storage device. > > Also, does anyone have a FreeBSD-friendly storage systems integrator or > other vendor they can reccomend, particularly one near the San Francisco > area? I keep contacting various vendors who then fail to get back to > me. :( Personally I have never seen a storage vendor who supports FreeBSD, but I deal with large enterprise players where Linux is only just in the last couple of years enjoying broad support. Maybe the smaller vendors are more likely to support FreeBSD. Are there any Bay area FBSD user's groups? You might try them. > > Thanks for all feedback and suggestions! > > Sincerely, > -danny -- Jonathan Fosburgh Storage Engineer/Architect