From owner-freebsd-fs@FreeBSD.ORG Tue Feb 13 17:28:29 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A885516A402; Tue, 13 Feb 2007 17:28:29 +0000 (UTC) (envelope-from chris@sigd.net) Received: from ms05.mailstreet2003.net (ms05.mailstreet2003.net [69.25.50.235]) by mx1.freebsd.org (Postfix) with ESMTP id 6073913C4A6; Tue, 13 Feb 2007 17:28:28 +0000 (UTC) (envelope-from chris@sigd.net) X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Tue, 13 Feb 2007 12:28:27 -0500 Message-ID: <6FC9F9894A9F8C49A722CF9F2132FC2204C9DAB7@ms05.mailstreet2003.net> In-Reply-To: <45D1F30A.6080403@freebsd.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: UFS2 with SAN Thread-Index: AcdPkxawNHXk2RCtSnSIXeJRpl2FGAAAHXnQ References: <45CD6FF5.8070007@freebsd.org> <20070213075627.63126.qmail@web34502.mail.mud.yahoo.com> <6FC9F9894A9F8C49A722CF9F2132FC2204C9DAB4@ms05.mailstreet2003.net> <6FC9F9894A9F8C49A722CF9F2132FC2204C9DAB5@ms05.mailstreet2003.net> <6FC9F9894A9F8C49A722CF9F2132FC2204C9DAB6@ms05.mailstreet2003.net> <45D1F30A.6080403@freebsd.org> From: "Chris Haulmark" To: "Eric Anderson" Cc: freebsd-fs@freebsd.org Subject: RE: UFS2 with SAN X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2007 17:28:29 -0000 > -----Original Message----- > From: Eric Anderson [mailto:anderson@freebsd.org] > Sent: Tuesday, February 13, 2007 11:19 AM > To: Chris Haulmark > Cc: freebsd-fs@freebsd.org > Subject: Re: UFS2 with SAN >=20 > On 02/13/07 11:00, Chris Haulmark wrote: > >> From: Jeff Mohler [mailto:speedtoys.racing@gmail.com] > >> Sent: Tuesday, February 13, 2007 10:40 AM > >> To: Chris Haulmark > >> Cc: Nicole Harrington; Eric Anderson; freebsd-fs@freebsd.org > >> Subject: Re: UFS2 with SAN > >> > >> Its pretty much that simple. > >> > >> You cannot share SAN data..well..you -can- make a LUN appear as a > >> shared NFS or CIFS share on a Netapp, but I havent tried it in a > >> while..but you -can-. > > > > I set up a Linux cluster with GFS on a single SAN file system. It > > included 8 nodes. >=20 >=20 > Yep, and there's a handful of other linux supported cluster file > systems > out there, some good, some not so good. >=20 >=20 > >> But in the normal world..you cannot...you would have to make it > >> available via NFS to other client.=C3' =C3' Thats the key = difference > between > >> SAN and NAS. > > > > No. SAN and NAS is different based on their access. SAN was > dependent > > on either SCSI or Fibre until iSCSI (IPSAN) was introduced. NAS = will > > always require an IP address. If you were using Fibre and wanted to > > use NAS, you would have to use iFCP or FCIP protocols. I do not = need > to > > explain the limitation of using SCSI with NAS. > > > > If your responses on this SAN thread would not be productive, please > > stay out. I am not interested to hear lectures about what is > impossible. > > > > I asked about if anyone has tried to use UFS2 with only one node to > > have write/read only while the rest would be read only. >=20 >=20 > Right, and the answer comes down to - UFS is not cluster aware, and so > it doesn't know to talk to other nodes to tell them to reload data = from > disk, nor does it know if two nodes are writing to the same file at = the > same time (think: appending to a log file from many nodes), locking, > etc. >=20 I figured that if UFS was my only solution, I would write up a script that would handle the talking to the other nodes to umount/mount the file system on demand. > Sharing the storage works (I have lots of FreeBSD boxen using the same > fiber backend, with disks visible to all the nodes). You can mount = the > file system RO on many systems just fine too. *BUT* as soon as you > mount an fs rw on another box, it's going to start making changes to > the > file system, and those changes will not be see on the other nodes. I > have indeed tried it. >=20 > You might look at tdfs (fuse module from Ivan Voras I believe). It's > probably horribly beta (not knocking it Ivan - I'm sure it's an = amazing > start), so lots of testing would need to be done. It's also most > likely > not high performance. Thanks for the tip! >=20 > Do you need one node to be doing *lots* of writes to the file system, > or > just nightly updates, etc? Just updates from the staging production. I doubt it would happen as often as the nightly updates. The client mentioned having a staging environment so I am sure that the site would be tested before doing a final update. It feels more likely to be weekly updates. Oliver, thanks too. Chris >=20 >=20 > Eric >=20 >=20 > >> On 2/13/07, Chris Haulmark wrote: > >> > >> > >>> -----Original Message----- > >>> From: Nicole Harrington [mailto:drumslayer2@yahoo.com] > >>> Sent: Tuesday, February 13, 2007 1:56 AM > >>> To: Eric Anderson; Chris Haulmark > >>> Cc: freebsd-fs@freebsd.org > >>> Subject: Re: UFS2 with SAN > >>> > >>> --- Eric Anderson wrote: > >>> > >>>> On 02/10/07 00:54, Chris Haulmark wrote: > >>>>>> -----Original Message----- > >>>>>> From: Eric Anderson [mailto:anderson@freebsd.org ] > >>>>>> Sent: Saturday, February 10, 2007 12:48 AM > >>>>>> To: Chris Haulmark > >>>>>> Cc: freebsd-fs@freebsd.org > >>>>>> Subject: Re: UFS2 with SAN > >>>>>> > >>>>>> On 02/09/07 19:30, Chris Haulmark wrote: > >>>>>>> Hello, > >>>>>>> > >>>>>>> I am looking into setting up a SAN with several > >>>> web servers that > >>>>>>> will be clustered.=C3' =C3' It would be a FC network > >>>> using Qlogic cards > >>>>>>> in each of those FreeBSD web servers.=C3' =C3' It would > >>>> be about 5+ > >>>>>>> of those web servers. > >>>>>>> > >>>>>>> I want to have the capability to share the same > >>>> web data across > >>>>>>> those web servers.=C3' =C3' I have scorched the entire > >>>> mailing list and > >>>>>>> found that there were some work on GFS porting > >>>> over to FreeBSD. > >>>>>>> It seems like that it is just all talk and if I > >>>> am wrong, could > >>>>>>> you have my head turned over to where I can find > >>>> out how to enable > >>>>>>> GFS on those FreeBSD systems. > >>>>>> GFS on FreeBSD is indeed dead.=C3' =C3' Not enough people > >>>> stepped up to help > >>>>>> port it. > >>>>> I really feared to hear that! > >>>>> > >>>>>>> If GFS is out of question, which file system am > >>>> I recommendeded > >>>>>>> to attempt to use for this SAN setup? > >>>>>> NFS. > >>>>>> > >>>>>>> My first thought to use UFS2 and attempt is to > >>>> allow only one web > >>>>>>> server to have a write/read access while the > >>>> reminder would be > >>>>>>> read only access. That should prevent from > >>>> lockings that is similar > >>>>>>> on NFS/NAS. > >>>>>> This will result it the read/write system seeing > >>>> the data ok, and the > >>>>>> rest getting corrupt data without knowing it, and > >>>> probably crashing. > >>>>>> UFS2 is not cluster aware.=C3' =C3' You could mount all > >>>> the hosts read only, > >>>>>> and > >>>>>> then update the mount point on one to rw, makes > >>>> changes, then back to > >>>>>> ro, then unmount/remount on the other boxes. > >>>>> That's my original idea if I do not have anything > >>>> else better to go > >>>>> with. > >>>>> > >>>>>> That's all still a kludge to simulate what NFS > >>>> will do for you.=C3' =C3' Why > >>>>>> won't NFS work for you? > >>>>> I have a client who wants to go from NAS to a true > >>>> SAN solution with > >>>>> full > >>>>> fibre channel network.=C3' =C3' I would hate to lose the > >>>> opportunity for this > >>>>> client > >>>>> to continue using FreeBSD as the choice of OS for > >>>> his web servers. > >>>>> Currently, > >>>>> his set up is using NAS with NFS.=C3' =C3' He complains of > >>>> locking files that > >>>>> occurs > >>>>> too often. > >>>>> > >>>>> I had hoped to find more better solution and make > >>>> this client much more > >>>>> happier > >>>>> with all the FreeBSD support that can be provided. > >>>> > >>>> Well, I'm not sure what issues they had, but have > >>>> had fantastic success > >>>> with NFS and FreeBSD.=C3' =C3' FreeBSD with the right > >>>> hardware and tweaks can > >>>> make some NetApp boxes look weak. *cough* WAFL > >>>> *cough* > >>>> > >>>> > >>>>>> I agree that it would be fantastic to have a > >>>> clustered file system for > >>>>>> FreeBSD, and I've done lot's of hunting and > >>>> nagging vendors to support > >>>>>> it - but it's just not there. > >>>>> We should get few bandwagons and get in circle. > >>>> It could be likely that > >>>>> I could > >>>>> provide access for the developers to test and get > >>>> whatever file system > >>>>> and other > >>>>> necessaries needed to be working. :) > >>>> > >>>> The problem isn't the environment or hardware, it's > >>>> developers skilled > >>>> to do the work.=C3' =C3' They're all either in NDA's, off > >>>> writing something > >>>> else, or just too busy to provide any amount of > >>>> input. > >>>> > >>>> Eric > >>> > >>> I have a set of servers NFS mounted to a Netapp and > >>> after hurs of tuning with netapp's help. (after > >>> getting through the idiots adking what FreeBSd was) > >>> =C3' =C3' I got very low performance. I was of course then told > >>> =C3' =C3' by Netapp to switch to Linux for better NFS support. > >> That is what I would like to avoid telling my client to do > >> The same thing. "Stay with NFS and tolerate it." > >> > >> I had hoped a SAN solution would be possible for > >> FreeBSD.=C3' =C3' So far, it appears that it is not possible to > >> share the same file system across several web servers. > >> > >> Chris > >>> =C3' =C3' I would love for any help with tuning this further, > >>> but I cannot say that FreeBSD with Netapp NFS will be > >>> great. Of course, I have not been able to test if > >>> indeed Linux would be any better. > >>> =C3' =C3' I will say however that I have a large number of > >>> small files which tends to not do well with NFS. > >>> > >>> > >>> =C3' =C3' Nicole > >>> > >> _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to "freebsd-fs- > unsubscribe@freebsd.org" > >