Date: Tue, 13 Feb 2007 11:19:06 -0600 From: Eric Anderson <anderson@freebsd.org> To: Chris Haulmark <chris@sigd.net> Cc: freebsd-fs@freebsd.org Subject: Re: UFS2 with SAN Message-ID: <45D1F30A.6080403@freebsd.org> In-Reply-To: <6FC9F9894A9F8C49A722CF9F2132FC2204C9DAB6@ms05.mailstreet2003.net> References: <45CD6FF5.8070007@freebsd.org> <20070213075627.63126.qmail@web34502.mail.mud.yahoo.com> <6FC9F9894A9F8C49A722CF9F2132FC2204C9DAB4@ms05.mailstreet2003.net> <a969fbd10702130839j727d055bu10c3ec80e38d2a3d@mail.gmail.com> <6FC9F9894A9F8C49A722CF9F2132FC2204C9DAB5@ms05.mailstreet2003.net> <6FC9F9894A9F8C49A722CF9F2132FC2204C9DAB6@ms05.mailstreet2003.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 02/13/07 11:00, Chris Haulmark wrote: >> From: Jeff Mohler [mailto:speedtoys.racing@gmail.com] >> Sent: Tuesday, February 13, 2007 10:40 AM >> To: Chris Haulmark >> Cc: Nicole Harrington; Eric Anderson; freebsd-fs@freebsd.org >> Subject: Re: UFS2 with SAN >> >> Its pretty much that simple. >> >> You cannot share SAN data..well..you -can- make a LUN appear as a >> shared NFS or CIFS share on a Netapp, but I havent tried it in a >> while..but you -can-. > > I set up a Linux cluster with GFS on a single SAN file system. It > included 8 nodes. Yep, and there's a handful of other linux supported cluster file systems out there, some good, some not so good. >> But in the normal world..you cannot...you would have to make it >> available via NFS to other client.  Thats the key difference between >> SAN and NAS. > > No. SAN and NAS is different based on their access. SAN was dependent > on either SCSI or Fibre until iSCSI (IPSAN) was introduced. NAS will > always require an IP address. If you were using Fibre and wanted to > use NAS, you would have to use iFCP or FCIP protocols. I do not need to > explain the limitation of using SCSI with NAS. > > If your responses on this SAN thread would not be productive, please > stay out. I am not interested to hear lectures about what is impossible. > > I asked about if anyone has tried to use UFS2 with only one node to > have write/read only while the rest would be read only. Right, and the answer comes down to - UFS is not cluster aware, and so it doesn't know to talk to other nodes to tell them to reload data from disk, nor does it know if two nodes are writing to the same file at the same time (think: appending to a log file from many nodes), locking, etc. Sharing the storage works (I have lots of FreeBSD boxen using the same fiber backend, with disks visible to all the nodes). You can mount the file system RO on many systems just fine too. *BUT* as soon as you mount an fs rw on another box, it's going to start making changes to the file system, and those changes will not be see on the other nodes. I have indeed tried it. You might look at tdfs (fuse module from Ivan Voras I believe). It's probably horribly beta (not knocking it Ivan - I'm sure it's an amazing start), so lots of testing would need to be done. It's also most likely not high performance. Do you need one node to be doing *lots* of writes to the file system, or just nightly updates, etc? Eric >> On 2/13/07, Chris Haulmark <chris@sigd.net> wrote: >> >> >>> -----Original Message----- >>> From: Nicole Harrington [mailto:drumslayer2@yahoo.com] >>> Sent: Tuesday, February 13, 2007 1:56 AM >>> To: Eric Anderson; Chris Haulmark >>> Cc: freebsd-fs@freebsd.org >>> Subject: Re: UFS2 with SAN >>> >>> --- Eric Anderson <anderson@freebsd.org> wrote: >>> >>>> On 02/10/07 00:54, Chris Haulmark wrote: >>>>>> -----Original Message----- >>>>>> From: Eric Anderson [mailto:anderson@freebsd.org ] >>>>>> Sent: Saturday, February 10, 2007 12:48 AM >>>>>> To: Chris Haulmark >>>>>> Cc: freebsd-fs@freebsd.org >>>>>> Subject: Re: UFS2 with SAN >>>>>> >>>>>> On 02/09/07 19:30, Chris Haulmark wrote: >>>>>>> Hello, >>>>>>> >>>>>>> I am looking into setting up a SAN with several >>>> web servers that >>>>>>> will be clustered.  It would be a FC network >>>> using Qlogic cards >>>>>>> in each of those FreeBSD web servers.  It would >>>> be about 5+ >>>>>>> of those web servers. >>>>>>> >>>>>>> I want to have the capability to share the same >>>> web data across >>>>>>> those web servers.  I have scorched the entire >>>> mailing list and >>>>>>> found that there were some work on GFS porting >>>> over to FreeBSD. >>>>>>> It seems like that it is just all talk and if I >>>> am wrong, could >>>>>>> you have my head turned over to where I can find >>>> out how to enable >>>>>>> GFS on those FreeBSD systems. >>>>>> GFS on FreeBSD is indeed dead.  Not enough people >>>> stepped up to help >>>>>> port it. >>>>> I really feared to hear that! >>>>> >>>>>>> If GFS is out of question, which file system am >>>> I recommendeded >>>>>>> to attempt to use for this SAN setup? >>>>>> NFS. >>>>>> >>>>>>> My first thought to use UFS2 and attempt is to >>>> allow only one web >>>>>>> server to have a write/read access while the >>>> reminder would be >>>>>>> read only access. That should prevent from >>>> lockings that is similar >>>>>>> on NFS/NAS. >>>>>> This will result it the read/write system seeing >>>> the data ok, and the >>>>>> rest getting corrupt data without knowing it, and >>>> probably crashing. >>>>>> UFS2 is not cluster aware.  You could mount all >>>> the hosts read only, >>>>>> and >>>>>> then update the mount point on one to rw, makes >>>> changes, then back to >>>>>> ro, then unmount/remount on the other boxes. >>>>> That's my original idea if I do not have anything >>>> else better to go >>>>> with. >>>>> >>>>>> That's all still a kludge to simulate what NFS >>>> will do for you.  Why >>>>>> won't NFS work for you? >>>>> I have a client who wants to go from NAS to a true >>>> SAN solution with >>>>> full >>>>> fibre channel network.  I would hate to lose the >>>> opportunity for this >>>>> client >>>>> to continue using FreeBSD as the choice of OS for >>>> his web servers. >>>>> Currently, >>>>> his set up is using NAS with NFS.  He complains of >>>> locking files that >>>>> occurs >>>>> too often. >>>>> >>>>> I had hoped to find more better solution and make >>>> this client much more >>>>> happier >>>>> with all the FreeBSD support that can be provided. >>>> >>>> Well, I'm not sure what issues they had, but have >>>> had fantastic success >>>> with NFS and FreeBSD.  FreeBSD with the right >>>> hardware and tweaks can >>>> make some NetApp boxes look weak. *cough* WAFL >>>> *cough* >>>> >>>> >>>>>> I agree that it would be fantastic to have a >>>> clustered file system for >>>>>> FreeBSD, and I've done lot's of hunting and >>>> nagging vendors to support >>>>>> it - but it's just not there. >>>>> We should get few bandwagons and get in circle. >>>> It could be likely that >>>>> I could >>>>> provide access for the developers to test and get >>>> whatever file system >>>>> and other >>>>> necessaries needed to be working. :) >>>> >>>> The problem isn't the environment or hardware, it's >>>> developers skilled >>>> to do the work.  They're all either in NDA's, off >>>> writing something >>>> else, or just too busy to provide any amount of >>>> input. >>>> >>>> Eric >>> >>> I have a set of servers NFS mounted to a Netapp and >>> after hurs of tuning with netapp's help. (after >>> getting through the idiots adking what FreeBSd was) >>>   I got very low performance. I was of course then told >>>   by Netapp to switch to Linux for better NFS support. >> That is what I would like to avoid telling my client to do >> The same thing. "Stay with NFS and tolerate it." >> >> I had hoped a SAN solution would be possible for >> FreeBSD.  So far, it appears that it is not possible to >> share the same file system across several web servers. >> >> Chris >>>   I would love for any help with tuning this further, >>> but I cannot say that FreeBSD with Netapp NFS will be >>> great. Of course, I have not been able to test if >>> indeed Linux would be any better. >>>   I will say however that I have a large number of >>> small files which tends to not do well with NFS. >>> >>> >>>   Nicole >>> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?45D1F30A.6080403>