Date: Mon, 9 Jun 2008 19:44:37 +0200 From: Wilko Bulte <wb@freebie.xs4all.nl> To: Andy Kosela <andy.kosela@gmail.com> Cc: msaad@datapipe.com, rv@openusenet.org, freebsd-stable@freebsd.org Subject: Re: Current status of support for high end SAN hardware Message-ID: <20080609174437.GB31050@freebie.xs4all.nl> In-Reply-To: <3cc535c80806090541g15c03c24mf1c47a9b92ff166a@mail.gmail.com> References: <3cc535c80806071158h44ec9be1pbe72ca6711016bde@mail.gmail.com> <484C4E6D.7060306@datapipe.com> <20080609103610.GA36227@riffraff.plig.net> <3cc535c80806090541g15c03c24mf1c47a9b92ff166a@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Quoting Andy Kosela, who wrote on Mon, Jun 09, 2008 at 02:41:24PM +0200 .. > On Mon, Jun 9, 2008 at 12:36 PM, Russell Vincent <rv@openusenet.org> wrote: > > > > The FreeBSD support for multipath/SAN is fairly poor. It's fiddly > > to get to work and boot times are a little variable (into the > > minutes) as it tries to discover the devices. Once it is configured > > and booted, it just works as long as things don't go wrong. SAN > > outages cause the machine to hang up until the issue is resolved > > (in which case it just seems to continue) or it doesn't recover at > > all and requires a reboot. Note that I don't spend a significant > > amount of time on this, so it may be that I could do things a little > > better. I have also not tested the failover stuff very well (I > > only upgraded this machine to 7-STABLE fairly recently). Disk > > access seems to be restricted to a single path at a time. Problem > > solving is very tricky as there is very little information to trace > > which path/disk refers to which fabric/storage device/LUN. > > > > Russell, > Thank you for your insights. It's good to see you have no problems > with isp(4) and Qlogic HBAs. Though I'm concerned about > multipathing. We run 6.x-RELEASE releases so it seems we have > to upgrade to 7.0-RELEASE to achieve that goal. gmultipath(8) > code is fairly new so I suppose it's not that mature yet as in > Linux. Unfortunately it is only an active/passive approach with > no load balancing (the active path is active until a BIO request is > failed with EIO or ENXIO) Well, it is worse than that: HP EVA arrays have 2 different multipath behaviours. VCS 3.x firmware is active/passive, whereas VCS 4.x is active/active. Active/active as well are all XCS firmware versions. All newer EVAs run XCS. To properly do multipathing on active/active EVAs your multipathing software should be aware of the ALUA stuff. On active/active EVAs you should fire your read commands to the right HSV controller, if you take the other HSV things will work, but the controller you sent the read to will have to do a proxy read to the HSV controller that "owns" the LUN. This will be slower that taking the right HSV in one go. It also consumes bandwith on the HSV controller's mirror port(s). Writes do not matter, they are mirrored to the write cache in each HSV anyway. Linux devicemapper is sort-of getting there now. For the longest time HP did not support devicemapper, and for good reasons too. To make things interesting I think I remember Matt (the isp(4) author) telling me that some ispfw(4) versions not getting it right. If the adapter firmware does not properly inform the isp(4) driver of SAN disruptions you are obviously doomed as a multipathing driver that depends on the underlying HBA drivers to feed SAN problems/status upstream. In short: if you use isp(4), always load ispfw(4) too. Getting all the components in a FC SAN to DTRT is unfortunately still a major pain in the backside. hth Wilko -- Wilko Bulte wilko@FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080609174437.GB31050>