Date: Mon, 29 Sep 2014 21:25:02 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Karl Denninger" <karl@denninger.net>, <freebsd-stable@freebsd.org> Subject: Re: MPS Message-ID: <018BC41041EE4DF589EF5D834AD45BAD@multiplay.co.uk> References: <5429BB41.8080609@denninger.net>
next in thread | previous in thread | raw e-mail | index | archive | help
I seem to remember detection being made parallel for speed, as there
can be lots of drives. The Avago guys may be able to shed more light.
Regards
Steve
----- Original Message -----
From: "Karl Denninger" <karl@denninger.net>
To: <freebsd-stable@freebsd.org>
Sent: Monday, September 29, 2014 9:04 PM
Subject: MPS
Has anyone found a set of configuration settings for the mps driver (LSI
SAS series cards) that gets the following situation under control?
I have a machine here with one of these cards and a SAS expander. The
expander has 16 ports, the card has 8 -- one connector attached to the
expander. So I have 20 potential drives associated with this adapter.
The issue is that the driver makes utterly no sense when it comes to
assigning drive designations. In theory it should enumerate in series;
that is, target 0 on the card should be da0, target 1 da1, etc. This
assumes there are no gaps in the attached drives; if there are then
things may move around, which is why some people wire drives.
Ok so far, but it doesn't behave that way!
This is the excerpt of what it returns:
root@Dbms-10:/boot # dmesg|grep mps
mps0: <LSI SAS2008> port 0xc000-0xc0ff mem
0xfbb3c000-0xfbb3ffff,0xfbb40000-0xfbb7ffff irq 30 at device 0.0 on pci3
mps0: Firmware: 19.00.00.00, Driver: 19.00.00.00-fbsd
mps0: IOCCapabilities:
1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
ses0 at mps0 bus 0 scbus0 target 8 lun 0
da0 at mps0 bus 0 scbus0 target 0 lun 0
da1 at mps0 bus 0 scbus0 target 1 lun 0
da2 at mps0 bus 0 scbus0 target 2 lun 0
da3 at mps0 bus 0 scbus0 target 3 lun 0
*da4 at mps0 bus 0 scbus0 target 6 lun 0**
**da5 at mps0 bus 0 scbus0 target 7 lun 0*
Those last two are the drives the card identifies as being on targets 0
and 1 when looked at in the BIOS. The other four drives (0-3) are on
the SAS expander -- and there's a two drive "gap" in the targets
identified as well which has zippo for logic associated with it.
Even more-oddly if I swap the plugs -- that is, plug the two boot disks
into the OTHER connector (putatively targets 4-7) and put the SAS
expander on the 0-3 port connector the BIOS *does* show the target shift
but the machine, when it boots, still places those two drives on targets
6 and 7!
There is nothing in /boot/loader.conf or /boot/device.hints that relates
to these devices at all.
I would like to wire down the various ports so Drive Bay #x corresponds
to da[x] but how can you when you don't know what order they will come
up in predicated on either the physical port they're occupying or, for
that matter, what the card's BIOS shows them as?
I looked in the driver's man page and saw nothing related to this; I get
around it by labeling the drives and then using "glabel list" to see
what da[x] number it grabbed if I need to pull a disk while the machine
is up, but this is a rather-unfriendly way to handle that.
--
Karl Denninger
karl@denninger.net <mailto:karl@denninger.net>
/The Market Ticker/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?018BC41041EE4DF589EF5D834AD45BAD>