Date: Thu, 03 May 2012 06:34:46 -0700 From: Matthew Jacob <mj@feral.com> To: Trent Nelson <trent@snakebite.org> Cc: freebsd-scsi@freebsd.org Subject: Re: Missing sysctl options for isp driver Message-ID: <4FA28976.2070607@feral.com> In-Reply-To: <20120503050956.GA18983@snakebite.org> References: <20120502203948.GA1141@snakebite.org> <4FA1DF1C.2030706@feral.com> <20120503050956.GA18983@snakebite.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> I, uh, I don't think I'm using target mode. I've got four Xyratex RS- > 1600-FC2 JBODs (16x146GB + 48x36GB) connected to two-but-soon-to-be-four > HP StorageWorks SAN Switch 2/16s. Each FreeBSD box has a minimum of two > FC HBA ports; each HBA port goes to a different switch, and zoning config > on the switches controls which disks each hosts sees (although more on > this later). > > So, uh, I think that constitutes fabric mode right? The switch reports > the HBA ports as F-Ports and, when zoning's enabled, all the FLOGI/PLOGI > stuff seems to work properly. > > (My SAN/FC-fu has been cobbled together from trial-and-error, so I'm not > entirely sure what target mode is or whether or not target mode bugs are > likely to affect me.) Target Mode is where the FreeBSD box can pretend to be a disk. Yes, that's fabric. > > >> Did you have specific bugs you were concerned with? > Well... as I'm still playing around with the setup, I'm not sure if some > of the oddities I'm seeing are bugs or not. Things seem to work when I've > got zoning enabled on the SAN switches. > > When zoning's disabled and the HBAs are plugged into the fabric, things > sort of go a bit ballistic... The isp driver doesn't seem to handle getting > 128 RSCNs (2 for each dual-ported FC drive) in one go very well. Mailbox > timeouts, hung `camcontrol rescan all`'s, lots of isp[n] dmesg errors and > warnings -- even the SAN switch console starts barfing errors. > > This sort of stuff (queue random dmesg snippets from earlier): > > (da37:isp1:0:88:0): got CAM status 0x4a > (da37:isp1:0:88:0): fatal error, failed to attach to device > (da37:isp1:0:88:0): removing device entry > (da2:isp0:0:9:0): Synchronize cache failed, status == 0x1b, scsi status == 0x0 > (da38:isp1:0:102:0): got CAM status 0x4a > (da38:isp1:0:102:0): fatal error, failed to attach to device > (da38:isp1:0:102:0): removing device entry > isp0: Chan 0 PLOGI 0x020500 failed > isp0: Chan 0 PLOGI 0x020600 failed > isp0: Chan 0 PLOGI 0x020800 failed > isp0: Mailbox Command 'BUS RESET' failed (COMMAND ERROR) > isp1: Mailbox Command 'BUS RESET' failed (COMMAND ERROR) > (da119:isp1:0:3:0): lost device - 0 outstanding, 0 refs > (da119:isp1:0:3:0): removing device entry > isp1: Chan 0 PLOGI 0x0201ca failed > isp1: Chan 0 PLOGI 0x0201cb failed > isp0: Chan 0 PLOGI 0x0101e0 failed > isp0: Chan 0 PLOGI 0x0101e1 failed > isp0: Chan 0 PortID 0x0102ad already at 512 handle 35 state 5 > isp0: Chan 0 PortID 0x0102ae already at 512 handle 34 state 5 > isp0: Chan 0 PortID 0x0102b1 already at 512 handle 33 state 5 > isp0: Chan 0 PortID 0x0102b2 already at 512 handle 32 state 5 > isp0: Chan 0 PortID 0x0102b3 already at 512 handle 31 state 5 > > If I plug the HBA directly into a port on the JBOD, I get no such issues; > all /dev/da* device nodes are created instantly and everything just seems > to work. (I presume HBA->JBOD would result in an FC-AL topology rather > than an F-port topology, which I figure translates into a different code > path in the isp driver.) > > When I enabled zoning and configured a HBA to only see a handful of disks, > everything behaved properly. I'm going to update to head and get a debug > build going with all the ISP debug bells and whistles; I'll report back on > HBA->fabric behaviour when zoning is off after that. > > (FWIW, I've got two HP-UX (PA-RISC and IA64) boxes with two HBAs also > plugged into the fabric; they don't go into fits like FreeBSD/isp seems > to when zoning is turned off.) > What kind of switches are you using? This is definitely wrong, and probably means that I need to do some work again here.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4FA28976.2070607>