From owner-freebsd-scsi@FreeBSD.ORG Sat Sep 15 04:09:07 2012 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.ORG Received: by hub.freebsd.org (Postfix, from userid 821) id 615B21065670; Sat, 15 Sep 2012 04:09:07 +0000 (UTC) Date: Sat, 15 Sep 2012 04:09:07 +0000 From: John To: FreeBSD iSCSI Message-ID: <20120915040907.GA5458@FreeBSD.org> References: <20120915022437.GA90210@FreeBSD.org> <20120915023329.GA55292@nargothrond.kdm.org> <20120915031305.GA97685@FreeBSD.org> <20120915032826.GA63349@nargothrond.kdm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120915032826.GA63349@nargothrond.kdm.org> User-Agent: Mutt/1.4.2.1i Cc: "Kenneth D. Merry" Subject: Re: How to force a reset of a device (disk) in an enclosre slot X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Sep 2012 04:09:07 -0000 ----- Kenneth D. Merry's Original Message ----- > On Sat, Sep 15, 2012 at 03:13:05 +0000, John wrote: > > ----- Kenneth D. Merry's Original Message ----- > > > On Sat, Sep 15, 2012 at 02:24:37 +0000, John wrote: > > > > Hi Folks, > > > > > > > > I've been poking around and can't seem to find a way to reset and > > > > hopefully acquire access to a disk device in an enclosure. For instance: > > > > > > > > FreeBSD 9.1-PRERELEASE > > > > > > > > # camcontrol smpphylist ses4 > > > > 37 PHYs: > > > > PHY Attached SAS Address > > > > 0 0x5000039368233602 (pass105,da98) > > > > 1 0x5000039368238e3e (pass106,da99) > > > > 2 0x500003936823bca2 (pass107,da100) > > > > 3 0x500003936819507e (pass108,da101) > > > > 4 0x5000039368197d5a (pass109,da102) > > > > 5 0x5000039368197c6e (pass110,da103) > > > > 6 0x500003936818770e (pass111,da104) > > > > 7 0x5000039368238eba (pass112,da105) > > > > 8 0x5000039368232f42 (pass113,da106) > > > > 9 0x0000000000000000 > > > > 10 0x500003936813c31e > > > > 11 0x5000039368233892 (pass114,da107) > > > > 12 0x500003936813c2ca (pass115,da108) > > > > ... > > > > > > > > Note, bay/slot 10 has a listed device address. If I were to pull the > > > > drive and re-insert it, it would show up (as da390 in this case). > > > > The above is after a fresh reboot. Note da106 to da107 skipping > > > > slot 10 (slot 9 is empty). > > > > > > > > The smp utils provide a similar view: > > > > > > > > # smp_discover /dev/ses4 > > > > phy 0:D:attached:[5000039368233602:00 t(SSP)] 6 Gbps > > > > phy 1:D:attached:[5000039368238e3e:00 t(SSP)] 6 Gbps > > > > phy 2:D:attached:[500003936823bca2:00 t(SSP)] 6 Gbps > > > > phy 3:D:attached:[500003936819507e:00 t(SSP)] 6 Gbps > > > > phy 4:D:attached:[5000039368197d5a:00 t(SSP)] 6 Gbps > > > > phy 5:D:attached:[5000039368197c6e:00 t(SSP)] 6 Gbps > > > > phy 6:D:attached:[500003936818770e:00 t(SSP)] 6 Gbps > > > > phy 7:D:attached:[5000039368238eba:00 t(SSP)] 6 Gbps > > > > phy 8:D:attached:[5000039368232f42:00 t(SSP)] 6 Gbps > > > > phy 10:D:attached:[500003936813c31e:00 t(SSP)] 6 Gbps > > > > phy 11:D:attached:[5000039368233892:00 t(SSP)] 6 Gbps > > > > phy 12:D:attached:[500003936813c2ca:00 t(SSP)] 6 Gbps > > > > ... > > > > > > > > The address of slot 10 matches. There is a disk in the slot - just > > > > isn't recognized and attached. > > > > > > > > Back to the basic question. How can I issue a command to the enclosure > > > > to force a re-initialization of the device to recover it without > > > > having to physically pull & insert it. Even if the device numbers > > > > are not sequential, I need access to the drive... > > > > > > You can try sending a link reset: > > > > > > camcontrol smppc ses4 -p 10 -o linkreset > > > > > > It may or may not work. You can also try disabling the PHY (-o disable) > > > and then sending a link reset to re-enable the link. You can also try a > > > hard reset (-o hardreset) > > > > Hi Ken, > > > > Well, I hadn't tried to actually disable the device. That did bring some > > reaction: > > > > # camcontrol smppc ses4 -p 10 -o disable > > # camcontrol smpphylist ses4 > > 37 PHYs: > > PHY Attached SAS Address > > 0 0x5000039368233602 (pass105,da98) > > .... > > 8 0x5000039368232f42 (pass113,da106) > > 9 0x0000000000000000 > > 10 0x0000000000000000 > > 11 0x5000039368233892 (pass114,da107) > > ... > > > > The device is gone. > > > > # camcontrol smppc ses4 -p 10 -o hardreset > > root@vprzfs01p:/root # camcontrol smpphylist ses4 > > 37 PHYs: > > PHY Attached SAS Address > > 0 0x5000039368233602 (pass105,da98) > > .... > > 8 0x5000039368232f42 (pass113,da106) > > 9 0x0000000000000000 > > 10 0x500003936813c31e > > 11 0x5000039368233892 (pass114,da107) > > ... > > > > The device is back, but not attached - This msg: > > > > kernel: mps1: mpssas_alloc_tm freezing simq > > kernel: mps1: mpssas_remove_complete on handle 0x0069, IOCStatus= 0x0 > > kernel: mps1: mpssas_free_tm releasing simq > > kernel: _mapping_add_new_device: failed to add the device with handle 0x0069 to persistent table because there is no free space available - entry 0 > > That message is harmless, it won't prevent the drive from attaching. > > > >From a debug statement in the driver: MaxPersistentEntries == 128, but I > > have more than 128 devices per LSI card and they normally all show up - > > though I do get a bunch of the above messages in dmesg.. > > You might try turning on some of the debugging in the mps(4) driver and > disabling and resetting the link again. > > Try: > > sysctl -w dev.mps.0.debug_level=0xf > > You might get a lot of output, so be prepared to reset it back to 4: > > sysctl -w dev.mps.0.debug_level=4 Hi Ken, I don't see anything obvious. Hopefully you're more familair with the code and have better eyes than I do... Here's everything from messages after the -o disable. There are some "unknown/unhandled"s showing up. kernel: mpssas_action func 0x4 kernel: mpssas_action func 0x15 kernel: mpssas_action func 0x91b kernel: sending SMP request to SAS address 0x5001438014a0a0bf kernel: mps_enqueue_request SMID 689 cm 0xffffff80013cd2c8 ccb 0 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 214 kernel: mps_complete_command cm 0xffffff80013cd2c8 calling cm_complete 0xffffffff805da470 data 0xfffffe002c7ca000 reply 0xffffff96f9f31800 kernel: SMP request to SAS address 0x5001438014a0a0bf completed successfully kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 215 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 215 kernel: mpssas_evt_handler kernel: kernel: 2 kernel: 0 kernel: SasDiscovery (0x16) kernel: 0x0 kernel: 1 kernel: Discovery Started kernel: 5 kernel: 0 kernel: mpssas_evt_handler kernel: kernel: 7 kernel: 0 kernel: SasDeviceStatusChange (0xf) kernel: 0x78 kernel: 0xffff kernel: Internal Device Reset kernel: 0x0 kernel: 0x0 kernel: 0x78 kernel: 0x500003936811feae kernel: mpssas_evt_handler kernel: kernel: 25 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x4 kernel: 0x47 kernel: 37 kernel: 22 kernel: 10 kernel: Responding (0x3) kernel: 5 kernel: 0x0078 kernel: LinkRate Unknown (0xa) kernel: TargetMissing kernel: 0x0079 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007a kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007b kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007c kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007d kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0089 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008a kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008b kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008c kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008d kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008e kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008f kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0090 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0093 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: mpssas_evt_handler kernel: kernel: 7 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x4 kernel: 0x47 kernel: 37 kernel: 4 kernel: 32 kernel: Responding (0x3) kernel: 5 kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: mpssas_evt_handler kernel: kernel: 7 kernel: 0 kernel: SasDeviceStatusChange (0xf) kernel: 0x78 kernel: 0xffff kernel: Cmp Internal Device Reset kernel: 0x0 kernel: 0x0 kernel: 0x78 kernel: 0x500003936811feae kernel: mpssas_evt_handler kernel: kernel: 6 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x9 kernel: 0x94 kernel: 37 kernel: 3 kernel: 33 kernel: Responding (0x3) kernel: 5 kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: mpssas_evt_handler kernel: kernel: 2 kernel: 0 kernel: SasDiscovery (0x16) kernel: 0x0 kernel: 0 kernel: Discovery Complete kernel: 5 kernel: 0 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 222 kernel: mps_intr_msi kernel: [16] kernel: SAS discovery start event kernel: [16] kernel: [f] kernel: Unhandled event 0xF kernel: [f] kernel: [1c] kernel: [1c] kernel: [1c] kernel: [1c] kernel: [f] kernel: Unhandled event 0xF kernel: [f] kernel: [1c] kernel: [1c] kernel: [16] kernel: SAS discovery stop event kernel: mpssas_discovery_end kernel: [16] kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 222 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 222 kernel: mpssas_evt_handler kernel: kernel: 2 kernel: 0 kernel: SasDiscovery (0x16) kernel: 0x0 kernel: 1 kernel: Discovery Started kernel: 5 kernel: 0 kernel: mpssas_evt_handler kernel: kernel: 25 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x4 kernel: 0x47 kernel: 37 kernel: 22 kernel: 10 kernel: Responding (0x3) kernel: 5 kernel: 0x0078 kernel: LinkRate Unknown (0x0) kernel: TargetGone kernel: 0x0079 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007a kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007b kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007c kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007d kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0089 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008a kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008b kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008c kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008d kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008e kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008f kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0090 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0093 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: mpssas_evt_handler kernel: kernel: 7 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x4 kernel: 0x47 kernel: 37 kernel: 4 kernel: 32 kernel: Responding (0x3) kernel: 5 kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: mpssas_evt_handler kernel: kernel: 2 kernel: 0 kernel: SasDiscovery (0x16) kernel: 0x0 kernel: 2 kernel: Discovery Complete kernel: 5 kernel: 0 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 226 kernel: mps_intr_msi kernel: [16] kernel: SAS discovery start event kernel: [16] kernel: [1c] kernel: mpssas_prepare_remove kernel: mpssas_alloc_tm freezing simq kernel: mpssas_alloc_tm tm_count 1 kernel: mpssas_rescan_target targetid 255 kernel: mpssas_rescan kernel: kernel: mps_enqueue_request SMID 2 cm 0xffffff8001396290 ccb 0 kernel: [1c] kernel: [1c] kernel: [1c] kernel: [16] kernel: SAS discovery stop event kernel: mpssas_discovery_end kernel: [16] kernel: mpssas_action func 0x4 kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 226 kernel: mps_complete_command cm 0xffffff8001396290 calling cm_complete 0xffffffff805d9550 data 0x78 reply 0xffffff96f9f31e00 kernel: mpssas_remove_device kernel: Reset aborted 0 commands kernel: mps_enqueue_request SMID 2 cm 0xffffff8001396290 ccb 0 kernel: clearing target 255 handle 0x0078 kernel: mps_complete_command cm 0xffffff8001396290 calling cm_complete 0xffffffff805d5500 data 0x78 reply 0xffffff96f9f31e80 kernel: mpssas_remove_complete kernel: mpssas_remove_complete on handle 0x0078, IOCStatus= 0x0 kernel: mpssas_free_tm releasing simq kernel: mpssas_free_tm tm_count 0 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 228 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 228 kernel: after disable kernel: mpssas_action func 0x4 kernel: mpssas_action func 0x15 kernel: mpssas_action func 0x91b kernel: sending SMP request to SAS address 0x5001438014a0a0bf kernel: mps_enqueue_request SMID 707 cm 0xffffff80013ce9d8 ccb 0 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 228 kernel: mps_complete_command cm 0xffffff80013ce9d8 calling cm_complete 0xffffffff805da470 data 0xfffffe002c7ca000 reply 0xffffff96f9f31f00 kernel: SMP request to SAS address 0x5001438014a0a0bf completed successfully kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 229 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 229 kernel: mpssas_evt_handler kernel: kernel: 1 kernel: 0 kernel: SasBroadcastPrimitive (0x17) kernel: 0x0 kernel: mpssas_evt_handler kernel: kernel: 1 kernel: 0 kernel: SasBroadcastPrimitive (0x17) kernel: 0x0 kernel: mpssas_evt_handler kernel: kernel: 1 kernel: 0 kernel: SasBroadcastPrimitive (0x17) kernel: 0x0 kernel: mpssas_evt_handler kernel: kernel: 1 kernel: 0 kernel: SasBroadcastPrimitive (0x17) kernel: 0x0 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 233 kernel: mps_intr_msi kernel: [17] kernel: Unhandled event 0x17 kernel: [17] kernel: [17] kernel: Unhandled event 0x17 kernel: [17] kernel: [17] kernel: Unhandled event 0x17 kernel: [17] kernel: [17] kernel: Unhandled event 0x17 kernel: [17] kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 233 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 233 kernel: mpssas_evt_handler kernel: kernel: 2 kernel: 0 kernel: SasDiscovery (0x16) kernel: 0x0 kernel: 1 kernel: Discovery Started kernel: 5 kernel: 0 kernel: mpssas_evt_handler kernel: kernel: 25 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x4 kernel: 0x47 kernel: 37 kernel: 22 kernel: 10 kernel: Responding (0x3) kernel: 5 kernel: 0x0078 kernel: 6.0Gbps (0xa0) kernel: NewTargetAdded kernel: 0x0079 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007a kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007b kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007c kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x007d kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0089 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008a kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008b kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008c kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008d kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008e kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x008f kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0090 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0093 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0006 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: mpssas_evt_handler kernel: kernel: 7 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x4 kernel: 0x47 kernel: 37 kernel: 4 kernel: 32 kernel: Responding (0x3) kernel: 5 kernel: 0x0094 kernel: 6.0Gbps (0xa0) kernel: PHYLinkStatusUnchanged kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: mpssas_evt_handler kernel: kernel: 6 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x9 kernel: 0x94 kernel: 37 kernel: 3 kernel: 33 kernel: Responding (0x3) kernel: 5 kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: mpssas_evt_handler kernel: kernel: 2 kernel: 0 kernel: SasDiscovery (0x16) kernel: 0x0 kernel: 2 kernel: Discovery Complete kernel: 5 kernel: 0 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 238 kernel: mps_intr_msi kernel: [16] kernel: SAS discovery start event kernel: [16] kernel: [1c] kernel: mps_config_get_sas_device_pg0 kernel: mps_enqueue_request SMID 701 cm 0xffffff80013ce228 ccb 0 kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 238 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 239 kernel: mps_enqueue_request SMID 693 cm 0xffffff80013cd7e8 ccb 0 kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 239 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 240 kernel: failed to add the device with handle 0x0078 to persistent table because there is no free space available - entry 0 kernel: mps_config_get_sas_device_pg0 kernel: mps_enqueue_request SMID 699 cm 0xffffff80013cdf98 ccb 0 kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 240 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 241 kernel: mps_enqueue_request SMID 703 cm 0xffffff80013ce4b8 ccb 0 kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 241 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 242 kernel: mps_config_get_sas_device_pg0 kernel: mps_enqueue_request SMID 571 cm 0xffffff80013c3b98 ccb 0 kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 242 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 243 kernel: mps_enqueue_request SMID 653 cm 0xffffff80013ca4a8 ccb 0 kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 243 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 244 kernel: SAS Address from SAS device page0 = 500003936811feae kernel: Found device <401,End Device> <6.0Gbps> <0x0078> <4/36> kernel: mpssas_rescan_target targetid 255 kernel: mpssas_rescan kernel: kernel: Target id 0xff added kernel: [1c] kernel: [1c] kernel: [1c] kernel: [1c] kernel: [1c] kernel: [16] kernel: SAS discovery stop event kernel: mpssas_discovery_end kernel: [16] kernel: mpssas_action func 0x4 kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 244 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 244 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 244 kernel: mpssas_evt_handler kernel: kernel: 2 kernel: 0 kernel: SasDiscovery (0x16) kernel: 0x0 kernel: 1 kernel: Discovery Started kernel: 5 kernel: 0 kernel: mpssas_evt_handler kernel: kernel: 6 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x4 kernel: 0x47 kernel: 37 kernel: 3 kernel: 33 kernel: Responding (0x3) kernel: 5 kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: mpssas_evt_handler kernel: kernel: 6 kernel: 0 kernel: SasTopologyChangeList (0x1c) kernel: 0x0 kernel: 0x9 kernel: 0x94 kernel: 37 kernel: 3 kernel: 33 kernel: Responding (0x3) kernel: 5 kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: 0x0000 kernel: LinkRate Unknown (0x0) kernel: Unknown Status kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 247 kernel: mps_intr_msi kernel: [16] kernel: SAS discovery start event kernel: [16] kernel: [1c] kernel: [1c] kernel: [1c] kernel: [1c] kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 247 kernel: mps_intr_msi kernel: mps_intr_locked sc 0xffffff8001353000 starting with replypostindex 247 kernel: mpssas_evt_handler kernel: kernel: 2 kernel: 0 kernel: SasDiscovery (0x16) kernel: 0x0 kernel: 0 kernel: Discovery Complete kernel: 5 kernel: 0 kernel: mps_intr_locked sc 0xffffff8001353000 writing postindex 248 kernel: [16] kernel: SAS discovery stop event kernel: mpssas_discovery_end kernel: [16] Thanks, -John