From owner-freebsd-scsi@freebsd.org Wed Oct 18 01:59:50 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2C04E4CE76 for ; Wed, 18 Oct 2017 01:59:49 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-49.reflexion.net [208.70.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2E0865EFF for ; Wed, 18 Oct 2017 01:59:48 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 31050 invoked from network); 18 Oct 2017 01:53:08 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 18 Oct 2017 01:53:08 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 17 Oct 2017 21:53:08 -0400 (EDT) Received: (qmail 6380 invoked from network); 18 Oct 2017 01:53:07 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 18 Oct 2017 01:53:07 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Tue, 17 Oct 2017 18:53:06 -0700 From: Shiva Bhanujan To: "Kenneth D. Merry" CC: "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uIAAihwAgAA6+IKAANcq6Q== Date: Wed, 18 Oct 2017 01:53:05 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local>, <20171017023126.GA6559@mithlond.kdm.org>, <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Oct 2017 01:59:50 -0000 Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? ________________________________ From: owner-freebsd-scsi=40freebsd.org = =5Bowner-freebsd-scsi=40freebsd.org=5D on behalf of Shiva Bhanujan = =5Bshiva.bhanujan=40quorum.net=5D Sent: Tuesday, October 17, 2017 6:08 AM To: Kenneth D. Merry Cc: freebsd-scsi=40freebsd.org Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Since I started having the SCSI errors, I ended up running sg_format to = format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete once the format is done, I can successfully format and partition using = gpart. The errors that show up when I try to run gpart for the first time are as = follows: (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da9:mpr0:0:17:0): CAM status: SCSI Status Error (da9:mpr0:0:17:0): SCSI status: Check Condition (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) (da9:mpr0:0:17:0): Error 22, Unretryable error It seems that it's the read that is failing, and is being tagged as an = illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG=5D Sent: Monday, October 16, 2017 7:31 PM To: Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: > Hi Ken, > > I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've = tried this w/ 11.1, w/ the same results. > Yes, the mpr(4) driver is in all recent FreeBSD releases. In looking at the dmesg, this is telling: (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete (da0:mpr0:0:8:0): Error 16, Unretryable error If the drives are in the process of formatting, I guess it may make sense for them to reject read commands. Otherwise, it makes no sense for a hard drive to reject reads. Are you able to check the status of the format? You should be able to send a test unit ready and figure out how far along the format is: camcontrol tur da0 -v And so on for each of the drives. Ken > ________________________________ > From: Kenneth D. Merry = =5Bken=40FreeBSD.ORG=5D > Sent: Monday, October 16, 2017 7:42 AM > To: Shiva Bhanujan > Cc: = freebsd-scsi=40freebsd.org > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 > > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > > Hello, > > > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the = following: > > > > > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) > > > > The error message above is missing the SCSI CDB. We need that in order to > figure out what command the drive is complaining about. > > The error message means that FreeBSD is sending a SCSI command that the > drive doesn't support. That can be benign, or it can cause a problem. > > So, what error does gpart give you when you have this problem? > > > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also = has the same result. > > > > > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 = doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > > > > > > Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 and = 11.0. I do see that there are others who have experienced these issues. = Can somebody please provide me some pointers > > as to why this is occurring? Or if there are some driver changes that = I might be able to incorporate? > > > > In general, the latest mpr(4) driver is the best one. The driver itself > generally doesn't send SCSI commands (there are a few exceptions), but > rather passes them through from the upper layers of CAM. > > > Please note, that I have gone through the mail titled =22scsi error at = SEAGATE ST1200MM0088 TT31=22 and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, if = the disks were written to using FreeBSD 10.2, which doesn't seem to have = SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated. > > > > Send the full error messages, and we may be able to figure out what's = going > on. > > Ken > -- > Kenneth Merry > = ken=40FreeBSD.ORG > ________________________________ -- Kenneth Merry ken=40FreeBSD.ORG ________________________________ _______________________________________________ freebsd-scsi=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-scsi To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org=22 ________________________________