From owner-freebsd-scsi@freebsd.org Thu Oct 19 21:17:03 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F3AFE47B12 for ; Thu, 19 Oct 2017 21:17:03 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C3A6381277 for ; Thu, 19 Oct 2017 21:17:02 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from [10.0.0.26] (mbp2013.int.kdm.org [10.0.0.26]) (authenticated bits=0) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPSA id v9JLH1xP069670 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Oct 2017 17:17:01 -0400 (EDT) (envelope-from ken@freebsd.org) From: Ken Merry Message-Id: <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> Mime-Version: 1.0 (Mac OS X Mail 11.0 \(3445.1.7\)) Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Date: Thu, 19 Oct 2017 17:17:00 -0400 In-Reply-To: <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> Cc: "freebsd-scsi@freebsd.org" To: Shiva Bhanujan References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> X-Mailer: Apple Mail (2.3445.1.7) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [96.89.93.250]); Thu, 19 Oct 2017 17:17:01 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Oct 2017 21:17:03 -0000 By the way, the message you referenced is here: = https://lists.freebsd.org/pipermail/freebsd-scsi/2017-January/007237.html = And there is more here: https://bugs.freenas.org/issues/14517 = Ken =E2=80=94=20 Ken Merry ken@FreeBSD.ORG > On Oct 19, 2017, at 5:15 PM, Ken Merry wrote: >=20 > What does sg_readcap =E2=80=9416 show for these drives? >=20 > If it has type 2 protection turned on, check to see what the DCIPZ = value is in the control mode page: >=20 > camcontrol modepage daX -v -m 10 >=20 > If that is set to 0, add a -e to the above command line and see if you = can set it to 1. That may or may not help anything. >=20 > The mpr(4) and mps(4) drivers try to support protection information if = it is turned on on the drive. So, they set the protection information = if protection information is turned on in the drive. For that reason, = setting the DCIPZ bit may not fix it. >=20 > There could be a problem with how that is implemented that=E2=80=99s = causing the drives to reject the command, but I=E2=80=99m not sure. >=20 > If it is, Steve (CCed) can help us debug it. >=20 > Ken > =E2=80=94=20 > Ken Merry > ken@FreeBSD.ORG >=20 >=20 >=20 >> On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan = wrote: >>=20 >> Sorry. perhaps I have failed to mention, the SCSI errors are only w/ = the seagate drives. These are of the model ST4000NM0023. >>=20 >>=20 >> From: Shiva Bhanujan >> Sent: Tuesday, October 17, 2017 6:53 PM >> To: Kenneth D. Merry >> Cc: freebsd-scsi@freebsd.org >> Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >>=20 >> Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? >>=20 >>=20 >>=20 >> From: owner-freebsd-scsi@freebsd.org = [owner-freebsd-scsi@freebsd.org = ] on behalf of Shiva Bhanujan = [shiva.bhanujan@quorum.net ] >> Sent: Tuesday, October 17, 2017 6:08 AM >> To: Kenneth D. Merry >> Cc: freebsd-scsi@freebsd.org >> Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >>=20 >> Since I started having the SCSI errors, I ended up running sg_format = to format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. >>=20 >> (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not = ready, format in progress) >> (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >>=20 >> once the format is done, I can successfully format and partition = using gpart. >>=20 >>=20 >> The errors that show up when I try to run gpart for the first time = are as follows: >>=20 >> (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >> (da9:mpr0:0:17:0): CAM status: SCSI Status Error >> (da9:mpr0:0:17:0): SCSI status: Check Condition >> (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) >> (da9:mpr0:0:17:0): Error 22, Unretryable error >>=20 >>=20 >> It seems that it's the read that is failing, and is being tagged as = an illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? >>=20 >>=20 >>=20 >>=20 >> ________________________________ >> From: Kenneth D. Merry [ken@FreeBSD.ORG ] >> Sent: Monday, October 16, 2017 7:31 PM >> To: Shiva Bhanujan >> Cc: freebsd-scsi@freebsd.org >> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >>=20 >> On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: >>> Hi Ken, >>>=20 >>> I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. >>>=20 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >>>=20 >>>=20 >>> My understanding is that FreeBSD 11.1 contains the mpr(4) driver? = I've tried this w/ 11.1, w/ the same results. >>>=20 >>=20 >> Yes, the mpr(4) driver is in all recent FreeBSD releases. >>=20 >> In looking at the dmesg, this is telling: >>=20 >> (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 >> (da0:mpr0:0:8:0): CAM status: SCSI Status Error >> (da0:mpr0:0:8:0): SCSI status: Check Condition >> (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not = ready, format in progress) >> (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >> (da0:mpr0:0:8:0): Error 16, Unretryable error >>=20 >> If the drives are in the process of formatting, I guess it may make = sense >> for them to reject read commands. Otherwise, it makes no sense for a = hard >> drive to reject reads. >>=20 >> Are you able to check the status of the format? You should be able to = send >> a test unit ready and figure out how far along the format is: >>=20 >> camcontrol tur da0 -v >>=20 >> And so on for each of the drives. >>=20 >> Ken >>=20 >>> ________________________________ >>> From: Kenneth D. Merry [ken@FreeBSD.ORG = = >] >>> Sent: Monday, October 16, 2017 7:42 AM >>> To: Shiva Bhanujan >>> Cc: freebsd-scsi@freebsd.org = = > >>> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic = SAS3008 PCI-Express Fusion-MPT SAS-3 >>>=20 >>> On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: >>>> Hello, >>>>=20 >>>> I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like = the following: >>>>=20 >>>>=20 >>>>=20 >>>> (da0:mpr0:0:0:0): CAM status: SCSI Status Error >>>>=20 >>>> (da0:mpr0:0:0:0): SCSI status: Check Condition >>>>=20 >>>> (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) >>>>=20 >>>=20 >>> The error message above is missing the SCSI CDB. We need that in = order to >>> figure out what command the drive is complaining about. >>>=20 >>> The error message means that FreeBSD is sending a SCSI command that = the >>> drive doesn't support. That can be benign, or it can cause a = problem. >>>=20 >>> So, what error does gpart give you when you have this problem? >>>=20 >>>> I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively = also has the same result. >>>>=20 >>>=20 >>>=20 >>>> Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD = 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either. >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 = and 11.0. I do see that there are others who have experienced these = issues. Can somebody please provide me some pointers >>>> as to why this is occurring? Or if there are some driver changes = that I might be able to incorporate? >>>>=20 >>>=20 >>> In general, the latest mpr(4) driver is the best one. The driver = itself >>> generally doesn't send SCSI commands (there are a few exceptions), = but >>> rather passes them through from the upper layers of CAM. >>>=20 >>>> Please note, that I have gone through the mail titled "scsi error = at SEAGATE ST1200MM0088 TT31" and have started sg_format on all the = SEAGATE disks. Having said that, I still need to figure out what would = happen, if the disks were written to using FreeBSD 10.2, which doesn't = seem to have SCSI errors, and when I try to upgrade to 10.3. Any help is = appreciated. >>>>=20 >>>=20 >>> Send the full error messages, and we may be able to figure out = what's going >>> on. >>>=20 >>> Ken >>> -- >>> Kenneth Merry >>> ken@FreeBSD.ORG = > = > >>> ________________________________ >>=20 >>=20 >> -- >> Kenneth Merry >> ken@FreeBSD.ORG = > >> ________________________________ >> _______________________________________________ >> freebsd-scsi@freebsd.org mailing = list >> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi = >> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org = " >=20 > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org"