From owner-freebsd-scsi@freebsd.org Tue Oct 17 02:31:30 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B5D2E4FCA5 for ; Tue, 17 Oct 2017 02:31:30 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D3F4C7D260 for ; Tue, 17 Oct 2017 02:31:29 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPS id v9H2VQ5Z006679 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 16 Oct 2017 22:31:26 -0400 (EDT) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.15.2/8.14.9/Submit) id v9H2VQQs006676; Mon, 16 Oct 2017 22:31:26 -0400 (EDT) (envelope-from ken) Date: Mon, 16 Oct 2017 22:31:26 -0400 From: "Kenneth D. Merry" To: Shiva Bhanujan Cc: "freebsd-scsi@freebsd.org" Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Message-ID: <20171017023126.GA6559@mithlond.kdm.org> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 16 Oct 2017 22:31:27 -0400 (EDT) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mithlond.kdm.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Oct 2017 02:31:30 -0000 On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: > Hi Ken, > > I've attached the output of dmesg. Here's the SCSI CDB for a sample drive, da3. > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've tried this w/ 11.1, w/ the same results. > Yes, the mpr(4) driver is in all recent FreeBSD releases. In looking at the dmesg, this is telling: (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete (da0:mpr0:0:8:0): Error 16, Unretryable error If the drives are in the process of formatting, I guess it may make sense for them to reject read commands. Otherwise, it makes no sense for a hard drive to reject reads. Are you able to check the status of the format? You should be able to send a test unit ready and figure out how far along the format is: camcontrol tur da0 -v And so on for each of the drives. Ken > ________________________________ > From: Kenneth D. Merry [ken@FreeBSD.ORG] > Sent: Monday, October 16, 2017 7:42 AM > To: Shiva Bhanujan > Cc: freebsd-scsi@freebsd.org > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 > > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > > Hello, > > > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the following: > > > > > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code) > > > > The error message above is missing the SCSI CDB. We need that in order to > figure out what command the drive is complaining about. > > The error message means that FreeBSD is sending a SCSI command that the > drive doesn't support. That can be benign, or it can cause a problem. > > So, what error does gpart give you when you have this problem? > > > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also has the same result. > > > > > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > > > > > > Is there a different version of the mpr driver I should be using? I haven't checked the differences between the mpr driver in 10.2 vs 10.3 and 11.0. I do see that there are others who have experienced these issues. Can somebody please provide me some pointers > > as to why this is occurring? Or if there are some driver changes that I might be able to incorporate? > > > > In general, the latest mpr(4) driver is the best one. The driver itself > generally doesn't send SCSI commands (there are a few exceptions), but > rather passes them through from the upper layers of CAM. > > > Please note, that I have gone through the mail titled "scsi error at SEAGATE ST1200MM0088 TT31" and have started sg_format on all the SEAGATE disks. Having said that, I still need to figure out what would happen, if the disks were written to using FreeBSD 10.2, which doesn't seem to have SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated. > > > > Send the full error messages, and we may be able to figure out what's going > on. > > Ken > -- > Kenneth Merry > ken@FreeBSD.ORG > ________________________________ -- Kenneth Merry ken@FreeBSD.ORG