From owner-freebsd-scsi@freebsd.org Sun Dec 6 12:06:12 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C7BA7669 for ; Sun, 6 Dec 2015 12:06:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D62A10A5 for ; Sun, 6 Dec 2015 12:06:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tB6C6BbP063139 for ; Sun, 6 Dec 2015 12:06:11 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 195033] [cam] [scsi] [patch] unclean SCSI attached HDD power-off on shut down Date: Sun, 06 Dec 2015 12:06:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: sneakywumpus@gmail.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Dec 2015 12:06:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195033 --- Comment #1 from Thomas Eberhardt --- I just found out about hw.mps.enable_ssu. When I set this to 3 (MPS_SSU_ENABLE_SSD_ENABLE_HDD) my disks power down cleanly (i.e. no Power-Off_Retract_Count increase). You can close this bug if you want. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Mon Dec 7 17:58:26 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E67D49B92CE for ; Mon, 7 Dec 2015 17:58:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D73A01CA9 for ; Mon, 7 Dec 2015 17:58:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tB7HwQSN041906 for ; Mon, 7 Dec 2015 17:58:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 195033] [cam] [scsi] [patch] unclean SCSI attached HDD power-off on shut down Date: Mon, 07 Dec 2015 17:58:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rpokala@panasas.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2015 17:58:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195033 Ravi Pokala changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rpokala@panasas.com --- Comment #2 from Ravi Pokala --- At the very least, hw.mps.enable_ssu should be documented in mps(4). There's probably also an argument that the default value of MPS_SSU_ENABLE_SSD_DISABLE_HDD should be changed to MPS_SSU_ENABLE_SSD_ENABLE_HDD. With 512e and SMR, nowadays even HDDs appreciate a chance to quiesce up before they're powered down. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Mon Dec 7 18:07:36 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 615629B9DA2 for ; Mon, 7 Dec 2015 18:07:36 +0000 (UTC) (envelope-from prateekrootkey@gmail.com) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 049FC1740 for ; Mon, 7 Dec 2015 18:07:36 +0000 (UTC) (envelope-from prateekrootkey@gmail.com) Received: by wmww144 with SMTP id w144so161166345wmw.0 for ; Mon, 07 Dec 2015 10:07:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=xwXWvN/YiQRCHuZmYzWalb4mhhztTLMe0O/K4TR8xG0=; b=kxz9WNhGnn514G5COhc13y6h+GBNiKw1JAuOr5P7gtxh7sOlf8VfcUYidnXALnVjrY JGDHmD7cCWT1CbvH+7eYvcJJoW22nMIbRNpI68wPKNH4mw93E75kz53eQRC2xL/L0DmT WhcxWVYl/nVaqdi0ytW/ktMQilELA0wMimYLBYSlJhsvdZyegmnBPkQ/4+SucqQp1rTf cHXK1N1BaXgbnWEjngaBLDacT9t5SOImjjWqO/bI00U0A4Z4+tqwA/Pz+NckiNCykFSD EcsO/lp6n1n63kqgobT1d7a2LBCW+I9APYK38uYcGGavRH30dJY66HaGBo+/l1oNRd/u K4cA== MIME-Version: 1.0 X-Received: by 10.28.45.216 with SMTP id t207mr23838559wmt.89.1449511654533; Mon, 07 Dec 2015 10:07:34 -0800 (PST) Received: by 10.27.16.7 with HTTP; Mon, 7 Dec 2015 10:07:34 -0800 (PST) Date: Mon, 7 Dec 2015 23:37:34 +0530 Message-ID: Subject: bad disk discovery From: prateek sethi To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2015 18:07:36 -0000 Hi , Is there any way or tool to find out that a disk which is not responding properly is really bad or not? Sometimes I have seen that there is lot of CDB error for a drive and system reboot makes every thing fine. What can be reasons for such kind of scenarios? I know smartctl is the one which can help. I have some couple of question regarding this . 1. What if disk does not support smartctl? 2. How I can do smartest use of smartctl command like which parameters can tell that the disk is actually bad? 3. What other test I can perform to make it sure that disk has completely gone? Please tell me correct place to ask this question if I am asking at wrong place. From owner-freebsd-scsi@freebsd.org Mon Dec 7 21:47:21 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 281359B914C for ; Mon, 7 Dec 2015 21:47:21 +0000 (UTC) (envelope-from scott4long@yahoo.com) Received: from nm16-vm6.bullet.mail.gq1.yahoo.com (nm16-vm6.bullet.mail.gq1.yahoo.com [98.137.177.254]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F30D41CFB for ; Mon, 7 Dec 2015 21:47:20 +0000 (UTC) (envelope-from scott4long@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1449524705; bh=kBjJPE2lyXDAHvT29Y69twjimbsEg9znUcmDzJ6MUmw=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject; b=g1u/dGLV3CMxtH1nT9wj/ynIdjegRVQtzgNq3sjBjM8I0vmI670b04RNQCwPZmdHt++8tOvi48RL4z+V3W+vgLk3BIqnkJIOyB52AGbCa3bsNZx5gAmrMYyv71m1mCuxY8eykRfLCd0kQXOjHv/b9/eMYUseyZ2KRkLubCWietNbLqba0USTf6TgQKOkEgLsu8KyMn7nKTadcGO7RcdQufKkGPg6q7bhmTo1sgL2N2SEKp7UrnGcdl3uUMID8qilsByeLXVWkgpPqZ7XjDG7Jqj+yazVQBaBxDUzHiIN/xgkXMHz8+36/xQBguxEoIWobr0yqwCOS3kdUnRvrphXMg== Received: from [98.137.12.55] by nm16.bullet.mail.gq1.yahoo.com with NNFMP; 07 Dec 2015 21:45:05 -0000 Received: from [208.71.42.200] by tm15.bullet.mail.gq1.yahoo.com with NNFMP; 07 Dec 2015 21:45:05 -0000 Received: from [127.0.0.1] by smtp211.mail.gq1.yahoo.com with NNFMP; 07 Dec 2015 21:45:05 -0000 X-Yahoo-Newman-Id: 58333.66122.bm@smtp211.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 6iLGOx0VM1n3eqnUjTu8EMJr1kgkKqPnbWm1mTf8_L82SF. 9QUcUyJtIQnHXf2eBAJ3rdWHuoYZYbMKw1pRWYyrU9PgUbFh6lCIpG6mRDmO ONX_yn.RuNaHnARSuFAaig6BqTVQu_KVSLKfO8HjGWEiNS_OplrzEcmE7Oxv ES0oMAXk4oZuTPSUbH9iUiWzDVyQCOGq94Tcg9th9SooW8Qqb4ROJfFQfUdx GcTdoWwq5Fwe2JjCS7295n4Pq9knfK4LcmjAJ8Ug1v.H_e8ipWN6DDrZOrRY kbpbEjli.O4GLZKHx3OFTe4zx.S2unWljB0OQPJ7eiJgAZIzBagKWLWqnb.3 UPlkTAVQ47Iu3iUT9oU38ZWKCCvYPhDir1.LajlGKfWQKWFHA93ycTkv4WY5 QCwLJmUoZPeewUlHErUTbPiNIJ5v4QGmENkpO36ydl_kMIuoEQZji7HJKZ6v sO4syHDynwqksLU8BW26rXoWcWk70QfxyTwPUaf4cqwikD7XynZt5syK_Q4a DN5P.CifPKCE3wvO19d00cIFTzFPdvDlGCxwCsiUO X-Yahoo-SMTP: clhABp.swBB7fs.LwIJpv3jkWgo2NU8- Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\)) Subject: Re: bad disk discovery From: Scott Long In-Reply-To: Date: Mon, 7 Dec 2015 14:45:17 -0700 Cc: freebsd-scsi@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <6A7832F8-53EB-4641-8EF6-E0E6175EB52D@yahoo.com> References: To: prateek sethi X-Mailer: Apple Mail (2.3096.5) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2015 21:47:21 -0000 Hi, If your situation is accurate and the disk is not responding properly to = regular commands then it=E2=80=99s unlikely that it will respond to SMART = commands either. Sometimes these situations are caused by a bad cable, bad controller, or buggy software/firmware, and only rarely will the standard statistics in = SMART pick up these kinds of errors. SMART is better at tracking wear rates = and error rates on the physical media, both HDD and SSD, but even then = it=E2=80=99s hard for it to be accurately predictive or even accurately diagnostic. For = your case, I recommend that you describe your hardware and software configuration = in more detail, and look for physical abnormalities in the cabling and = connections. Once that is ruled and and the rest of us know what kind of hardware = you=E2=80=99re dealing with, we might be able to make better commendations. Scott > On Dec 7, 2015, at 11:07 AM, prateek sethi = wrote: >=20 > Hi , >=20 > Is there any way or tool to find out that a disk which is not = responding > properly is really bad or not? Sometimes I have seen that there is lot = of > CDB error for a drive and system reboot makes every thing fine. What = can be > reasons for such kind of scenarios? >=20 > I know smartctl is the one which can help. I have some couple of = question > regarding this . >=20 > 1. What if disk does not support smartctl? > 2. How I can do smartest use of smartctl command like which parameters = can > tell that the disk is actually bad? > 3. What other test I can perform to make it sure that disk has = completely > gone? >=20 >=20 > Please tell me correct place to ask this question if I am asking at = wrong > place. > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Tue Dec 8 13:00:39 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6B329D321A for ; Tue, 8 Dec 2015 13:00:39 +0000 (UTC) (envelope-from prateekrootkey@gmail.com) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 360321DB8 for ; Tue, 8 Dec 2015 13:00:39 +0000 (UTC) (envelope-from prateekrootkey@gmail.com) Received: by wmuu63 with SMTP id u63so180016903wmu.0 for ; Tue, 08 Dec 2015 05:00:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gT14hBlFfaeqJDb12Uat+Y/325TmIylxutvdKOUWS28=; b=bg6SS3UPDyPNuwrB4M9mROVPGGRzA60i8uJhhL9QWBPhBgd1eSjU0USUjFFNUXkJ5/ 3MrXdgwiOBaR30JkSQECwSpyjyVPhOz+2k7V92PvtTPnFmK0IOgmHBM29FPBBTvO+u1v aqpdirdV4QMg4bDhnEG9MGRzHKjmuaabrgN841kbt932fFqSoAvmDV3GjdoWoiZY8Pp9 jLB5OZBE5RS+xdmY5NCemR4G1oqgH97CelXjInzPPlDmeq1Q41dcWJaJ3igz4dAuq6/j RMaCDF7cgTvlE8LaJkDgCY7Kqpv5xC+qSEkOaKYRhBNifPdPWU/qCEdZjmvu51Dklbmi xZvQ== MIME-Version: 1.0 X-Received: by 10.28.85.129 with SMTP id j123mr4663109wmb.77.1449579637781; Tue, 08 Dec 2015 05:00:37 -0800 (PST) Received: by 10.27.16.7 with HTTP; Tue, 8 Dec 2015 05:00:37 -0800 (PST) In-Reply-To: <6A7832F8-53EB-4641-8EF6-E0E6175EB52D@yahoo.com> References: <6A7832F8-53EB-4641-8EF6-E0E6175EB52D@yahoo.com> Date: Tue, 8 Dec 2015 18:30:37 +0530 Message-ID: Subject: Re: bad disk discovery From: prateek sethi To: Scott Long Cc: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 13:00:39 -0000 Hi Scott, Thanks for the your quick response. I have different set of hardware . So that's why I want to know how I can debug it myself . Is there anyway or procedure using that I can findout about the situation or the reason for CDB errors or disk command failure? Right now I am giving detail about the setup where I am getting this issue = . I am using LSI SAS2008 controller and connected with supermicro Enclosure with freebsd 9.3. 16 different disks are there but only one disk is having problem. That means contoller and cable are fine. Faulty disk info are like:-. *smartctl output is:-* smartctl -x /dev/da23 =3D=3D=3D START OF INFORMATION SECTION =3D=3D=3D Vendor: SEAGATE Product: ST3600057SS Revision: 000B Rotation Rate: 15000 rpm Form Factor: 3.5 inches Logical Unit id: 0x5000c5007725173f Serial number: 6SL8YLPC0000N5030DY7 Device type: disk Transport protocol: SAS Local Time is: Tue Dec 8 18:20:45 2015 IST *device is NOT READY (e.g. spun down, busy)* *Logs:-* Dec 8 14:12:01 N1 kernel: da23 at mps0 bus 0 scbus0 target 148 lun 0 Dec 8 14:12:01 N1 kernel: da23: Fixed Direct Access SCSI-5 device Dec 8 14:12:01 N1 kernel: da23: Serial Number 6SL8YLPC0000N5030DY7 Dec 8 14:12:01 N1 kernel: da23: 600.000MB/s transfers Dec 8 14:12:01 N1 kernel: da23: Command Queueing enabled Dec 8 14:12:01 N1 kernel: da23: *Attempt to query device size failed: NOT READY, Logical unit not ready, cause n* Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: Element descriptor: 'Slot 24' Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: SAS Device Slot Element: 1 Phys at Slot 23 *driver versions:-* dev.mps.0.firmware_version: 15.00.00.00 dev.mps.0.driver_version: 16.00.00.00-fbsd On Tue, Dec 8, 2015 at 3:15 AM, Scott Long wrote: > Hi, > > If your situation is accurate and the disk is not responding properly to > regular > commands then it=E2=80=99s unlikely that it will respond to SMART command= s either. > Sometimes these situations are caused by a bad cable, bad controller, or > buggy software/firmware, and only rarely will the standard statistics in > SMART > pick up these kinds of errors. SMART is better at tracking wear rates an= d > error rates on the physical media, both HDD and SSD, but even then it=E2= =80=99s > hard > for it to be accurately predictive or even accurately diagnostic. For > your case, > I recommend that you describe your hardware and software configuration in > more detail, and look for physical abnormalities in the cabling and > connections. > Once that is ruled and and the rest of us know what kind of hardware you= =E2=80=99re > dealing with, we might be able to make better commendations. > > Scott > > > On Dec 7, 2015, at 11:07 AM, prateek sethi > wrote: > > > > Hi , > > > > Is there any way or tool to find out that a disk which is not respondin= g > > properly is really bad or not? Sometimes I have seen that there is lot = of > > CDB error for a drive and system reboot makes every thing fine. What ca= n > be > > reasons for such kind of scenarios? > > > > I know smartctl is the one which can help. I have some couple of questi= on > > regarding this . > > > > 1. What if disk does not support smartctl? > > 2. How I can do smartest use of smartctl command like which parameters > can > > tell that the disk is actually bad? > > 3. What other test I can perform to make it sure that disk has complete= ly > > gone? > > > > > > Please tell me correct place to ask this question if I am asking at wro= ng > > place. > > _______________________________________________ > > freebsd-scsi@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > > From owner-freebsd-scsi@freebsd.org Tue Dec 8 13:26:52 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0545E9D4658; Tue, 8 Dec 2015 13:26:52 +0000 (UTC) (envelope-from mikej@mikej.com) Received: from mx2.paymentallianceintl.com (mx2.paymentallianceintl.com [216.26.158.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx2.paymentallianceintl.com", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D13351B1C; Tue, 8 Dec 2015 13:26:51 +0000 (UTC) (envelope-from mikej@mikej.com) Received: from firewall.mikej.com (162-230-214-65.lightspeed.lsvlky.sbcglobal.net [162.230.214.65]) by mx2.paymentallianceintl.com (8.15.1/8.15.1) with ESMTPS id tB8DQhcA087691 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 8 Dec 2015 08:26:43 -0500 (EST) (envelope-from mikej@mikej.com) X-Authentication-Warning: mx2.paymentallianceintl.com: Host 162-230-214-65.lightspeed.lsvlky.sbcglobal.net [162.230.214.65] claimed to be firewall.mikej.com Received: from mail.mikej.com (firewall.mikej.com [192.168.6.63]) by firewall.mikej.com (8.15.2/8.15.2) with ESMTP id tB8DQPSu087326; Tue, 8 Dec 2015 08:26:25 -0500 (EST) (envelope-from mikej@mikej.com) DKIM-Filter: OpenDKIM Filter v2.10.3 firewall.mikej.com tB8DQPSu087326 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mikej.com; s=mail; t=1449581186; bh=z3rmZncuKnDn/f8ahsIWaBKcx7AV8vZ7I1i+oIHIPRI=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=fQXbcsewBWphj/g8syfQQsPFycLfVwd6Q9rokE1nyqW0QCAUa9L1Su8lPHNsJTlz8 ek2idpuCjbQ/pkhW0MQ3nLzmlSb6WuPGE1L8bA8XQjnn+UWObMsD/FNzvUWu0UeZVC 9k/WaAf5LbopquCCCm+cqL78JOWUn22gUI17zQJhJoNTmF7noSv4SRDV9PQq6Amaru F9tsf3cneAb6P8+UcBZUQpktXZKu2cPfwfn2jT/EdaRowEytT/J/XJ0UtlL/Nxb/xq UoAz1ewVoqG81KbzymvGwCsoBPyxGl8h7ZZ0wZy5aJsYst0EyUm2V479detGWorEc4 wHGymgTFDx6HQ== X-Authentication-Warning: firewall.mikej.com: Host firewall.mikej.com [192.168.6.63] claimed to be mail.mikej.com MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 08 Dec 2015 08:26:24 -0500 From: Michael Jung To: prateek sethi Cc: Scott Long , freebsd-scsi@freebsd.org, owner-freebsd-scsi@freebsd.org Subject: Re: bad disk discovery In-Reply-To: References: <6A7832F8-53EB-4641-8EF6-E0E6175EB52D@yahoo.com> Message-ID: X-Sender: mikej@mikej.com User-Agent: Roundcube Webmail/1.1.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 13:26:52 -0000 On 2015-12-08 08:00, prateek sethi wrote: > Hi Scott, > Thanks for the your quick response. > > I have different set of hardware . So that's why I want to know how I > can > debug it myself . Is there anyway or procedure using that I can findout > about the situation or the reason for CDB errors or disk command > failure? > > Right now I am giving detail about the setup where I am getting this > issue . > > I am using LSI SAS2008 controller and connected with supermicro > Enclosure > with freebsd 9.3. 16 different disks are there but only one disk is > having > problem. That means contoller and cable are fine. > > Faulty disk info are like:-. > > *smartctl output is:-* > > smartctl -x /dev/da23 > > === START OF INFORMATION SECTION === > Vendor: SEAGATE > Product: ST3600057SS > Revision: 000B > Rotation Rate: 15000 rpm > Form Factor: 3.5 inches > Logical Unit id: 0x5000c5007725173f > Serial number: 6SL8YLPC0000N5030DY7 > Device type: disk > Transport protocol: SAS > Local Time is: Tue Dec 8 18:20:45 2015 IST > *device is NOT READY (e.g. spun down, busy)* > > *Logs:-* > > Dec 8 14:12:01 N1 kernel: da23 at mps0 bus 0 scbus0 target 148 lun 0 > Dec 8 14:12:01 N1 kernel: da23: Fixed > Direct > Access SCSI-5 device > Dec 8 14:12:01 N1 kernel: da23: Serial Number 6SL8YLPC0000N5030DY7 > Dec 8 14:12:01 N1 kernel: da23: 600.000MB/s transfers > Dec 8 14:12:01 N1 kernel: da23: Command Queueing enabled > Dec 8 14:12:01 N1 kernel: da23: *Attempt to query device size failed: > NOT > READY, Logical unit not ready, cause n* > Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: Element descriptor: 'Slot > 24' > Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: SAS Device Slot Element: > 1 > Phys at Slot 23 > > *driver versions:-* > > dev.mps.0.firmware_version: 15.00.00.00 > dev.mps.0.driver_version: 16.00.00.00-fbsd > > > > > > > On Tue, Dec 8, 2015 at 3:15 AM, Scott Long > wrote: > >> Hi, >> >> If your situation is accurate and the disk is not responding properly >> to >> regular >> commands then it’s unlikely that it will respond to SMART commands >> either. >> Sometimes these situations are caused by a bad cable, bad controller, >> or >> buggy software/firmware, and only rarely will the standard statistics >> in >> SMART >> pick up these kinds of errors. SMART is better at tracking wear rates >> and >> error rates on the physical media, both HDD and SSD, but even then >> it’s >> hard >> for it to be accurately predictive or even accurately diagnostic. For >> your case, >> I recommend that you describe your hardware and software configuration >> in >> more detail, and look for physical abnormalities in the cabling and >> connections. >> Once that is ruled and and the rest of us know what kind of hardware >> you’re >> dealing with, we might be able to make better commendations. >> >> Scott >> >> > On Dec 7, 2015, at 11:07 AM, prateek sethi >> wrote: >> > >> > Hi , >> > >> > Is there any way or tool to find out that a disk which is not responding >> > properly is really bad or not? Sometimes I have seen that there is lot of >> > CDB error for a drive and system reboot makes every thing fine. What can >> be >> > reasons for such kind of scenarios? >> > >> > I know smartctl is the one which can help. I have some couple of question >> > regarding this . >> > >> > 1. What if disk does not support smartctl? >> > 2. How I can do smartest use of smartctl command like which parameters >> can >> > tell that the disk is actually bad? >> > 3. What other test I can perform to make it sure that disk has completely >> > gone? >> > >> > >> > Please tell me correct place to ask this question if I am asking at wrong >> > place. >> > _______________________________________________ >> > freebsd-scsi@freebsd.org mailing list >> > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >> > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" >> >> > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" Have you simply moved the drive to another slot - does the problem follow the drive? Unlikely but it could be a backplane issue. I don't know about version 15 firmware, I have always used version 16 firmware with 9.x to match the driver version. From owner-freebsd-scsi@freebsd.org Tue Dec 8 16:05:24 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F86B9D3A07; Tue, 8 Dec 2015 16:05:24 +0000 (UTC) (envelope-from prateekrootkey@gmail.com) Received: from mail-wm0-x22f.google.com (mail-wm0-x22f.google.com [IPv6:2a00:1450:400c:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C6591E28; Tue, 8 Dec 2015 16:05:24 +0000 (UTC) (envelope-from prateekrootkey@gmail.com) Received: by wmww144 with SMTP id w144so35642876wmw.0; Tue, 08 Dec 2015 08:05:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=59VcFUEhMq8erY1qLzmkDEISAlSxFnCexh4CfRvveB4=; b=0Vc3wo8myPr9paNliD46eLgvZnLykoLbLkrNd4wUKfXOl4Ay2m59NCSGngCKeLN65g umJp55YbqBpNgniyr2f5O61J2dy9r5e7PRix+jo7tUiJCUI5w5DyieKMeU/qacE2OvnR 0fi6zQ/JYoU/x7g1eeJCpidyFOR5L/nfO3L0asLbBGJHWBo6rcICpSdtyLGW3SUhkf0J JQJWCrfnrBZtWRXioeBdIa7w6EY/21QC/6zsuLW4Ia1tgYxn5fQujZKySjchXEtSBaoI uWx58bslfZ/wVTUfDsAWrGd9SpGikAXSvLLTTmQoE/XHOXRAHM30IE+kh6x5QuLHahC5 bl6Q== MIME-Version: 1.0 X-Received: by 10.28.45.216 with SMTP id t207mr5661237wmt.89.1449590722048; Tue, 08 Dec 2015 08:05:22 -0800 (PST) Received: by 10.27.16.7 with HTTP; Tue, 8 Dec 2015 08:05:21 -0800 (PST) In-Reply-To: References: <6A7832F8-53EB-4641-8EF6-E0E6175EB52D@yahoo.com> Date: Tue, 8 Dec 2015 21:35:21 +0530 Message-ID: Subject: Re: bad disk discovery From: prateek sethi To: Michael Jung Cc: Scott Long , freebsd-scsi@freebsd.org, owner-freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 16:05:24 -0000 Yes, I have tried that one but issue is still there. Other disk are working fine with the same configuration that means firmware should not be a problem. On Tue, Dec 8, 2015 at 6:56 PM, Michael Jung wrote: > On 2015-12-08 08:00, prateek sethi wrote: > >> Hi Scott, >> Thanks for the your quick response. >> >> I have different set of hardware . So that's why I want to know how I ca= n >> debug it myself . Is there anyway or procedure using that I can findout >> about the situation or the reason for CDB errors or disk command failure= ? >> >> Right now I am giving detail about the setup where I am getting this >> issue . >> >> I am using LSI SAS2008 controller and connected with supermicro Enclosur= e >> with freebsd 9.3. 16 different disks are there but only one disk is havi= ng >> problem. That means contoller and cable are fine. >> >> Faulty disk info are like:-. >> >> *smartctl output is:-* >> >> smartctl -x /dev/da23 >> >> =3D=3D=3D START OF INFORMATION SECTION =3D=3D=3D >> Vendor: SEAGATE >> Product: ST3600057SS >> Revision: 000B >> Rotation Rate: 15000 rpm >> Form Factor: 3.5 inches >> Logical Unit id: 0x5000c5007725173f >> Serial number: 6SL8YLPC0000N5030DY7 >> Device type: disk >> Transport protocol: SAS >> Local Time is: Tue Dec 8 18:20:45 2015 IST >> *device is NOT READY (e.g. spun down, busy)* >> >> *Logs:-* >> >> Dec 8 14:12:01 N1 kernel: da23 at mps0 bus 0 scbus0 target 148 lun 0 >> Dec 8 14:12:01 N1 kernel: da23: Fixed Direct >> Access SCSI-5 device >> Dec 8 14:12:01 N1 kernel: da23: Serial Number 6SL8YLPC0000N5030DY7 >> Dec 8 14:12:01 N1 kernel: da23: 600.000MB/s transfers >> Dec 8 14:12:01 N1 kernel: da23: Command Queueing enabled >> Dec 8 14:12:01 N1 kernel: da23: *Attempt to query device size failed: N= OT >> READY, Logical unit not ready, cause n* >> Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: Element descriptor: 'Slot >> 24' >> Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: SAS Device Slot Element: 1 >> Phys at Slot 23 >> >> *driver versions:-* >> >> >> dev.mps.0.firmware_version: 15.00.00.00 >> dev.mps.0.driver_version: 16.00.00.00-fbsd >> >> >> >> >> >> >> On Tue, Dec 8, 2015 at 3:15 AM, Scott Long wrote: >> >> Hi, >>> >>> If your situation is accurate and the disk is not responding properly t= o >>> regular >>> commands then it=E2=80=99s unlikely that it will respond to SMART comma= nds >>> either. >>> Sometimes these situations are caused by a bad cable, bad controller, o= r >>> buggy software/firmware, and only rarely will the standard statistics i= n >>> SMART >>> pick up these kinds of errors. SMART is better at tracking wear rates >>> and >>> error rates on the physical media, both HDD and SSD, but even then it= =E2=80=99s >>> hard >>> for it to be accurately predictive or even accurately diagnostic. For >>> your case, >>> I recommend that you describe your hardware and software configuration = in >>> more detail, and look for physical abnormalities in the cabling and >>> connections. >>> Once that is ruled and and the rest of us know what kind of hardware >>> you=E2=80=99re >>> dealing with, we might be able to make better commendations. >>> >>> Scott >>> >>> > On Dec 7, 2015, at 11:07 AM, prateek sethi >>> wrote: >>> > >>> > Hi , >>> > >>> > Is there any way or tool to find out that a disk which is not >>> responding >>> > properly is really bad or not? Sometimes I have seen that there is lo= t >>> of >>> > CDB error for a drive and system reboot makes every thing fine. What >>> can >>> be >>> > reasons for such kind of scenarios? >>> > >>> > I know smartctl is the one which can help. I have some couple of >>> question >>> > regarding this . >>> > >>> > 1. What if disk does not support smartctl? >>> > 2. How I can do smartest use of smartctl command like which parameter= s >>> can >>> > tell that the disk is actually bad? >>> > 3. What other test I can perform to make it sure that disk has >>> completely >>> > gone? >>> > >>> > >>> > Please tell me correct place to ask this question if I am asking at >>> wrong >>> > place. >>> > _______________________________________________ >>> > freebsd-scsi@freebsd.org mailing list >>> > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >>> > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.or= g >>> " >>> >>> >>> _______________________________________________ >> freebsd-scsi@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" >> > > > Have you simply moved the drive to another slot - does the problem follow > the drive? > Unlikely but it could be a backplane issue. > > I don't know about version 15 firmware, I have always used version 16 > firmware > with 9.x to match the driver version. > > From owner-freebsd-scsi@freebsd.org Tue Dec 8 16:20:51 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 360519D4750 for ; Tue, 8 Dec 2015 16:20:51 +0000 (UTC) (envelope-from stephen.mcconnell@avagotech.com) Received: from mail-pf0-x22a.google.com (mail-pf0-x22a.google.com [IPv6:2607:f8b0:400e:c00::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 11BFC1F5B for ; Tue, 8 Dec 2015 16:20:50 +0000 (UTC) (envelope-from stephen.mcconnell@avagotech.com) Received: by pfbg73 with SMTP id g73so14635938pfb.1 for ; Tue, 08 Dec 2015 08:20:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=avagotech.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=coCg5xU4j7BrZi4H7VUew/eAxF1LSn/97NMMyLClPTI=; b=UiPjVrggTA0VxEE6JYbPpw+WJESZhzh4lDbqKBUCckzZL2wnj8FModGK5js/hr1FK9 n+gJK3AvrTEbYBYIP0y617M3Vs6jKWuzGkcZNnw9Rz1ShL4Ttjvc5GRrQq5qp+Jit44G +ULYc3pLFifdaJ5NqvFPL5aHS5VDbZnQZzr4I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=coCg5xU4j7BrZi4H7VUew/eAxF1LSn/97NMMyLClPTI=; b=mndtCeVmO87sH9eUQE1YPAtunIPCbDfUiT1MUyAkHWJQ5L86JssiQQ+4B6kcUnlr+4 ykTJqYZCaTgM18jrzByFFnl6yd9+rjfDG6cIbpRThy9zQH9uNtjgIUsgcOhWOnHYKwxW GPMkW7a/alBkMJ80d/Qv9QQQZ3XL6H2R/Q7+pp3dZxlgYd9nmo7PdGPTBuLernAG+yXV fAndo8cwLF5N+0hKByUT/zMTwXE7nzNbPRMBQxQja8HkNl8dwlUluoBMhk35CwEVcsVV xR2pYCakAle1dsiTVGEHpkd1WCPbf29uaRKhVzZ/ABRGHuqA8dQH+Geal8RT12/DgMDF E1Sw== X-Gm-Message-State: ALoCoQlGUCPJ9ISMmrtq04IXE73R1qScgaG9hAAGPnxccFRa7j449ZpufvMfwuMzzIgn6yND1nTpmxZ1hFQ01ExjC7BvyKxRg1zh6wxiuoFLaOMaKhlHm30= X-Received: by 10.98.14.26 with SMTP id w26mr6107849pfi.110.1449591650313; Tue, 08 Dec 2015 08:20:50 -0800 (PST) From: Stephen Mcconnell References: <6A7832F8-53EB-4641-8EF6-E0E6175EB52D@yahoo.com> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJO1PHZ3S5C87fllREIl8yX+V/IRAJvH4HyAm7AR3ACs4vlkAGgDtQvnXxyjhA= Date: Tue, 8 Dec 2015 09:20:48 -0700 Message-ID: <48445286cfa5082c78581b2c1e8afb66@mail.gmail.com> Subject: RE: bad disk discovery To: prateek sethi , Michael Jung Cc: freebsd-scsi@freebsd.org, owner-freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 16:20:51 -0000 Try looking through the system log to see if there is any debug output from the mps driver, or you can send it to me and I'll take a look. It might give us a clue as to what's going on. Steve McConnell > -----Original Message----- > From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd- > scsi@freebsd.org] On Behalf Of prateek sethi > Sent: Tuesday, December 08, 2015 9:05 AM > To: Michael Jung > Cc: freebsd-scsi@freebsd.org; owner-freebsd-scsi@freebsd.org > Subject: Re: bad disk discovery > > Yes, I have tried that one but issue is still there. Other disk are > working fine > with the same configuration that means firmware should not be a problem. > > > On Tue, Dec 8, 2015 at 6:56 PM, Michael Jung wrote: > > > On 2015-12-08 08:00, prateek sethi wrote: > > > >> Hi Scott, > >> Thanks for the your quick response. > >> > >> I have different set of hardware . So that's why I want to know how I > >> can debug it myself . Is there anyway or procedure using that I can > >> findout about the situation or the reason for CDB errors or disk > >> command > failure? > >> > >> Right now I am giving detail about the setup where I am getting this > >> issue . > >> > >> I am using LSI SAS2008 controller and connected with supermicro > >> Enclosure with freebsd 9.3. 16 different disks are there but only one > >> disk is having problem. That means contoller and cable are fine. > >> > >> Faulty disk info are like:-. > >> > >> *smartctl output is:-* > >> > >> smartctl -x /dev/da23 > >> > >> =3D=3D=3D START OF INFORMATION SECTION =3D=3D=3D > >> Vendor: SEAGATE > >> Product: ST3600057SS > >> Revision: 000B > >> Rotation Rate: 15000 rpm > >> Form Factor: 3.5 inches > >> Logical Unit id: 0x5000c5007725173f > >> Serial number: 6SL8YLPC0000N5030DY7 > >> Device type: disk > >> Transport protocol: SAS > >> Local Time is: Tue Dec 8 18:20:45 2015 IST > >> *device is NOT READY (e.g. spun down, busy)* > >> > >> *Logs:-* > >> > >> Dec 8 14:12:01 N1 kernel: da23 at mps0 bus 0 scbus0 target 148 lun 0 > >> Dec 8 14:12:01 N1 kernel: da23: Fixed > >> Direct Access SCSI-5 device Dec 8 14:12:01 N1 kernel: da23: Serial > >> Number 6SL8YLPC0000N5030DY7 Dec 8 14:12:01 N1 kernel: da23: > >> 600.000MB/s transfers Dec 8 14:12:01 N1 kernel: da23: Command > >> Queueing enabled Dec 8 14:12:01 N1 kernel: da23: *Attempt to query > >> device size failed: NOT READY, Logical unit not ready, cause n* Dec > >> 8 14:12:01 N1 kernel: ses1: da23,pass26: Element descriptor: 'Slot > >> 24' > >> Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: SAS Device Slot > >> Element: 1 Phys at Slot 23 > >> > >> *driver versions:-* > >> > >> > >> dev.mps.0.firmware_version: 15.00.00.00 > >> dev.mps.0.driver_version: 16.00.00.00-fbsd > >> > >> > >> > >> > >> > >> > >> On Tue, Dec 8, 2015 at 3:15 AM, Scott Long > wrote: > >> > >> Hi, > >>> > >>> If your situation is accurate and the disk is not responding > >>> properly to regular commands then it=E2=80=99s unlikely that it will = respond > >>> to SMART commands either. > >>> Sometimes these situations are caused by a bad cable, bad > >>> controller, or buggy software/firmware, and only rarely will the > >>> standard statistics in SMART pick up these kinds of errors. SMART > >>> is better at tracking wear rates and error rates on the physical > >>> media, both HDD and SSD, but even then it=E2=80=99s hard for it to be > >>> accurately predictive or even accurately diagnostic. For your case, > >>> I recommend that you describe your hardware and software > >>> configuration in more detail, and look for physical abnormalities in > >>> the cabling and connections. > >>> Once that is ruled and and the rest of us know what kind of hardware > >>> you=E2=80=99re dealing with, we might be able to make better commenda= tions. > >>> > >>> Scott > >>> > >>> > On Dec 7, 2015, at 11:07 AM, prateek sethi > >>> > > >>> wrote: > >>> > > >>> > Hi , > >>> > > >>> > Is there any way or tool to find out that a disk which is not > >>> responding > >>> > properly is really bad or not? Sometimes I have seen that there is > >>> > lot > >>> of > >>> > CDB error for a drive and system reboot makes every thing fine. > >>> > What > >>> can > >>> be > >>> > reasons for such kind of scenarios? > >>> > > >>> > I know smartctl is the one which can help. I have some couple of > >>> question > >>> > regarding this . > >>> > > >>> > 1. What if disk does not support smartctl? > >>> > 2. How I can do smartest use of smartctl command like which > >>> > parameters > >>> can > >>> > tell that the disk is actually bad? > >>> > 3. What other test I can perform to make it sure that disk has > >>> completely > >>> > gone? > >>> > > >>> > > >>> > Please tell me correct place to ask this question if I am asking > >>> > at > >>> wrong > >>> > place. > >>> > _______________________________________________ > >>> > freebsd-scsi@freebsd.org mailing list > >>> > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > >>> > To unsubscribe, send any mail to > >>> > "freebsd-scsi-unsubscribe@freebsd.org > >>> " > >>> > >>> > >>> _______________________________________________ > >> freebsd-scsi@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > >> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org= " > >> > > > > > > Have you simply moved the drive to another slot - does the problem > > follow the drive? > > Unlikely but it could be a backplane issue. > > > > I don't know about version 15 firmware, I have always used version 16 > > firmware with 9.x to match the driver version. > > > > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Wed Dec 9 08:35:30 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BC929D3383 for ; Wed, 9 Dec 2015 08:35:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DB9219B4 for ; Wed, 9 Dec 2015 08:35:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tB98ZTjI050818 for ; Wed, 9 Dec 2015 08:35:29 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 195033] [cam] [scsi] [patch] unclean SCSI attached HDD power-off on shut down Date: Wed, 09 Dec 2015 08:35:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rpokala@panasas.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2015 08:35:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195033 --- Comment #3 from Ravi Pokala --- https://reviews.freebsd.org/D4456 is up for review. It both documents the enable_ssu knob, and changes the default value such that SSU is sent to both HDDs and SSDs. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Thu Dec 10 12:20:54 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79F3C9D72D5; Thu, 10 Dec 2015 12:20:54 +0000 (UTC) (envelope-from maxg@mellanox.com) Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3on0075.outbound.protection.outlook.com [157.55.234.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89C6E13BA; Thu, 10 Dec 2015 12:20:52 +0000 (UTC) (envelope-from maxg@mellanox.com) Received: from AM3PR05CA0054.eurprd05.prod.outlook.com (10.162.114.22) by DB4PR05MB541.eurprd05.prod.outlook.com (10.242.155.142) with Microsoft SMTP Server (TLS) id 15.1.355.16; Thu, 10 Dec 2015 12:20:43 +0000 Received: from AM1FFO11FD035.protection.gbl (2a01:111:f400:7e00::111) by AM3PR05CA0054.outlook.office365.com (2a01:111:e400:52b7::22) with Microsoft SMTP Server (TLS) id 15.1.355.16 via Frontend Transport; Thu, 10 Dec 2015 12:20:43 +0000 Authentication-Results: spf=pass (sender IP is 193.47.165.134) smtp.mailfrom=mellanox.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=pass action=none header.from=mellanox.com; Received-SPF: Pass (protection.outlook.com: domain of mellanox.com designates 193.47.165.134 as permitted sender) receiver=protection.outlook.com; client-ip=193.47.165.134; helo=mtlcas13.mtl.com; Received: from mtlcas13.mtl.com (193.47.165.134) by AM1FFO11FD035.mail.protection.outlook.com (10.174.64.224) with Microsoft SMTP Server (TLS) id 15.1.337.8 via Frontend Transport; Thu, 10 Dec 2015 12:20:41 +0000 Received: from MTLCAS13.mtl.com (10.0.8.78) by mtlcas13.mtl.com (10.0.8.78) with Microsoft SMTP Server (TLS) id 15.0.775.38; Thu, 10 Dec 2015 14:20:37 +0200 Received: from MTLCAS01.mtl.com (10.0.8.71) by MTLCAS13.mtl.com (10.0.8.78) with Microsoft SMTP Server (TLS) id 15.0.775.38 via Frontend Transport; Thu, 10 Dec 2015 14:20:37 +0200 Received: from [10.222.32.225] (10.222.32.225) by MTLCAS01.mtl.com (10.0.8.71) with Microsoft SMTP Server (TLS) id 14.3.123.3; Thu, 10 Dec 2015 14:20:25 +0200 To: , Sagi Grimberg , Oren Duer , Hans Petter Selasky , From: Max Gurtovoy Subject: splitting iovecs to bios Message-ID: <56696E03.8050202@mellanox.com> Date: Thu, 10 Dec 2015 14:20:19 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1255"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.222.32.225] X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; AM1FFO11FD035; 1:CoVvXxpIKf3OjH7y8IX/Eck1IkCjDV1v0/VmFb+JK2/Jc9xhnjUm8YqZ8zROvw4HM2JsZEhjR+ToVnZcKO6dOz0GZFWtw3ObloUTAQNZGGgg463c+qwlpnmM1fqtkBXTZOmd2CGiMKbai2J5wXovCw7N7mqelsDS3slTD42+DR+ed3GXxZ4cG5f/X1EWIgwQ4gJTFrrzAV/Aba4WB/cr3zuh22s/YvF/UbLFZjR5rs3Iyvhh9yBFQG7G8cDIn5TCZofZHJZffwFEjeI7Rw92cKeD36d1fDZ/bTlEJW2s3S0EqGPTuj9pTPhQM+h86qbkvdWfJgyV4eZmIYs/fX19D4jvZgsopmSq9VdVQz1AbAM= X-Forefront-Antispam-Report: CIP:193.47.165.134; CTRY:IL; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(438002)(164054003)(53754006)(199003)(189002)(230700001)(3846002)(1220700001)(80316001)(6116002)(586003)(1096002)(47776003)(50466002)(87266999)(36756003)(50986999)(11100500001)(64126003)(450100001)(189998001)(54356999)(107886002)(5004730100002)(6806005)(86362001)(5001770100001)(99136001)(229853001)(97736004)(106466001)(87936001)(33656002)(4001350100001)(5008740100001)(65956001)(65816999)(59896002)(65806001)(77096005)(83506001)(92566002)(3940600001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB4PR05MB541; H:mtlcas13.mtl.com; FPR:; SPF:Pass; PTR:ErrorRetry; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; DB4PR05MB541; 2:IdrhbVgZ1A9jB3sZDS/igSOoDhQMxePw4+U6wjng+Tp+SksIgADtWdyPbeZHSvrQubUomMyPg6XoKCdkqzD5FAI931d1kkI/AGfYmecupTW4Obyko9U/8P1IkI8i6h2i62GmEysC38hVpg6g3J2n5w==; 3:Kp7lOpjYDdQaZekFhllffd3CBDz9BduDnN9uJvVRW8dk4RNPj1/O6zwdv53I0WMXg7p2IhABuVPJS5PGc9Ty8yB11c6I7JVUncUkVZQwETZ8Tml1rMGMVZeGVDVOVgzM4tv+ZQ0XOwNG3FfjgmtG9AYRUAXrVcFet1p14GXU5tBh3cebm1h0yeZkp4KRj4+7rWX3fi3TslLCLTICD2av4Z1TLkKiwl9V9ldyCigV+hMe6Z7gmwq472lzNYz/TjKD8t8wcndxe/OSM8vGarA+Vw==; 25:ovoK2V69xQWKaZiEx5CArhRV4w4I2RFMCcK+/4IK+9PXYroQe2HVV9V+qHbC68p+LmEJ9pD4RUYNQ3/hiLGJ1Z0rPK7jvXxmKQNT1U4xKhfYHPgVR0bSGWf2XlbjVOvC5+ol3KXyO7cqwkAp70WdgXACif9c0fQMsDAyjhwMQPbsUUt4yJZcOnCdZGQr54zrRsZ2SsVEQ24Yet/bHYG4qEdjt0xFjEvYPdoKdXeyDyKBCGFoGyHAqaEdIKznEv3l13zG9GBfp6OyvuPBFcTJIA== X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501001); SRVR:DB4PR05MB541; X-Microsoft-Exchange-Diagnostics: 1; DB4PR05MB541; 20:ygjIBYUI3PIwhnQ0Yip09fWybUufSBQZzfuZd4MxO1gJRcF/I+zz7HnCuc4aQVFitZCK9GGh1zNpGm2FbTAze7SnrbVafctXRXZywYJG+6Rm9I3UtfNpja7cxed5qvBuQbUDBcuC1SEySQMtiFbkniLQwctnqKMeNZcqdpgwNElF7vsAriJiJe9PfcGcjPfFHl7heJ6zukBJoa0T3gLnvt8x/N4A18ioHzzIm6fiuYr2E8TGynlLJvWW3lZWtQsHmRm2PMhxjNUYl+QZ+rxBYsLDcV/4W8wbzj+VLUv2tX6niNEblYB9lp6oOfr1Dd13y5dL75Csz6HthFw63FoCrPqdpjmFdQ87HOMjxUOC17oCNh4o3ycF1THhgEnUs+cqGaPoSKILvEzz2IVEVLGLOlP98B8vEjqNSKHT6B1AuO9qvq68z7/CLZ+0bsOyw3eyBQVGXMbQOg4VDLIS8eGZXodFJJtdw7xpHV9P8MELZiYk4Ma/OFiZUkvQQ3XtdSlK; 4:KTqgpnQeAppJWLGFNti/M6wP0zst3gQlmLYKiiSOOqVTafrOGzkvhZIFl+jM3YtpMnQQUWB8R6+HdRrXN2ZvNw2eoYDGl/CXfTklN4g2NQ4lj8wPj44UWAJNoataXu71v8g5NHyfTZaPsw1X6pPikGrxMrCsUklV9KMKuXn13Dg6xgHpOENjGgAE4ornu8XC1CRD3EALwQZjuhCkmHXRO2PYHfuMYWiVpgKpji/4pLY2FJnNmAyFstNuK1KNW8c2onvUl53cCJPlEuI6b2SvlRxZxNK0AXtOUWbKMede0QQFxunLJ1SwxYLCMq2lYKsflzRFPud0HkWBVfYmcho0btlGCQheJ6qwOw69eos57F4pDC4ZsCm0pXMjvNYhzNJG X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(520078)(8121501046)(5005006)(10201501046)(3002001); SRVR:DB4PR05MB541; BCL:0; PCL:0; RULEID:; SRVR:DB4PR05MB541; X-Forefront-PRVS: 078693968A X-Microsoft-Exchange-Diagnostics: =?windows-1255?Q?1; DB4PR05MB541; 23:SueQiPHTqb9ms/thhb4aApKaF5/7hm5gm2o/R+?= =?windows-1255?Q?S2nHfNZI7XppmkUJJ7szU5tPyvUwDtdGo2m6MZwrKaR+9CKn4ALxG/Bn?= =?windows-1255?Q?YQ1K/LDkAwSRxrMLbTweRwrWve9Y70mFqXDBOECQ0ge3sLftA7o0EPFZ?= =?windows-1255?Q?v4h++poRZ7JsKNGT+RMtp+JMVK/EnfO9xMcLmaT5ZKsof7sCCGgPeoi9?= =?windows-1255?Q?fFkFhlUZXSjUlhACffndzuN9IjxahzYT4aujG4t1N95dS6KPnrp90ivg?= =?windows-1255?Q?YNX2kC8pc5yEXt6YP/gQWytqboi7cYIrOdOtWGBrBIdHbRAH1H9gAYBX?= =?windows-1255?Q?kxZgRu9pTeADhmI5DoJ2Z03fFfUDOVjmXXNVkQMhSm5D2Nytcph9Ob3W?= =?windows-1255?Q?t2ppv2hhQ2jSN+3KgBWD93FyIvdHEizxCFa/IfuyXuPnGArZYrfjiIEN?= =?windows-1255?Q?EDay3BVKIoLWSSRqVzR4zce4ueigEDihCJyxJpiIs00104JjRS/AhIgV?= =?windows-1255?Q?T8Yvkrdv4FxGhfJcYXVlsOvQQwaAqlz95OfPJaIMahiCvlVG9jWBDy2u?= =?windows-1255?Q?bx3RyvAVfxWYzowViao/u5MS9+dMUPZVrKjTn9g+TPFAZegpTFrDvTx4?= =?windows-1255?Q?+XO/iwd3w7re4AaGGpnfYJUlIs9GLG+mTQ/rtRimBn/lHeRY9tWOr0N1?= =?windows-1255?Q?EH5Z0j7BbecuFdRbt8r/mX8BqQlSPv5e5afcUFXzyW8S0hF/3BlsJZVO?= =?windows-1255?Q?T6eGiqeWrKr/tTLzNEzmcx8LzmsIpcQvbuyfcdgm7Uy1R7vzxGsgWgSJ?= =?windows-1255?Q?lilBMWa6Zk7eS9uu5OV76L75iV2ETHc9sf/8thuXKZRScIaDiB69RJC9?= =?windows-1255?Q?Z0NJRonXKo14MQNS070fd4NK78EVsgfOra2NP9QFFZK9k3cXaopWvY+/?= =?windows-1255?Q?ZjVW/gaB6vhvko3MEf7BsbLHPXQyBT0diVZOr5KFAAwvcmJM2iE5GLGE?= =?windows-1255?Q?7AXSaqicZtt8GSp/VyUPbDFMEhKSGYJ5YyHx/n+wK+Dj6We1GCzR2DDC?= =?windows-1255?Q?9vpR9k4A8d9tahvn6RYQbemeY9JWmhAPpXqoSTprh0D+pGp43vOzf1Es?= =?windows-1255?Q?AzcEJOPRYKQocygDOXhrTiJR65Min/4DLzg8hB4SwLLmdVsxuWcB9NgP?= =?windows-1255?Q?pE4LJWO6iIiv0AG3rVIhkDSHXxJkHgkqa/iuARgpDK3qjFTNCaGpEGez?= =?windows-1255?Q?mLAu3VCZ3LAmk9VQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; DB4PR05MB541; 5:pNDsZVtdK1hjE0/1jY3ZLFjaMzmdOs9atPxpkOj9f1PtfYQWBhgeKqKo8/yzVMZeQ7clRGVP3H9NMKptM+qIvI2jBQlyRQgrORdikf+JSREuM8OD5YmdHTlDV9UaFK3GdFPajmWUr4v/R1/37a8uNA==; 24:Ji2sZTGcmR1Qaqu/e71mzOXJHQO4mf2LJUjGEZQ5lbjwxbSzcTrsD+TAobp/F6f5gGvfjSHyPSkCBXnJAL9KnkKr8urnPPMUbu86/PEkKHw= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Dec 2015 12:20:41.1852 (UTC) X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a652971c-7d2e-4d9b-a6a4-d149256f461b; Ip=[193.47.165.134]; Helo=[mtlcas13.mtl.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR05MB541 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 12:20:54 -0000 Hi all, I'm developing an iSER (iSCSI extensions for RDMA) driver for FreeBSD 11-Current and I encounter some weired behaviour while testing IOs with iovcnt > 1. I wrote a small test program that creates an iovec struct (let's say in size of 4) and each iov_base starts from page begining and 4k len: 0 iov_base 0xe25000 iov_len 4096 1 iov_base 0xe26000 iov_len 4096 2 iov_base 0xe27000 iov_len 4096 3 iov_base 0xe28000 iov_len 4096 I use readv/writev to send this iovec data. I saw that my driver get 4 different bio requests even thought this vector is aligned. This is surprising becasue in other OS I get only 1 bio. I have noticed the the physio function in sys/kern/kern_physio.c is praparing one bio for each iovec entry. is there a reason for this kind of implementation ? is there an option to send this array using 1 bio (some flag) ? We can improve performance if we send it in 1 bio instead of 4. My driver supports BIO_UNMAPPED. Thanks, Max Gurtovoy. Mellanox Technologies. From owner-freebsd-scsi@freebsd.org Thu Dec 10 12:58:19 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9AD9A9D5C76 for ; Thu, 10 Dec 2015 12:58:19 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-vk0-x233.google.com (mail-vk0-x233.google.com [IPv6:2607:f8b0:400c:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6135D198A for ; Thu, 10 Dec 2015 12:58:19 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: by vkbs1 with SMTP id s1so80270359vkb.1 for ; Thu, 10 Dec 2015 04:58:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bkSwYYi3AUuIzQFlh/Igjl2ABZ8QNnEdHnBYqClEtn8=; b=jYixfqZmCBKZPh0NdOVAT7cf5fnK4t1x6oTUbDouHl/CfQk33Kv4dnDBVU2oy/kCZT SUGwSQYHHzsO17j6LRBbwyLFIoxAUTI1+Qd1+rpxELMXOxyxzZgqivFJ+s9PqaCq3Kbg xez+wvjPx7/cOLD70Tm9hHKFCF+Pn2v5RLF50= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=bkSwYYi3AUuIzQFlh/Igjl2ABZ8QNnEdHnBYqClEtn8=; b=Izg9UZ/wedyWUFjT7VxPACyxw2MNrZANKfa8w1lgSkT2+JawEineS8RgChZpqPMMzW CCPVb/VH2o4JBUZgvH13++y4fhDCVmttc1AYuwHO7uNLrEUZTU52lezK8Vs4EEkP1dWL qh5GGjzAk/XMWr/n/vDDxpdsLzoxKH4JqcoKrpKbAJhdPUbF+oxIljFi01MOgiplYZVv ZCLT8v0bmiOCMBSTs7ATUKeIyQyuxYj9x8q1eEhWh/iQQlNl5muv4CrMibAGACN4vzyC FLrv1T090UWrGDJosul5hZJyPh0RchOFILNTqjZUtfJseVANWdnV5OWxj+40nN8tSdSF qXHQ== X-Gm-Message-State: ALoCoQnEl4VkM40+i6NZixp+NCXLe4ScbrH3wlb/xpcnY1beEdw6m7kZJQqvBlxQr4o0QKsv4vjf7yOmf+I4Th/CgM3VOKkXTQ== MIME-Version: 1.0 X-Received: by 10.129.81.147 with SMTP id f141mr4375443ywb.176.1449752298177; Thu, 10 Dec 2015 04:58:18 -0800 (PST) Received: by 10.37.13.147 with HTTP; Thu, 10 Dec 2015 04:58:18 -0800 (PST) In-Reply-To: <80BB5907-CC31-4F06-9C70-E6F7834FF28E@panasas.com> References: <80BB5907-CC31-4F06-9C70-E6F7834FF28E@panasas.com> Date: Thu, 10 Dec 2015 05:58:18 -0700 Message-ID: Subject: Re: Accessing static drive info w/o ATA identify and lockup with camcontrol identify From: Kevin Bowling To: Ravi Pokala Cc: "freebsd-scsi@freebsd.org" , Sean Bruno Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 12:58:19 -0000 Thanks Ravi! Rotation rate is probably the most important thing and thankfully easy as you suggested: https://reviews.freebsd.org/D4483. Would be nice to land that in 10.3 so I can swap the SaltStack module over to it. Next in priority would be getting firmware version.. In 'camcontrol identify' it looks like: 'firmware revision DXM9203Q' In 'camcontrol inquiry' it's part of the device string like: 'pass0: ACS-2 ATA SATA 3.x device' Do you have any suggestions for wiring that into geom disk? And lastly, yes bus speeds would be nice (especially to spot hard/soft misconfiguration). It shows up like 'protocol ATA/ATAPI-9 SATA 3.x' in camcontrol identify and can be seen in the inquiry string above too. Do you have design suggestions on that? Regards, On Fri, Dec 4, 2015 at 9:23 AM, Ravi Pokala wrote: > >Date: Fri, 4 Dec 2015 02:13:31 -0700 > >From: Kevin Bowling > >To: freebsd-scsi@freebsd.org > >Subject: Accessing static drive info w/o ATA identify and lockup with > > camcontrol identify > >Message-ID: > > QA0jKsbuH9wSOTjDiiQdmJhmKJg@mail.gmail.com> > >Content-Type: text/plain; charset=UTF-8 > > > >... > > > >#2 This all came about because I want to poll device information like disk > >model, serial number, speeds, etc. Common use cases would be > configuration > >management systems and inventory databases. Issuing an ATA identify seems > >a bit much and could trigger unwanted HW errata like above. I'm wondering > >if it would be better to cache the data on interface change in sysctls or > >something. The data is static, read only, but we need to account for disk > >swaps. > > At least some of it is already available via `geom disk list': > > [daneel:~] rpokala% geom disk list ada5 > Geom name: ada5 > Providers: > 1. Name: ada5 > Mediasize: 2000398934016 (1.8T) > Sectorsize: 512 > Stripesize: 4096 > Stripeoffset: 0 > Mode: r1w1e1 > descr: WDC WD20EFRX-68EUZN0 > lunid: 50014ee20b9f7c0f > ident: WD-WCC4M1VN0P7L > fwsectors: 63 > fwheads: 16 > > > > GEOM keeps this data in a "struct disk", which is populated during drive > attach. You can see that the drive model ("descr") and serial number > ("ident") are already listed by GEOM. "struct disk" also already has a > rotation-rate field; if that's what you mean by "speeds", it should be > trivial for me to add that to the GEOM output as well. > > If by "speeds", you're talking about the connection's bus speed (i.e. > 3/6/12Gbps), then that's probably more complicated. > > -Ravi (rpokala@) > > >Regards, > >Kevin > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > From owner-freebsd-scsi@freebsd.org Thu Dec 10 15:02:15 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F10619D6796; Thu, 10 Dec 2015 15:02:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7376818F3; Thu, 10 Dec 2015 15:02:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tBAF2AlF026079 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 10 Dec 2015 17:02:10 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tBAF2AlF026079 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tBAF2Awr026078; Thu, 10 Dec 2015 17:02:10 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 10 Dec 2015 17:02:10 +0200 From: Konstantin Belousov To: Max Gurtovoy Cc: freebsd-scsi@freebsd.org, Sagi Grimberg , Oren Duer , Hans Petter Selasky , freebsd-geom@freebsd.org Subject: Re: splitting iovecs to bios Message-ID: <20151210150210.GY82577@kib.kiev.ua> References: <56696E03.8050202@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <56696E03.8050202@mellanox.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 15:02:16 -0000 On Thu, Dec 10, 2015 at 02:20:19PM +0200, Max Gurtovoy wrote: > Hi all, > > I'm developing an iSER (iSCSI extensions for RDMA) driver for FreeBSD > 11-Current and I encounter some weired behaviour while testing IOs with > iovcnt > 1. > I wrote a small test program that creates an iovec struct (let's say in > size of 4) and each iov_base starts from page begining and 4k len: > 0 iov_base 0xe25000 iov_len 4096 > 1 iov_base 0xe26000 iov_len 4096 > 2 iov_base 0xe27000 iov_len 4096 > 3 iov_base 0xe28000 iov_len 4096 > > I use readv/writev to send this iovec data. > I saw that my driver get 4 different bio requests even thought this > vector is aligned. This is surprising becasue in other OS I get only 1 bio. > I have noticed the the physio function in sys/kern/kern_physio.c is > praparing one bio for each iovec entry. > > is there a reason for this kind of implementation ? is there an option > to send this array using 1 bio (some flag) ? > We can improve performance if we send it in 1 bio instead of 4. > My driver supports BIO_UNMAPPED. There might be indeed a reason, it could be that some drivers expect blocking to be done by the userspace. The drivers could have some restrictions on transfer sizes and atomicity of transfer, which would be broken by the unconditional merge. I cannot give you an example of such driver, known block-aware drivers like sa(4) only require the bio size to be multiple of the basic block size. OTOH, I see no issue with adding a SI_PHYSIOMERGE flag and doing the merges for the driver in physio(), when unmapped request has consequtive iov elements ending and starting at the page boundary. From owner-freebsd-scsi@freebsd.org Thu Dec 10 16:10:26 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CFC69D6A55 for ; Thu, 10 Dec 2015 16:10:26 +0000 (UTC) (envelope-from rpokala@panasas.com) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0053.outbound.protection.outlook.com [65.55.169.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6FFB1942; Thu, 10 Dec 2015 16:10:25 +0000 (UTC) (envelope-from rpokala@panasas.com) Received: from CY1PR08MB1803.namprd08.prod.outlook.com (10.162.218.25) by CY1PR08MB1804.namprd08.prod.outlook.com (10.162.218.26) with Microsoft SMTP Server (TLS) id 15.1.355.16; Thu, 10 Dec 2015 16:10:17 +0000 Received: from CY1PR08MB1803.namprd08.prod.outlook.com ([10.162.218.25]) by CY1PR08MB1803.namprd08.prod.outlook.com ([10.162.218.25]) with mapi id 15.01.0355.012; Thu, 10 Dec 2015 16:10:17 +0000 From: "Pokala, Ravi" To: Kevin Bowling , Ravi Pokala CC: "freebsd-scsi@freebsd.org" , Sean Bruno Subject: Re: Accessing static drive info w/o ATA identify and lockup with camcontrol identify Thread-Topic: Accessing static drive info w/o ATA identify and lockup with camcontrol identify Thread-Index: AQHRM2VBeCKuqTynrkez8plTZtfm2A== Date: Thu, 10 Dec 2015 16:10:16 +0000 Message-ID: References: <80BB5907-CC31-4F06-9C70-E6F7834FF28E@panasas.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/0.0.0.151105 authentication-results: spf=none (sender IP is ) smtp.mailfrom=rpokala@panasas.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [24.6.178.251] x-microsoft-exchange-diagnostics: 1; CY1PR08MB1804; 5:XOQnetABBxYGVSAkBVI6scoB/arFQKji/1/1/+0NgZg/LHarnPTNKRMQGvfX8E4q7/ozCaKFmkfBPsKIKqHh2K21mPOH/7gOVkZdjt9yJp+eNRjMu2Rk1aNrlm4MVkU8YoqM09bGx2mr6T4y2y3pgQ==; 24:V8OTccj1HgCNalPMChUu8hTFgRphZWnxh18BwBeKRs2TFMCdEkWA16TQc3PRdSYm8sJrJxTkCRw+NXflx0IncBMU+o8k7y6ui6JfdtNwfGI= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR08MB1804; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(520078)(8121501046)(5005006)(10201501046)(3002001); SRVR:CY1PR08MB1804; BCL:0; PCL:0; RULEID:; SRVR:CY1PR08MB1804; x-forefront-prvs: 078693968A x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(199003)(46034005)(13464003)(189002)(377424004)(40224003)(36756003)(4001350100001)(81156007)(5001770100001)(97736004)(87936001)(77096005)(82746002)(66066001)(86362001)(83506001)(19580405001)(19580395003)(33656002)(2950100001)(5890100001)(92566002)(2900100001)(15975445007)(83716003)(5002640100001)(4001150100001)(106356001)(10400500002)(40100003)(101416001)(6116002)(3846002)(102836003)(1220700001)(1096002)(586003)(5008740100001)(99286002)(5004730100002)(11100500001)(50986999)(105586002)(106116001)(54356999)(122556002)(5001960100002)(189998001)(76176999)(7059030)(104396002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR08MB1804; H:CY1PR08MB1803.namprd08.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: panasas.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <92A1DA67A761DC44A975AF8DF0A832B4@namprd08.prod.outlook.com> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Dec 2015 16:10:16.5902 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR08MB1804 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 16:10:26 -0000 LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCg0KDQpGcm9tOiBLZXZpbiBCb3dsaW5nIDxrZXZp bi5ib3dsaW5nQGtldjAwOS5jb20+DQpEYXRlOiAyMDE1LTEyLTEwLCBUaHVyc2RheSBhdCAwNDo1 OA0KVG86IFJhdmkgUG9rYWxhIDxycG9rYWxhQG1hYy5jb20+DQpDYzogImZyZWVic2Qtc2NzaUBm cmVlYnNkLm9yZyIgPGZyZWVic2Qtc2NzaUBmcmVlYnNkLm9yZz4sIFNlYW4gQnJ1bm8gPHNicnVu b0BmcmVlYnNkLm9yZz4NClN1YmplY3Q6IFJlOiBBY2Nlc3Npbmcgc3RhdGljIGRyaXZlIGluZm8g dy9vIEFUQSBpZGVudGlmeSBhbmQgbG9ja3VwIHdpdGggY2FtY29udHJvbCBpZGVudGlmeQ0KDQo+ VGhhbmtzIFJhdmkhDQo+DQo+Um90YXRpb24gcmF0ZSBpcyBwcm9iYWJseSB0aGUgbW9zdCBpbXBv cnRhbnQgdGhpbmcgYW5kIHRoYW5rZnVsbHkgZWFzeSBhcyB5b3Ugc3VnZ2VzdGVkOg0KPg0KPmh0 dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9ENDQ4My4gIFdvdWxkIGJlIG5pY2UgdG8gbGFuZCB0 aGF0IGluIDEwLjMgc28gSSBjYW4gc3dhcCB0aGUgU2FsdFN0YWNrIG1vZHVsZSBvdmVyIHRvIGl0 Lg0KDQpBcyBJIGp1c3QgY29tbWVudGVkIHRoZXJlLCBJIHdhcyBhY3R1YWxseSBnb2luZyB0byBs b29rIGF0IHRoaXMgdGhpcyB3ZWVrZW5kLiBNeSBjb25jZXJuIGlzIHRoYXQgdGhlIGRfcm90YXRp b25fcmF0ZSBkb2VzIG5vdCBzdHJpY3RseSBtYXAgdG8gdGhlIGFjdHVhbCByb3RhdGlvbiByYXRl IC0gdGhlcmUgYXJlIHNvbWUgc3BlY2lhbCBjYXNlcyBhbmQgcmVzZXJ2ZWQgdmFsdWVzLiBJIHdh cyB0aGlua2luZyBzb21ldGhpbmcgbW9yZSBsaWtlIHRoaXM6DQoNCiAgICBzYnVmX3ByaW50Zihz YiwgIiVzPHJvdGF0aW9ucmF0ZT4iLCBpbmRlbnQpOw0KICAgIGlmIChkcC0+ZF9yb3RhdGF0aW9u X3JhdGUgPT0gMCkNCiAgICAgICAgc2J1Zl9wcmludGYoc2IsICJ1bmtub3duIik7DQogICAgZWxz ZSBpZiAoZHAtPmRfcm90YXRpb25fcmF0ZSA9PSAxKQ0KICAgICAgICBzYnVmX3ByaW50ZihzYiwg IjAiKTsNCiAgICBlbHNlIGlmICgoZHAtPmRfcm90YXRpb25fcmF0ZSA+PSAweDA0MSkgJiYgKGRw LT5kX3JvdGF0aW9uX3JhdGUgPD0gMHhmZmZlKSkNCiAgICAgICAgc2J1Zl9wcmludGYoc2IsICIl dSIsIGRwLT5kX3JvdGF0aW9uX3JhdGUpOw0KICAgIGVsc2UNCiAgICAgICAgc2J1Zl9wcmludGYo c2IsICJpbnZhbGlkIik7DQogICAgc2J1Zl9wcmludGYoc2IsICI8L3JvdGF0aW9ucmF0ZT5cbiIp Ow0KDQoNClRoYXQgd291bGQgYmUgbW9yZSBhY2N1cmF0ZSwgYnV0IHNsaWdodGx5IGhhcmRlciB0 byBwYXJzZSAoZGVwZW5kaW5nIG9uIHdoYXQncyBwYXJzaW5nIHRoZSBYTUwpLiBJIGRvbid0IGhh dmUgYSBzdHJvbmcgZmVlbGluZyBhYm91dCB0aGlzOyB3aGF0IGRvIG90aGVyIHBlb3BsZSB0aGlu az8gU2hvdWxkIGl0IGp1c3QgcmV0dXJuIHRoZSB2YWx1ZSBwcm92aWRlZCBieSB0aGUgZHJpdmUs IG9yIHNob3VsZCBpdCBkbyBzb21lIGludGVycHJldGF0aW9uPw0KDQo+TmV4dCBpbiBwcmlvcml0 eSB3b3VsZCBiZSBnZXR0aW5nIGZpcm13YXJlIHZlcnNpb24uLg0KPg0KPkluICdjYW1jb250cm9s IGlkZW50aWZ5JyBpdCBsb29rcyBsaWtlOiAgJ2Zpcm13YXJlIHJldmlzaW9uICAgICBEWE05MjAz UScNCj4NCj5JbiAnY2FtY29udHJvbCBpbnF1aXJ5JyBpdCdzIHBhcnQgb2YgdGhlIGRldmljZSBz dHJpbmcgbGlrZTogJ3Bhc3MwOiA8U0FNU1VORyBNWjdXRDQ4MEhDR00tMDAwMDMgRFhNOTIwM1E+ IEFDUy0yIEFUQSBTQVRBIDMueCBkZXZpY2UnDQo+DQo+RG8geW91IGhhdmUgYW55IHN1Z2dlc3Rp b25zIGZvciB3aXJpbmcgdGhhdCBpbnRvIGdlb20gZGlzaz8NCg0KSW50ZXJlc3RpbmcuIEkgbmV2 ZXIgbm90aWNlZCB0aGF0IGZpcm13YXJlIHdhc24ndCBhbHJlYWR5IGluY2x1ZGVkIGluICJzdHJ1 Y3QgZGlzayIsIGxpa2UgZHJpdmUgbW9kZWwgYW5kIHNlcmlhbCBudW1iZXIgYWxyZWFkeSBhcmUu IEl0IHNob3VsZCBiZSB0cml2aWFsIHRvIGFkZCwgYnV0IGtlZXBpbmcgYSBjb3B5IG9mIHRoZSBz dHJpbmcgd2lsbCBvZiBjb3Vyc2UgbWFrZSAic3RydWN0IGRpc2siIGxhcmdlci4gVGhhdCBtaWdo dCBoYXZlIGltcGxpY2F0aW9ucyBvbiBLQkkgY29tcGF0aWJpbGl0eSBmb3Igb3V0LW9mLXRyZWUg ZHJpdmVycy4uLj8gQWdhaW4sIEknbSBub3Qgc3VyZS4NCg0KDQo+QW5kIGxhc3RseSwgeWVzIGJ1 cyBzcGVlZHMgd291bGQgYmUgbmljZSAoZXNwZWNpYWxseSB0byBzcG90IGhhcmQvc29mdCBtaXNj b25maWd1cmF0aW9uKS4gIEl0IHNob3dzIHVwIGxpa2UgJ3Byb3RvY29sICAgICAgICAgICAgICBB VEEvQVRBUEktOSBTQVRBIDMueCcgaW4gY2FtY29udHJvbCBpZGVudGlmeSBhbmQgY2FuIGJlIHNl ZW4gaW4gdGhlIGlucXVpcnkgc3RyaW5nIGFib3ZlIHRvby4gIERvIHlvdSBoYXZlIGRlc2lnbiBz dWdnZXN0aW9ucyBvbiB0aGF0Pw0KDQpJJ20gc3VyZSB0aGUgc3RyaW5nIHRoYXQncyBnZW5lcmF0 ZWQgYnkgQ0FNIGF0IGF0dGFjaC10aW1lIGNvdWxkIGJlIHByZXNlcnZlZC4gQWdhaW4sIHRoZXJl J3MgdGhlIG1lbW9yeSBhbmQgS0JJIGlzc3VlcyBvZiBhZGRpbmcgYW5vdGhlciBzdHJpbmcgdG8g InN0cnVjdCBkaXNrIi4NCg0KLVJhdmkNCg0K From owner-freebsd-scsi@freebsd.org Thu Dec 10 16:16:01 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 442979D6E38 for ; Thu, 10 Dec 2015 16:16:01 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 174E61C57 for ; Thu, 10 Dec 2015 16:16:01 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by obciw8 with SMTP id iw8so62493051obc.1 for ; Thu, 10 Dec 2015 08:15:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=BYxfcj/XkpwCa9WP7HHVId7295cAvJXJCkH6hUG5MzQ=; b=pjRsAMaiTL+7BkJODeS1kSU7MHtu8OteXXYRNfxL5cI63rapuNI2L5ZpDzDpjCS+8P FVsVBJvPNRs2INQHS/FUk/gZnyUcYLjxs5816QZPIoDO6A1RsfRyqGk2zvk4YyKOBxNa s+Skz0D9AKEo9U3HfSzmpf66hZ2N1MLhUtD6jpohbvK15rEXM45qtLIGzbU2n8ojiEXB t/NPNyZX2EXNA0xnCIRKG23mL+2NsCDhP5yrNJTXatcseb2+XYbIdYWRbn2VP6oN+sxQ DKtNJnSHsJhBVV9cvk5j6MI6w6gl00+GCUVbgsb3piVsiQiOPZIf8c+3fYWVBUHxUIOh yqcA== MIME-Version: 1.0 X-Received: by 10.182.106.13 with SMTP id gq13mr10359026obb.38.1449764159335; Thu, 10 Dec 2015 08:15:59 -0800 (PST) Sender: asomers@gmail.com Received: by 10.202.0.7 with HTTP; Thu, 10 Dec 2015 08:15:59 -0800 (PST) In-Reply-To: References: <80BB5907-CC31-4F06-9C70-E6F7834FF28E@panasas.com> Date: Thu, 10 Dec 2015 09:15:59 -0700 X-Google-Sender-Auth: 0pOjLrSYOwLHLli-FMBo7rUoUYo Message-ID: Subject: Re: Accessing static drive info w/o ATA identify and lockup with camcontrol identify From: Alan Somers To: "Pokala, Ravi" Cc: Kevin Bowling , Ravi Pokala , "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 16:16:01 -0000 On Thu, Dec 10, 2015 at 9:10 AM, Pokala, Ravi wrote: > -----Original Message----- > > > From: Kevin Bowling > Date: 2015-12-10, Thursday at 04:58 > To: Ravi Pokala > Cc: "freebsd-scsi@freebsd.org" , Sean Bruno > Subject: Re: Accessing static drive info w/o ATA identify and lockup with= camcontrol identify > >>Thanks Ravi! >> >>Rotation rate is probably the most important thing and thankfully easy as= you suggested: >> >>https://reviews.freebsd.org/D4483. Would be nice to land that in 10.3 so= I can swap the SaltStack module over to it. > > As I just commented there, I was actually going to look at this this week= end. My concern is that the d_rotation_rate does not strictly map to the ac= tual rotation rate - there are some special cases and reserved values. I wa= s thinking something more like this: > > sbuf_printf(sb, "%s", indent); > if (dp->d_rotatation_rate =3D=3D 0) > sbuf_printf(sb, "unknown"); > else if (dp->d_rotation_rate =3D=3D 1) > sbuf_printf(sb, "0"); > else if ((dp->d_rotation_rate >=3D 0x041) && (dp->d_rotation_rate <= =3D 0xfffe)) > sbuf_printf(sb, "%u", dp->d_rotation_rate); > else > sbuf_printf(sb, "invalid"); > sbuf_printf(sb, "\n"); > > > That would be more accurate, but slightly harder to parse (depending on w= hat's parsing the XML). I don't have a strong feeling about this; what do o= ther people think? Should it just return the value provided by the drive, o= r should it do some interpretation? > >>Next in priority would be getting firmware version.. >> >>In 'camcontrol identify' it looks like: 'firmware revision DXM9203Q' >> >>In 'camcontrol inquiry' it's part of the device string like: 'pass0: ACS-2 ATA SATA 3.x device' >> >>Do you have any suggestions for wiring that into geom disk? > > Interesting. I never noticed that firmware wasn't already included in "st= ruct disk", like drive model and serial number already are. It should be tr= ivial to add, but keeping a copy of the string will of course make "struct = disk" larger. That might have implications on KBI compatibility for out-of-= tree drivers...? Again, I'm not sure. This information is already perserved in CAM, if not in GEOM. For SCSI disks, look at (struct cam_device).inq_data.revision. For ATA disks, look at (struct ccb_getdev).ident_data.revision. > > >>And lastly, yes bus speeds would be nice (especially to spot hard/soft mi= sconfiguration). It shows up like 'protocol ATA/ATAPI-9 SATA = 3.x' in camcontrol identify and can be seen in the inquiry string above too= . Do you have design suggestions on that? > > I'm sure the string that's generated by CAM at attach-time could be prese= rved. Again, there's the memory and KBI issues of adding another string to = "struct disk". > > -Ravi > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Thu Dec 10 16:25:56 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FA809D76C3 for ; Thu, 10 Dec 2015 16:25:56 +0000 (UTC) (envelope-from rpokala@mac.com) Received: from mr11p00im-asmtp002.me.com (mr11p00im-asmtp002.me.com [17.110.69.253]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B5221435; Thu, 10 Dec 2015 16:25:56 +0000 (UTC) (envelope-from rpokala@mac.com) Received: from [192.168.1.4] (c-24-6-178-251.hsd1.ca.comcast.net [24.6.178.251]) by mr11p00im-asmtp002.me.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Mar 31 2015)) with ESMTPSA id <0NZ500AHSHN6OW20@mr11p00im-asmtp002.me.com>; Thu, 10 Dec 2015 16:25:55 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2015-12-10_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1011 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1510270003 definitions=main-1512100277 User-Agent: Microsoft-MacOutlook/0.0.0.151105 Date: Thu, 10 Dec 2015 08:25:53 -0800 Subject: Re: Accessing static drive info w/o ATA identify and lockup with camcontrol identify From: Ravi Pokala Sender: "Pokala, Ravi" To: Alan Somers Cc: Kevin Bowling , Ravi Pokala , "freebsd-scsi@freebsd.org" Message-id: Thread-topic: Accessing static drive info w/o ATA identify and lockup with camcontrol identify References: <80BB5907-CC31-4F06-9C70-E6F7834FF28E@panasas.com> In-reply-to: MIME-version: 1.0 Content-type: text/plain; charset=UTF-8 Content-transfer-encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 16:25:56 -0000 -----Original Message----- From: on behalf of Alan Somers Date: 2015-12-10, Thursday at 08:15 To: Ravi Pokala Cc: Kevin Bowling , Ravi Pokala , "freebsd-scsi@freebsd.org" Subject: Re: Accessing static drive info w/o ATA identify and lockup with camcontrol identify >On Thu, Dec 10, 2015 at 9:10 AM, Pokala, Ravi wrote: >>Interesting. I never noticed that firmware wasn't already included in "struct disk", like drive model and serial number already are. It should be trivial to add, but keeping a copy of the string will of course make "struct disk" larger. That might have implications on KBI compatibility for out-of-tree drivers...? Again, I'm not sure. > > >This information is already perserved in CAM, if not in GEOM. For >SCSI disks, look at (struct cam_device).inq_data.revision. For ATA >disks, look at (struct ccb_getdev).ident_data.revision. Right, but it sounded like Kevin was looking for a way to get this from userland. That means through something like `geom disk list' (or the raw XML from which that is parsed). -Ravi From owner-freebsd-scsi@freebsd.org Thu Dec 10 17:13:57 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03A3B9D68F3 for ; Thu, 10 Dec 2015 17:13:57 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C20DF10BD for ; Thu, 10 Dec 2015 17:13:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by obcse5 with SMTP id se5so64142916obc.3 for ; Thu, 10 Dec 2015 09:13:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=DTvFIf3kx2NhdrboJDA54Ag95eHYvwNQhHXXlDKZt48=; b=dlf92jjY7F8COOz9QEaM4YCT7H2+XVTnKTeyGN/nuMXkWM6lRtSPN67W3DT9z8ceZ+ WTj01agjPO0Xc1JgOFbqJUUH/1CaestsMa5iLnfHV1yhi8AbIPoRsPqicPaHwdhxX4X2 q/Mn0Y9dabT+n/f4bTFhftkxeL3awacoMDYP9Vlq8XinWoaSe5lJqkM5RLdroL1AlZBd L58fUx6eXSBSH9oRR+u0dXs/1Ks0uxikAXMAeGRfVfhOWb3haJ/IpMuiLR19Fkl33D7R LII18U4hbas2f1sQRpsdNS8NHPmZUMwFjRqLzFGhJ5xRLGtNaOsKgTDrbLlfoibPHEiV K0GQ== MIME-Version: 1.0 X-Received: by 10.60.92.138 with SMTP id cm10mr10211025oeb.64.1449767636175; Thu, 10 Dec 2015 09:13:56 -0800 (PST) Sender: asomers@gmail.com Received: by 10.202.0.7 with HTTP; Thu, 10 Dec 2015 09:13:56 -0800 (PST) In-Reply-To: References: <80BB5907-CC31-4F06-9C70-E6F7834FF28E@panasas.com> Date: Thu, 10 Dec 2015 10:13:56 -0700 X-Google-Sender-Auth: xXpqayMDlhy7eY2BeLoNKWaNTlM Message-ID: Subject: Re: Accessing static drive info w/o ATA identify and lockup with camcontrol identify From: Alan Somers To: Ravi Pokala Cc: Kevin Bowling , "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 17:13:57 -0000 On Thu, Dec 10, 2015 at 9:25 AM, Ravi Pokala wrote: > -----Original Message----- > > > From: on behalf of Alan Somers > Date: 2015-12-10, Thursday at 08:15 > To: Ravi Pokala > Cc: Kevin Bowling , Ravi Pokala , "freebsd-scsi@freebsd.org" > Subject: Re: Accessing static drive info w/o ATA identify and lockup with= camcontrol identify > >>On Thu, Dec 10, 2015 at 9:10 AM, Pokala, Ravi wrote= : >>>Interesting. I never noticed that firmware wasn't already included in "s= truct disk", like drive model and serial number already are. It should be t= rivial to add, but keeping a copy of the string will of course make "struct= disk" larger. That might have implications on KBI compatibility for out-of= -tree drivers...? Again, I'm not sure. >> >> >>This information is already perserved in CAM, if not in GEOM. For >>SCSI disks, look at (struct cam_device).inq_data.revision. For ATA >>disks, look at (struct ccb_getdev).ident_data.revision. > > Right, but it sounded like Kevin was looking for a way to get this from u= serland. That means through something like `geom disk list' (or the raw XML= from which that is parsed). > > -Ravi > Actually, it is possible to get that info in userland, though a little awkward. You can do 'cam_open_device("da13")' to get a struct cam_device. Getting the ccb_getdev is harder though. Look at get_cgd() from camcontrol.c. Even though it looks like it's sending a CCB, the CCB actually gets processed by CAM and doesn't result in any I/O hitting the disk. -Alan From owner-freebsd-scsi@freebsd.org Thu Dec 10 18:38:27 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 779CB9D6261 for ; Thu, 10 Dec 2015 18:38:27 +0000 (UTC) (envelope-from stephen.mcconnell@avagotech.com) Received: from mail-pf0-x231.google.com (mail-pf0-x231.google.com [IPv6:2607:f8b0:400e:c00::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4625F1119 for ; Thu, 10 Dec 2015 18:38:27 +0000 (UTC) (envelope-from stephen.mcconnell@avagotech.com) Received: by pfu207 with SMTP id 207so52045731pfu.2 for ; Thu, 10 Dec 2015 10:38:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=avagotech.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:cc:content-type; bh=2VHDjDhi0gkX4FvxqDOgcoNYuTkHw0f0hrrWn9DbUTE=; b=Wj2/auMMfB/hGSQHyfddWQTnekgsEG2nA6SsoiiatenwUDSIgi2P0FcPxlmZmHGDkt 0x88eWR4/iVN7Ex0ZHsLdAWSZpQ6ZSEfX1xQozprarIyz2eUtk2EajAmhlPzZfPBprPl bOCuS3G50yiAUrvp0zSSJHhR6Y3lScB/BJT70= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc:content-type; bh=2VHDjDhi0gkX4FvxqDOgcoNYuTkHw0f0hrrWn9DbUTE=; b=RzUjd+rzegyptKhwOwmMbj3+6r4iQtnteiZdawCsJ4ayZKDKfax9Gh2n3xjVv5lxmc aTAXSy6msS0tlHarDETkkxZxaRXnXu6xImFSX7YSXua3tZpQUBsSeDB7OjNj/r2dUCq+ WG2h2ubXp+h7SY9+hBXWmQHWR+oBYpWPhcABgHCLZTJSFNaw+Vk1pS9HYaFHCuBnWQut YtXG74OlnwwMorSR48GLpuERGl6zb0/7dlTUmgyYlq+AO/9BleFfh959fvPCIqLeTwLA iy9Jc9mbaIPSot7zrbwyVLAILNFimuYaU2HbZrRbqSjFI9i0tjuCLz+gVylmLTkkbeiO H1JA== X-Gm-Message-State: ALoCoQkzwkmZNvhHxq/V0YG7q5F8TLCvO/jvetSAG/ovEHGj3lSPDTyCc/fxwLDnwQPl5jgZtJBJk97djPDe+9qWMxXX1f4HzlCp0b/CuFp/EpUU6nKyI9k= X-Received: by 10.98.14.26 with SMTP id w26mr9057922pfi.110.1449772706550; Thu, 10 Dec 2015 10:38:26 -0800 (PST) From: Stephen Mcconnell References: <6A7832F8-53EB-4641-8EF6-E0E6175EB52D@yahoo.com> <48445286cfa5082c78581b2c1e8afb66@mail.gmail.com> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQJO1PHZ3S5C87fllREIl8yX+V/IRAJvH4HyAm7AR3ACs4vlkAGgDtQvAt9zZZUBXS4HTJ1d12MA Date: Thu, 10 Dec 2015 11:38:25 -0700 Message-ID: Subject: RE: bad disk discovery To: prateek sethi Cc: Michael Jung , freebsd-scsi@freebsd.org, owner-freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 18:38:27 -0000 All I can see from that log is thousands of the same error for and Inquiry command. It looks like a bad drive to me. Did you say that you tried moving the drive and the problem still happens for that drive in a different slot? Steve *From:* prateek sethi [mailto:prateekrootkey@gmail.com] *Sent:* Tuesday, December 08, 2015 11:01 PM *To:* Stephen Mcconnell *Cc:* Michael Jung; freebsd-scsi@freebsd.org; owner-freebsd-scsi@freebsd.or= g *Subject:* Re: bad disk discovery I have seen logs and mainly it is saying that *Logical unit not ready, cause not reportable*. I am attaching logs related to da22 and da23.( Previously disk was da22 after reboot it has become da23.) On Tue, Dec 8, 2015 at 9:50 PM, Stephen Mcconnell < stephen.mcconnell@avagotech.com> wrote: Try looking through the system log to see if there is any debug output from the mps driver, or you can send it to me and I'll take a look. It might give us a clue as to what's going on. Steve McConnell > -----Original Message----- > From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd- > scsi@freebsd.org] On Behalf Of prateek sethi > Sent: Tuesday, December 08, 2015 9:05 AM > To: Michael Jung > Cc: freebsd-scsi@freebsd.org; owner-freebsd-scsi@freebsd.org > Subject: Re: bad disk discovery > > Yes, I have tried that one but issue is still there. Other disk are > working fine > with the same configuration that means firmware should not be a problem. > > > On Tue, Dec 8, 2015 at 6:56 PM, Michael Jung wrote: > > > On 2015-12-08 08:00, prateek sethi wrote: > > > >> Hi Scott, > >> Thanks for the your quick response. > >> > >> I have different set of hardware . So that's why I want to know how I > >> can debug it myself . Is there anyway or procedure using that I can > >> findout about the situation or the reason for CDB errors or disk > >> command > failure? > >> > >> Right now I am giving detail about the setup where I am getting this > >> issue . > >> > >> I am using LSI SAS2008 controller and connected with supermicro > >> Enclosure with freebsd 9.3. 16 different disks are there but only one > >> disk is having problem. That means contoller and cable are fine. > >> > >> Faulty disk info are like:-. > >> > >> *smartctl output is:-* > >> > >> smartctl -x /dev/da23 > >> > >> =3D=3D=3D START OF INFORMATION SECTION =3D=3D=3D > >> Vendor: SEAGATE > >> Product: ST3600057SS > >> Revision: 000B > >> Rotation Rate: 15000 rpm > >> Form Factor: 3.5 inches > >> Logical Unit id: 0x5000c5007725173f > >> Serial number: 6SL8YLPC0000N5030DY7 > >> Device type: disk > >> Transport protocol: SAS > >> Local Time is: Tue Dec 8 18:20:45 2015 IST > >> *device is NOT READY (e.g. spun down, busy)* > >> > >> *Logs:-* > >> > >> Dec 8 14:12:01 N1 kernel: da23 at mps0 bus 0 scbus0 target 148 lun 0 > >> Dec 8 14:12:01 N1 kernel: da23: Fixed > >> Direct Access SCSI-5 device Dec 8 14:12:01 N1 kernel: da23: Serial > >> Number 6SL8YLPC0000N5030DY7 Dec 8 14:12:01 N1 kernel: da23: > >> 600.000MB/s transfers Dec 8 14:12:01 N1 kernel: da23: Command > >> Queueing enabled Dec 8 14:12:01 N1 kernel: da23: *Attempt to query > >> device size failed: NOT READY, Logical unit not ready, cause n* Dec > >> 8 14:12:01 N1 kernel: ses1: da23,pass26: Element descriptor: 'Slot > >> 24' > >> Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: SAS Device Slot > >> Element: 1 Phys at Slot 23 > >> > >> *driver versions:-* > >> > >> > >> dev.mps.0.firmware_version: 15.00.00.00 > >> dev.mps.0.driver_version: 16.00.00.00-fbsd > >> > >> > >> > >> > >> > >> > >> On Tue, Dec 8, 2015 at 3:15 AM, Scott Long > wrote: > >> > >> Hi, > >>> > >>> If your situation is accurate and the disk is not responding > >>> properly to regular commands then it=E2=80=99s unlikely that it will = respond > >>> to SMART commands either. > >>> Sometimes these situations are caused by a bad cable, bad > >>> controller, or buggy software/firmware, and only rarely will the > >>> standard statistics in SMART pick up these kinds of errors. SMART > >>> is better at tracking wear rates and error rates on the physical > >>> media, both HDD and SSD, but even then it=E2=80=99s hard for it to be > >>> accurately predictive or even accurately diagnostic. For your case, > >>> I recommend that you describe your hardware and software > >>> configuration in more detail, and look for physical abnormalities in > >>> the cabling and connections. > >>> Once that is ruled and and the rest of us know what kind of hardware > >>> you=E2=80=99re dealing with, we might be able to make better commenda= tions. > >>> > >>> Scott > >>> > >>> > On Dec 7, 2015, at 11:07 AM, prateek sethi > >>> > > >>> wrote: > >>> > > >>> > Hi , > >>> > > >>> > Is there any way or tool to find out that a disk which is not > >>> responding > >>> > properly is really bad or not? Sometimes I have seen that there is > >>> > lot > >>> of > >>> > CDB error for a drive and system reboot makes every thing fine. > >>> > What > >>> can > >>> be > >>> > reasons for such kind of scenarios? > >>> > > >>> > I know smartctl is the one which can help. I have some couple of > >>> question > >>> > regarding this . > >>> > > >>> > 1. What if disk does not support smartctl? > >>> > 2. How I can do smartest use of smartctl command like which > >>> > parameters > >>> can > >>> > tell that the disk is actually bad? > >>> > 3. What other test I can perform to make it sure that disk has > >>> completely > >>> > gone? > >>> > > >>> > > >>> > Please tell me correct place to ask this question if I am asking > >>> > at > >>> wrong > >>> > place. > >>> > _______________________________________________ > >>> > freebsd-scsi@freebsd.org mailing list > >>> > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > >>> > To unsubscribe, send any mail to > >>> > "freebsd-scsi-unsubscribe@freebsd.org > >>> " > >>> > >>> > >>> _______________________________________________ > >> freebsd-scsi@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > >> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org= " > >> > > > > > > Have you simply moved the drive to another slot - does the problem > > follow the drive? > > Unlikely but it could be a backplane issue. > > > > I don't know about version 15 firmware, I have always used version 16 > > firmware with 9.x to match the driver version. > > > > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Thu Dec 10 21:44:04 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 946809D7CD9 for ; Thu, 10 Dec 2015 21:44:04 +0000 (UTC) (envelope-from rpokala@panasas.com) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0079.outbound.protection.outlook.com [207.46.100.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F15810D7; Thu, 10 Dec 2015 21:44:03 +0000 (UTC) (envelope-from rpokala@panasas.com) Received: from CY1PR08MB1803.namprd08.prod.outlook.com (10.162.218.25) by CY1PR08MB1803.namprd08.prod.outlook.com (10.162.218.25) with Microsoft SMTP Server (TLS) id 15.1.355.16; Thu, 10 Dec 2015 21:43:55 +0000 Received: from CY1PR08MB1803.namprd08.prod.outlook.com ([10.162.218.25]) by CY1PR08MB1803.namprd08.prod.outlook.com ([10.162.218.25]) with mapi id 15.01.0355.012; Thu, 10 Dec 2015 21:43:55 +0000 From: "Pokala, Ravi" To: Alan Somers , Ravi Pokala CC: Kevin Bowling , "freebsd-scsi@freebsd.org" Subject: Re: Accessing static drive info w/o ATA identify and lockup with camcontrol identify Thread-Topic: Accessing static drive info w/o ATA identify and lockup with camcontrol identify Thread-Index: AQHRM2VBeCKuqTynrkez8plTZtfm2A== Date: Thu, 10 Dec 2015 21:43:54 +0000 Message-ID: <7F867A13-89E5-40C1-B13C-AA989F7B2A45@panasas.com> References: <80BB5907-CC31-4F06-9C70-E6F7834FF28E@panasas.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/0.0.0.151105 authentication-results: spf=none (sender IP is ) smtp.mailfrom=rpokala@panasas.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [64.80.217.3] x-microsoft-exchange-diagnostics: 1; CY1PR08MB1803; 5:YjhATCj/TIz/LWQ18VL28vBvaiXJ3q29lw6pvl0jZHsXatgMJ3T+fm2eBc2Dg8lBNjCj/c6cJyqJX8hAl2U+8/tR+kwDiZs97XP48xCpMao1qLyRxLDNUQ5jy/AvAheu5XLDZbUcT8g+SEYonHNw0A==; 24:dWQ3NPDFkkxKbl6p0dW+GwnyDezU2MFcPNVexCLizecudGOcahfP4kmO9UdTfCurlwjZLx7BWksBv/eTZAoqMJCpbgPWdqhfWCjgcuKA9RQ= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR08MB1803; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(520078)(8121501046)(5005006)(10201501046)(3002001); SRVR:CY1PR08MB1803; BCL:0; PCL:0; RULEID:; SRVR:CY1PR08MB1803; x-forefront-prvs: 078693968A x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(13464003)(377424004)(199003)(189002)(4001150100001)(101416001)(6116002)(5008740100001)(93886004)(83716003)(40100003)(99286002)(2900100001)(106356001)(106116001)(33656002)(105586002)(54356999)(76176999)(2950100001)(50986999)(19580395003)(5004730100002)(5002640100001)(122556002)(1096002)(92566002)(86362001)(586003)(11100500001)(102836003)(66066001)(10400500002)(87936001)(5001770100001)(81156007)(3846002)(5001960100002)(97736004)(19580405001)(36756003)(82746002)(189998001)(83506001)(77096005)(1220700001)(4001350100001)(7059030)(104396002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR08MB1803; H:CY1PR08MB1803.namprd08.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: panasas.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Dec 2015 21:43:54.6703 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR08MB1803 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 21:44:04 -0000 LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCg0KDQpGcm9tOiA8YXNvbWVyc0BnbWFpbC5jb20+ IG9uIGJlaGFsZiBvZiBBbGFuIFNvbWVycyA8YXNvbWVyc0BmcmVlYnNkLm9yZz4NCkRhdGU6IDIw MTUtMTItMTAsIFRodXJzZGF5IGF0IDA5OjEzDQpUbzogUmF2aSBQb2thbGEgPHJwb2thbGFAbWFj LmNvbT4NCkNjOiBLZXZpbiBCb3dsaW5nIDxrZXZpbi5ib3dsaW5nQGtldjAwOS5jb20+LCAiZnJl ZWJzZC1zY3NpQGZyZWVic2Qub3JnIiA8ZnJlZWJzZC1zY3NpQGZyZWVic2Qub3JnPg0KU3ViamVj dDogUmU6IEFjY2Vzc2luZyBzdGF0aWMgZHJpdmUgaW5mbyB3L28gQVRBIGlkZW50aWZ5IGFuZCBs b2NrdXAgd2l0aCBjYW1jb250cm9sIGlkZW50aWZ5DQoNCj5BY3R1YWxseSwgaXQgaXMgcG9zc2li bGUgdG8gZ2V0IHRoYXQgaW5mbyBpbiB1c2VybGFuZCwgdGhvdWdoIGEgbGl0dGxlDQo+QXdrd2Fy ZC4NCg0KT2ggc3VyZS4gRm9yIHZhcmlvdXMgaGlzdG9yaWNhbCByZWFzb25zLCBJIGdldCBpZGVu dCBpbmZvcm1hdGlvbiBieSBwYXNzaW5nIGFuIElERU5USUZZX0RFVklDRSBjb21tYW5kIGZyb20g dXNlcmxhbmQgdmlhIHBhc3N0aHJvdWdoIGlvY3RsLCB0aGVuIGV4dHJhY3Qgd2hhdCBJIG5lZWQg ZnJvbSB0aGVyZS4gSXQncyBhIGhlYXZ5IGhhbW1lciwgYnV0IGl0IHdvcmtzLg0KDQoNCj5Zb3Ug Y2FuIGRvICdjYW1fb3Blbl9kZXZpY2UoImRhMTMiKScgdG8gZ2V0IGEgc3RydWN0DQo+Y2FtX2Rl dmljZS4gIEdldHRpbmcgdGhlIGNjYl9nZXRkZXYgaXMgaGFyZGVyIHRob3VnaC4gIExvb2sgYXQN Cj5nZXRfY2dkKCkgZnJvbSBjYW1jb250cm9sLmMuICBFdmVuIHRob3VnaCBpdCBsb29rcyBsaWtl IGl0J3Mgc2VuZGluZyBhDQo+Q0NCLCB0aGUgQ0NCIGFjdHVhbGx5IGdldHMgcHJvY2Vzc2VkIGJ5 IENBTSBhbmQgZG9lc24ndCByZXN1bHQgaW4gYW55DQo+SS9PIGhpdHRpbmcgdGhlIGRpc2suDQoN ClllYWgsIEkgc2F3IHRoYXQgd2hlbiBJIHdhcyBsb29raW5nIGF0IHRoaXMgdGhpcyBtb3JuaW5n LiBJdCBsb29rcyBsaWtlIHRoZXJlIGFyZSBxdWl0ZSBhIGZldyBDQU0gaW9jdGxzIHdoaWNoIGp1 c3QgcmV0dXJuIGluZm9ybWF0aW9uIHRoYXQncyBhbHJlYWR5IGluIHRoZSBrZXJuZWwsIHdpdGhv dXQgc2VuZGluZyBhbnkgY29tbWFuZHMgdG8gYW55IGRldmljZXMuDQoNCi1SYXZpDQoNCj4tQWxh bg0K From owner-freebsd-scsi@freebsd.org Fri Dec 11 05:28:31 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1C839D69F6; Fri, 11 Dec 2015 05:28:31 +0000 (UTC) (envelope-from prateekrootkey@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46999178E; Fri, 11 Dec 2015 05:28:31 +0000 (UTC) (envelope-from prateekrootkey@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id n186so13626605wmn.1; Thu, 10 Dec 2015 21:28:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MCzXPgs9+dXU6FpR3M9Jcm+tyZTxbgb6SJkXJGq0TDI=; b=GTL8h8tdn3ler6LE60Xkb754WlO6VvwhQeAwpRkvYpofwRFYxx/BSGETEWCEBr9LgR dZy0maYVnydfyIf6QcHS9Ncg/YYdU0jKZ2N+aBFJeIEIQ+T/DmRd5cn09q4K6510V8oG cOE/dXdEcKSmG6FF5jr9bGNe3+wfGSyEimtWAfHjfGuYCDj371ikpF+hgJsYxYha+KWg 7MU7KZXgU5OsdCF7aHFlCnkrdEqjshxQo6eeFi8L2+mH5tlGW4nMBRhrraDRPxfqdm04 YkpsJXXAB3e8xD1YFkhwYkTEPHmnjDjeej7ZAK52QJKcOsJb4qMp1vuxIBaVpBB1E8Hg 1/hw== MIME-Version: 1.0 X-Received: by 10.28.4.212 with SMTP id 203mr3353297wme.89.1449811709561; Thu, 10 Dec 2015 21:28:29 -0800 (PST) Received: by 10.27.16.7 with HTTP; Thu, 10 Dec 2015 21:28:29 -0800 (PST) In-Reply-To: References: <6A7832F8-53EB-4641-8EF6-E0E6175EB52D@yahoo.com> <48445286cfa5082c78581b2c1e8afb66@mail.gmail.com> Date: Fri, 11 Dec 2015 10:58:29 +0530 Message-ID: Subject: Re: bad disk discovery From: prateek sethi To: Stephen Mcconnell Cc: Michael Jung , freebsd-scsi@freebsd.org, owner-freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 05:28:31 -0000 Yes, I after moving the disk also problem is there. But same error for Inquiry command can't be a reason for a bad disk. I tried couple of other commands also and for all of them it says *Logical unit not ready . * So error *Logical unit not ready *can be a bad disk symbol? or if not then what does this error actually tell? On Fri, Dec 11, 2015 at 12:08 AM, Stephen Mcconnell < stephen.mcconnell@avagotech.com> wrote: > All I can see from that log is thousands of the same error for and Inquir= y > command. It looks like a bad drive to me. Did you say that you tried > moving the drive and the problem still happens for that drive in a > different slot? > > > > Steve > > > > *From:* prateek sethi [mailto:prateekrootkey@gmail.com] > *Sent:* Tuesday, December 08, 2015 11:01 PM > *To:* Stephen Mcconnell > *Cc:* Michael Jung; freebsd-scsi@freebsd.org; > owner-freebsd-scsi@freebsd.org > > *Subject:* Re: bad disk discovery > > > > I have seen logs and mainly it is saying that *Logical unit not ready, > cause not reportable*. > > I am attaching logs related to da22 and da23.( Previously disk was da22 > after reboot it has become da23.) > > > > On Tue, Dec 8, 2015 at 9:50 PM, Stephen Mcconnell < > stephen.mcconnell@avagotech.com> wrote: > > Try looking through the system log to see if there is any debug output fr= om > the mps driver, or you can send it to me and I'll take a look. It might > give us a clue as to what's going on. > > Steve McConnell > > > > -----Original Message----- > > From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd- > > scsi@freebsd.org] On Behalf Of prateek sethi > > Sent: Tuesday, December 08, 2015 9:05 AM > > To: Michael Jung > > Cc: freebsd-scsi@freebsd.org; owner-freebsd-scsi@freebsd.org > > Subject: Re: bad disk discovery > > > > Yes, I have tried that one but issue is still there. Other disk are > > working fine > > with the same configuration that means firmware should not be a problem= . > > > > > > On Tue, Dec 8, 2015 at 6:56 PM, Michael Jung wrote: > > > > > On 2015-12-08 08:00, prateek sethi wrote: > > > > > >> Hi Scott, > > >> Thanks for the your quick response. > > >> > > >> I have different set of hardware . So that's why I want to know how = I > > >> can debug it myself . Is there anyway or procedure using that I can > > >> findout about the situation or the reason for CDB errors or disk > > >> command > > failure? > > >> > > >> Right now I am giving detail about the setup where I am getting this > > >> issue . > > >> > > >> I am using LSI SAS2008 controller and connected with supermicro > > >> Enclosure with freebsd 9.3. 16 different disks are there but only on= e > > >> disk is having problem. That means contoller and cable are fine. > > >> > > >> Faulty disk info are like:-. > > >> > > >> *smartctl output is:-* > > >> > > >> smartctl -x /dev/da23 > > >> > > >> =3D=3D=3D START OF INFORMATION SECTION =3D=3D=3D > > >> Vendor: SEAGATE > > >> Product: ST3600057SS > > >> Revision: 000B > > >> Rotation Rate: 15000 rpm > > >> Form Factor: 3.5 inches > > >> Logical Unit id: 0x5000c5007725173f > > >> Serial number: 6SL8YLPC0000N5030DY7 > > >> Device type: disk > > >> Transport protocol: SAS > > >> Local Time is: Tue Dec 8 18:20:45 2015 IST > > >> *device is NOT READY (e.g. spun down, busy)* > > >> > > >> *Logs:-* > > >> > > >> Dec 8 14:12:01 N1 kernel: da23 at mps0 bus 0 scbus0 target 148 lun = 0 > > >> Dec 8 14:12:01 N1 kernel: da23: Fixed > > >> Direct Access SCSI-5 device Dec 8 14:12:01 N1 kernel: da23: Serial > > >> Number 6SL8YLPC0000N5030DY7 Dec 8 14:12:01 N1 kernel: da23: > > >> 600.000MB/s transfers Dec 8 14:12:01 N1 kernel: da23: Command > > >> Queueing enabled Dec 8 14:12:01 N1 kernel: da23: *Attempt to query > > >> device size failed: NOT READY, Logical unit not ready, cause n* Dec > > >> 8 14:12:01 N1 kernel: ses1: da23,pass26: Element descriptor: 'Slot > > >> 24' > > >> Dec 8 14:12:01 N1 kernel: ses1: da23,pass26: SAS Device Slot > > >> Element: 1 Phys at Slot 23 > > >> > > >> *driver versions:-* > > >> > > >> > > >> dev.mps.0.firmware_version: 15.00.00.00 > > >> dev.mps.0.driver_version: 16.00.00.00-fbsd > > >> > > >> > > >> > > >> > > >> > > >> > > >> On Tue, Dec 8, 2015 at 3:15 AM, Scott Long > > wrote: > > >> > > >> Hi, > > >>> > > >>> If your situation is accurate and the disk is not responding > > >>> properly to regular commands then it=E2=80=99s unlikely that it wil= l respond > > >>> to SMART commands either. > > >>> Sometimes these situations are caused by a bad cable, bad > > >>> controller, or buggy software/firmware, and only rarely will the > > >>> standard statistics in SMART pick up these kinds of errors. SMART > > >>> is better at tracking wear rates and error rates on the physical > > >>> media, both HDD and SSD, but even then it=E2=80=99s hard for it to = be > > >>> accurately predictive or even accurately diagnostic. For your case= , > > >>> I recommend that you describe your hardware and software > > >>> configuration in more detail, and look for physical abnormalities i= n > > >>> the cabling and connections. > > >>> Once that is ruled and and the rest of us know what kind of hardwar= e > > >>> you=E2=80=99re dealing with, we might be able to make better commen= dations. > > >>> > > >>> Scott > > >>> > > >>> > On Dec 7, 2015, at 11:07 AM, prateek sethi > > >>> > > > >>> wrote: > > >>> > > > >>> > Hi , > > >>> > > > >>> > Is there any way or tool to find out that a disk which is not > > >>> responding > > >>> > properly is really bad or not? Sometimes I have seen that there i= s > > >>> > lot > > >>> of > > >>> > CDB error for a drive and system reboot makes every thing fine. > > >>> > What > > >>> can > > >>> be > > >>> > reasons for such kind of scenarios? > > >>> > > > >>> > I know smartctl is the one which can help. I have some couple of > > >>> question > > >>> > regarding this . > > >>> > > > >>> > 1. What if disk does not support smartctl? > > >>> > 2. How I can do smartest use of smartctl command like which > > >>> > parameters > > >>> can > > >>> > tell that the disk is actually bad? > > >>> > 3. What other test I can perform to make it sure that disk has > > >>> completely > > >>> > gone? > > >>> > > > >>> > > > >>> > Please tell me correct place to ask this question if I am asking > > >>> > at > > >>> wrong > > >>> > place. > > >>> > _______________________________________________ > > >>> > freebsd-scsi@freebsd.org mailing list > > >>> > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > >>> > To unsubscribe, send any mail to > > >>> > "freebsd-scsi-unsubscribe@freebsd.org > > >>> " > > >>> > > >>> > > >>> _______________________________________________ > > >> freebsd-scsi@freebsd.org mailing list > > >> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > >> To unsubscribe, send any mail to " > freebsd-scsi-unsubscribe@freebsd.org" > > >> > > > > > > > > > Have you simply moved the drive to another slot - does the problem > > > follow the drive? > > > Unlikely but it could be a backplane issue. > > > > > > I don't know about version 15 firmware, I have always used version 16 > > > firmware with 9.x to match the driver version. > > > > > > > > _______________________________________________ > > freebsd-scsi@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > > > From owner-freebsd-scsi@freebsd.org Fri Dec 11 19:56:41 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDE0E9D765A for ; Fri, 11 Dec 2015 19:56:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE59C1159 for ; Fri, 11 Dec 2015 19:56:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tBBJufJJ085517 for ; Fri, 11 Dec 2015 19:56:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 204901] GEOM doesn't see new disk capacity after VMDK resize without a reboot Date: Fri, 11 Dec 2015 19:56:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pi@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 19:56:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204901 --- Comment #1 from Kurt Jaeger --- See also: https://lists.freebsd.org/pipermail/freebsd-current/2015-November/058644.html -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Fri Dec 11 21:51:55 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78B0CA042F6 for ; Fri, 11 Dec 2015 21:51:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A9AE15EF for ; Fri, 11 Dec 2015 21:51:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tBBLpt29025923 for ; Fri, 11 Dec 2015 21:51:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 195033] [cam] [scsi] [patch] unclean SCSI attached HDD power-off on shut down Date: Fri, 11 Dec 2015 21:51:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 21:51:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195033 --- Comment #4 from commit-hook@freebsd.org --- A commit references this bug: Author: rpokala Date: Fri Dec 11 21:51:00 UTC 2015 New revision: 292123 URL: https://svnweb.freebsd.org/changeset/base/292123 Log: [PR 195033] Document mps.enable_ssu mps(4) sends StartStopUnit to SATA direct-access devices during shutdown. Document the tunables which control that behavior. PR: 195033 Reviewed by: scottl Approved by: jhb MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D4456 Changes: head/share/man/man4/mps.4 -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Fri Dec 11 21:53:07 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B8C4A0441D for ; Fri, 11 Dec 2015 21:53:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DA771778 for ; Fri, 11 Dec 2015 21:53:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id tBBLr7Dj028367 for ; Fri, 11 Dec 2015 21:53:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 195033] [cam] [scsi] [patch] unclean SCSI attached HDD power-off on shut down Date: Fri, 11 Dec 2015 21:53:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rpokala@panasas.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 21:53:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195033 --- Comment #5 from Ravi Pokala --- (In reply to Thomas Eberhardt from comment #1) After discussing w/ scottl@, we decided *not* to change the default value of enable_ssu; I just documented it. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-scsi@freebsd.org Fri Dec 11 22:35:34 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A61D8A04D0F for ; Fri, 11 Dec 2015 22:35:34 +0000 (UTC) (envelope-from Mykel@mWare.ca) Received: from Vice.ServerNorth.net (vice.ServerNorth.net [209.44.123.194]) by mx1.freebsd.org (Postfix) with ESMTP id 5C842143E for ; Fri, 11 Dec 2015 22:35:33 +0000 (UTC) (envelope-from Mykel@mWare.ca) Received: from mail.servernorth.net (localhost [127.0.0.1]) by Vice.ServerNorth.net (Postfix) with ESMTP id BD16856571 for ; Fri, 11 Dec 2015 17:34:18 -0500 (EST) Received: from myke@servernorth.net by mail.servernorth.net (Archiveopteryx 3.1.4) with esmtpsa id 1449873257-81299-81295/9/2; Fri, 11 Dec 2015 17:34:17 -0500 To: freebsd-scsi@freebsd.org From: Mykel@mWare.ca Subject: Informal(?) sesX messages Message-Id: <566B4F68.2040807@mWare.ca> Date: Fri, 11 Dec 2015 17:34:16 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:42.0) Gecko/20100101 Thunderbird/42.0 Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Sender: myke@servernorth.net X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 22:35:34 -0000 Hi all, please CC me on reply as I'm not subscribed to this list. I've got one of those Supermicro 72-drive monster machines, all ZFS'd up. https://www.supermicro.com/products/system/4u/6048/SSG-6048R-E1CR72L.cfm And before & after replacing a faulty SAS Expander and a pair of cables=20 (gobs of WRITE/ABORT errors), I'm still occasionally seeing these kernel=20 messages (in groups), and I'm not sure if they're benign, or pointing to=20 a SAS expander event... or what. I admit, this is my first time dealing=20 with a machine with SAS expanders, so I'm a bit out of my depth in=20 diagnosis thereof. Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: Element descriptor: = 'Slot00' Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: SAS Device Slot Element:=20 1 Phys at Slot 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844bd449 Every now and then (couple times per week) I'll get messages like that=20 for a number of drives, but not all, and not particularly specific to=20 any particular drive - but always spinning rust, not SSDs. It seems to=20 happen under high load, but I'm not completely certain of that. - Smartctl is happy with all the drives. - We've got redundant power supplies and have rotated them around, run=20 with spares - ZFS has logged zero checksum errors, many scrubs, massive=20 writes/copies/send|recvs & bonnies later. - The messages come when we're using either istgt, ctld, local=20 bonnie++s, dd'ing 0s, or just scrubbing. - Seem to only occur while there's regular activity on the pools, NOT=20 when importing/exporting/snapshotting. - Tried without SSDs connected, and with TRIM off, no change (saw a post=20 from a few years ago with the older version of the LSI SAS card having=20 some issues with that.) - It's a new machine, we're still commissioning it, but plan to press it=20 into service as a Vmware storage machine later next week - I've compiled and fired up sesd, but haven't had any messages go by=20 recently - OS is on a pair of Intel SSDs using the motherboard's controller,=20 unrelated/unaffected. Root on ZFS tho. Do I have anything to worry about? Are these normal things to see=20 popping up sporadically? Could this be a firmware bug in the expander(s)=20 or with the driver? I don't have any important data on there at this time, so=20 data-destructive testing is possible right now. Including bunch of info that might be helpful for dx. Thanks for any advice/suggestion! Myke PS: Please CC me in your reply. PPS: That's the kind of hostname you get when you don't give me=20 something better, or tell me that you don't care ;) FreeBSD ZFS-AF.$CLIENT.fqdn 10.2-RELEASE-p7 FreeBSD 10.2-RELEASE-p7 #0:=20 Mon Nov 2 14:19:39 UTC 2015=20 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 (The device IDs are a bit skewed as I hot-rearranged a number of them=20 (camcontrol stopping and reimporting the zpools), but it'll do it after=20 a reboot as well.) Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: Element descriptor: = 'Slot00' Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: SAS Device Slot Element:=20 1 Phys at Slot 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844bd449 Dec 11 16:06:54 ZFS-AF kernel: ses5: da4,pass4: Element descriptor: = 'Slot01' Dec 11 16:06:54 ZFS-AF kernel: ses5: da4,pass4: SAS Device Slot Element:=20 1 Phys at Slot 1 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844bc9b1 Dec 11 16:06:54 ZFS-AF kernel: ses5: da2,pass2: Element descriptor: = 'Slot02' Dec 11 16:06:54 ZFS-AF kernel: ses5: da2,pass2: SAS Device Slot Element:=20 1 Phys at Slot 2 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844d6ea1 Dec 11 16:06:54 ZFS-AF kernel: ses5: da1,pass1: Element descriptor: = 'Slot03' Dec 11 16:06:54 ZFS-AF kernel: ses5: da1,pass1: SAS Device Slot Element:=20 1 Phys at Slot 3 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844b9785 Dec 11 16:06:54 ZFS-AF kernel: ses5: da32,pass39: Element descriptor:=20 'Slot04' Dec 11 16:06:54 ZFS-AF kernel: ses5: da32,pass39: SAS Device Slot=20 Element: 1 Phys at Slot 4 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844bda21 Dec 11 16:06:54 ZFS-AF kernel: ses5: da33,pass40: Element descriptor:=20 'Slot05' Dec 11 16:06:54 ZFS-AF kernel: ses5: da33,pass40: SAS Device Slot=20 Element: 1 Phys at Slot 5 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844d050d Dec 11 16:06:54 ZFS-AF kernel: ses5: da34,pass41: Element descriptor:=20 'Slot06' Dec 11 16:06:54 ZFS-AF kernel: ses5: da34,pass41: SAS Device Slot=20 Element: 1 Phys at Slot 6 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844c81cd Dec 11 16:06:54 ZFS-AF kernel: ses5: da35,pass42: Element descriptor:=20 'Slot07' Dec 11 16:06:54 ZFS-AF kernel: ses5: da35,pass42: SAS Device Slot=20 Element: 1 Phys at Slot 7 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844bf42d Dec 11 16:06:54 ZFS-AF kernel: ses5: da36,pass43: Element descriptor:=20 'Slot08' Dec 11 16:06:54 ZFS-AF kernel: ses5: da36,pass43: SAS Device Slot=20 Element: 1 Phys at Slot 8 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844cf71d Dec 11 16:06:54 ZFS-AF kernel: ses5: da37,pass44: Element descriptor:=20 'Slot09' Dec 11 16:06:54 ZFS-AF kernel: ses5: da37,pass44: SAS Device Slot=20 Element: 1 Phys at Slot 9 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844bde8d Dec 11 16:06:54 ZFS-AF kernel: ses5: da38,pass45: Element descriptor:=20 'Slot10' Dec 11 16:06:54 ZFS-AF kernel: ses5: da38,pass45: SAS Device Slot=20 Element: 1 Phys at Slot 10 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844d66c9 Dec 11 16:06:54 ZFS-AF kernel: ses5: da39,pass46: Element descriptor:=20 'Slot11' Dec 11 16:06:54 ZFS-AF kernel: ses5: da39,pass46: SAS Device Slot=20 Element: 1 Phys at Slot 11 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None=20 ) Target( SSP ) Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f=20 addr 5000c500844cf01d [root@ZFS-AF ~]# camcontrol devlist at scbus0 target 13 lun 0 (da12,pass13= ) at scbus0 target 14 lun 0 (da14,pass15= ) at scbus0 target 15 lun 0 (da16,pass17= ) at scbus0 target 23 lun 0 (da19,pass20= ) at scbus0 target 32 lun 0 (ses0,pass8) at scbus1 target 8 lun 0 (pass9,da8) at scbus1 target 9 lun 0 (da10,pass11) at scbus1 target 10 lun 0 (da9,pass10) at scbus1 target 11 lun 0 (pass12,da11= ) at scbus1 target 12 lun 0 (da22,pass28= ) at scbus1 target 13 lun 0 (da13,pass14= ) at scbus1 target 14 lun 0 (da15,pass16= ) at scbus1 target 15 lun 0 (da17,pass18= ) at scbus1 target 20 lun 0 (da23,pass29= ) at scbus1 target 21 lun 0 (da21,pass27= ) at scbus1 target 22 lun 0 (da18,pass19= ) at scbus1 target 23 lun 0 (da20,pass26= ) at scbus1 target 32 lun 0 (ses1,pass21= ) at scbus6 target 0 lun 0 (ses2,pass22) at scbus11 target 0 lun 0 (ada0,pass23= ) at scbus12 target 0 lun 0 (ada1,pass24= ) at scbus13 target 0 lun 0 (ses3,pass25= ) at scbus14 target 8 lun 0 (da6,pass6) at scbus14 target 9 lun 0 (da5,pass5) at scbus14 target 10 lun 0 (da3,pass3) at scbus14 target 11 lun 0 (da0,pass0) at scbus14 target 12 lun 0 (pass30,da2= 4) at scbus14 target 13 lun 0 (pass31,da2= 5) at scbus14 target 14 lun 0 (pass32,da2= 6) at scbus14 target 15 lun 0 (pass33,da2= 7) at scbus14 target 16 lun 0 (pass34,da2= 8) at scbus14 target 17 lun 0 (pass35,da2= 9) at scbus14 target 18 lun 0 (pass36,da3= 0) at scbus14 target 19 lun 0 (pass37,da3= 1) at scbus14 target 20 lun 0 (ses4,pass3= 8) at scbus14 target 39 lun 0 (da7,pass7) at scbus14 target 40 lun 0 (da4,pass4) at scbus14 target 41 lun 0 (da2,pass2) at scbus14 target 42 lun 0 (da1,pass1) at scbus14 target 43 lun 0 (pass39,da3= 2) at scbus14 target 44 lun 0 (pass40,da3= 3) at scbus14 target 45 lun 0 (pass41,da3= 4) at scbus14 target 46 lun 0 (pass42,da3= 5) at scbus14 target 47 lun 0 (pass43,da3= 6) at scbus14 target 48 lun 0 (pass44,da3= 7) at scbus14 target 49 lun 0 (pass45,da3= 8) at scbus14 target 50 lun 0 (pass46,da3= 9) at scbus14 target 63 lun 0 (ses5,pass4= 7) [root@ZFS-AF ~]# [root@ZFS-AF ~]# smp_discover ses0 phy 21:D:attached:[5000c500844c64e5:00 t(SSP)] 12 Gbps phy 22:D:attached:[5000c500844d8de5:00 t(SSP)] 12 Gbps phy 23:D:attached:[5000c500844d6865:00 t(SSP)] 12 Gbps phy 31:D:attached:[5000c500844c9a3d:00 t(SSP)] 12 Gbps phy 40:U:attached:[5003048018c77101:03 i(SSP+STP+SMP)] 12 Gbps phy 41:U:attached:[5003048018c77101:02 i(SSP+STP+SMP)] 12 Gbps phy 42:U:attached:[5003048018c77101:01 i(SSP+STP+SMP)] 12 Gbps phy 43:U:attached:[5003048018c77101:00 i(SSP+STP+SMP)] 12 Gbps phy 44:U:attached:[5003048018c77101:07 i(SSP+STP+SMP)] 12 Gbps phy 45:U:attached:[5003048018c77101:06 i(SSP+STP+SMP)] 12 Gbps phy 46:U:attached:[5003048018c77101:05 i(SSP+STP+SMP)] 12 Gbps phy 47:U:attached:[5003048018c77101:04 i(SSP+STP+SMP)] 12 Gbps phy 48:D:attached:[50030480090b8d3d:00 V i(SMP) t(SSP)] 12 Gbps [root@ZFS-AF ~]# smp_discover ses1 phy 0:U:attached:[500304801970b401:00 i(SSP+STP+SMP)] 12 Gbps phy 1:U:attached:[500304801970b401:01 i(SSP+STP+SMP)] 12 Gbps phy 2:U:attached:[500304801970b401:03 i(SSP+STP+SMP)] 12 Gbps phy 3:U:attached:[500304801970b401:02 i(SSP+STP+SMP)] 12 Gbps phy 4:D:attached:[5000c5007788ad75:00 t(SSP)] 12 Gbps phy 5:D:attached:[5000c5007799d161:00 t(SSP)] 12 Gbps phy 6:D:attached:[5000c5007788fd71:00 t(SSP)] 12 Gbps phy 7:D:attached:[5000c5007788d7f9:00 t(SSP)] 12 Gbps phy 8:D:attached:[50030480090b8d88:00 t(SATA)] 12 Gbps phy 9:D:attached:[5000c500844c2185:00 t(SSP)] 12 Gbps phy 10:D:attached:[5000c500844d7ca1:00 t(SSP)] 12 Gbps phy 11:D:attached:[5000c500844c5db5:00 t(SSP)] 12 Gbps phy 16:D:attached:[50030480090b8d90:00 t(SATA)] 12 Gbps phy 17:D:attached:[50030480090b8d91:00 t(SATA)] 12 Gbps phy 18:D:attached:[50030480090b8d92:00 t(SATA)] 12 Gbps phy 19:D:attached:[5000c500844d028d:00 t(SSP)] 12 Gbps phy 22:U:disabled phy 23:U:disabled phy 24:U:attached:[500304801970b401:06 i(SSP+STP+SMP)] 12 Gbps phy 25:U:attached:[500304801970b401:07 i(SSP+STP+SMP)] 12 Gbps phy 26:U:attached:[500304801970b401:05 i(SSP+STP+SMP)] 12 Gbps phy 27:U:attached:[500304801970b401:04 i(SSP+STP+SMP)] 12 Gbps phy 31:U:disabled phy 32:U:disabled phy 36:D:attached:[50030480090b8dbd:00 V i(SMP) t(SSP)] 12 Gbps [root@ZFS-AF ~]# smp_discover ses4 phy 4:D:attached:[5000c500844bb641:00 t(SSP)] 12 Gbps phy 5:D:attached:[5000c500844ba865:00 t(SSP)] 12 Gbps phy 6:D:disabled phy 7:D:disabled phy 8:D:disabled phy 9:D:disabled phy 10:D:disabled phy 11:D:disabled phy 12:D:attached:[5000c500844c6699:00 t(SSP)] 12 Gbps phy 13:D:attached:[5000c500844bd9d5:00 t(SSP)] 12 Gbps phy 14:D:attached:[5000c500844d8c29:00 t(SSP)] 12 Gbps phy 15:D:attached:[5000c500844ba3c1:00 t(SSP)] 12 Gbps phy 16:D:attached:[5000c500844ca251:00 t(SSP)] 12 Gbps phy 17:D:attached:[5000c500844bcc31:00 t(SSP)] 12 Gbps phy 18:D:attached:[5000c500844bda45:00 t(SSP)] 12 Gbps phy 19:D:attached:[5000c500844c8579:00 t(SSP)] 12 Gbps phy 20:D:attached:[5000c500844c7ee1:00 t(SSP)] 12 Gbps phy 21:D:attached:[5000c500844d01fd:00 t(SSP)] 12 Gbps phy 22:U:disabled phy 23:U:disabled phy 24:U:attached:[500304801970b101:02 i(SSP+STP+SMP)] 12 Gbps phy 25:U:attached:[500304801970b101:03 i(SSP+STP+SMP)] 12 Gbps phy 26:U:attached:[500304801970b101:01 i(SSP+STP+SMP)] 12 Gbps phy 27:U:attached:[500304801970b101:00 i(SSP+STP+SMP)] 12 Gbps phy 28:D:attached:[500304801ea2dfbd:00 V i(SMP) t(SSP)] 12 Gbps [root@ZFS-AF ~]# smp_discover ses5 phy 0:D:attached:[5000c500844bd449:00 t(SSP)] 12 Gbps phy 1:D:attached:[5000c500844bc9b1:00 t(SSP)] 12 Gbps phy 2:D:attached:[5000c500844d6ea1:00 t(SSP)] 12 Gbps phy 3:D:attached:[5000c500844b9785:00 t(SSP)] 12 Gbps phy 20:D:attached:[5000c500844bda21:00 t(SSP)] 12 Gbps phy 21:D:attached:[5000c500844d050d:00 t(SSP)] 12 Gbps phy 22:D:attached:[5000c500844c81cd:00 t(SSP)] 12 Gbps phy 23:D:attached:[5000c500844bf42d:00 t(SSP)] 12 Gbps phy 24:D:attached:[5000c500844cf71d:00 t(SSP)] 12 Gbps phy 25:D:attached:[5000c500844bde8d:00 t(SSP)] 12 Gbps phy 26:D:attached:[5000c500844d66c9:00 t(SSP)] 12 Gbps phy 27:D:attached:[5000c500844cf01d:00 t(SSP)] 12 Gbps phy 40:U:attached:[500304801970b102:07 i(SSP+STP+SMP)] 12 Gbps phy 41:U:attached:[500304801970b102:06 i(SSP+STP+SMP)] 12 Gbps phy 42:U:attached:[500304801970b102:05 i(SSP+STP+SMP)] 12 Gbps phy 43:U:attached:[500304801970b102:04 i(SSP+STP+SMP)] 12 Gbps phy 48:D:attached:[500304801ea2df3d:00 V i(SMP) t(SSP)] 12 Gbps [root@ZFS-AF ~]# mpr0@pci0:2:0:0: class=3D0x010700 card=3D0x080815d9 chip=3D0x00971= 000=20 rev=3D0x02 hdr=3D0x00 vendor =3D 'LSI Logic / Symbios Logic' device =3D 'SAS3008 PCI-Express Fusion-MPT SAS-3' class =3D mass storage subclass =3D SAS mpr1@pci0:3:0:0: class=3D0x010700 card=3D0x080815d9 chip=3D0x00971= 000=20 rev=3D0x02 hdr=3D0x00 vendor =3D 'LSI Logic / Symbios Logic' device =3D 'SAS3008 PCI-Express Fusion-MPT SAS-3' class =3D mass storage subclass =3D SAS mpr2@pci0:133:0:0: class=3D0x010700 card=3D0x080815d9 chip=3D0x00971= 000=20 rev=3D0x02 hdr=3D0x00 vendor =3D 'LSI Logic / Symbios Logic' device =3D 'SAS3008 PCI-Express Fusion-MPT SAS-3' class =3D mass storage subclass =3D SAS From owner-freebsd-scsi@freebsd.org Fri Dec 11 22:44:40 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E438B9D7575 for ; Fri, 11 Dec 2015 22:44:40 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B3DA7198E for ; Fri, 11 Dec 2015 22:44:40 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by obc18 with SMTP id 18so93185640obc.2 for ; Fri, 11 Dec 2015 14:44:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=8vt+vr0bQJZ2bvKgNha3ksi7nPCJhuOsensI5+mCG2A=; b=U9otsWw3ipb6/4NG3smXJM4QCwGwTdKF1nP2/a+vOzqFRutBSsnTQBHSIM21D7iyqr JE+3LMdQPq7mc+uMbwio69RA6bMZTSQ0yTIUvgRHwga1G9e3F/XpVZRF2WqnFMWf7WXs B0+ZnjNUyqUCdTYzTwmjF7u2ch+shX0SGCsXlDR1zs9syIyvgUD164Gz2ud9S6cG40/9 FtL+gltmg/aCCw0xeWgUb1UVjaRq4cTMaqlFrvWmsQgKDO7DjM0a1mn6M5CklVqgat8E Gz1Q9ocYSiu0CoDecupqofiEEYyyvZ5IY0nVaiFOfaA9l3P46Ph/2VN1QI6QzJUsSo6L KJrQ== MIME-Version: 1.0 X-Received: by 10.182.171.105 with SMTP id at9mr17178851obc.49.1449873880118; Fri, 11 Dec 2015 14:44:40 -0800 (PST) Sender: asomers@gmail.com Received: by 10.202.0.7 with HTTP; Fri, 11 Dec 2015 14:44:40 -0800 (PST) In-Reply-To: <566B4F68.2040807@mWare.ca> References: <566B4F68.2040807@mWare.ca> Date: Fri, 11 Dec 2015 15:44:40 -0700 X-Google-Sender-Auth: yjwZwWWQMJoYKWCTA__YxYxKI5Y Message-ID: Subject: Re: Informal(?) sesX messages From: Alan Somers To: Mykel@mware.ca Cc: FreeBSD-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 22:44:41 -0000 On Fri, Dec 11, 2015 at 3:34 PM, wrote: > Hi all, please CC me on reply as I'm not subscribed to this list. > > I've got one of those Supermicro 72-drive monster machines, all ZFS'd up. > https://www.supermicro.com/products/system/4u/6048/SSG-6048R-E1CR72L.cfm > > And before & after replacing a faulty SAS Expander and a pair of cables > (gobs of WRITE/ABORT errors), I'm still occasionally seeing these kernel > messages (in groups), and I'm not sure if they're benign, or pointing to a > SAS expander event... or what. I admit, this is my first time dealing with a > machine with SAS expanders, so I'm a bit out of my depth in diagnosis > thereof. > > Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: Element descriptor: 'Slot00' > Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: SAS Device Slot Element: 1 > Phys at Slot 0 > Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 > Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None ) > Target( SSP ) > Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f addr > 5000c500844bd449 > These look like device arrival notifications. If you scroll up, do you see any departure notifications? They should look like this: mps0: mpssas_prepare_remove: Sending reset for target ID 10 da0 at mps0 bus 0 scbus0 target 10 lun 0 da0: s/n JPW930HQ15H26H detached mps0: Unfreezing devq for target ID 10 xpt_release_devq(): requested 1 > present 0 (da0:mps0:0:10:0): Periph destroyed Also, could you post your HBA and expander firmware versions? For the HBA, use "sysctl dev.mps.0.firmware_version". For the expander, install sg3_utils and do "sg_inq --hex --len=64 ses0". The firmware version is the dotted quad at the end. # sg_inq --hex --len=64 ses0 00 0d 00 05 02 34 00 40 02 41 49 43 20 43 4f 52 50 ....4.@.AIC CORP 10 53 41 53 20 36 47 20 45 78 70 61 6e 64 65 72 20 SAS 6G Expander 20 30 62 30 31 78 33 36 2d 31 2e 31 31 2e 31 2e 31 0b01x36-1.11.1.1 30 00 20 20 20 20 20 20 20 -Alan From owner-freebsd-scsi@freebsd.org Sat Dec 12 03:02:08 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 838E49D8CFE for ; Sat, 12 Dec 2015 03:02:08 +0000 (UTC) (envelope-from Mykel@mWare.ca) Received: from Vice.ServerNorth.net (vice.ServerNorth.net [209.44.123.194]) by mx1.freebsd.org (Postfix) with ESMTP id 0D373132E; Sat, 12 Dec 2015 03:02:07 +0000 (UTC) (envelope-from Mykel@mWare.ca) Received: from mail.servernorth.net (localhost [127.0.0.1]) by Vice.ServerNorth.net (Postfix) with ESMTP id CE9135646A; Fri, 11 Dec 2015 22:02:04 -0500 (EST) Received: from myke@servernorth.net by mail.servernorth.net (Archiveopteryx 3.1.4) with esmtpsa id 1449889323-60207-60202/9/8; Fri, 11 Dec 2015 22:02:03 -0500 Subject: Re: Informal(?) sesX messages References: <566B4F68.2040807@mWare.ca> To: freebsd-scsi@freebsd.org From: Mykel@mWare.ca Message-Id: <566B8E2A.8070404@mWare.ca> Date: Fri, 11 Dec 2015 22:02:02 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:42.0) Gecko/20100101 Thunderbird/42.0 Mime-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Sender: myke@servernorth.net X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2015 03:02:08 -0000 On 15-12-11 17:44, Alan Somers wrote: > On Fri, Dec 11, 2015 at 3:34 PM, wrote: >> Hi all, please CC me on reply as I'm not subscribed to this list. >> >> I've got one of those Supermicro 72-drive monster machines, all ZFS'd = up. >> https://www.supermicro.com/products/system/4u/6048/SSG-6048R-E1CR72L.c= fm >> >> And before & after replacing a faulty SAS Expander and a pair of = cables >> (gobs of WRITE/ABORT errors), I'm still occasionally seeing these = kernel >> messages (in groups), and I'm not sure if they're benign, or pointing = to a >> SAS expander event... or what. I admit, this is my first time dealing = with a >> machine with SAS expanders, so I'm a bit out of my depth in diagnosis >> thereof. >> >> Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: Element descriptor: = 'Slot00' >> Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: SAS Device Slot = Element: 1 >> Phys at Slot 0 >> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 >> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( = None ) >> Target( SSP ) >> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f = addr >> 5000c500844bd449 >> > These look like device arrival notifications. If you scroll up, do > you see any departure notifications? They should look like this: > > mps0: mpssas_prepare_remove: Sending reset for target ID 10 > da0 at mps0 bus 0 scbus0 target 10 lun 0 > da0: s/n JPW930HQ15H26H detached > mps0: Unfreezing devq for target ID 10 > xpt_release_devq(): requested 1 > present 0 > (da0:mps0:0:10:0): Periph destroyed > > Also, could you post your HBA and expander firmware versions? For the > HBA, use "sysctl dev.mps.0.firmware_version". For the expander, > install sg3_utils and do "sg_inq --hex --len=3D64 ses0". The firmware > version is the dotted quad at the end. > > # sg_inq --hex --len=3D64 ses0 > 00 0d 00 05 02 34 00 40 02 41 49 43 20 43 4f 52 50 ....4.@.AI= C CORP > 10 53 41 53 20 36 47 20 45 78 70 61 6e 64 65 72 20 SAS 6G = Expander > 20 30 62 30 31 78 33 36 2d 31 2e 31 31 2e 31 2e 31 0b01x36-1.= 11.1.1 > 30 00 20 20 20 20 20 20 20 > > -Alan I can say, without doubt, that I do NOT have any preceding=20 detachments... which is why I'm so baffled by the messages. If the=20 devices aren't de/reattaching, what's the point of these informal/benign=20 ones? I am familiar with them from other hot-swap and disk failure=20 scenarios in other machines. Could this be a driver bug not logging the disconnection? But when I=20 hot-unplugged them, I do see that in dmesg. Or does SAS do something where it might renegotiate or reconfigure the=20 lanes, and I'm just seeing it do that? Thanks, Myke dev.mpr.0.driver_version: 09.255.01.00-fbsd dev.mpr.0.firmware_version: 06.00.00.00 dev.mpr.1.driver_version: 09.255.01.00-fbsd dev.mpr.1.firmware_version: 08.00.00.00 dev.mpr.2.driver_version: 09.255.01.00-fbsd dev.mpr.2.firmware_version: 08.00.00.00 [root@ZFS-AF ~]# sg_inq --hex --len=3D64 ses0 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI 10 53 41 53 33 78 34 38 20 20 20 20 20 20 20 20 20 SAS3x48 20 30 37 30 31 78 34 38 2d 36 36 2e 37 2e 31 2e 31 0701x48-66.7.1.= 1 30 37 00 20 20 20 20 20 20 7. [root@ZFS-AF ~]# sg_inq --hex --len=3D64 ses1 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI 10 53 41 53 33 78 33 36 20 20 20 20 20 20 20 20 20 SAS3x36 20 30 37 30 31 78 33 36 2d 36 36 2e 37 2e 31 2e 31 0701x36-66.7.1.= 1 30 37 00 20 20 20 20 20 20 7. [root@ZFS-AF ~]# sg_inq --hex --len=3D64 ses2 SCSI INQUIRY failed on ses2, res=3D-1 [root@ZFS-AF ~]# sg_inq --hex --len=3D64 ses3 SCSI INQUIRY failed on ses3, res=3D-1 [root@ZFS-AF ~]# sg_inq --hex --len=3D64 ses4 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI 10 53 41 53 33 78 32 38 20 20 20 20 20 20 20 20 20 SAS3x28 20 30 37 30 31 78 32 38 2d 36 36 2e 37 2e 31 2e 31 0701x28-66.7.1.= 1 30 37 00 20 20 20 20 20 20 7. [root@ZFS-AF ~]# sg_inq --hex --len=3D64 ses5 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI 10 53 41 53 33 78 34 38 20 20 20 20 20 20 20 20 20 SAS3x48 20 30 37 30 31 78 34 38 2d 36 36 2e 37 2e 31 2e 31 0701x48-66.7.1.= 1 30 37 00 20 20 20 20 20 20 7. [root@ZFS-AF ~]# And here's dmesg after fresh reboot: [root@ZFS-AF ~]# dmesg Copyright (c) 1992-2015 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.2-RELEASE-p7 #0: Mon Nov 2 14:19:39 UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Xeon(R) CPU E5-2609 v3 @ 1.90GHz (1900.04-MHz K8-class CPU) Origin=3D"GenuineIntel" Id=3D0x306f2 Family=3D0x6 Model=3D0x3f = Stepping=3D2 Features=3D0xbfebfbff Features2=3D0x7ffefbff,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,= TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND> AMD Features=3D0x2c100800 AMD Features2=3D0x21 Structured Extended=20 Features=3D0x37ab XSAVE Features=3D0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory =3D 137438953472 (131072 MB) avail memory =3D 133408280576 (127228 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 2 package(s) x 6 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 cpu4 (AP): APIC ID: 8 cpu5 (AP): APIC ID: 10 cpu6 (AP): APIC ID: 16 cpu7 (AP): APIC ID: 18 cpu8 (AP): APIC ID: 20 cpu9 (AP): APIC ID: 22 cpu10 (AP): APIC ID: 24 cpu11 (AP): APIC ID: 26 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard random: initialized module_register_init: MOD_LOAD (vesa, 0xffffffff80db8e60, 0) error 19 kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci255: on pcib0 pcib1: on acpi0 pci127: on pcib1 pcib2: port 0xcf8-0xcff on acpi0 pci0: on pcib2 pcib3: irq 26 at device 1.0 on pci0 pci1: on pcib3 pcib4: irq 32 at device 2.0 on pci0 pci2: on pcib4 mpr0: port 0x6000-0x60ff mem=20 0xc7400000-0xc740ffff irq 32 at device 0.0 on pci2 mpr0: IOCFacts : MsgVersion: 0x205 HeaderVersion: 0x2300 IOCNumber: 0 IOCExceptions: 0x0 MaxChainDepth: 128 NumberOfPorts: 1 RequestCredit: 3072 ProductID: 0x2221 IOCRequestFrameSize: 32 MaxInitiators: 1 MaxTargets: 256 MaxSasExpanders: 48 MaxEnclosures: 16 HighPriorityCredit: 128 MaxReplyDescriptorPostQueueDepth: 65504 ReplyFrameSize: 32 MaxVolumes: 0 MaxDevHandle: 313 MaxPersistentEntries: 128 mpr0: Firmware: 06.00.00.00, Driver: 09.255.01.00-fbsd mpr0: IOCCapabilities:=20 7a85c pcib5: irq 32 at device 2.2 on pci0 pci3: on pcib5 mpr1: port 0x5000-0x50ff mem=20 0xc7240000-0xc724ffff,0xc7200000-0xc723ffff irq 34 at device 0.0 on pci3 mpr1: IOCFacts : MsgVersion: 0x205 HeaderVersion: 0x2300 IOCNumber: 0 IOCExceptions: 0x0 MaxChainDepth: 128 NumberOfPorts: 1 RequestCredit: 3072 ProductID: 0x2221 IOCRequestFrameSize: 32 MaxInitiators: 1 MaxTargets: 256 MaxSasExpanders: 48 MaxEnclosures: 16 HighPriorityCredit: 128 MaxReplyDescriptorPostQueueDepth: 65504 ReplyFrameSize: 32 MaxVolumes: 0 MaxDevHandle: 313 MaxPersistentEntries: 128 mpr1: Firmware: 08.00.00.00, Driver: 09.255.01.00-fbsd mpr1: IOCCapabilities:=20 7a85c pcib6: irq 40 at device 3.0 on pci0 pci4: on pcib6 pcib7: irq 40 at device 3.2 on pci0 pci5: on pcib7 ix0: =20 port 0x4020-0x403f mem 0xc7c00000-0xc7dfffff,0xc7e04000-0xc7e07fff irq=20 42 at device 0.0 on pci5 ix0: Using MSIX interrupts with 9 vectors ix0: Ethernet address: 0c:c4:7a:35:12:c0 ix0: PCI Express Bus: Speed 5.0GT/s Width x8 ix1: =20 port 0x4000-0x401f mem 0xc7a00000-0xc7bfffff,0xc7e00000-0xc7e03fff irq=20 45 at device 0.1 on pci5 ix1: Using MSIX interrupts with 9 vectors ix1: Ethernet address: 0c:c4:7a:35:12:c1 ix1: PCI Express Bus: Speed 5.0GT/s Width x8 pci0: at device 17.0 (no driver attached) ahci0: port=20 0x7110-0x7117,0x7100-0x7103,0x70f0-0x70f7,0x70e0-0x70e3,0x7020-0x703f=20 mem 0xc7538000-0xc75387ff irq 16 at device 17.4 on pci0 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahciem0: on ahci0 xhci0: mem 0xc7500000-0xc750ffff irq=20 19 at device 20.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) ehci0: mem 0xc7534000-0xc75343ff irq=20 18 at device 26.0 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci0 pcib8: irq 16 at device 28.0 on pci0 pci6: on pcib8 pcib9: irq 16 at device 28.4 on pci0 pci7: on pcib9 pcib10: at device 0.0 on pci7 pci8: on pcib10 vgapci0: port 0x3000-0x307f mem=20 0xc6000000-0xc6ffffff,0xc7000000-0xc701ffff irq 16 at device 0.0 on pci8 vgapci0: Boot video device ehci1: mem 0xc7533000-0xc75333ff irq=20 18 at device 29.0 on pci0 usbus2: EHCI version 1.0 usbus2 on ehci1 isab0: at device 31.0 on pci0 isa0: on isab0 ahci1: port=20 0x7070-0x7077,0x7060-0x7063,0x7050-0x7057,0x7040-0x7043,0x7000-0x701f=20 mem 0xc7532000-0xc75327ff irq 16 at device 31.2 on pci0 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich5: at channel 1 on ahci1 ahcich6: at channel 2 on ahci1 ahcich7: at channel 3 on ahci1 ahcich8: at channel 4 on ahci1 ahcich9: at channel 5 on ahci1 ahciem1: on ahci1 pcib11: on acpi0 pci128: on pcib11 pcib12: at device 0.0 on pci128 pci129: on pcib12 pcib13: irq 50 at device 1.0 on pci128 pci130: on pcib13 pcib14: irq 56 at device 2.0 on pci128 pci131: on pcib14 em0: port 0xf020-0xf03f mem=20 0xfbea0000-0xfbebffff,0xfbe80000-0xfbe9ffff irq 56 at device 0.0 on = pci131 em0: Using an MSI interrupt em0: Ethernet address: 00:26:55:d3:1b:3a em1: port 0xf000-0xf01f mem=20 0xfbe40000-0xfbe5ffff,0xfbe20000-0xfbe3ffff irq 60 at device 0.1 on = pci131 em1: Using an MSI interrupt em1: Ethernet address: 00:26:55:d3:1b:3b pcib15: irq 56 at device 2.2 on pci128 pci132: on pcib15 ix2: =20 mem 0xfb800000-0xfb9fffff,0xfba04000-0xfba07fff irq 61 at device 0.0 on=20 pci132 ix2: Using MSIX interrupts with 9 vectors ix2: Ethernet address: a0:36:9f:5d:40:a8 ix2: PCI Express Bus: Speed 5.0GT/s Width x8 ix3: =20 mem 0xfb600000-0xfb7fffff,0xfba00000-0xfba03fff irq 58 at device 0.1 on=20 pci132 ix3: Using MSIX interrupts with 9 vectors ix3: Ethernet address: a0:36:9f:5d:40:aa ix3: PCI Express Bus: Speed 5.0GT/s Width x8 pcib16: irq 64 at device 3.0 on pci128 pci133: on pcib16 mpr2: port 0xe000-0xe0ff mem=20 0xfbc40000-0xfbc4ffff,0xfbc00000-0xfbc3ffff irq 64 at device 0.0 on = pci133 mpr2: IOCFacts : MsgVersion: 0x205 HeaderVersion: 0x2300 IOCNumber: 0 IOCExceptions: 0x0 MaxChainDepth: 128 NumberOfPorts: 1 RequestCredit: 3072 ProductID: 0x2221 IOCRequestFrameSize: 32 MaxInitiators: 1 MaxTargets: 256 MaxSasExpanders: 48 MaxEnclosures: 16 HighPriorityCredit: 128 MaxReplyDescriptorPostQueueDepth: 65504 ReplyFrameSize: 32 MaxVolumes: 0 MaxDevHandle: 313 MaxPersistentEntries: 128 mpr2: Firmware: 08.00.00.00, Driver: 09.255.01.00-fbsd mpr2: IOCCapabilities:=20 7a85c acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 ipmi0: port 0xca2,0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi orm0: at iomem=20 0xc0000-0xc7fff,0xc8000-0xc87ff,0xc8800-0xc97ff,0xc9800-0xca7ff,0xca800-0= xcb7ff,0xcb800-0xcc7ff,0xcc800-0xcd7ff=20 on isa0 sc0: at flags 0x100 on isa0 sc0: CGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3d0-0x3db iomem 0xb8000-0xbffff on isa0 ppc0: cannot reserve I/O port range est0: on cpu0 est1: on cpu1 est2: on cpu2 est3: on cpu3 est4: on cpu4 est5: on cpu5 est6: on cpu6 est7: on cpu7 est8: on cpu8 est9: on cpu9 est10: on cpu10 est11: on cpu11 random: unblocking device. usbus0: 5.0Gbps Super Speed USB v3.0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec usbus1: 480Mbps High Speed USB v2.0 usbus2: 480Mbps High Speed USB v2.0 ugen0.1: <0x8086> at usbus0 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ipmi0: IPMI device rev. 1, firmware rev. 2.14, version 2.0 ipmi0: Number of channels 2 ipmi0: Attached watchdog uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub0: 21 ports with 21 removable, self powered ugen0.2: at usbus0 uhub3: on=20 usbus0 mpr0: SAS Address from SAS device page0 =3D 5000c500844c64e5 mpr0: Found device <401,End Device> <12.0Gbps> handle<0x000a>=20 enclosureHandle<0x0002> slot 5 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 =3D 5000c500844d8de5 mpr0: Found device <401,End Device> <12.0Gbps> handle<0x000b>=20 enclosureHandle<0x0002> slot 6 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 =3D 5000c500844d6865 mpr0: Found device <401,End Device> <12.0Gbps> handle<0x000c>=20 enclosureHandle<0x0002> slot 7 mpr0: At enclosure level 0 and connector name ( ) mpr1: SAS Address for SATA device =3D 9683a928e2b4aab3 mpr0: SAS Address from SAS device page0 =3D 5000c500844c9a3d mpr0: Found device <401,End Device> <12.0Gbps> handle<0x000d>=20 enclosureHandle<0x0002> slot 15 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 =3D 50030480090b8d3d mpr0: Found device <4411,End Device> <12.0Gbps>=20 handle<0x000e> enclosureHandle<0x0002> slot 24 mpr0: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844bb641 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x000b>=20 enclosureHandle<0x0002> slot 0 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address for SATA device =3D 9684b128e2b4aab3 mpr2: SAS Address from SAS device page0 =3D 5000c500844ba865 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x000c>=20 enclosureHandle<0x0002> slot 1 mpr1: SAS Address for SATA device =3D 9687a328e2b4aab3 ugen1.2: at usbus1 uhub4: =20 on usbus1 ugen2.2: at usbus2 uhub5: =20 on usbus2 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844c6699 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0011>=20 enclosureHandle<0x0002> slot 2 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address for SATA device =3D 9586a528e2b4aab3 mpr2: SAS Address from SAS device page0 =3D 5000c500844bd9d5 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0012>=20 enclosureHandle<0x0002> slot 3 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844d8c29 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0013>=20 enclosureHandle<0x0002> slot 4 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 5000c5007788ad75 mpr1: mpr2: SAS Address from SAS device page0 =3D 5000c500844ba3c1 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0014>=20 enclosureHandle<0x0002> slot 5 mpr2: At enclosure level 0 and connector name ( ) Found device <401,End Device> <12.0Gbps> handle<0x000a>=20 enclosureHandle<0x0002> slot 0 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844ca251 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0015>=20 enclosureHandle<0x0002> slot 6 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844bcc31 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0016>=20 enclosureHandle<0x0002> slot 7 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 5000c5007799d161 mpr1: Found device <401,End Device> <12.0Gbps> handle<0x000b>=20 enclosureHandle<0x0002> slot 1 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844bda45 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0017>=20 enclosureHandle<0x0002> slot 8 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 5000c5007788fd71 mpr1: Found device <401,End Device> <12.0Gbps> handle<0x000c>=20 enclosureHandle<0x0002> slot 2 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844c8579 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0018>=20 enclosureHandle<0x0002> slot 9 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 5000c5007788d7f9 mpr1: Found device <401,End Device> <12.0Gbps> handle<0x000d>=20 enclosureHandle<0x0002> slot 3 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844c7ee1 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0019>=20 enclosureHandle<0x0002> slot 10 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 50030480090b8d88 mpr2: SAS Address from SAS device page0 =3D 5000c500844d01fd mpr2: Found device <401,End Device> <12.0Gbps> handle<0x001a>=20 enclosureHandle<0x0002> slot 11 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SATA device =3D 9683a928e2b4aab3 mpr1: Found device <81,End Device> <12.0Gbps> handle<0x000e>=20 enclosureHandle<0x0002> slot 4 mpr1: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 5000c500844c2185 mpr1: Found device <401,End Device> <12.0Gbps> handle<0x000f>=20 enclosureHandle<0x0002> slot 5 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 500304801ea2dfbd mpr2: Found device <4411,End Device> <12.0Gbps>=20 handle<0x001f> enclosureHandle<0x0002> slot 12 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 5000c500844d7ca1 mpr1: Found device <401,End Device> <12.0Gbps> handle<0x0010>=20 enclosureHandle<0x0002> slot 6 mpr1: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 5000c500844c5db5 mpr1: Found device <401,End Device> <12.0Gbps> handle<0x0011>=20 enclosureHandle<0x0002> slot 7 mpr1: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 50030480090b8d90 mpr2: SAS Address from SAS device page0 =3D 5000c500844bd449 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x000d>=20 enclosureHandle<0x0003> slot 0 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SATA device =3D 9684b128e2b4aab3 mpr1: Found device <81,End Device> <12.0Gbps> handle<0x0012>=20 enclosureHandle<0x0002> slot 12 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844bc9b1 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x000e>=20 enclosureHandle<0x0003> slot 1 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 50030480090b8d91 mpr2: SAS Address from SAS device page0 =3D 5000c500844d6ea1 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x000f>=20 enclosureHandle<0x0003> slot 2 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SATA device =3D 9687a328e2b4aab3 mpr1: Found device <81,End Device> <12.0Gbps> handle<0x0013>=20 enclosureHandle<0x0002> slot 13 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844b9785 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0010>=20 enclosureHandle<0x0003> slot 3 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 50030480090b8d92 mpr2: SAS Address from SAS device page0 =3D 5000c500844bda21 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x001b>=20 enclosureHandle<0x0003> slot 4 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SATA device =3D 9586a528e2b4aab3 mpr1: Found device <81,End Device> <12.0Gbps> handle<0x0014>=20 enclosureHandle<0x0002> slot 14 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844d050d mpr2: Found device <401,End Device> <12.0Gbps> handle<0x001c>=20 enclosureHandle<0x0003> slot 5 mpr2: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 5000c500844d028d mpr1: Found device <401,End Device> <12.0Gbps> handle<0x0015>=20 enclosureHandle<0x0002> slot 15 mpr1: At enclosure level 0 and connector name ( ) mpr1: SAS Address from SAS device page0 =3D 50030480090b8dbd mpr1: Found device <4411,End Device> <12.0Gbps>=20 handle<0x0016> enclosureHandle<0x0002> slot 24 mpr1: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844c81cd mpr2: Found device <401,End Device> <12.0Gbps> handle<0x001d>=20 enclosureHandle<0x0003> slot 6 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844bf42d mpr2: Found device <401,End Device> <12.0Gbps> handle<0x001e>=20 enclosureHandle<0x0003> slot 7 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844cf71d mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0020>=20 enclosureHandle<0x0003> slot 8 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844bde8d mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0021>=20 enclosureHandle<0x0003> slot 9 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844d66c9 mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0022>=20 enclosureHandle<0x0003> slot 10 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 5000c500844cf01d mpr2: Found device <401,End Device> <12.0Gbps> handle<0x0023>=20 enclosureHandle<0x0003> slot 11 mpr2: At enclosure level 0 and connector name ( ) mpr2: SAS Address from SAS device page0 =3D 500304801ea2df3d mpr2: Found device <4411,End Device> <12.0Gbps>=20 handle<0x0024> enclosureHandle<0x0003> slot 24 mpr2: At enclosure level 0 and connector name ( ) uhub4: 6 ports with 6 removable, self powered uhub5: 8 ports with 8 removable, self powered uhub3: 3 ports with 2 removable, bus powered ugen0.3: at usbus0 ukbd0: on=20 usbus0 kbd0 at ukbd0 ugen0.4: at usbus0 uhub6: =20 on usbus0 uhub6: 4 ports with 3 removable, self powered ugen0.5: at usbus0 ukbd1: =20 on usbus0 kbd2 at ukbd1 da0 at mpr0 bus 0 scbus0 target 13 lun 0 da0: Fixed Direct Access SPC-4 SCSI device da0: Serial Number Z4F0BTFL0000R551Y6M9 da0: 1200.000MB/s transfers da0: Command Queueing enabled da0: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da1 at mpr0 bus 0 scbus0 target 14 lun 0 da2 at mpr0 bus 0 scbus0 target 15 lun 0 da1: Fixed Direct Access SPC-4 SCSI device da1: Serial Number Z4F0BFJS0000R548WQDM da1: 1200.000MB/s transfers da1: Command Queueing enabled da1: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da2: Fixed Direct Access SPC-4 SCSI device da2: Serial Number Z4F0BG670000R548ZWZP da2: 1200.000MB/s transfers da2: Command Queueing enabled da2: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da3 at mpr0 bus 0 scbus0 target 23 lun 0 da4 at mpr1 bus 0 scbus1 target 8 lun 0 da3: Fixed Direct Access SPC-4 SCSI device da3: Serial Number Z4F0BSKY0000R548Z0E2 da3: 1200.000MB/s transfers da3: Command Queueing enabled da3: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da11 at mpr1 bus 0 scbus1 target 15 lun 0 da11: Fixed Direct Access SPC-4 SCSI device da4: Fixed Direct Access SPC-4 SCSI device da9 at mpr1 bus 0 scbus1 target 13 lun 0 da9: Fixed Direct Access SPC-4 SCSI device da9: Serial Number Z4F0BVZP0000R551Y333 da9: 1200.000MB/s transfers da9: Command Queueing enabled da9: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da11: Serial Number Z4F0BTLX0000R551NW3V da11: 1200.000MB/s transfers da11: Command Queueing enabled da11: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da5 at mpr1 bus 0 scbus1 target 9 lun 0 da4: Serial Number S7M016SG0000S439LVPG da4: 1200.000MB/s transfers da4: Command Queueing enabled da4: 572325MB (1172123568 512 byte sectors: 255H 63S/T 72961C) da10 at mpr1 bus 0 scbus1 target 14 lun 0 da6 at mpr1 bus 0 scbus1 target 10 lun 0 da5: Fixed Direct Access SPC-4 SCSI device da5: Serial Number S7M01AYK0000J503B7CT da5: 1200.000MB/s transfers da5: Command Queueing enabled da5: 572325MB (1172123568 512 byte sectors: 255H 63S/T 72961C) da10: Fixed Direct Access SPC-4 SCSI device da6: Fixed Direct Access SPC-4 SCSI device da6: Serial Number S7M016TK0000J448E26D da6: 1200.000MB/s transfers da6: Command Queueing enabled da6: 572325MB (1172123568 512 byte sectors: 255H 63S/T 72961C) da10: Serial Number Z4F0BFW70000R548WR5D da10: 1200.000MB/s transfers da10: Command Queueing enabled da10: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da8 at mpr1 bus 0 scbus1 target 12 lun 0 da8: Fixed Direct Access SPC-4 SCSI device da8: Serial Number S2L2NWAG701294P da8: 1200.000MB/s transfers da8: Command Queueing enabled da8: 915715MB (1875385008 512 byte sectors: 255H 63S/T 116737C) da7 at mpr1 bus 0 scbus1 target 11 lun 0 da13 at mpr1 bus 0 scbus1 target 21 lun 0 da13: Fixed Direct Access SPC-4 SCSI device da13: Serial Number S2L2NWAG701298J da13: 1200.000MB/s transfers da13: Command Queueing enabled da13: 915715MB (1875385008 512 byte sectors: 255H 63S/T 116737C) da16 at mpr2 bus 0 scbus14 target 8 lun 0 da16: Fixed Direct Access SPC-4 SCSI device da16: Serial Number Z4F0BMKW0000R551NVS4 da16: 1200.000MB/s transfers da16: Command Queueing enabled da16: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da19 at mpr2 bus 0 scbus14 target 11 lun 0 da19: Fixed Direct Access SPC-4 SCSI device da19: Serial Number Z4F0BMC60000R551NWQB da19: 1200.000MB/s transfers da19: Command Queueing enabled da19: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da14 at mpr1 bus 0 scbus1 target 22 lun 0 da14: Fixed Direct Access SPC-4 SCSI device da14: Serial Number S2L2NWAG701287L da14: 1200.000MB/s transfers da14: Command Queueing enabled da14: 915715MB (1875385008 512 byte sectors: 255H 63S/T 116737C) da17 at mpr2 bus 0 scbus14 target 9 lun 0 da17: Fixed Direct Access SPC-4 SCSI device da17: Serial Number Z4F0BMTX0000R551NUEH da17: 1200.000MB/s transfers da17: Command Queueing enabled da17: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da20 at mpr2 bus 0 scbus14 target 12 lun 0 da20: Fixed Direct Access SPC-4 SCSI device da20: Serial Number Z4F0BFL20000R548HJTN da20: 1200.000MB/s transfers da20: Command Queueing enabled da20: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da23 at mpr2 bus 0 scbus14 target 15 lun 0 da23: Fixed Direct Access SPC-4 SCSI device da23: Serial Number Z4F0BM690000R551NSDP da23: 1200.000MB/s transfers da23: Command Queueing enabled da23: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da26 at mpr2 bus 0 scbus14 target 18 lun 0 da26: Fixed Direct Access SPC-4 SCSI device da26: Serial Number Z4F0BT4B0000R551NVJD da26: 1200.000MB/s transfers da26: Command Queueing enabled da26: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da28 at mpr2 bus 0 scbus14 target 39 lun 0 da28: Fixed Direct Access SPC-4 SCSI device da28: Serial Number Z4F0BMEA0000R551PA2G da28: 1200.000MB/s transfers da28: Command Queueing enabled da28: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da31 at mpr2 bus 0 scbus14 target 42 lun 0 da31: Fixed Direct Access SPC-4 SCSI device da31: Serial Number Z4F0BN5C0000R551NW9C da31: 1200.000MB/s transfers da31: Command Queueing enabled da31: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da34 at mpr2 bus 0 scbus14 target 45 lun 0 da34: Fixed Direct Access SPC-4 SCSI device da34: Serial Number Z4F0BT3M0000R551NTPG da34: 1200.000MB/s transfers da34: Command Queueing enabled da34: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da37 at mpr2 bus 0 scbus14 target 48 lun 0 da37: Fixed Direct Access SPC-4 SCSI device da37: Serial Number Z4F0BMGK0000R551NWLG da37: 1200.000MB/s transfers da37: Command Queueing enabled da37: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da7: Fixed Direct Access SPC-4 SCSI device da7: Serial Number S7M016V90000J449GN30 da7: 1200.000MB/s transfers da7: Command Queueing enabled da7: 572325MB (1172123568 512 byte sectors: 255H 63S/T 72961C) da15 at mpr1 bus 0 scbus1 target 23 lun 0 da22 at mpr2 bus 0 scbus14 target 14 lun 0 da22: Fixed Direct Access SPC-4 SCSI device da15: Fixed Direct Access SPC-4 SCSI device da15: Serial Number Z4F0BRB00000R551NWNR da15: 1200.000MB/s transfers da15: Command Queueing enabled da15: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da18 at mpr2 bus 0 scbus14 target 10 lun 0 da18: Fixed Direct Access SPC-4 SCSI device da18: Serial Number Z4F0BTDW0000R544PAJB da18: 1200.000MB/s transfers da18: Command Queueing enabled da18: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da21 at mpr2 bus 0 scbus14 target 13 lun 0 da21: Fixed Direct Access SPC-4 SCSI device da21: Serial Number Z4F0BMX90000R551NVLQ da21: 1200.000MB/s transfers da21: Command Queueing enabled da21: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da24 at mpr2 bus 0 scbus14 target 16 lun 0 da24: Fixed Direct Access SPC-4 SCSI device da24: Serial Number Z4F0BLZD0000R551NWQ5 da24: 1200.000MB/s transfers da24: Command Queueing enabled da24: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da27 at mpr2 bus 0 scbus14 target 19 lun 0 da27: Fixed Direct Access SPC-4 SCSI device da27: Serial Number Z4F0BRBA0000R551NUC6 da27: 1200.000MB/s transfers da27: Command Queueing enabled da27: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da29 at mpr2 bus 0 scbus14 target 40 lun 0 da29: Fixed Direct Access SPC-4 SCSI device da29: Serial Number Z4F0BM7E0000R551P9UJ da29: 1200.000MB/s transfers da29: Command Queueing enabled da29: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da32 at mpr2 bus 0 scbus14 target 43 lun 0 da32: Fixed Direct Access SPC-4 SCSI device da32: Serial Number Z4F0BLZF0000R551NUWR da32: 1200.000MB/s transfers da32: Command Queueing enabled da32: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da35 at mpr2 bus 0 scbus14 target 46 lun 0 da35: Fixed Direct Access SPC-4 SCSI device da35: Serial Number Z4F0BLN00000R551P9CY da35: 1200.000MB/s transfers da35: Command Queueing enabled da35: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da38 at mpr2 bus 0 scbus14 target 49 lun 0 da38: Fixed Direct Access SPC-4 SCSI device da38: Serial Number Z4F0BG6Y0000R548HL0Y da38: 1200.000MB/s transfers da38: Command Queueing enabled da38: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da22: Serial Number Z4F0BSEP0000R548YZLB da22: 1200.000MB/s transfers da22: Command Queueing enabled da22: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da25 at mpr2 bus 0 scbus14 target 17 lun 0 da25: Fixed Direct Access SPC-4 SCSI device da25: Serial Number Z4F0BT2T0000R551NSSW da25: 1200.000MB/s transfers da25: Command Queueing enabled da25: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da30 at mpr2 bus 0 scbus14 target 41 lun 0 da30: Fixed Direct Access SPC-4 SCSI device da30: Serial Number Z4F0BG2Z0000R548HL43 da30: 1200.000MB/s transfers da30: Command Queueing enabled da30: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da33 at mpr2 bus 0 scbus14 target 44 lun 0 da33: Fixed Direct Access SPC-4 SCSI device da33: Serial Number Z4F0BR9X0000R551P9JR da33: 1200.000MB/s transfers da33: Command Queueing enabled da33: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da36 at mpr2 bus 0 scbus14 target 47 lun 0 da36: Fixed Direct Access SPC-4 SCSI device da36: Serial Number Z4F0BRKH0000R548WQAT da36: 1200.000MB/s transfers da36: Command Queueing enabled da36: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da39 at mpr2 bus 0 scbus14 target 50 lun 0 da39: Fixed Direct Access SPC-4 SCSI device da39: Serial Number Z4F0BRP30000R548YYMT da39: 1200.000MB/s transfers da39: Command Queueing enabled da39: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C) da12 at mpr1 bus 0 scbus1 target 20 lun 0 da12: Fixed Direct Access SPC-4 SCSI device da12: Serial Number S2L2NWAG701295X da12: 1200.000MB/s transfers da12: Command Queueing enabled da12: 915715MB (1875385008 512 byte sectors: 255H 63S/T 116737C) ada0 at ahcich8 bus 0 scbus11 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number BTWA518309N3080BGN ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada0: Command Queueing enabled ada0: 76319MB (156301488 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad20 ada1 at ahcich9 bus 0 scbus12 target 0 lun 0 ada1: ACS-2 ATA SATA 3.x device ada1: Serial Number BTWA518309N0080BGN ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada1: Command Queueing enabled ada1: 76319MB (156301488 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad22 ses0 at mpr0 bus 0 scbus0 target 32 lun 0 ses0: Fixed Enclosure Services SPC-3 SCSI device ses0: Serial Number=20 \M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\= M^?\M^?\M^?\M^?\M^?\M^? ses0: 1200.000MB/s transfers ses0: Command Queueing enabled ses0: SCSI-3 ENC Device ses1 at mpr1 bus 0 scbus1 target 32 lun 0 ses1: Fixed Enclosure Services SPC-3 SCSI device ses1: Serial Number=20 \M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\= M^?\M^?\M^?\M^?\M^?\M^? ses1: 1200.000MB/s transfers ses1: Command Queueing enabled ses1: SCSI-3 ENC Device ses2 at ahciem0 bus 0 scbus6 target 0 lun 0 ses2: SEMB S-E-S 2.00 device ses2: SEMB SES Device ses3 at ahciem1 bus 0 scbus13 target 0 lun 0 ses3: SEMB S-E-S 2.00 device ses3: SEMB SES Device ses4 at mpr2 bus 0 scbus14 target 20 lun 0 ses4: Fixed Enclosure Services SPC-3 SCSI device ses4: Serial Number=20 \M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\= M^?\M^?\M^?\M^?\M^?\M^? ses4: 1200.000MB/s transfers ses4: Command Queueing enabled ses4: SCSI-3 ENC Device ses5 at mpr2 bus 0 scbus14 target 63 lun 0 ses5: Fixed Enclosure Services SPC-3 SCSI device ses5: Serial Number=20 \M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\M^?\= M^?\M^?\M^?\M^?\M^?\M^? ses5: 1200.000MB/s transfers ses5: Command Queueing enabled ses5: SCSI-3 ENC Device SMP: AP CPU #10 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #8 Launched! Timecounter "TSC" frequency 1900035932 Hz quality 1000 GEOM_MIRROR: Device mirror/swap launched (2/2). Trying to mount root from zfs:ZFS-AF/ROOT/default []... uhid0: on=20 usbus0 ums0: =20 on usbus0 ums0: 3 buttons and [Z] coordinates ID=3D0 ix0: link state changed to UP lagg0: link state changed to UP ix0: link state changed to DOWN lagg0: link state changed to DOWN ix0: link state changed to UP lagg0: link state changed to UP ix1: link state changed to UP ix2: link state changed to UP ix3: link state changed to UP [root@ZFS-AF ~]# From owner-freebsd-scsi@freebsd.org Sat Dec 12 03:55:40 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CAC26A040F4 for ; Sat, 12 Dec 2015 03:55:40 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ob0-x22d.google.com (mail-ob0-x22d.google.com [IPv6:2607:f8b0:4003:c01::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D6BE12E8 for ; Sat, 12 Dec 2015 03:55:40 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by obbsd4 with SMTP id sd4so47792505obb.0 for ; Fri, 11 Dec 2015 19:55:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GjK01q0Lu21u4HeL0Ng9V0tLjAqr0PUWORCTRQyq24k=; b=iOPDIwGLoDZECW+Z+06gyHEjQ7LLK3zwSzwE4/uAN5rUgtvJgBqa2vNk2fL8ct4fVV HyxwQdpgHAX3nWR0yA3sRUXEmfHh3V92yy0FgccY566eoQQ0XRGjdFKEpgC8nZki930l nfvCdvZnAYVHJLyz1sQQHRGBDfs/st8zG7xhwPL27cSAU7iVZLu1vXcXgRkJLEYp8b7A fYjjDEh5F++ipLhtaWKa77cc7V6BA0bXP5llfJzag+2barln+WwsJ71LUMhtTKEfE7+C 3pn+OSsHEji6BX7D3yjbEJ6X2IIx90Bloq/m2TiKpJE+6eZho+LEQvGmOutYvDxD7rBy mW2w== MIME-Version: 1.0 X-Received: by 10.60.131.40 with SMTP id oj8mr16633456oeb.31.1449892539550; Fri, 11 Dec 2015 19:55:39 -0800 (PST) Sender: asomers@gmail.com Received: by 10.202.0.7 with HTTP; Fri, 11 Dec 2015 19:55:39 -0800 (PST) In-Reply-To: <566B8E2A.8070404@mWare.ca> References: <566B4F68.2040807@mWare.ca> <566B8E2A.8070404@mWare.ca> Date: Fri, 11 Dec 2015 20:55:39 -0700 X-Google-Sender-Auth: 6ebS-Eon9ZtaCBX3cAhgWwumu48 Message-ID: Subject: Re: Informal(?) sesX messages From: Alan Somers To: Mykel@mware.ca Cc: FreeBSD-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2015 03:55:41 -0000 On Fri, Dec 11, 2015 at 8:02 PM, wrote: > On 15-12-11 17:44, Alan Somers wrote: >> >> On Fri, Dec 11, 2015 at 3:34 PM, wrote: >>> >>> Hi all, please CC me on reply as I'm not subscribed to this list. >>> >>> I've got one of those Supermicro 72-drive monster machines, all ZFS'd up. >>> https://www.supermicro.com/products/system/4u/6048/SSG-6048R-E1CR72L.cfm >>> >>> And before & after replacing a faulty SAS Expander and a pair of cables >>> (gobs of WRITE/ABORT errors), I'm still occasionally seeing these kernel >>> messages (in groups), and I'm not sure if they're benign, or pointing to >>> a >>> SAS expander event... or what. I admit, this is my first time dealing >>> with a >>> machine with SAS expanders, so I'm a bit out of my depth in diagnosis >>> thereof. >>> >>> Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: Element descriptor: >>> 'Slot00' >>> Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: SAS Device Slot Element: >>> 1 >>> Phys at Slot 0 >>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 >>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None ) >>> Target( SSP ) >>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f addr >>> 5000c500844bd449 >>> >> These look like device arrival notifications. If you scroll up, do >> you see any departure notifications? They should look like this: >> >> mps0: mpssas_prepare_remove: Sending reset for target ID 10 >> da0 at mps0 bus 0 scbus0 target 10 lun 0 >> da0: s/n JPW930HQ15H26H detached >> mps0: Unfreezing devq for target ID 10 >> xpt_release_devq(): requested 1 > present 0 >> (da0:mps0:0:10:0): Periph destroyed >> >> Also, could you post your HBA and expander firmware versions? For the >> HBA, use "sysctl dev.mps.0.firmware_version". For the expander, >> install sg3_utils and do "sg_inq --hex --len=64 ses0". The firmware >> version is the dotted quad at the end. >> >> # sg_inq --hex --len=64 ses0 >> 00 0d 00 05 02 34 00 40 02 41 49 43 20 43 4f 52 50 ....4.@.AIC >> CORP >> 10 53 41 53 20 36 47 20 45 78 70 61 6e 64 65 72 20 SAS 6G >> Expander >> 20 30 62 30 31 78 33 36 2d 31 2e 31 31 2e 31 2e 31 >> 0b01x36-1.11.1.1 >> 30 00 20 20 20 20 20 20 20 >> >> -Alan > > > I can say, without doubt, that I do NOT have any preceding detachments... > which is why I'm so baffled by the messages. If the devices aren't > de/reattaching, what's the point of these informal/benign ones? I am > familiar with them from other hot-swap and disk failure scenarios in other > machines. > > Could this be a driver bug not logging the disconnection? But when I > hot-unplugged them, I do see that in dmesg. > Or does SAS do something where it might renegotiate or reconfigure the > lanes, and I'm just seeing it do that? > > Thanks, > > Myke > > > dev.mpr.0.driver_version: 09.255.01.00-fbsd > dev.mpr.0.firmware_version: 06.00.00.00 > dev.mpr.1.driver_version: 09.255.01.00-fbsd > dev.mpr.1.firmware_version: 08.00.00.00 > dev.mpr.2.driver_version: 09.255.01.00-fbsd > dev.mpr.2.firmware_version: 08.00.00.00 > > [root@ZFS-AF ~]# sg_inq --hex --len=64 ses0 > 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI > 10 53 41 53 33 78 34 38 20 20 20 20 20 20 20 20 20 SAS3x48 > 20 30 37 30 31 78 34 38 2d 36 36 2e 37 2e 31 2e 31 0701x48-66.7.1.1 > 30 37 00 20 20 20 20 20 20 7. > [root@ZFS-AF ~]# sg_inq --hex --len=64 ses1 > 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI > 10 53 41 53 33 78 33 36 20 20 20 20 20 20 20 20 20 SAS3x36 > 20 30 37 30 31 78 33 36 2d 36 36 2e 37 2e 31 2e 31 0701x36-66.7.1.1 > 30 37 00 20 20 20 20 20 20 7. > [root@ZFS-AF ~]# sg_inq --hex --len=64 ses2 > SCSI INQUIRY failed on ses2, res=-1 > [root@ZFS-AF ~]# sg_inq --hex --len=64 ses3 > SCSI INQUIRY failed on ses3, res=-1 > [root@ZFS-AF ~]# sg_inq --hex --len=64 ses4 > 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI > 10 53 41 53 33 78 32 38 20 20 20 20 20 20 20 20 20 SAS3x28 > 20 30 37 30 31 78 32 38 2d 36 36 2e 37 2e 31 2e 31 0701x28-66.7.1.1 > 30 37 00 20 20 20 20 20 20 7. > [root@ZFS-AF ~]# sg_inq --hex --len=64 ses5 > 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI > 10 53 41 53 33 78 34 38 20 20 20 20 20 20 20 20 20 SAS3x48 > 20 30 37 30 31 78 34 38 2d 36 36 2e 37 2e 31 2e 31 0701x48-66.7.1.1 > 30 37 00 20 20 20 20 20 20 7. > [root@ZFS-AF ~]# > > > And here's dmesg after fresh reboot: Well, that's weird. Your firmware versions look OK, though you might want to upgrade mpr0 just to be consistent. The next thing I would check, if I were you, would be devctl messages. Edit /etc/syslog.conf and change devd's loglevel to INFO, then HUP syslogd. Now every devctl message should get logged in /var/log/devd.log. That will tell you more precisely than dmesg whether there are any arrival or departure events. -Alan From owner-freebsd-scsi@freebsd.org Sat Dec 12 06:48:52 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45369A1462F for ; Sat, 12 Dec 2015 06:48:52 +0000 (UTC) (envelope-from mykel@mWare.ca) Received: from Vice.ServerNorth.net (vice.ServerNorth.net [209.44.123.194]) by mx1.freebsd.org (Postfix) with ESMTP id 1FEA41C45; Sat, 12 Dec 2015 06:48:50 +0000 (UTC) (envelope-from mykel@mWare.ca) Received: from mail.servernorth.net (localhost [127.0.0.1]) by Vice.ServerNorth.net (Postfix) with ESMTP id F305D564C3; Sat, 12 Dec 2015 01:48:47 -0500 (EST) Received: from mykel@mWare.ca by mail.servernorth.net (Archiveopteryx 3.1.4) with esmtpsa id 1449902926-24972-24971/9/8; Sat, 12 Dec 2015 01:48:46 -0500 Subject: Re: Informal(?) sesX messages To: Alan Somers References: <566B4F68.2040807@mWare.ca> <566B8E2A.8070404@mWare.ca> Cc: freebsd-scsi@freebsd.org From: mykel@mWare.ca Message-Id: <566BC34D.2020404@mware.ca> Date: Sat, 12 Dec 2015 01:48:45 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 Mime-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2015 06:48:52 -0000 On 2015/12/11 22:55, Alan Somers wrote: > On Fri, Dec 11, 2015 at 8:02 PM, wrote: >> On 15-12-11 17:44, Alan Somers wrote: >>> On Fri, Dec 11, 2015 at 3:34 PM, wrote: >>>> Hi all, please CC me on reply as I'm not subscribed to this list. >>>> >>>> I've got one of those Supermicro 72-drive monster machines, all ZFS'd up. >>>> https://www.supermicro.com/products/system/4u/6048/SSG-6048R-E1CR72L.cfm >>>> >>>> And before & after replacing a faulty SAS Expander and a pair of cables >>>> (gobs of WRITE/ABORT errors), I'm still occasionally seeing these kernel >>>> messages (in groups), and I'm not sure if they're benign, or pointing to >>>> a >>>> SAS expander event... or what. I admit, this is my first time dealing >>>> with a >>>> machine with SAS expanders, so I'm a bit out of my depth in diagnosis >>>> thereof. >>>> >>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: Element descriptor: >>>> 'Slot00' >>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: SAS Device Slot Element: >>>> 1 >>>> Phys at Slot 0 >>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 >>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None ) >>>> Target( SSP ) >>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f addr >>>> 5000c500844bd449 >>>> >>> These look like device arrival notifications. If you scroll up, do >>> you see any departure notifications? They should look like this: >>> >>> mps0: mpssas_prepare_remove: Sending reset for target ID 10 >>> da0 at mps0 bus 0 scbus0 target 10 lun 0 >>> da0: s/n JPW930HQ15H26H detached >>> mps0: Unfreezing devq for target ID 10 >>> xpt_release_devq(): requested 1 > present 0 >>> (da0:mps0:0:10:0): Periph destroyed >>> >>> Also, could you post your HBA and expander firmware versions? >>> >>> -Alan >> >> I can say, without doubt, that I do NOT have any preceding detachments... >> which is why I'm so baffled by the messages. If the devices aren't >> de/reattaching, what's the point of these informal/benign ones? I am >> familiar with them from other hot-swap and disk failure scenarios in other >> machines. >> >> Could this be a driver bug not logging the disconnection? But when I >> hot-unplugged them, I do see that in dmesg. >> Or does SAS do something where it might renegotiate or reconfigure the >> lanes, and I'm just seeing it do that? >> >> Thanks, >> >> Myke >> >> >> dev.mpr.0.driver_version: 09.255.01.00-fbsd >> dev.mpr.0.firmware_version: 06.00.00.00 >> dev.mpr.1.driver_version: 09.255.01.00-fbsd >> dev.mpr.1.firmware_version: 08.00.00.00 >> dev.mpr.2.driver_version: 09.255.01.00-fbsd >> dev.mpr.2.firmware_version: 08.00.00.00 >> >> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses0 >> 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI >> 10 53 41 53 33 78 34 38 20 20 20 20 20 20 20 20 20 SAS3x48 >> 20 30 37 30 31 78 34 38 2d 36 36 2e 37 2e 31 2e 31 0701x48-66.7.1.1 >> 30 37 00 20 20 20 20 20 20 7. >> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses1 >> 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI >> 10 53 41 53 33 78 33 36 20 20 20 20 20 20 20 20 20 SAS3x36 >> 20 30 37 30 31 78 33 36 2d 36 36 2e 37 2e 31 2e 31 0701x36-66.7.1.1 >> 30 37 00 20 20 20 20 20 20 7. >> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses2 >> SCSI INQUIRY failed on ses2, res=-1 >> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses3 >> SCSI INQUIRY failed on ses3, res=-1 >> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses4 >> 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI >> 10 53 41 53 33 78 32 38 20 20 20 20 20 20 20 20 20 SAS3x28 >> 20 30 37 30 31 78 32 38 2d 36 36 2e 37 2e 31 2e 31 0701x28-66.7.1.1 >> 30 37 00 20 20 20 20 20 20 7. >> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses5 >> 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI >> 10 53 41 53 33 78 34 38 20 20 20 20 20 20 20 20 20 SAS3x48 >> 20 30 37 30 31 78 34 38 2d 36 36 2e 37 2e 31 2e 31 0701x48-66.7.1.1 >> 30 37 00 20 20 20 20 20 20 7. >> [root@ZFS-AF ~]# >> >> >> And here's dmesg after fresh reboot: > Well, that's weird. Your firmware versions look OK, though you might > want to upgrade mpr0 just to be consistent. The next thing I would > check, if I were you, would be devctl messages. Edit /etc/syslog.conf > and change devd's loglevel to INFO, then HUP syslogd. Now every > devctl message should get logged in /var/log/devd.log. That will tell > you more precisely than dmesg whether there are any arrival or > departure events. > > -Alan Huh, I never noticed the 6 vs. 8; curiously, mpr0 and mpr1 are the two connected to the front expander... and where I've never seen an issue. Tho perhaps I scrambled which cards are serving was which in my testing - I also moved mpr2 to sit on the other CPU's PCI bus. I've added the devd log, although I haven't been able to trigger the event yet anyway. Tried to assert hw.mpr.2.debug_level, however it seems like hw.mpr doesn't exist. Finally, I haven't the slightest clue how to update the firmware; the Avago site only has a product brochure for the 3008 anyway :( From owner-freebsd-scsi@freebsd.org Sat Dec 12 15:41:01 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B094DA149B3 for ; Sat, 12 Dec 2015 15:41:01 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ob0-x236.google.com (mail-ob0-x236.google.com [IPv6:2607:f8b0:4003:c01::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7DFF613C5 for ; Sat, 12 Dec 2015 15:41:01 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by obc18 with SMTP id 18so102983035obc.2 for ; Sat, 12 Dec 2015 07:41:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=TIu58hViSc2U+IeFwWit3b3F9x5Ra83TbRjZ79p8roU=; b=ipfaw5F6b7p0p58VMOK+SFKGVlKl8sDNxng7PQH0WoUxzoTQOemZN5lZ0XQKmKbKHu KO4oUPw/EAxyUSqiZjSYnatrTXUKPXhqr0mNU7R8e5AIEUCjkLeU/rOS6NXwS+0kHedI LiFXC3aIG3Fqn+nOgPPtKIA53QRS8J4TCJflPq029Y55mcFTTqJ9YXdCo74Nn5GN++s5 0WMrDDFA6025up0mvCix6531wNDUN+U9Vx5urHP1jJof/73cJeMw8egoylIJrBvpl4O8 5LLYadYlSz6UHJxdp7gFn++s653NMv7YCR9P/n5IU4lpYYCPapGTrOiDL3RAPrYaaGHY eRnA== MIME-Version: 1.0 X-Received: by 10.60.15.197 with SMTP id z5mr18618945oec.57.1449934860657; Sat, 12 Dec 2015 07:41:00 -0800 (PST) Sender: asomers@gmail.com Received: by 10.202.0.7 with HTTP; Sat, 12 Dec 2015 07:41:00 -0800 (PST) In-Reply-To: <566BC34D.2020404@mware.ca> References: <566B4F68.2040807@mWare.ca> <566B8E2A.8070404@mWare.ca> <566BC34D.2020404@mware.ca> Date: Sat, 12 Dec 2015 08:41:00 -0700 X-Google-Sender-Auth: CtEC2-E4SRT5CT-_DZpWbikEm6k Message-ID: Subject: Re: Informal(?) sesX messages From: Alan Somers To: Mike Geiger Cc: FreeBSD-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2015 15:41:01 -0000 On Fri, Dec 11, 2015 at 11:48 PM, wrote: > On 2015/12/11 22:55, Alan Somers wrote: >> >> On Fri, Dec 11, 2015 at 8:02 PM, wrote: >>> >>> On 15-12-11 17:44, Alan Somers wrote: >>>> >>>> On Fri, Dec 11, 2015 at 3:34 PM, wrote: >>>>> >>>>> Hi all, please CC me on reply as I'm not subscribed to this list. >>>>> >>>>> I've got one of those Supermicro 72-drive monster machines, all ZFS'd >>>>> up. >>>>> >>>>> https://www.supermicro.com/products/system/4u/6048/SSG-6048R-E1CR72L.cfm >>>>> >>>>> And before & after replacing a faulty SAS Expander and a pair of cables >>>>> (gobs of WRITE/ABORT errors), I'm still occasionally seeing these >>>>> kernel >>>>> messages (in groups), and I'm not sure if they're benign, or pointing >>>>> to >>>>> a >>>>> SAS expander event... or what. I admit, this is my first time dealing >>>>> with a >>>>> machine with SAS expanders, so I'm a bit out of my depth in diagnosis >>>>> thereof. >>>>> >>>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: Element descriptor: >>>>> 'Slot00' >>>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: da7,pass7: SAS Device Slot >>>>> Element: >>>>> 1 >>>>> Phys at Slot 0 >>>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: SAS device type 1 id 0 >>>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: protocols: Initiator( None >>>>> ) >>>>> Target( SSP ) >>>>> Dec 11 16:06:54 ZFS-AF kernel: ses5: phy 0: parent 500304801ea2df3f >>>>> addr >>>>> 5000c500844bd449 >>>>> >>>> These look like device arrival notifications. If you scroll up, do >>>> you see any departure notifications? They should look like this: >>>> >>>> mps0: mpssas_prepare_remove: Sending reset for target ID 10 >>>> da0 at mps0 bus 0 scbus0 target 10 lun 0 >>>> da0: s/n JPW930HQ15H26H detached >>>> mps0: Unfreezing devq for target ID 10 >>>> xpt_release_devq(): requested 1 > present 0 >>>> (da0:mps0:0:10:0): Periph destroyed >>>> >>>> Also, could you post your HBA and expander firmware versions? >>>> >>>> -Alan >>> >>> >>> I can say, without doubt, that I do NOT have any preceding detachments... >>> which is why I'm so baffled by the messages. If the devices aren't >>> de/reattaching, what's the point of these informal/benign ones? I am >>> familiar with them from other hot-swap and disk failure scenarios in >>> other >>> machines. >>> >>> Could this be a driver bug not logging the disconnection? But when I >>> hot-unplugged them, I do see that in dmesg. >>> Or does SAS do something where it might renegotiate or reconfigure the >>> lanes, and I'm just seeing it do that? >>> >>> Thanks, >>> >>> Myke >>> >>> >>> dev.mpr.0.driver_version: 09.255.01.00-fbsd >>> dev.mpr.0.firmware_version: 06.00.00.00 >>> dev.mpr.1.driver_version: 09.255.01.00-fbsd >>> dev.mpr.1.firmware_version: 08.00.00.00 >>> dev.mpr.2.driver_version: 09.255.01.00-fbsd >>> dev.mpr.2.firmware_version: 08.00.00.00 >>> >>> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses0 >>> 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI >>> 10 53 41 53 33 78 34 38 20 20 20 20 20 20 20 20 20 SAS3x48 >>> 20 30 37 30 31 78 34 38 2d 36 36 2e 37 2e 31 2e 31 >>> 0701x48-66.7.1.1 >>> 30 37 00 20 20 20 20 20 20 7. >>> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses1 >>> 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI >>> 10 53 41 53 33 78 33 36 20 20 20 20 20 20 20 20 20 SAS3x36 >>> 20 30 37 30 31 78 33 36 2d 36 36 2e 37 2e 31 2e 31 >>> 0701x36-66.7.1.1 >>> 30 37 00 20 20 20 20 20 20 7. >>> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses2 >>> SCSI INQUIRY failed on ses2, res=-1 >>> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses3 >>> SCSI INQUIRY failed on ses3, res=-1 >>> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses4 >>> 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI >>> 10 53 41 53 33 78 32 38 20 20 20 20 20 20 20 20 20 SAS3x28 >>> 20 30 37 30 31 78 32 38 2d 36 36 2e 37 2e 31 2e 31 >>> 0701x28-66.7.1.1 >>> 30 37 00 20 20 20 20 20 20 7. >>> [root@ZFS-AF ~]# sg_inq --hex --len=64 ses5 >>> 00 0d 00 05 02 33 00 40 02 4c 53 49 20 20 20 20 20 ....3.@.LSI >>> 10 53 41 53 33 78 34 38 20 20 20 20 20 20 20 20 20 SAS3x48 >>> 20 30 37 30 31 78 34 38 2d 36 36 2e 37 2e 31 2e 31 >>> 0701x48-66.7.1.1 >>> 30 37 00 20 20 20 20 20 20 7. >>> [root@ZFS-AF ~]# >>> >>> >>> And here's dmesg after fresh reboot: >> >> Well, that's weird. Your firmware versions look OK, though you might >> want to upgrade mpr0 just to be consistent. The next thing I would >> check, if I were you, would be devctl messages. Edit /etc/syslog.conf >> and change devd's loglevel to INFO, then HUP syslogd. Now every >> devctl message should get logged in /var/log/devd.log. That will tell >> you more precisely than dmesg whether there are any arrival or >> departure events. >> >> -Alan > > Huh, I never noticed the 6 vs. 8; curiously, mpr0 and mpr1 are the two > connected to the front expander... and where I've never seen an issue. Tho > perhaps I scrambled which cards are serving was which in my testing - I also > moved mpr2 to sit on the other CPU's PCI bus. > > I've added the devd log, although I haven't been able to trigger the event > yet anyway. > Tried to assert hw.mpr.2.debug_level, however it seems like hw.mpr doesn't > exist. hw.mpr.debug_level is a tunable which, if set at boot time, will affect all mpr cards. What you want is dev.mpr.2.debug_level, a runtime-controllabel sysctl. > > Finally, I haven't the slightest clue how to update the firmware; the Avago > site only has a product brochure for the 3008 anyway :( It's fairly annoying. First, you must figure out which card you have. 3008 is the name of your chip. Your card is probably a 9300-9i. If so, go to this URL and click on firmware. If you download "Installer_P10_for_UEFI" then you can install it through the EFI shell. But they also have an installer that runs in FreeBSD. To use that, download both "Installer_P10_for_FreeBSD" AND "Installer_P10_for_MSDOS_and_Windows". Unzip the latter and extract the .bin file. Then unzip the former and run the executable contained within, providing the path to the .bin file obtained from the latter. You'll need a reboot afterwards. http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-9300-8i#downloads -Alan