From owner-freebsd-scsi@freebsd.org Mon Oct 16 10:45:46 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0DC00E36263 for ; Mon, 16 Oct 2017 10:45:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D78F876C1C for ; Mon, 16 Oct 2017 10:45:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9GAjjWr008551 for ; Mon, 16 Oct 2017 10:45:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 222898] [iscsi]: panic: page fault - system crashes while working as iSCSI target Date: Mon, 16 Oct 2017 10:45:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: emz@norma.perm.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Oct 2017 10:45:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222898 --- Comment #8 from emz@norma.perm.ru --- Well, I took a risk and installed a stock r315520. Unfortunately, the behav= ior is identical: crash within dozens of seconds after starting ctld, same panic message on a debug kernel: =3D=3D=3DCut=3D=3D=3D WARNING: 213.152.134.114 (iqn.1991-05.com.microsoft:worker85): no ping reply (NOP-Out) after 5 seconds; dropping connect ion WARNING: 213.152.138.158 (iqn.1991-05.com.microsoft:worker265): no ping rep= ly (NOP-Out) after 5 seconds; dropping connec tion WARNING: 213.152.134.62 (iqn.1991-05.com.microsoft:worker33): no ping reply (NOP-Out) after 5 seconds; dropping connecti on WARNING: 213.152.134.114 (iqn.1991-05.com.microsoft:worker85): no ping reply (NOP-Out) after 5 seconds; dropping connect ion panic: destroying session with 4 outstanding PDUs cpuid =3D 24 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe105789a= 8f0 vpanic() at vpanic+0x186/frame 0xfffffe105789a970 kassert_panic() at kassert_panic+0x126/frame 0xfffffe105789a9e0 icl_soft_conn_close() at icl_soft_conn_close+0x20a/frame 0xfffffe105789aa10 cfiscsi_maintenance_thread() at cfiscsi_maintenance_thread+0x100/frame 0xfffffe105789aa70 fork_exit() at fork_exit+0x84/frame 0xfffffe105789aab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe105789aab0 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- Uptime: 1m55s =3D=3D=3DCut=3D=3D=3D I am running a non-debug version of r315520 right now (so there's no either explicit regression or improvement over the r310734), it's still tending to crash in random moments of time when loaded by 299 initiators. As a workaround I will try to diminish the load by transferring some initia= tors to a spare SAN. Crashdumps, binaries and infos: http://files2.enaza.ru/ctld-stuck/r315520/ In the same time I'm still willing to help in any way in resolving this, including the access on a production system, over ssh/IPMI/whatever. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-scsi@freebsd.org Mon Oct 16 10:47:55 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC22DE362E0 for ; Mon, 16 Oct 2017 10:47:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CAC4176CA7 for ; Mon, 16 Oct 2017 10:47:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9GAlt60012447 for ; Mon, 16 Oct 2017 10:47:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 222898] [iscsi]: panic: page fault - system crashes while working as iSCSI target Date: Mon, 16 Oct 2017 10:47:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: emz@norma.perm.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Oct 2017 10:47:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222898 --- Comment #9 from emz@norma.perm.ru --- Follow-up: it's stock r315520 except the CTL_MAX_PORTS set to 1024, I decid= ed to mention this in case if it matters (but mav@ said that it's very unlikely that this change could affect anything). --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-scsi@freebsd.org Mon Oct 16 15:05:16 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6115E3CC01 for ; Mon, 16 Oct 2017 15:05:16 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5AF0383F1D for ; Mon, 16 Oct 2017 15:05:16 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPS id v9GEgVdk095108 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 16 Oct 2017 10:42:31 -0400 (EDT) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.15.2/8.14.9/Submit) id v9GEgVND095107; Mon, 16 Oct 2017 10:42:31 -0400 (EDT) (envelope-from ken) Date: Mon, 16 Oct 2017 10:42:31 -0400 From: "Kenneth D. Merry" To: Shiva Bhanujan Cc: "freebsd-scsi@freebsd.org" Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Message-ID: <20171016144231.GA94858@mithlond.kdm.org> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 16 Oct 2017 10:42:31 -0400 (EDT) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mithlond.kdm.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Oct 2017 15:05:16 -0000 On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > Hello, > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the following: > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code) > The error message above is missing the SCSI CDB. We need that in order to figure out what command the drive is complaining about. The error message means that FreeBSD is sending a SCSI command that the drive doesn't support. That can be benign, or it can cause a problem. So, what error does gpart give you when you have this problem? > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also has the same result. > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > Is there a different version of the mpr driver I should be using? I haven't checked the differences between the mpr driver in 10.2 vs 10.3 and 11.0. I do see that there are others who have experienced these issues. Can somebody please provide me some pointers > as to why this is occurring? Or if there are some driver changes that I might be able to incorporate? > In general, the latest mpr(4) driver is the best one. The driver itself generally doesn't send SCSI commands (there are a few exceptions), but rather passes them through from the upper layers of CAM. > Please note, that I have gone through the mail titled "scsi error at SEAGATE ST1200MM0088 TT31" and have started sg_format on all the SEAGATE disks. Having said that, I still need to figure out what would happen, if the disks were written to using FreeBSD 10.2, which doesn't seem to have SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated. > Send the full error messages, and we may be able to figure out what's going on. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@freebsd.org Tue Oct 17 01:26:12 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A2C5E4B913 for ; Tue, 17 Oct 2017 01:26:12 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-35.reflexion.net [208.70.210.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1A9775F3E for ; Tue, 17 Oct 2017 01:26:11 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 28080 invoked from network); 17 Oct 2017 01:19:30 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 17 Oct 2017 01:19:30 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Mon, 16 Oct 2017 21:19:30 -0400 (EDT) Received: (qmail 3474 invoked from network); 17 Oct 2017 01:19:29 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 17 Oct 2017 01:19:29 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Mon, 16 Oct 2017 18:19:28 -0700 From: Shiva Bhanujan To: "Kenneth D. Merry" CC: "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uA== Date: Tue, 17 Oct 2017 01:19:27 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local>, <20171016144231.GA94858@mithlond.kdm.org> In-Reply-To: <20171016144231.GA94858@mithlond.kdm.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] Content-Type: multipart/mixed; boundary="_004_3A5A10BE32AC9E45B4A22F89FC90EC0701C366610EQLEXC01Quorum_" MIME-Version: 1.0 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Oct 2017 01:26:12 -0000 --_004_3A5A10BE32AC9E45B4A22F89FC90EC0701C366610EQLEXC01Quorum_ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi Ken, I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've = tried this w/ 11.1, w/ the same results. Regards, Shiva ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG=5D Sent: Monday, October 16, 2017 7:42 AM To: Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > Hello, > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the = following: > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) > The error message above is missing the SCSI CDB. We need that in order to figure out what command the drive is complaining about. The error message means that FreeBSD is sending a SCSI command that the drive doesn't support. That can be benign, or it can cause a problem. So, what error does gpart give you when you have this problem? > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also = has the same result. > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 = doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 and = 11.0. I do see that there are others who have experienced these issues. = Can somebody please provide me some pointers > as to why this is occurring? Or if there are some driver changes that I = might be able to incorporate? > In general, the latest mpr(4) driver is the best one. The driver itself generally doesn't send SCSI commands (there are a few exceptions), but rather passes them through from the upper layers of CAM. > Please note, that I have gone through the mail titled =22scsi error at = SEAGATE ST1200MM0088 TT31=22 and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, if = the disks were written to using FreeBSD 10.2, which doesn't seem to have = SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated. > Send the full error messages, and we may be able to figure out what's going on. Ken -- Kenneth Merry ken=40FreeBSD.ORG ________________________________ --_004_3A5A10BE32AC9E45B4A22F89FC90EC0701C366610EQLEXC01Quorum_ Content-Type: text/plain; name="dmesg.txt" Content-Description: dmesg.txt Content-Disposition: attachment; filename="dmesg.txt"; size=86671; creation-date="Tue, 17 Oct 2017 01:18:35 GMT"; modification-date="Tue, 17 Oct 2017 01:18:35 GMT" Content-Transfer-Encoding: base64 KGRhMTA6bXByMDowOjE4OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhOTptcHIw OjA6MTc6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAwIDAwIDAwIDAxIDAwIDAwIAoo ZGE5Om1wcjA6MDoxNzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhOTptcHIw OjA6MTc6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhOTptcHIwOjA6MTc6MCk6 IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9w ZXJhdGlvbiBjb2RlKQooZGE5Om1wcjA6MDoxNzowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVy cm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAwMCAw MCAwMSAwMCAwMCAKKGRhOTptcHIwOjA6MTc6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVy cm9yCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTk6 bXByMDowOjE3OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFs aWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhOTptcHIwOjA6MTc6MCk6IEVycm9yIDIyLCBV bnJldHJ5YWJsZSBlcnJvcgooZGE5Om1wcjA6MDoxNzowKTogUkVBRCgxNikuIENEQjogODggMDAg MDAgMDAgMDAgMDEgZDEgYzAgYmEgMDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTk6bXByMDowOjE3 OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE5Om1wcjA6MDoxNzowKTogU0NT SSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE5Om1wcjA6MDoxNzowKTogU0NTSSBzZW5zZTog SUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUp CihkYTk6bXByMDowOjE3OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhOTptcHIw OjA6MTc6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJjIDAwIDAw IDAwIDAxIDAwIDAwIDAwIAooZGE5Om1wcjA6MDoxNzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0 dXMgRXJyb3IKKGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24K KGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAo SW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE5Om1wcjA6MDoxNzowKTogRXJyb3Ig MjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTg6bXByMDowOjE2OjApOiBSRUFEKDEwKS4gQ0RCOiAy OCAwMCAwMCAwMCAwMCAwMCAwMCAwMSAwMCAwMCAKKGRhODptcHIwOjA6MTY6MCk6IENBTSBzdGF0 dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHN0YXR1czogQ2hl Y2sgQ29uZGl0aW9uCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVF U1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhODptcHIwOjA6 MTY6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE4Om1wcjA6MDoxNjowKTogUkVB RCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMDAgMDAgMDEgMDAgMDAgCihkYTg6bXByMDowOjE2 OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE4Om1wcjA6MDoxNjowKTogU0NT SSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzZW5zZTog SUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUp CihkYTg6bXByMDowOjE2OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhODptcHIw OjA6MTY6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJhIDAwIDAw IDAwIDAxIDAwIDAwIDAwIAooZGE4Om1wcjA6MDoxNjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0 dXMgRXJyb3IKKGRhODptcHIwOjA6MTY6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24K KGRhODptcHIwOjA6MTY6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAo SW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE4Om1wcjA6MDoxNjowKTogRXJyb3Ig MjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTg6bXByMDowOjE2OjApOiBSRUFEKDE2KS4gQ0RCOiA4 OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAwMCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhODptcHIw OjA6MTY6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTg6bXByMDowOjE2OjAp OiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHNl bnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24g Y29kZSkKKGRhODptcHIwOjA6MTY6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE3 Om1wcjA6MDoxNTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMDAgMDAgMDEgMDAg MDAgCihkYTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE3 Om1wcjA6MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE3Om1wcjA6MDox NTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1h bmQgb3BlcmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1OjApOiBFcnJvciAyMiwgVW5yZXRyeWFi bGUgZXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAy IDAwIDAwIDAxIDAwIDAwIAooZGE3Om1wcjA6MDoxNTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0 dXMgRXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24K KGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAo SW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE3Om1wcjA6MDoxNTowKTogRXJyb3Ig MjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTc6bXByMDowOjE1OjApOiBSRUFEKDE2KS4gQ0RCOiA4 OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAwMCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhNzptcHIw OjA6MTU6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTc6bXByMDowOjE1OjAp OiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTc6bXByMDowOjE1OjApOiBTQ1NJIHNl bnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24g Y29kZSkKKGRhNzptcHIwOjA6MTU6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE3 Om1wcjA6MDoxNTowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMg MDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVzOiBTQ1NJ IFN0YXR1cyBFcnJvcgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRp dGlvbgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1OjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFJFQUQoMTApLiBD REI6IDI4IDAwIDAwIDAwIDAwIDAwIDAwIDAxIDAwIDAwIAooZGE2Om1wcjA6MDoxNDowKTogQ0FN IHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc3RhdHVz OiBDaGVjayBDb25kaXRpb24KKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwg UkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE2Om1w cjA6MDoxNDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTY6bXByMDowOjE0OjAp OiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAwMCAwMCAwMSAwMCAwMCAKKGRhNjptcHIw OjA6MTQ6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTY6bXByMDowOjE0OjAp OiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHNl bnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24g Y29kZSkKKGRhNjptcHIwOjA6MTQ6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE2 Om1wcjA6MDoxNDowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEg MDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTY6bXByMDowOjE0OjApOiBDQU0gc3RhdHVzOiBTQ1NJ IFN0YXR1cyBFcnJvcgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRp dGlvbgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTY6bXByMDowOjE0OjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFJFQUQoMTYpLiBD REI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJjIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE2 Om1wcjA6MDoxNDowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNjptcHIwOjA6 MTQ6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNjptcHIwOjA6MTQ6MCk6IFND U0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJh dGlvbiBjb2RlKQooZGE2Om1wcjA6MDoxNDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTU6bXByMDowOjEzOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAwMCAwMCAw MSAwMCAwMCAKKGRhNTptcHIwOjA6MTM6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9y CihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTU6bXBy MDowOjEzOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQg Y29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNTptcHIwOjA6MTM6MCk6IEVycm9yIDIyLCBVbnJl dHJ5YWJsZSBlcnJvcgooZGE1Om1wcjA6MDoxMzowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAg MDAgMDIgMDAgMDAgMDEgMDAgMDAgCihkYTU6bXByMDowOjEzOjApOiBDQU0gc3RhdHVzOiBTQ1NJ IFN0YXR1cyBFcnJvcgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRp dGlvbgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTU6bXByMDowOjEzOjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFJFQUQoMTYpLiBD REI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJhIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE1 Om1wcjA6MDoxMzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNTptcHIwOjA6 MTM6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNTptcHIwOjA6MTM6MCk6IFND U0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJh dGlvbiBjb2RlKQooZGE1Om1wcjA6MDoxMzowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTU6bXByMDowOjEzOjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBj MCBiYyAwMCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhNTptcHIwOjA6MTM6MCk6IENBTSBzdGF0dXM6 IFNDU0kgU3RhdHVzIEVycm9yCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1Qg YXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNTptcHIwOjA6MTM6 MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjowKTogUkVBRCgx MCkuIENEQjogMjggMDAgMDAgMDAgMDAgMDAgMDAgMDEgMDAgMDAgCihkYTQ6bXByMDowOjEyOjAp OiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBz dGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBzZW5zZTogSUxM RUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihk YTQ6bXByMDowOjEyOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNDptcHIwOjA6 MTI6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAyIDAwIDAwIDAxIDAwIDAwIAooZGE0 Om1wcjA6MDoxMjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNDptcHIwOjA6 MTI6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNDptcHIwOjA6MTI6MCk6IFND U0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJh dGlvbiBjb2RlKQooZGE0Om1wcjA6MDoxMjowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTQ6bXByMDowOjEyOjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBj MCBiYSAwMCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhNDptcHIwOjA6MTI6MCk6IENBTSBzdGF0dXM6 IFNDU0kgU3RhdHVzIEVycm9yCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1Qg YXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNDptcHIwOjA6MTI6 MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjowKTogUkVBRCgx NikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMDAgMDAgMDAgMDEgMDAgMDAgMDAg CihkYTQ6bXByMDowOjEyOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE0Om1w cjA6MDoxMjowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE0Om1wcjA6MDoxMjow KTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQg b3BlcmF0aW9uIGNvZGUpCihkYTQ6bXByMDowOjEyOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAwIDAw IDAwIDAxIDAwIDAwIAooZGEzOm1wcjA6MDoxMTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMg RXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRh MzptcHIwOjA6MTE6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52 YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEzOm1wcjA6MDoxMTowKTogRXJyb3IgMjIs IFVucmV0cnlhYmxlIGVycm9yCihkYTM6bXByMDowOjExOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAw MCAwMCAwMCAwMiAwMCAwMCAwMSAwMCAwMCAKKGRhMzptcHIwOjA6MTE6MCk6IENBTSBzdGF0dXM6 IFNDU0kgU3RhdHVzIEVycm9yCihkYTM6bXByMDowOjExOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTM6bXByMDowOjExOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1Qg YXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMzptcHIwOjA6MTE6 MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEzOm1wcjA6MDoxMTowKTogUkVBRCgx NikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMDAgMDAgMDAgMDEgMDAgMDAgMDAg CihkYTM6bXByMDowOjExOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEzOm1w cjA6MDoxMTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEzOm1wcjA6MDoxMTow KTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQg b3BlcmF0aW9uIGNvZGUpCihkYTM6bXByMDowOjExOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAx IGQxIGMwIGJjIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGEzOm1wcjA6MDoxMTowKTogQ0FNIHN0 YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFNDU0kgc3RhdHVzOiBD aGVjayBDb25kaXRpb24KKGRhMzptcHIwOjA6MTE6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVR VUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEzOm1wcjA6 MDoxMTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTI6bXByMDowOjEwOjApOiBS RUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAwMCAwMCAwMSAwMCAwMCAKKGRhMjptcHIwOjA6 MTA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTI6bXByMDowOjEwOjApOiBT Q1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTI6bXByMDowOjEwOjApOiBTQ1NJIHNlbnNl OiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29k ZSkKKGRhMjptcHIwOjA6MTA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEyOm1w cjA6MDoxMDowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMDAgMDAgMDEgMDAgMDAg CihkYTI6bXByMDowOjEwOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEyOm1w cjA6MDoxMDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEyOm1wcjA6MDoxMDow KTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQg b3BlcmF0aW9uIGNvZGUpCihkYTI6bXByMDowOjEwOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAx IGQxIGMwIGJhIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGEyOm1wcjA6MDoxMDowKTogQ0FNIHN0 YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc3RhdHVzOiBD aGVjayBDb25kaXRpb24KKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVR VUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEyOm1wcjA6 MDoxMDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTI6bXByMDowOjEwOjApOiBS RUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAwMCAwMCAwMCAwMSAwMCAw MCAwMCAKKGRhMjptcHIwOjA6MTA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihk YTI6bXByMDowOjEwOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTI6bXByMDow OjEwOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29t bWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMjptcHIwOjA6MTA6MCk6IEVycm9yIDIyLCBVbnJldHJ5 YWJsZSBlcnJvcgooZGExOm1wcjA6MDo5OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAw MCAwMCAwMCAwMSAwMCAwMCAKKGRhMTptcHIwOjA6OTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0 dXMgRXJyb3IKKGRhMTptcHIwOjA6OTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgoo ZGExOm1wcjA6MDo5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTptcHIwOjA6OTowKTogRXJyb3IgMjIs IFVucmV0cnlhYmxlIGVycm9yCihkYTE6bXByMDowOjk6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAw IDAwIDAwIDAyIDAwIDAwIDAxIDAwIDAwIAooZGExOm1wcjA6MDo5OjApOiBDQU0gc3RhdHVzOiBT Q1NJIFN0YXR1cyBFcnJvcgooZGExOm1wcjA6MDo5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29u ZGl0aW9uCihkYTE6bXByMDowOjk6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6 MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGExOm1wcjA6MDo5OjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTptcHIwOjA6OTowKTogUkVBRCgxNikuIENE QjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTE6 bXByMDowOjk6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTE6bXByMDowOjk6 MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTptcHIwOjA6OTowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTE6bXByMDowOjk6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEx Om1wcjA6MDo5OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAw MCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhMTptcHIwOjA6OTowKTogQ0FNIHN0YXR1czogU0NTSSBT dGF0dXMgRXJyb3IKKGRhMTptcHIwOjA6OTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlv bgooZGExOm1wcjA6MDo5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAg KEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTptcHIwOjA6OTowKTogRXJyb3Ig MjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTA6bXByMDowOjg6MCk6IFJFQUQoMTApLiBDREI6IDI4 IDAwIDAwIDAwIDAwIDAwIDAwIDAxIDAwIDAwIAooZGEwOm1wcjA6MDo4OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBh c2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEwOm1wcjA6MDo4OjAp OiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMDptcHIwOjA6ODowKTogUkVBRCgxMCku IENEQjogMjggMDAgMDAgMDAgMDIgMDAgMDAgMDEgMDAgMDAgCihkYTA6bXByMDowOjg6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc3RhdHVz OiBDaGVjayBDb25kaXRpb24KKGRhMDptcHIwOjA6ODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBS RVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTA6bXBy MDowOjg6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEwOm1wcjA6MDo4OjApOiBS RUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAwMCAwMCAwMCAwMSAwMCAw MCAwMCAKKGRhMDptcHIwOjA6ODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh MDptcHIwOjA6ODowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEwOm1wcjA6MDo4 OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFu ZCBvcGVyYXRpb24gY29kZSkKKGRhMDptcHIwOjA6ODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTA6bXByMDowOjg6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAx IGQxIGMwIGJjIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGEwOm1wcjA6MDo4OjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHN0YXR1czogQ2hl Y2sgQ29uZGl0aW9uCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVT VCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEwOm1wcjA6MDo4 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBSRUFE KDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRhMTE6bXByMDowOjE5 OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMTptcHIwOjA6MTk6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHNlbnNl OiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29k ZSkKKGRhMTE6bXByMDowOjE5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTE6 bXByMDowOjE5OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMiAyMiAwMCAwMSAwMCAw MCAKKGRhMTE6bXByMDowOjE5OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEx MTptcHIwOjA6MTk6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTE6bXByMDow OjE5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29t bWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTE6bXByMDowOjE5OjApOiBFcnJvciAyMiwgVW5yZXRy eWFibGUgZXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAw MCAwMCAwMSBkMSBjMCBiYSAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhMTE6bXByMDowOjE5OjAp OiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMTptcHIwOjA6MTk6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHNlbnNlOiBJ TExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkK KGRhMTE6bXByMDowOjE5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTE6bXBy MDowOjE5OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAyMiAw MCAwMCAwMSAwMCAwMCAwMCAKKGRhMTE6bXByMDowOjE5OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0 YXR1cyBFcnJvcgooZGExMTptcHIwOjA6MTk6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRp b24KKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIw LDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTE6bXByMDowOjE5OjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBSRUFEKDEwKS4g Q0RCOiAyOCAwMCAwMCAwMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRhMTE6bXByMDowOjE5OjApOiBD QU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMTptcHIwOjA6MTk6MCk6IFNDU0kgc3Rh dHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHNlbnNlOiBJTExF R0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRh MTE6bXByMDowOjE5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTE6bXByMDow OjE5OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAyMiAwMCAwMSAwMCAwMCAKKGRh MTE6bXByMDowOjE5OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMTptcHIw OjA6MTk6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTE6bXByMDowOjE5OjAp OiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBv cGVyYXRpb24gY29kZSkKKGRhMTE6bXByMDowOjE5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmYyAy MiAwMCAwMSAwMCAwMCAKKGRhMTE6bXByMDowOjE5OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1 cyBFcnJvcgooZGExMTptcHIwOjA6MTk6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24K KGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAg KEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTE6bXByMDowOjE5OjApOiBFcnJv ciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBSRUFEKDEwKS4gQ0RC OiAyOCAwMCAwMCA5ZiBmZSAyMiAwMCAwMSAwMCAwMCAKKGRhMTE6bXByMDowOjE5OjApOiBDQU0g c3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMTptcHIwOjA6MTk6MCk6IFNDU0kgc3RhdHVz OiBDaGVjayBDb25kaXRpb24KKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTE6 bXByMDowOjE5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTA6bXByMDowOjE4 OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRhMTA6 bXByMDowOjE4OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMDptcHIwOjA6 MTg6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTA6bXByMDowOjE4OjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhMTA6bXByMDowOjE4OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJy b3IKKGRhMTA6bXByMDowOjE4OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMiAyMiAw MCAwMSAwMCAwMCAKKGRhMTA6bXByMDowOjE4OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBF cnJvcgooZGExMDptcHIwOjA6MTg6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRh MTA6bXByMDowOjE4OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTA6bXByMDowOjE4OjApOiBFcnJvciAy MiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBSRUFEKDE2KS4gQ0RCOiA4 OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhMTA6bXBy MDowOjE4OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMDptcHIwOjA6MTg6 MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJ IHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRp b24gY29kZSkKKGRhMTA6bXByMDowOjE4OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IK KGRhMTA6bXByMDowOjE4OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBj MCBiYyAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhMTA6bXByMDowOjE4OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMDptcHIwOjA6MTg6MCk6IFNDU0kgc3RhdHVzOiBDaGVj ayBDb25kaXRpb24KKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVF U1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTA6bXByMDow OjE4OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBS RUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRhMTA6bXByMDow OjE4OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMDptcHIwOjA6MTg6MCk6 IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHNl bnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24g Y29kZSkKKGRhMTA6bXByMDowOjE4OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRh MTA6bXByMDowOjE4OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAyMiAwMCAwMSAw MCAwMCAKKGRhMTA6bXByMDowOjE4OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgoo ZGExMDptcHIwOjA6MTg6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTA6bXBy MDowOjE4OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQg Y29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTA6bXByMDowOjE4OjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAw MCA5ZiBmYyAyMiAwMCAwMSAwMCAwMCAKKGRhMTA6bXByMDowOjE4OjApOiBDQU0gc3RhdHVzOiBT Q1NJIFN0YXR1cyBFcnJvcgooZGExMDptcHIwOjA6MTg6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBD b25kaXRpb24KKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1Qg YXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTA6bXByMDowOjE4 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBSRUFE KDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmZSAyMiAwMCAwMSAwMCAwMCAKKGRhMTA6bXByMDowOjE4 OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExMDptcHIwOjA6MTg6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHNlbnNl OiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29k ZSkKKGRhMTA6bXByMDowOjE4OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhOTpt cHIwOjA6MTc6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIGEwIDAwIDIyIDAwIDAxIDAwIDAw IAooZGE5Om1wcjA6MDoxNzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhOTpt cHIwOjA6MTc6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhOTptcHIwOjA6MTc6 MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5k IG9wZXJhdGlvbiBjb2RlKQooZGE5Om1wcjA6MDoxNzowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMiAy MiAwMCAwMSAwMCAwMCAKKGRhOTptcHIwOjA6MTc6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTk6bXByMDowOjE3OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhOTptcHIwOjA6MTc6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE5Om1wcjA6MDoxNzowKTogUkVBRCgxNikuIENEQjogODgg MDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTk6bXByMDow OjE3OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE5Om1wcjA6MDoxNzowKTog U0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE5Om1wcjA6MDoxNzowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTk6bXByMDowOjE3OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhOTpt cHIwOjA6MTc6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJjIDIy IDAwIDAwIDAxIDAwIDAwIDAwIAooZGE5Om1wcjA6MDoxNzowKTogQ0FNIHN0YXR1czogU0NTSSBT dGF0dXMgRXJyb3IKKGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRp b24KKGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAs MCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE5Om1wcjA6MDoxNzowKTogRXJy b3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDEwKS4gQ0RC OiAyOCAwMCAwMCAwMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRhOTptcHIwOjA6MTc6MCk6IENBTSBz dGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHN0YXR1czog Q2hlY2sgQ29uZGl0aW9uCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJF UVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhOTptcHIw OjA6MTc6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE5Om1wcjA6MDoxNzowKTog UkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTk6bXByMDow OjE3OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE5Om1wcjA6MDoxNzowKTog U0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE5Om1wcjA6MDoxNzowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTk6bXByMDowOjE3OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhOTpt cHIwOjA6MTc6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZjIDIyIDAwIDAxIDAwIDAw IAooZGE5Om1wcjA6MDoxNzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhOTpt cHIwOjA6MTc6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhOTptcHIwOjA6MTc6 MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5k IG9wZXJhdGlvbiBjb2RlKQooZGE5Om1wcjA6MDoxNzowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmZSAy MiAwMCAwMSAwMCAwMCAKKGRhOTptcHIwOjA6MTc6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTk6bXByMDowOjE3OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhOTptcHIwOjA6MTc6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE4Om1wcjA6MDoxNjowKTogUkVBRCgxMCkuIENEQjogMjgg MDAgMDAgYTAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTg6bXByMDowOjE2OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzdGF0dXM6IENoZWNr IENvbmRpdGlvbgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNU IGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTg6bXByMDowOjE2 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhODptcHIwOjA6MTY6MCk6IFJFQUQo MTApLiBDREI6IDI4IDAwIDAwIGEwIDAyIDIyIDAwIDAxIDAwIDAwIAooZGE4Om1wcjA6MDoxNjow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhODptcHIwOjA6MTY6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhODptcHIwOjA6MTY6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE4Om1wcjA6MDoxNjowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTg6bXByMDow OjE2OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAyMiAwMCAw MCAwMSAwMCAwMCAwMCAKKGRhODptcHIwOjA6MTY6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTg6bXByMDowOjE2OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhODptcHIwOjA6MTY6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE4Om1wcjA6MDoxNjowKTogUkVBRCgxNikuIENEQjogODgg MDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTg6bXByMDow OjE2OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE4Om1wcjA6MDoxNjowKTog U0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTg6bXByMDowOjE2OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhODpt cHIwOjA6MTY6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAwIDIyIDAwIDAxIDAwIDAw IAooZGE4Om1wcjA6MDoxNjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhODpt cHIwOjA6MTY6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhODptcHIwOjA6MTY6 MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5k IG9wZXJhdGlvbiBjb2RlKQooZGE4Om1wcjA6MDoxNjowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTg6bXByMDowOjE2OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAy MiAwMCAwMSAwMCAwMCAKKGRhODptcHIwOjA6MTY6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTg6bXByMDowOjE2OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhODptcHIwOjA6MTY6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE4Om1wcjA6MDoxNjowKTogUkVBRCgxMCkuIENEQjogMjgg MDAgMDAgOWYgZmMgMjIgMDAgMDEgMDAgMDAgCihkYTg6bXByMDowOjE2OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzdGF0dXM6IENoZWNr IENvbmRpdGlvbgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNU IGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTg6bXByMDowOjE2 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhODptcHIwOjA6MTY6MCk6IFJFQUQo MTApLiBDREI6IDI4IDAwIDAwIDlmIGZlIDIyIDAwIDAxIDAwIDAwIAooZGE4Om1wcjA6MDoxNjow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhODptcHIwOjA6MTY6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhODptcHIwOjA6MTY6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE4Om1wcjA6MDoxNjowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTc6bXByMDow OjE1OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRh NzptcHIwOjA6MTU6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTc6bXByMDow OjE1OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTc6bXByMDowOjE1OjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhNzptcHIwOjA6MTU6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJv cgooZGE3Om1wcjA6MDoxNTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDIgMjIgMDAg MDEgMDAgMDAgCihkYTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE3Om1w cjA6MDoxNTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1OjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAw IDAwIDAwIDAxIGQxIGMwIGJhIDIyIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE3Om1wcjA6MDoxNTow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE3Om1wcjA6MDoxNTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTc6bXByMDow OjE1OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAyMiAwMCAw MCAwMSAwMCAwMCAwMCAKKGRhNzptcHIwOjA6MTU6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTc6bXByMDowOjE1OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTc6bXByMDowOjE1OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNzptcHIwOjA6MTU6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE3Om1wcjA6MDoxNTowKTogUkVBRCgxMCkuIENEQjogMjgg MDAgMDAgMDAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNr IENvbmRpdGlvbgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNU IGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQo MTApLiBDREI6IDI4IDAwIDAwIDAwIDAyIDIyIDAwIDAxIDAwIDAwIAooZGE3Om1wcjA6MDoxNTow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE3Om1wcjA6MDoxNTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTc6bXByMDow OjE1OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmYyAyMiAwMCAwMSAwMCAwMCAKKGRh NzptcHIwOjA6MTU6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTc6bXByMDow OjE1OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTc6bXByMDowOjE1OjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhNzptcHIwOjA6MTU6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJv cgooZGE3Om1wcjA6MDoxNTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmUgMjIgMDAg MDEgMDAgMDAgCihkYTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE3Om1w cjA6MDoxNTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1OjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAw IGEwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGE2Om1wcjA6MDoxNDowKTogQ0FNIHN0YXR1czogU0NT SSBTdGF0dXMgRXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25k aXRpb24KKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6 MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE2Om1wcjA6MDoxNDowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTY6bXByMDowOjE0OjApOiBSRUFEKDEwKS4g Q0RCOiAyOCAwMCAwMCBhMCAwMiAyMiAwMCAwMSAwMCAwMCAKKGRhNjptcHIwOjA6MTQ6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNjpt cHIwOjA6MTQ6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE2Om1wcjA6MDoxNDow KTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMjIgMDAgMDAgMDEg MDAgMDAgMDAgCihkYTY6bXByMDowOjE0OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE2Om1w cjA6MDoxNDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTY6bXByMDowOjE0OjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAw IDAwIDAwIDAxIGQxIGMwIGJjIDIyIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE2Om1wcjA6MDoxNDow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE2Om1wcjA6MDoxNDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTY6bXByMDow OjE0OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRh NjptcHIwOjA6MTQ6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTY6bXByMDow OjE0OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTY6bXByMDowOjE0OjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhNjptcHIwOjA6MTQ6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJv cgooZGE2Om1wcjA6MDoxNDowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMjIgMDAg MDEgMDAgMDAgCihkYTY6bXByMDowOjE0OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE2Om1w cjA6MDoxNDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTY6bXByMDowOjE0OjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAw IDlmIGZjIDIyIDAwIDAxIDAwIDAwIAooZGE2Om1wcjA6MDoxNDowKTogQ0FNIHN0YXR1czogU0NT SSBTdGF0dXMgRXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25k aXRpb24KKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6 MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE2Om1wcjA6MDoxNDowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTY6bXByMDowOjE0OjApOiBSRUFEKDEwKS4g Q0RCOiAyOCAwMCAwMCA5ZiBmZSAyMiAwMCAwMSAwMCAwMCAKKGRhNjptcHIwOjA6MTQ6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNjpt cHIwOjA6MTQ6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE1Om1wcjA6MDoxMzow KTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTU6bXBy MDowOjEzOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE1Om1wcjA6MDoxMzow KTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTU6bXByMDowOjEzOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRh NTptcHIwOjA6MTM6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIGEwIDAyIDIyIDAwIDAxIDAw IDAwIAooZGE1Om1wcjA6MDoxMzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh NTptcHIwOjA6MTM6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNTptcHIwOjA6 MTM6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGE1Om1wcjA6MDoxMzowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTU6bXByMDowOjEzOjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAw MCAwMSBkMSBjMCBiYSAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhNTptcHIwOjA6MTM6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNTpt cHIwOjA6MTM6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE1Om1wcjA6MDoxMzow KTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMjIgMDAgMDAgMDEg MDAgMDAgMDAgCihkYTU6bXByMDowOjEzOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE1Om1w cjA6MDoxMzowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTU6bXByMDowOjEzOjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAw IDAwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGE1Om1wcjA6MDoxMzowKTogQ0FNIHN0YXR1czogU0NT SSBTdGF0dXMgRXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25k aXRpb24KKGRhNTptcHIwOjA6MTM6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6 MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE1Om1wcjA6MDoxMzowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTU6bXByMDowOjEzOjApOiBSRUFEKDEwKS4g Q0RCOiAyOCAwMCAwMCAwMCAwMiAyMiAwMCAwMSAwMCAwMCAKKGRhNTptcHIwOjA6MTM6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNTpt cHIwOjA6MTM6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE1Om1wcjA6MDoxMzow KTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmMgMjIgMDAgMDEgMDAgMDAgCihkYTU6bXBy MDowOjEzOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE1Om1wcjA6MDoxMzow KTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTU6bXByMDowOjEzOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRh NTptcHIwOjA6MTM6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZlIDIyIDAwIDAxIDAw IDAwIAooZGE1Om1wcjA6MDoxMzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh NTptcHIwOjA6MTM6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNTptcHIwOjA6 MTM6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGE1Om1wcjA6MDoxMzowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTQ6bXByMDowOjEyOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAw MCAyMiAwMCAwMSAwMCAwMCAKKGRhNDptcHIwOjA6MTI6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3Rh dHVzIEVycm9yCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAg KEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNDptcHIwOjA6MTI6MCk6IEVycm9y IDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjowKTogUkVBRCgxMCkuIENEQjog MjggMDAgMDAgYTAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTQ6bXByMDowOjEyOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTQ6bXByMDow OjEyOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNDptcHIwOjA6MTI6MCk6IFJF QUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJhIDIyIDAwIDAwIDAxIDAwIDAw IDAwIAooZGE0Om1wcjA6MDoxMjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh NDptcHIwOjA6MTI6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNDptcHIwOjA6 MTI6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGE0Om1wcjA6MDoxMjowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTQ6bXByMDowOjEyOjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAw MCAwMSBkMSBjMCBiYyAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhNDptcHIwOjA6MTI6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNDpt cHIwOjA6MTI6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjow KTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTQ6bXBy MDowOjEyOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE0Om1wcjA6MDoxMjow KTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTQ6bXByMDowOjEyOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRh NDptcHIwOjA6MTI6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAyIDIyIDAwIDAxIDAw IDAwIAooZGE0Om1wcjA6MDoxMjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh NDptcHIwOjA6MTI6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNDptcHIwOjA6 MTI6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGE0Om1wcjA6MDoxMjowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTQ6bXByMDowOjEyOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBm YyAyMiAwMCAwMSAwMCAwMCAKKGRhNDptcHIwOjA6MTI6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3Rh dHVzIEVycm9yCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAg KEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNDptcHIwOjA6MTI6MCk6IEVycm9y IDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjowKTogUkVBRCgxMCkuIENEQjog MjggMDAgMDAgOWYgZmUgMjIgMDAgMDEgMDAgMDAgCihkYTQ6bXByMDowOjEyOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTQ6bXByMDow OjEyOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFJF QUQoMTApLiBDREI6IDI4IDAwIDAwIGEwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGEzOm1wcjA6MDox MTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMzptcHIwOjA6MTE6MCk6IFNDU0kgc2Vuc2U6 IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2Rl KQooZGEzOm1wcjA6MDoxMTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTM6bXBy MDowOjExOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMiAyMiAwMCAwMSAwMCAwMCAK KGRhMzptcHIwOjA6MTE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTM6bXBy MDowOjExOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTM6bXByMDowOjExOjAp OiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBv cGVyYXRpb24gY29kZSkKKGRhMzptcHIwOjA6MTE6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBl cnJvcgooZGEzOm1wcjA6MDoxMTowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEg ZDEgYzAgYmEgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTM6bXByMDowOjExOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTM6bXByMDow OjExOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFJF QUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJjIDIyIDAwIDAwIDAxIDAwIDAw IDAwIAooZGEzOm1wcjA6MDoxMTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh MzptcHIwOjA6MTE6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMzptcHIwOjA6 MTE6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGEzOm1wcjA6MDoxMTowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTM6bXByMDowOjExOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAw MCAyMiAwMCAwMSAwMCAwMCAKKGRhMzptcHIwOjA6MTE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3Rh dHVzIEVycm9yCihkYTM6bXByMDowOjExOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTM6bXByMDowOjExOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAg KEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMzptcHIwOjA6MTE6MCk6IEVycm9y IDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEzOm1wcjA6MDoxMTowKTogUkVBRCgxMCkuIENEQjog MjggMDAgMDAgMDAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTM6bXByMDowOjExOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTM6bXByMDow OjExOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFJF QUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZjIDIyIDAwIDAxIDAwIDAwIAooZGEzOm1wcjA6MDox MTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMzptcHIwOjA6MTE6MCk6IFNDU0kgc2Vuc2U6 IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2Rl KQooZGEzOm1wcjA6MDoxMTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTM6bXBy MDowOjExOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmZSAyMiAwMCAwMSAwMCAwMCAK KGRhMzptcHIwOjA6MTE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTM6bXBy MDowOjExOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTM6bXByMDowOjExOjAp OiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBv cGVyYXRpb24gY29kZSkKKGRhMzptcHIwOjA6MTE6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBl cnJvcgooZGEyOm1wcjA6MDoxMDowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDAgMjIg MDAgMDEgMDAgMDAgCihkYTI6bXByMDowOjEwOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBF cnJvcgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEy Om1wcjA6MDoxMDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZh bGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTI6bXByMDowOjEwOjApOiBFcnJvciAyMiwg VW5yZXRyeWFibGUgZXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAw IDAwIGEwIDAyIDIyIDAwIDAxIDAwIDAwIAooZGEyOm1wcjA6MDoxMDowKTogQ0FNIHN0YXR1czog U0NTSSBTdGF0dXMgRXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBD b25kaXRpb24KKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBh c2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEyOm1wcjA6MDoxMDow KTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTI6bXByMDowOjEwOjApOiBSRUFEKDE2 KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAyMiAwMCAwMCAwMSAwMCAwMCAwMCAK KGRhMjptcHIwOjA6MTA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTI6bXBy MDowOjEwOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTI6bXByMDowOjEwOjAp OiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBv cGVyYXRpb24gY29kZSkKKGRhMjptcHIwOjA6MTA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBl cnJvcgooZGEyOm1wcjA6MDoxMDowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEg ZDEgYzAgYmMgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTI6bXByMDowOjEwOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTI6bXByMDow OjEwOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFJF QUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGEyOm1wcjA6MDox MDowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc2Vuc2U6 IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2Rl KQooZGEyOm1wcjA6MDoxMDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTI6bXBy MDowOjEwOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAyMiAwMCAwMSAwMCAwMCAK KGRhMjptcHIwOjA6MTA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTI6bXBy MDowOjEwOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTI6bXByMDowOjEwOjAp OiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBv cGVyYXRpb24gY29kZSkKKGRhMjptcHIwOjA6MTA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBl cnJvcgooZGEyOm1wcjA6MDoxMDowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmMgMjIg MDAgMDEgMDAgMDAgCihkYTI6bXByMDowOjEwOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBF cnJvcgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEy Om1wcjA6MDoxMDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZh bGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTI6bXByMDowOjEwOjApOiBFcnJvciAyMiwg VW5yZXRyeWFibGUgZXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAw IDAwIDlmIGZlIDIyIDAwIDAxIDAwIDAwIAooZGEyOm1wcjA6MDoxMDowKTogQ0FNIHN0YXR1czog U0NTSSBTdGF0dXMgRXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBD b25kaXRpb24KKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBh c2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEyOm1wcjA6MDoxMDow KTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTE6bXByMDowOjk6MCk6IFJFQUQoMTAp LiBDREI6IDI4IDAwIDAwIGEwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGExOm1wcjA6MDo5OjApOiBD QU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExOm1wcjA6MDo5OjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTE6bXByMDowOjk6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwg UkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGExOm1w cjA6MDo5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTptcHIwOjA6OTowKTog UkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTE6bXByMDow Ojk6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTE6bXByMDowOjk6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTptcHIwOjA6OTowKTogU0NTSSBzZW5zZTog SUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUp CihkYTE6bXByMDowOjk6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGExOm1wcjA6 MDo5OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAyMiAwMCAw MCAwMSAwMCAwMCAwMCAKKGRhMTptcHIwOjA6OTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMg RXJyb3IKKGRhMTptcHIwOjA6OTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEx Om1wcjA6MDo5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFs aWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTptcHIwOjA6OTowKTogRXJyb3IgMjIsIFVu cmV0cnlhYmxlIGVycm9yCihkYTE6bXByMDowOjk6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAw IDAwIDAwIDAxIGQxIGMwIGJjIDIyIDAwIDAwIDAxIDAwIDAwIDAwIAooZGExOm1wcjA6MDo5OjAp OiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExOm1wcjA6MDo5OjApOiBTQ1NJIHN0 YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTE6bXByMDowOjk6MCk6IFNDU0kgc2Vuc2U6IElMTEVH QUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEx Om1wcjA6MDo5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTptcHIwOjA6OTow KTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTE6bXBy MDowOjk6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTE6bXByMDowOjk6MCk6 IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTptcHIwOjA6OTowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTE6bXByMDowOjk6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGExOm1w cjA6MDo5OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAyMiAwMCAwMSAwMCAwMCAK KGRhMTptcHIwOjA6OTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTptcHIw OjA6OTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGExOm1wcjA6MDo5OjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhMTptcHIwOjA6OTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTE6bXByMDowOjk6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZjIDIyIDAwIDAx IDAwIDAwIAooZGExOm1wcjA6MDo5OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgoo ZGExOm1wcjA6MDo5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTE6bXByMDow Ojk6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGExOm1wcjA6MDo5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFi bGUgZXJyb3IKKGRhMTptcHIwOjA6OTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmUg MjIgMDAgMDEgMDAgMDAgCihkYTE6bXByMDowOjk6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTE6bXByMDowOjk6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRh MTptcHIwOjA6OTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZh bGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTE6bXByMDowOjk6MCk6IEVycm9yIDIyLCBV bnJldHJ5YWJsZSBlcnJvcgooZGEwOm1wcjA6MDo4OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAw MCBhMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRhMDptcHIwOjA6ODowKTogQ0FNIHN0YXR1czogU0NT SSBTdGF0dXMgRXJyb3IKKGRhMDptcHIwOjA6ODowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRp dGlvbgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIw LDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMDptcHIwOjA6ODowKTogRXJy b3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTA6bXByMDowOjg6MCk6IFJFQUQoMTApLiBDREI6 IDI4IDAwIDAwIGEwIDAyIDIyIDAwIDAxIDAwIDAwIAooZGEwOm1wcjA6MDo4OjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHN0YXR1czogQ2hl Y2sgQ29uZGl0aW9uCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVT VCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEwOm1wcjA6MDo4 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMDptcHIwOjA6ODowKTogUkVBRCgx NikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMjIgMDAgMDAgMDEgMDAgMDAgMDAg CihkYTA6bXByMDowOjg6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTA6bXBy MDowOjg6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMDptcHIwOjA6ODowKTog U0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3Bl cmF0aW9uIGNvZGUpCihkYTA6bXByMDowOjg6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJv cgooZGEwOm1wcjA6MDo4OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBj MCBiYyAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhMDptcHIwOjA6ODowKTogQ0FNIHN0YXR1czog U0NTSSBTdGF0dXMgRXJyb3IKKGRhMDptcHIwOjA6ODowKTogU0NTSSBzdGF0dXM6IENoZWNrIENv bmRpdGlvbgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNj OjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMDptcHIwOjA6ODowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTA6bXByMDowOjg6MCk6IFJFQUQoMTApLiBD REI6IDI4IDAwIDAwIDAwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGEwOm1wcjA6MDo4OjApOiBDQU0g c3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHN0YXR1czog Q2hlY2sgQ29uZGl0aW9uCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVR VUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEwOm1wcjA6 MDo4OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMDptcHIwOjA6ODowKTogUkVB RCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTA6bXByMDowOjg6 MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTA6bXByMDowOjg6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMDptcHIwOjA6ODowKTogU0NTSSBzZW5zZTogSUxM RUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihk YTA6bXByMDowOjg6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEwOm1wcjA6MDo4 OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmYyAyMiAwMCAwMSAwMCAwMCAKKGRhMDpt cHIwOjA6ODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMDptcHIwOjA6ODow KTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHNl bnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24g Y29kZSkKKGRhMDptcHIwOjA6ODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTA6 bXByMDowOjg6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZlIDIyIDAwIDAxIDAwIDAw IAooZGEwOm1wcjA6MDo4OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEwOm1w cjA6MDo4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTA6bXByMDowOjg6MCk6 IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9w ZXJhdGlvbiBjb2RlKQooZGEwOm1wcjA6MDo4OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJy b3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIGEwIDAwIDIyIDAw IDAxIDAwIDAwIAooZGE3Om1wcjA6MDoxNTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJy b3IKKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNzpt cHIwOjA6MTU6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxp ZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE3Om1wcjA6MDoxNTowKTogRXJyb3IgMjIsIFVu cmV0cnlhYmxlIGVycm9yCihkYTc6bXByMDowOjE1OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAw MCBhMCAwMiAyMiAwMCAwMSAwMCAwMCAKKGRhNzptcHIwOjA6MTU6MCk6IENBTSBzdGF0dXM6IFND U0kgU3RhdHVzIEVycm9yCihkYTc6bXByMDowOjE1OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29u ZGl0aW9uCihkYTc6bXByMDowOjE1OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNj OjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNzptcHIwOjA6MTU6MCk6 IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE3Om1wcjA6MDoxNTowKTogUkVBRCgxNiku IENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihk YTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE3Om1wcjA6 MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE3Om1wcjA6MDoxNTowKTog U0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3Bl cmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJy b3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQx IGMwIGJjIDIyIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE3Om1wcjA6MDoxNTowKTogQ0FNIHN0YXR1 czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc3RhdHVzOiBDaGVj ayBDb25kaXRpb24KKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVT VCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE3Om1wcjA6MDox NTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTExOm1wcjA6MDoxOTowKTogUkVB RCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMDAgMDAgMDEgMDAgMDAgCihkYTExOm1wcjA6MDox OTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBT Q1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTExOm1wcjA6MDoxOTowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTExOm1wcjA6MDoxOTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEx Om1wcjA6MDoxOTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMDAgMDAgMDEgMDAg MDAgCihkYTExOm1wcjA6MDoxOTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh MTE6bXByMDowOjE5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTExOm1wcjA6 MDoxOTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNv bW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTExOm1wcjA6MDoxOTowKTogRXJyb3IgMjIsIFVucmV0 cnlhYmxlIGVycm9yCihkYTExOm1wcjA6MDoxOTowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAg MDAgMDAgMDEgZDEgYzAgYmEgMDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTExOm1wcjA6MDoxOTow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJ IHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTExOm1wcjA6MDoxOTowKTogU0NTSSBzZW5zZTog SUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUp CihkYTExOm1wcjA6MDoxOTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTExOm1w cjA6MDoxOTowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMDAg MDAgMDAgMDEgMDAgMDAgMDAgCihkYTExOm1wcjA6MDoxOTowKTogQ0FNIHN0YXR1czogU0NTSSBT dGF0dXMgRXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0 aW9uCihkYTExOm1wcjA6MDoxOTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTExOm1wcjA6MDoxOTowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEwOm1wcjA6MDoxODowKTogUkVBRCgxMCku IENEQjogMjggMDAgMDAgMDAgMDAgMDAgMDAgMDEgMDAgMDAgCihkYTEwOm1wcjA6MDoxODowKTog Q0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHN0 YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5zZTogSUxM RUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihk YTEwOm1wcjA6MDoxODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEwOm1wcjA6 MDoxODowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMDAgMDAgMDEgMDAgMDAgCihk YTEwOm1wcjA6MDoxODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTA6bXBy MDowOjE4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTEwOm1wcjA6MDoxODow KTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQg b3BlcmF0aW9uIGNvZGUpCihkYTEwOm1wcjA6MDoxODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTEwOm1wcjA6MDoxODowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAg MDEgZDEgYzAgYmEgMDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTEwOm1wcjA6MDoxODowKTogQ0FN IHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5zZTogSUxMRUdB TCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTEw Om1wcjA6MDoxODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEwOm1wcjA6MDox ODowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMDAgMDAgMDAg MDEgMDAgMDAgMDAgCihkYTEwOm1wcjA6MDoxODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMg RXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJ bnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTEwOm1wcjA6MDoxODowKTogRXJyb3Ig MjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDEwKS4gQ0RCOiAy OCAwMCAwMCAwMCAwMCAwMCAwMCAwMSAwMCAwMCAKKGRhOTptcHIwOjA6MTc6MCk6IENBTSBzdGF0 dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHN0YXR1czogQ2hl Y2sgQ29uZGl0aW9uCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVF U1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhOTptcHIwOjA6 MTc6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE5Om1wcjA6MDoxNzowKTogUkVB RCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMDAgMDAgMDEgMDAgMDAgCihkYTk6bXByMDowOjE3 OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE5Om1wcjA6MDoxNzowKTogU0NT SSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE5Om1wcjA6MDoxNzowKTogU0NTSSBzZW5zZTog SUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUp CihkYTk6bXByMDowOjE3OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhOTptcHIw OjA6MTc6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJhIDAwIDAw IDAwIDAxIDAwIDAwIDAwIAooZGE5Om1wcjA6MDoxNzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0 dXMgRXJyb3IKKGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24K KGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAo SW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE5Om1wcjA6MDoxNzowKTogRXJyb3Ig MjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDE2KS4gQ0RCOiA4 OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAwMCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhOTptcHIw OjA6MTc6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTk6bXByMDowOjE3OjAp OiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHNl bnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24g Y29kZSkKKGRhOTptcHIwOjA6MTc6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE4 Om1wcjA6MDoxNjowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMDAgMDAgMDEgMDAg MDAgCihkYTg6bXByMDowOjE2OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE4 Om1wcjA6MDoxNjowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE4Om1wcjA6MDox NjowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1h bmQgb3BlcmF0aW9uIGNvZGUpCihkYTg6bXByMDowOjE2OjApOiBFcnJvciAyMiwgVW5yZXRyeWFi bGUgZXJyb3IKKGRhODptcHIwOjA6MTY6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAy IDAwIDAwIDAxIDAwIDAwIAooZGE4Om1wcjA6MDoxNjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0 dXMgRXJyb3IKKGRhODptcHIwOjA6MTY6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24K KGRhODptcHIwOjA6MTY6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAo SW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE4Om1wcjA6MDoxNjowKTogRXJyb3Ig MjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTg6bXByMDowOjE2OjApOiBSRUFEKDE2KS4gQ0RCOiA4 OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAwMCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhODptcHIw OjA6MTY6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTg6bXByMDowOjE2OjAp OiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHNl bnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24g Y29kZSkKKGRhODptcHIwOjA6MTY6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE4 Om1wcjA6MDoxNjowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMg MDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTg6bXByMDowOjE2OjApOiBDQU0gc3RhdHVzOiBTQ1NJ IFN0YXR1cyBFcnJvcgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRp dGlvbgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTg6bXByMDowOjE2OjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQoMTApLiBD REI6IDI4IDAwIDAwIDAwIDAwIDAwIDAwIDAxIDAwIDAwIAooZGE3Om1wcjA6MDoxNTowKTogQ0FN IHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc3RhdHVz OiBDaGVjayBDb25kaXRpb24KKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwg UkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE3Om1w cjA6MDoxNTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTc6bXByMDowOjE1OjAp OiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAwMCAwMCAwMSAwMCAwMCAKKGRhNzptcHIw OjA6MTU6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTc6bXByMDowOjE1OjAp OiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTc6bXByMDowOjE1OjApOiBTQ1NJIHNl bnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24g Y29kZSkKKGRhNzptcHIwOjA6MTU6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE3 Om1wcjA6MDoxNTowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEg MDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVzOiBTQ1NJ IFN0YXR1cyBFcnJvcgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRp dGlvbgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1OjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQoMTYpLiBD REI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJjIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE3 Om1wcjA6MDoxNTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNzptcHIwOjA6 MTU6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNzptcHIwOjA6MTU6MCk6IFND U0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJh dGlvbiBjb2RlKQooZGE3Om1wcjA6MDoxNTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTY6bXByMDowOjE0OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAwMCAwMCAw MSAwMCAwMCAKKGRhNjptcHIwOjA6MTQ6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9y CihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTY6bXBy MDowOjE0OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQg Y29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNjptcHIwOjA6MTQ6MCk6IEVycm9yIDIyLCBVbnJl dHJ5YWJsZSBlcnJvcgooZGE2Om1wcjA6MDoxNDowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAg MDAgMDIgMDAgMDAgMDEgMDAgMDAgCihkYTY6bXByMDowOjE0OjApOiBDQU0gc3RhdHVzOiBTQ1NJ IFN0YXR1cyBFcnJvcgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRp dGlvbgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTY6bXByMDowOjE0OjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFJFQUQoMTYpLiBD REI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJhIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE2 Om1wcjA6MDoxNDowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNjptcHIwOjA6 MTQ6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNjptcHIwOjA6MTQ6MCk6IFND U0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJh dGlvbiBjb2RlKQooZGE2Om1wcjA6MDoxNDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTY6bXByMDowOjE0OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBj MCBiYyAwMCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhNjptcHIwOjA6MTQ6MCk6IENBTSBzdGF0dXM6 IFNDU0kgU3RhdHVzIEVycm9yCihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1Qg YXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNjptcHIwOjA6MTQ6 MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE1Om1wcjA6MDoxMzowKTogUkVBRCgx MCkuIENEQjogMjggMDAgMDAgMDAgMDAgMDAgMDAgMDEgMDAgMDAgCihkYTU6bXByMDowOjEzOjAp OiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBz dGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBzZW5zZTogSUxM RUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihk YTU6bXByMDowOjEzOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNTptcHIwOjA6 MTM6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAyIDAwIDAwIDAxIDAwIDAwIAooZGE1 Om1wcjA6MDoxMzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNTptcHIwOjA6 MTM6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNTptcHIwOjA6MTM6MCk6IFND U0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJh dGlvbiBjb2RlKQooZGE1Om1wcjA6MDoxMzowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTU6bXByMDowOjEzOjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBj MCBiYSAwMCAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhNTptcHIwOjA6MTM6MCk6IENBTSBzdGF0dXM6 IFNDU0kgU3RhdHVzIEVycm9yCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1Qg YXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNTptcHIwOjA6MTM6 MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE1Om1wcjA6MDoxMzowKTogUkVBRCgx NikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMDAgMDAgMDAgMDEgMDAgMDAgMDAg CihkYTU6bXByMDowOjEzOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE1Om1w cjA6MDoxMzowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE1Om1wcjA6MDoxMzow KTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQg b3BlcmF0aW9uIGNvZGUpCihkYTU6bXByMDowOjEzOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKKGRhNDptcHIwOjA6MTI6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAwIDAw IDAwIDAxIDAwIDAwIAooZGE0Om1wcjA6MDoxMjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMg RXJyb3IKKGRhNDptcHIwOjA6MTI6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRh NDptcHIwOjA6MTI6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52 YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE0Om1wcjA6MDoxMjowKTogRXJyb3IgMjIs IFVucmV0cnlhYmxlIGVycm9yCihkYTQ6bXByMDowOjEyOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAw MCAwMCAwMCAwMiAwMCAwMCAwMSAwMCAwMCAKKGRhNDptcHIwOjA6MTI6MCk6IENBTSBzdGF0dXM6 IFNDU0kgU3RhdHVzIEVycm9yCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1Qg YXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNDptcHIwOjA6MTI6 MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjowKTogUkVBRCgx NikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMDAgMDAgMDAgMDEgMDAgMDAgMDAg CihkYTQ6bXByMDowOjEyOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE0Om1w cjA6MDoxMjowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE0Om1wcjA6MDoxMjow KTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQg b3BlcmF0aW9uIGNvZGUpCihkYTQ6bXByMDowOjEyOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKKGRhNDptcHIwOjA6MTI6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAx IGQxIGMwIGJjIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE0Om1wcjA6MDoxMjowKTogQ0FNIHN0 YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNDptcHIwOjA6MTI6MCk6IFNDU0kgc3RhdHVzOiBD aGVjayBDb25kaXRpb24KKGRhNDptcHIwOjA6MTI6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVR VUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE0Om1wcjA6 MDoxMjowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTM6bXByMDowOjExOjApOiBS RUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAwMCAwMCAwMSAwMCAwMCAKKGRhMzptcHIwOjA6 MTE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTM6bXByMDowOjExOjApOiBT Q1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTM6bXByMDowOjExOjApOiBTQ1NJIHNlbnNl OiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29k ZSkKKGRhMzptcHIwOjA6MTE6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEzOm1w cjA6MDoxMTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMDAgMDAgMDEgMDAgMDAg CihkYTM6bXByMDowOjExOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEzOm1w cjA6MDoxMTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEzOm1wcjA6MDoxMTow KTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQg b3BlcmF0aW9uIGNvZGUpCihkYTM6bXByMDowOjExOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAx IGQxIGMwIGJhIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGEzOm1wcjA6MDoxMTowKTogQ0FNIHN0 YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFNDU0kgc3RhdHVzOiBD aGVjayBDb25kaXRpb24KKGRhMzptcHIwOjA6MTE6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVR VUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEzOm1wcjA6 MDoxMTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTM6bXByMDowOjExOjApOiBS RUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAwMCAwMCAwMCAwMSAwMCAw MCAwMCAKKGRhMzptcHIwOjA6MTE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihk YTM6bXByMDowOjExOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTM6bXByMDow OjExOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29t bWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMzptcHIwOjA6MTE6MCk6IEVycm9yIDIyLCBVbnJldHJ5 YWJsZSBlcnJvcgooZGEyOm1wcjA6MDoxMDowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAg MDAgMDAgMDAgMDEgMDAgMDAgCihkYTI6bXByMDowOjEwOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0 YXR1cyBFcnJvcgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlv bgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCww IChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTI6bXByMDowOjEwOjApOiBFcnJv ciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFJFQUQoMTApLiBDREI6 IDI4IDAwIDAwIDAwIDAyIDAwIDAwIDAxIDAwIDAwIAooZGEyOm1wcjA6MDoxMDowKTogQ0FNIHN0 YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc3RhdHVzOiBD aGVjayBDb25kaXRpb24KKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVR VUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEyOm1wcjA6 MDoxMDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTI6bXByMDowOjEwOjApOiBS RUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAwMCAwMCAwMCAwMSAwMCAw MCAwMCAKKGRhMjptcHIwOjA6MTA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihk YTI6bXByMDowOjEwOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTI6bXByMDow OjEwOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29t bWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMjptcHIwOjA6MTA6MCk6IEVycm9yIDIyLCBVbnJldHJ5 YWJsZSBlcnJvcgooZGEyOm1wcjA6MDoxMDowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAg MDAgMDEgZDEgYzAgYmMgMDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTI6bXByMDowOjEwOjApOiBD QU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzdGF0 dXM6IENoZWNrIENvbmRpdGlvbgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzZW5zZTogSUxMRUdB TCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTI6 bXByMDowOjEwOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTptcHIwOjA6OTow KTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMDAgMDAgMDEgMDAgMDAgCihkYTE6bXBy MDowOjk6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTE6bXByMDowOjk6MCk6 IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTptcHIwOjA6OTowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTE6bXByMDowOjk6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGExOm1w cjA6MDo5OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAwMCAwMCAwMSAwMCAwMCAK KGRhMTptcHIwOjA6OTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTptcHIw OjA6OTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGExOm1wcjA6MDo5OjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhMTptcHIwOjA6OTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTE6bXByMDowOjk6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMw IGJhIDAwIDAwIDAwIDAxIDAwIDAwIDAwIAooZGExOm1wcjA6MDo5OjApOiBDQU0gc3RhdHVzOiBT Q1NJIFN0YXR1cyBFcnJvcgooZGExOm1wcjA6MDo5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29u ZGl0aW9uCihkYTE6bXByMDowOjk6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6 MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGExOm1wcjA6MDo5OjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTptcHIwOjA6OTowKTogUkVBRCgxNikuIENE QjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTE6 bXByMDowOjk6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTE6bXByMDowOjk6 MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTptcHIwOjA6OTowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTE6bXByMDowOjk6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEw Om1wcjA6MDo4OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAwMCAwMCAwMSAwMCAw MCAKKGRhMDptcHIwOjA6ODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMDpt cHIwOjA6ODowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEwOm1wcjA6MDo4OjAp OiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBv cGVyYXRpb24gY29kZSkKKGRhMDptcHIwOjA6ODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVy cm9yCihkYTA6bXByMDowOjg6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAyIDAwIDAw IDAxIDAwIDAwIAooZGEwOm1wcjA6MDo4OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTA6bXBy MDowOjg6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBj b21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEwOm1wcjA6MDo4OjApOiBFcnJvciAyMiwgVW5yZXRy eWFibGUgZXJyb3IKKGRhMDptcHIwOjA6ODowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAg MDAgMDEgZDEgYzAgYmEgMDAgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTA6bXByMDowOjg6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc3RhdHVz OiBDaGVjayBDb25kaXRpb24KKGRhMDptcHIwOjA6ODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBS RVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTA6bXBy MDowOjg6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEwOm1wcjA6MDo4OjApOiBS RUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAwMCAwMCAwMCAwMSAwMCAw MCAwMCAKKGRhMDptcHIwOjA6ODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh MDptcHIwOjA6ODowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEwOm1wcjA6MDo4 OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFu ZCBvcGVyYXRpb24gY29kZSkKKGRhMDptcHIwOjA6ODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTExOm1wcjA6MDoxOTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDAg MjIgMDAgMDEgMDAgMDAgCihkYTExOm1wcjA6MDoxOTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0 dXMgRXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTExOm1wcjA6MDoxOTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCww IChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTExOm1wcjA6MDoxOTowKTogRXJy b3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTExOm1wcjA6MDoxOTowKTogUkVBRCgxMCkuIENE QjogMjggMDAgMDAgYTAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTExOm1wcjA6MDoxOTowKTogQ0FN IHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTExOm1wcjA6MDoxOTowKTogU0NTSSBzZW5zZTogSUxMRUdB TCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTEx Om1wcjA6MDoxOTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTExOm1wcjA6MDox OTowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMjIgMDAgMDAg MDEgMDAgMDAgMDAgCihkYTExOm1wcjA6MDoxOTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMg RXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTExOm1wcjA6MDoxOTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJ bnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTExOm1wcjA6MDoxOTowKTogRXJyb3Ig MjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTExOm1wcjA6MDoxOTowKTogUkVBRCgxNikuIENEQjog ODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTExOm1w cjA6MDoxOTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTE6bXByMDowOjE5 OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTExOm1wcjA6MDoxOTowKTogU0NT SSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0 aW9uIGNvZGUpCihkYTExOm1wcjA6MDoxOTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihkYTExOm1wcjA6MDoxOTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMjIgMDAg MDEgMDAgMDAgCihkYTExOm1wcjA6MDoxOTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJy b3IKKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTEx Om1wcjA6MDoxOTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZh bGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTExOm1wcjA6MDoxOTowKTogRXJyb3IgMjIs IFVucmV0cnlhYmxlIGVycm9yCihkYTExOm1wcjA6MDoxOTowKTogUkVBRCgxMCkuIENEQjogMjgg MDAgMDAgMDAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTExOm1wcjA6MDoxOTowKTogQ0FNIHN0YXR1 czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHN0YXR1czogQ2hl Y2sgQ29uZGl0aW9uCihkYTExOm1wcjA6MDoxOTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTExOm1wcjA6 MDoxOTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTExOm1wcjA6MDoxOTowKTog UkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmMgMjIgMDAgMDEgMDAgMDAgCihkYTExOm1wcjA6 MDoxOTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTE6bXByMDowOjE5OjAp OiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTExOm1wcjA6MDoxOTowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTExOm1wcjA6MDoxOTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihk YTExOm1wcjA6MDoxOTowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmUgMjIgMDAgMDEg MDAgMDAgCihkYTExOm1wcjA6MDoxOTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IK KGRhMTE6bXByMDowOjE5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTExOm1w cjA6MDoxOTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTExOm1wcjA6MDoxOTowKTogRXJyb3IgMjIsIFVu cmV0cnlhYmxlIGVycm9yCihkYTEwOm1wcjA6MDoxODowKTogUkVBRCgxMCkuIENEQjogMjggMDAg MDAgYTAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTEwOm1wcjA6MDoxODowKTogQ0FNIHN0YXR1czog U0NTSSBTdGF0dXMgRXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNU IGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTEwOm1wcjA6MDox ODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEwOm1wcjA6MDoxODowKTogUkVB RCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTEwOm1wcjA6MDox ODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBT Q1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTEwOm1wcjA6MDoxODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEw Om1wcjA6MDoxODowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEg MjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTEwOm1wcjA6MDoxODowKTogQ0FNIHN0YXR1czogU0NT SSBTdGF0dXMgRXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29u ZGl0aW9uCihkYTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFz YzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTEwOm1wcjA6MDoxODow KTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEwOm1wcjA6MDoxODowKTogUkVBRCgx NikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMjIgMDAgMDAgMDEgMDAgMDAgMDAg CihkYTEwOm1wcjA6MDoxODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTA6 bXByMDowOjE4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTEwOm1wcjA6MDox ODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1h bmQgb3BlcmF0aW9uIGNvZGUpCihkYTEwOm1wcjA6MDoxODowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTEwOm1wcjA6MDoxODowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAg MDAgMjIgMDAgMDEgMDAgMDAgCihkYTEwOm1wcjA6MDoxODowKTogQ0FNIHN0YXR1czogU0NTSSBT dGF0dXMgRXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0 aW9uCihkYTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTEwOm1wcjA6MDoxODowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEwOm1wcjA6MDoxODowKTogUkVBRCgxMCku IENEQjogMjggMDAgMDAgMDAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTEwOm1wcjA6MDoxODowKTog Q0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHN0 YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5zZTogSUxM RUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihk YTEwOm1wcjA6MDoxODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTEwOm1wcjA6 MDoxODowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmMgMjIgMDAgMDEgMDAgMDAgCihk YTEwOm1wcjA6MDoxODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTA6bXBy MDowOjE4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTEwOm1wcjA6MDoxODow KTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQg b3BlcmF0aW9uIGNvZGUpCihkYTEwOm1wcjA6MDoxODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTEwOm1wcjA6MDoxODowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmUg MjIgMDAgMDEgMDAgMDAgCihkYTEwOm1wcjA6MDoxODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0 dXMgRXJyb3IKKGRhMTA6bXByMDowOjE4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTEwOm1wcjA6MDoxODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCww IChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTEwOm1wcjA6MDoxODowKTogRXJy b3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDEwKS4gQ0RC OiAyOCAwMCAwMCBhMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRhOTptcHIwOjA6MTc6MCk6IENBTSBz dGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHN0YXR1czog Q2hlY2sgQ29uZGl0aW9uCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJF UVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhOTptcHIw OjA6MTc6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE5Om1wcjA6MDoxNzowKTog UkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTk6bXByMDow OjE3OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE5Om1wcjA6MDoxNzowKTog U0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE5Om1wcjA6MDoxNzowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTk6bXByMDowOjE3OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhOTpt cHIwOjA6MTc6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJhIDIy IDAwIDAwIDAxIDAwIDAwIDAwIAooZGE5Om1wcjA6MDoxNzowKTogQ0FNIHN0YXR1czogU0NTSSBT dGF0dXMgRXJyb3IKKGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRp b24KKGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAs MCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE5Om1wcjA6MDoxNzowKTogRXJy b3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDE2KS4gQ0RC OiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhOTpt cHIwOjA6MTc6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTk6bXByMDowOjE3 OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTk6bXByMDowOjE3OjApOiBTQ1NJ IHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRp b24gY29kZSkKKGRhOTptcHIwOjA6MTc6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgoo ZGE5Om1wcjA6MDoxNzowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMjIgMDAgMDEg MDAgMDAgCihkYTk6bXByMDowOjE3OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgoo ZGE5Om1wcjA6MDoxNzowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE5Om1wcjA6 MDoxNzowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNv bW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTk6bXByMDowOjE3OjApOiBFcnJvciAyMiwgVW5yZXRy eWFibGUgZXJyb3IKKGRhOTptcHIwOjA6MTc6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAw IDAyIDIyIDAwIDAxIDAwIDAwIAooZGE5Om1wcjA6MDoxNzowKTogQ0FNIHN0YXR1czogU0NTSSBT dGF0dXMgRXJyb3IKKGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRp b24KKGRhOTptcHIwOjA6MTc6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAs MCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE5Om1wcjA6MDoxNzowKTogRXJy b3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTk6bXByMDowOjE3OjApOiBSRUFEKDEwKS4gQ0RC OiAyOCAwMCAwMCA5ZiBmYyAyMiAwMCAwMSAwMCAwMCAKKGRhOTptcHIwOjA6MTc6MCk6IENBTSBz dGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHN0YXR1czog Q2hlY2sgQ29uZGl0aW9uCihkYTk6bXByMDowOjE3OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJF UVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhOTptcHIw OjA6MTc6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE5Om1wcjA6MDoxNzowKTog UkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmUgMjIgMDAgMDEgMDAgMDAgCihkYTk6bXByMDow OjE3OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE5Om1wcjA6MDoxNzowKTog U0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE5Om1wcjA6MDoxNzowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTk6bXByMDowOjE3OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhODpt cHIwOjA6MTY6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIGEwIDAwIDIyIDAwIDAxIDAwIDAw IAooZGE4Om1wcjA6MDoxNjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhODpt cHIwOjA6MTY6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhODptcHIwOjA6MTY6 MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5k IG9wZXJhdGlvbiBjb2RlKQooZGE4Om1wcjA6MDoxNjowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTg6bXByMDowOjE2OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMiAy MiAwMCAwMSAwMCAwMCAKKGRhODptcHIwOjA6MTY6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTg6bXByMDowOjE2OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhODptcHIwOjA6MTY6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE4Om1wcjA6MDoxNjowKTogUkVBRCgxNikuIENEQjogODgg MDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTg6bXByMDow OjE2OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE4Om1wcjA6MDoxNjowKTog U0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTg6bXByMDowOjE2OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhODpt cHIwOjA6MTY6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJjIDIy IDAwIDAwIDAxIDAwIDAwIDAwIAooZGE4Om1wcjA6MDoxNjowKTogQ0FNIHN0YXR1czogU0NTSSBT dGF0dXMgRXJyb3IKKGRhODptcHIwOjA6MTY6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRp b24KKGRhODptcHIwOjA6MTY6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAs MCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE4Om1wcjA6MDoxNjowKTogRXJy b3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTg6bXByMDowOjE2OjApOiBSRUFEKDEwKS4gQ0RC OiAyOCAwMCAwMCAwMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRhODptcHIwOjA6MTY6MCk6IENBTSBz dGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHN0YXR1czog Q2hlY2sgQ29uZGl0aW9uCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJF UVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhODptcHIw OjA6MTY6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE4Om1wcjA6MDoxNjowKTog UkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTg6bXByMDow OjE2OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE4Om1wcjA6MDoxNjowKTog U0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE4Om1wcjA6MDoxNjowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTg6bXByMDowOjE2OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhODpt cHIwOjA6MTY6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZjIDIyIDAwIDAxIDAwIDAw IAooZGE4Om1wcjA6MDoxNjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhODpt cHIwOjA6MTY6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhODptcHIwOjA6MTY6 MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5k IG9wZXJhdGlvbiBjb2RlKQooZGE4Om1wcjA6MDoxNjowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTg6bXByMDowOjE2OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmZSAy MiAwMCAwMSAwMCAwMCAKKGRhODptcHIwOjA6MTY6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTg6bXByMDowOjE2OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTg6bXByMDowOjE2OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhODptcHIwOjA6MTY6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE3Om1wcjA6MDoxNTowKTogUkVBRCgxMCkuIENEQjogMjgg MDAgMDAgYTAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNr IENvbmRpdGlvbgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNU IGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQo MTApLiBDREI6IDI4IDAwIDAwIGEwIDAyIDIyIDAwIDAxIDAwIDAwIAooZGE3Om1wcjA6MDoxNTow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE3Om1wcjA6MDoxNTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTc6bXByMDow OjE1OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYSAyMiAwMCAw MCAwMSAwMCAwMCAwMCAKKGRhNzptcHIwOjA6MTU6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTc6bXByMDowOjE1OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTc6bXByMDowOjE1OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNzptcHIwOjA6MTU6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE3Om1wcjA6MDoxNTowKTogUkVBRCgxNikuIENEQjogODgg MDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTc6bXByMDow OjE1OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE3Om1wcjA6MDoxNTowKTog U0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzZW5z ZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNv ZGUpCihkYTc6bXByMDowOjE1OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNzpt cHIwOjA6MTU6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAwIDIyIDAwIDAxIDAwIDAw IAooZGE3Om1wcjA6MDoxNTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNzpt cHIwOjA6MTU6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNzptcHIwOjA6MTU6 MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5k IG9wZXJhdGlvbiBjb2RlKQooZGE3Om1wcjA6MDoxNTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTc6bXByMDowOjE1OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMiAy MiAwMCAwMSAwMCAwMCAKKGRhNzptcHIwOjA6MTU6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTc6bXByMDowOjE1OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTc6bXByMDowOjE1OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNzptcHIwOjA6MTU6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE3Om1wcjA6MDoxNTowKTogUkVBRCgxMCkuIENEQjogMjgg MDAgMDAgOWYgZmMgMjIgMDAgMDEgMDAgMDAgCihkYTc6bXByMDowOjE1OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzdGF0dXM6IENoZWNr IENvbmRpdGlvbgooZGE3Om1wcjA6MDoxNTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNU IGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTc6bXByMDowOjE1 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFJFQUQo MTApLiBDREI6IDI4IDAwIDAwIDlmIGZlIDIyIDAwIDAxIDAwIDAwIAooZGE3Om1wcjA6MDoxNTow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNzptcHIwOjA6MTU6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE3Om1wcjA6MDoxNTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTY6bXByMDow OjE0OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRh NjptcHIwOjA6MTQ6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTY6bXByMDow OjE0OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTY6bXByMDowOjE0OjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhNjptcHIwOjA6MTQ6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJv cgooZGE2Om1wcjA6MDoxNDowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDIgMjIgMDAg MDEgMDAgMDAgCihkYTY6bXByMDowOjE0OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE2Om1w cjA6MDoxNDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTY6bXByMDowOjE0OjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAw IDAwIDAwIDAxIGQxIGMwIGJhIDIyIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE2Om1wcjA6MDoxNDow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE2Om1wcjA6MDoxNDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTY6bXByMDow OjE0OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAyMiAwMCAw MCAwMSAwMCAwMCAwMCAKKGRhNjptcHIwOjA6MTQ6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVz IEVycm9yCihkYTY6bXByMDowOjE0OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTY6bXByMDowOjE0OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKElu dmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNjptcHIwOjA6MTQ6MCk6IEVycm9yIDIy LCBVbnJldHJ5YWJsZSBlcnJvcgooZGE2Om1wcjA6MDoxNDowKTogUkVBRCgxMCkuIENEQjogMjgg MDAgMDAgMDAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTY6bXByMDowOjE0OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzdGF0dXM6IENoZWNr IENvbmRpdGlvbgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNU IGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTY6bXByMDowOjE0 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFJFQUQo MTApLiBDREI6IDI4IDAwIDAwIDAwIDAyIDIyIDAwIDAxIDAwIDAwIAooZGE2Om1wcjA6MDoxNDow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNjptcHIwOjA6MTQ6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE2Om1wcjA6MDoxNDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTY6bXByMDow OjE0OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmYyAyMiAwMCAwMSAwMCAwMCAKKGRh NjptcHIwOjA6MTQ6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTY6bXByMDow OjE0OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTY6bXByMDowOjE0OjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhNjptcHIwOjA6MTQ6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJv cgooZGE2Om1wcjA6MDoxNDowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmUgMjIgMDAg MDEgMDAgMDAgCihkYTY6bXByMDowOjE0OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE2Om1wcjA6MDoxNDowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE2Om1w cjA6MDoxNDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTY6bXByMDowOjE0OjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAw IGEwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGE1Om1wcjA6MDoxMzowKTogQ0FNIHN0YXR1czogU0NT SSBTdGF0dXMgRXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25k aXRpb24KKGRhNTptcHIwOjA6MTM6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6 MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE1Om1wcjA6MDoxMzowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTU6bXByMDowOjEzOjApOiBSRUFEKDEwKS4g Q0RCOiAyOCAwMCAwMCBhMCAwMiAyMiAwMCAwMSAwMCAwMCAKKGRhNTptcHIwOjA6MTM6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNTpt cHIwOjA6MTM6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE1Om1wcjA6MDoxMzow KTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMjIgMDAgMDAgMDEg MDAgMDAgMDAgCihkYTU6bXByMDowOjEzOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE1Om1w cjA6MDoxMzowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTU6bXByMDowOjEzOjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAw IDAwIDAwIDAxIGQxIGMwIGJjIDIyIDAwIDAwIDAxIDAwIDAwIDAwIAooZGE1Om1wcjA6MDoxMzow KTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFNDU0kg c3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNTptcHIwOjA6MTM6MCk6IFNDU0kgc2Vuc2U6IElM TEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQoo ZGE1Om1wcjA6MDoxMzowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTU6bXByMDow OjEzOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAyMiAwMCAwMSAwMCAwMCAKKGRh NTptcHIwOjA6MTM6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTU6bXByMDow OjEzOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTU6bXByMDowOjEzOjApOiBT Q1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVy YXRpb24gY29kZSkKKGRhNTptcHIwOjA6MTM6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJv cgooZGE1Om1wcjA6MDoxMzowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDIgMjIgMDAg MDEgMDAgMDAgCihkYTU6bXByMDowOjEzOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE1Om1wcjA6MDoxMzowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE1Om1w cjA6MDoxMzowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTU6bXByMDowOjEzOjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAw IDlmIGZjIDIyIDAwIDAxIDAwIDAwIAooZGE1Om1wcjA6MDoxMzowKTogQ0FNIHN0YXR1czogU0NT SSBTdGF0dXMgRXJyb3IKKGRhNTptcHIwOjA6MTM6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25k aXRpb24KKGRhNTptcHIwOjA6MTM6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6 MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE1Om1wcjA6MDoxMzowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTU6bXByMDowOjEzOjApOiBSRUFEKDEwKS4g Q0RCOiAyOCAwMCAwMCA5ZiBmZSAyMiAwMCAwMSAwMCAwMCAKKGRhNTptcHIwOjA6MTM6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTU6bXByMDowOjEzOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNTpt cHIwOjA6MTM6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjow KTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTQ6bXBy MDowOjEyOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE0Om1wcjA6MDoxMjow KTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTQ6bXByMDowOjEyOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRh NDptcHIwOjA6MTI6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIGEwIDAyIDIyIDAwIDAxIDAw IDAwIAooZGE0Om1wcjA6MDoxMjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh NDptcHIwOjA6MTI6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNDptcHIwOjA6 MTI6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGE0Om1wcjA6MDoxMjowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTQ6bXByMDowOjEyOjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAw MCAwMSBkMSBjMCBiYSAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhNDptcHIwOjA6MTI6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNDpt cHIwOjA6MTI6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjow KTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMjIgMDAgMDAgMDEg MDAgMDAgMDAgCihkYTQ6bXByMDowOjEyOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJv cgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE0Om1w cjA6MDoxMjowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlk IGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTQ6bXByMDowOjEyOjApOiBFcnJvciAyMiwgVW5y ZXRyeWFibGUgZXJyb3IKKGRhNDptcHIwOjA6MTI6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAw IDAwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGE0Om1wcjA6MDoxMjowKTogQ0FNIHN0YXR1czogU0NT SSBTdGF0dXMgRXJyb3IKKGRhNDptcHIwOjA6MTI6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25k aXRpb24KKGRhNDptcHIwOjA6MTI6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6 MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGE0Om1wcjA6MDoxMjowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTQ6bXByMDowOjEyOjApOiBSRUFEKDEwKS4g Q0RCOiAyOCAwMCAwMCAwMCAwMiAyMiAwMCAwMSAwMCAwMCAKKGRhNDptcHIwOjA6MTI6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTQ6bXByMDowOjEyOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhNDpt cHIwOjA6MTI6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGE0Om1wcjA6MDoxMjow KTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgOWYgZmMgMjIgMDAgMDEgMDAgMDAgCihkYTQ6bXBy MDowOjEyOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGE0Om1wcjA6MDoxMjow KTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGE0Om1wcjA6MDoxMjowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTQ6bXByMDowOjEyOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRh NDptcHIwOjA6MTI6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZlIDIyIDAwIDAxIDAw IDAwIAooZGE0Om1wcjA6MDoxMjowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh NDptcHIwOjA6MTI6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhNDptcHIwOjA6 MTI6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGE0Om1wcjA6MDoxMjowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTM6bXByMDowOjExOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAw MCAyMiAwMCAwMSAwMCAwMCAKKGRhMzptcHIwOjA6MTE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3Rh dHVzIEVycm9yCihkYTM6bXByMDowOjExOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTM6bXByMDowOjExOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAg KEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMzptcHIwOjA6MTE6MCk6IEVycm9y IDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEzOm1wcjA6MDoxMTowKTogUkVBRCgxMCkuIENEQjog MjggMDAgMDAgYTAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTM6bXByMDowOjExOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTM6bXByMDow OjExOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMzptcHIwOjA6MTE6MCk6IFJF QUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJhIDIyIDAwIDAwIDAxIDAwIDAw IDAwIAooZGEzOm1wcjA6MDoxMTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh MzptcHIwOjA6MTE6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMzptcHIwOjA6 MTE6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGEzOm1wcjA6MDoxMTowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTM6bXByMDowOjExOjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAw MCAwMSBkMSBjMCBiYyAyMiAwMCAwMCAwMSAwMCAwMCAwMCAKKGRhMzptcHIwOjA6MTE6MCk6IENB TSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTM6bXByMDowOjExOjApOiBTQ1NJIHN0YXR1 czogQ2hlY2sgQ29uZGl0aW9uCihkYTM6bXByMDowOjExOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMzpt cHIwOjA6MTE6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEzOm1wcjA6MDoxMTow KTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgMDAgMDAgMjIgMDAgMDEgMDAgMDAgCihkYTM6bXBy MDowOjExOjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEzOm1wcjA6MDoxMTow KTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBz ZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9u IGNvZGUpCihkYTM6bXByMDowOjExOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRh MzptcHIwOjA6MTE6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAyIDIyIDAwIDAxIDAw IDAwIAooZGEzOm1wcjA6MDoxMTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh MzptcHIwOjA6MTE6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMzptcHIwOjA6 MTE6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGEzOm1wcjA6MDoxMTowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTM6bXByMDowOjExOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBm YyAyMiAwMCAwMSAwMCAwMCAKKGRhMzptcHIwOjA6MTE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3Rh dHVzIEVycm9yCihkYTM6bXByMDowOjExOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTM6bXByMDowOjExOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAg KEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMzptcHIwOjA6MTE6MCk6IEVycm9y IDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEzOm1wcjA6MDoxMTowKTogUkVBRCgxMCkuIENEQjog MjggMDAgMDAgOWYgZmUgMjIgMDAgMDEgMDAgMDAgCihkYTM6bXByMDowOjExOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGEzOm1wcjA6MDoxMTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTM6bXByMDow OjExOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFJF QUQoMTApLiBDREI6IDI4IDAwIDAwIGEwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGEyOm1wcjA6MDox MDowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc2Vuc2U6 IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2Rl KQooZGEyOm1wcjA6MDoxMDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTI6bXBy MDowOjEwOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMiAyMiAwMCAwMSAwMCAwMCAK KGRhMjptcHIwOjA6MTA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTI6bXBy MDowOjEwOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTI6bXByMDowOjEwOjAp OiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBv cGVyYXRpb24gY29kZSkKKGRhMjptcHIwOjA6MTA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBl cnJvcgooZGEyOm1wcjA6MDoxMDowKTogUkVBRCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDEg ZDEgYzAgYmEgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTI6bXByMDowOjEwOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTI6bXByMDow OjEwOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFJF QUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAxIGQxIGMwIGJjIDIyIDAwIDAwIDAxIDAwIDAw IDAwIAooZGEyOm1wcjA6MDoxMDowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh MjptcHIwOjA6MTA6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMjptcHIwOjA6 MTA6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21t YW5kIG9wZXJhdGlvbiBjb2RlKQooZGEyOm1wcjA6MDoxMDowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTI6bXByMDowOjEwOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAw MCAyMiAwMCAwMSAwMCAwMCAKKGRhMjptcHIwOjA6MTA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3Rh dHVzIEVycm9yCihkYTI6bXByMDowOjEwOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTI6bXByMDowOjEwOjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAg KEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMjptcHIwOjA6MTA6MCk6IEVycm9y IDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEyOm1wcjA6MDoxMDowKTogUkVBRCgxMCkuIENEQjog MjggMDAgMDAgMDAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTI6bXByMDowOjEwOjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGEyOm1wcjA6MDoxMDowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFV RVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTI6bXByMDow OjEwOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFJF QUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZjIDIyIDAwIDAxIDAwIDAwIAooZGEyOm1wcjA6MDox MDowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMjptcHIwOjA6MTA6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMjptcHIwOjA6MTA6MCk6IFNDU0kgc2Vuc2U6 IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2Rl KQooZGEyOm1wcjA6MDoxMDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTI6bXBy MDowOjEwOjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCA5ZiBmZSAyMiAwMCAwMSAwMCAwMCAK KGRhMjptcHIwOjA6MTA6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTI6bXBy MDowOjEwOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTI6bXByMDowOjEwOjAp OiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBv cGVyYXRpb24gY29kZSkKKGRhMjptcHIwOjA6MTA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBl cnJvcgooZGExOm1wcjA6MDo5OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMCAyMiAw MCAwMSAwMCAwMCAKKGRhMTptcHIwOjA6OTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJy b3IKKGRhMTptcHIwOjA6OTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGExOm1w cjA6MDo5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQg Y29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTptcHIwOjA6OTowKTogRXJyb3IgMjIsIFVucmV0 cnlhYmxlIGVycm9yCihkYTE6bXByMDowOjk6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIGEw IDAyIDIyIDAwIDAxIDAwIDAwIAooZGExOm1wcjA6MDo5OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0 YXR1cyBFcnJvcgooZGExOm1wcjA6MDo5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9u CihkYTE6bXByMDowOjk6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAo SW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGExOm1wcjA6MDo5OjApOiBFcnJvciAy MiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTptcHIwOjA6OTowKTogUkVBRCgxNikuIENEQjogODgg MDAgMDAgMDAgMDAgMDEgZDEgYzAgYmEgMjIgMDAgMDAgMDEgMDAgMDAgMDAgCihkYTE6bXByMDow Ojk6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTE6bXByMDowOjk6MCk6IFND U0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMTptcHIwOjA6OTowKTogU0NTSSBzZW5zZTog SUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUp CihkYTE6bXByMDowOjk6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGExOm1wcjA6 MDo5OjApOiBSRUFEKDE2KS4gQ0RCOiA4OCAwMCAwMCAwMCAwMCAwMSBkMSBjMCBiYyAyMiAwMCAw MCAwMSAwMCAwMCAwMCAKKGRhMTptcHIwOjA6OTowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMg RXJyb3IKKGRhMTptcHIwOjA6OTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEx Om1wcjA6MDo5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFs aWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTptcHIwOjA6OTowKTogRXJyb3IgMjIsIFVu cmV0cnlhYmxlIGVycm9yCihkYTE6bXByMDowOjk6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAw IDAwIDAwIDIyIDAwIDAxIDAwIDAwIAooZGExOm1wcjA6MDo5OjApOiBDQU0gc3RhdHVzOiBTQ1NJ IFN0YXR1cyBFcnJvcgooZGExOm1wcjA6MDo5OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0 aW9uCihkYTE6bXByMDowOjk6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAs MCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGExOm1wcjA6MDo5OjApOiBFcnJv ciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMTptcHIwOjA6OTowKTogUkVBRCgxMCkuIENEQjog MjggMDAgMDAgMDAgMDIgMjIgMDAgMDEgMDAgMDAgCihkYTE6bXByMDowOjk6MCk6IENBTSBzdGF0 dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTE6bXByMDowOjk6MCk6IFNDU0kgc3RhdHVzOiBDaGVj ayBDb25kaXRpb24KKGRhMTptcHIwOjA6OTowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNU IGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTE6bXByMDowOjk6 MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGExOm1wcjA6MDo5OjApOiBSRUFEKDEw KS4gQ0RCOiAyOCAwMCAwMCA5ZiBmYyAyMiAwMCAwMSAwMCAwMCAKKGRhMTptcHIwOjA6OTowKTog Q0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMTptcHIwOjA6OTowKTogU0NTSSBzdGF0 dXM6IENoZWNrIENvbmRpdGlvbgooZGExOm1wcjA6MDo5OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FM IFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMTpt cHIwOjA6OTowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTE6bXByMDowOjk6MCk6 IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDlmIGZlIDIyIDAwIDAxIDAwIDAwIAooZGExOm1wcjA6 MDo5OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGExOm1wcjA6MDo5OjApOiBT Q1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTE6bXByMDowOjk6MCk6IFNDU0kgc2Vuc2U6 IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2Rl KQooZGExOm1wcjA6MDo5OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMDptcHIw OjA6ODowKTogUkVBRCgxMCkuIENEQjogMjggMDAgMDAgYTAgMDAgMjIgMDAgMDEgMDAgMDAgCihk YTA6bXByMDowOjg6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTA6bXByMDow Ojg6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMDptcHIwOjA6ODowKTogU0NT SSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0 aW9uIGNvZGUpCihkYTA6bXByMDowOjg6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgoo ZGEwOm1wcjA6MDo4OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCBhMCAwMiAyMiAwMCAwMSAw MCAwMCAKKGRhMDptcHIwOjA6ODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRh MDptcHIwOjA6ODowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEwOm1wcjA6MDo4 OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFu ZCBvcGVyYXRpb24gY29kZSkKKGRhMDptcHIwOjA6ODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxl IGVycm9yCihkYTA6bXByMDowOjg6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAwIDAwIDAx IGQxIGMwIGJhIDIyIDAwIDAwIDAxIDAwIDAwIDAwIAooZGEwOm1wcjA6MDo4OjApOiBDQU0gc3Rh dHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHN0YXR1czogQ2hl Y2sgQ29uZGl0aW9uCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVT VCBhc2M6MjAsMCAoSW52YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEwOm1wcjA6MDo4 OjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGRhMDptcHIwOjA6ODowKTogUkVBRCgx NikuIENEQjogODggMDAgMDAgMDAgMDAgMDEgZDEgYzAgYmMgMjIgMDAgMDAgMDEgMDAgMDAgMDAg CihkYTA6bXByMDowOjg6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTA6bXBy MDowOjg6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMDptcHIwOjA6ODowKTog U0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoyMCwwIChJbnZhbGlkIGNvbW1hbmQgb3Bl cmF0aW9uIGNvZGUpCihkYTA6bXByMDowOjg6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJv cgooZGEwOm1wcjA6MDo4OjApOiBSRUFEKDEwKS4gQ0RCOiAyOCAwMCAwMCAwMCAwMCAyMiAwMCAw MSAwMCAwMCAKKGRhMDptcHIwOjA6ODowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IK KGRhMDptcHIwOjA6ODowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEwOm1wcjA6 MDo4OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVFU1QgYXNjOjIwLDAgKEludmFsaWQgY29t bWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMDptcHIwOjA6ODowKTogRXJyb3IgMjIsIFVucmV0cnlh YmxlIGVycm9yCihkYTA6bXByMDowOjg6MCk6IFJFQUQoMTApLiBDREI6IDI4IDAwIDAwIDAwIDAy IDIyIDAwIDAxIDAwIDAwIAooZGEwOm1wcjA6MDo4OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1 cyBFcnJvcgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihk YTA6bXByMDowOjg6MCk6IFNDU0kgc2Vuc2U6IElMTEVHQUwgUkVRVUVTVCBhc2M6MjAsMCAoSW52 YWxpZCBjb21tYW5kIG9wZXJhdGlvbiBjb2RlKQooZGEwOm1wcjA6MDo4OjApOiBFcnJvciAyMiwg VW5yZXRyeWFibGUgZXJyb3IKKGRhMDptcHIwOjA6ODowKTogUkVBRCgxMCkuIENEQjogMjggMDAg MDAgOWYgZmMgMjIgMDAgMDEgMDAgMDAgCihkYTA6bXByMDowOjg6MCk6IENBTSBzdGF0dXM6IFND U0kgU3RhdHVzIEVycm9yCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25k aXRpb24KKGRhMDptcHIwOjA6ODowKTogU0NTSSBzZW5zZTogSUxMRUdBTCBSRVFVRVNUIGFzYzoy MCwwIChJbnZhbGlkIGNvbW1hbmQgb3BlcmF0aW9uIGNvZGUpCihkYTA6bXByMDowOjg6MCk6IEVy cm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgooZGEwOm1wcjA6MDo4OjApOiBSRUFEKDEwKS4gQ0RC OiAyOCAwMCAwMCA5ZiBmZSAyMiAwMCAwMSAwMCAwMCAKKGRhMDptcHIwOjA6ODowKTogQ0FNIHN0 YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMDptcHIwOjA6ODowKTogU0NTSSBzdGF0dXM6IENo ZWNrIENvbmRpdGlvbgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHNlbnNlOiBJTExFR0FMIFJFUVVF U1QgYXNjOjIwLDAgKEludmFsaWQgY29tbWFuZCBvcGVyYXRpb24gY29kZSkKKGRhMDptcHIwOjA6 ODowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihkYTA6bXByMDowOjg6MCk6IFdSSVRF KDYpLiBDREI6IDBhIDAwIDAwIDAwIDAxIDAwIAooZGEwOm1wcjA6MDo4OjApOiBDQU0gc3RhdHVz OiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEwOm1wcjA6MDo4OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sg Q29uZGl0aW9uCihkYTA6bXByMDowOjg6MCk6IFNDU0kgc2Vuc2U6IE5PVCBSRUFEWSBhc2M6NCw0 IChMb2dpY2FsIHVuaXQgbm90IHJlYWR5LCBmb3JtYXQgaW4gcHJvZ3Jlc3MpCihkYTA6bXByMDow Ojg6MCk6IFByb2dyZXNzOiA5JSAoNjI1Ni82NTUzNikgY29tcGxldGUKKGRhMDptcHIwOjA6ODow KTogRXJyb3IgMTYsIFVucmV0cnlhYmxlIGVycm9yCg== --_004_3A5A10BE32AC9E45B4A22F89FC90EC0701C366610EQLEXC01Quorum_-- From owner-freebsd-scsi@freebsd.org Tue Oct 17 02:31:30 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B5D2E4FCA5 for ; Tue, 17 Oct 2017 02:31:30 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D3F4C7D260 for ; Tue, 17 Oct 2017 02:31:29 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPS id v9H2VQ5Z006679 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 16 Oct 2017 22:31:26 -0400 (EDT) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.15.2/8.14.9/Submit) id v9H2VQQs006676; Mon, 16 Oct 2017 22:31:26 -0400 (EDT) (envelope-from ken) Date: Mon, 16 Oct 2017 22:31:26 -0400 From: "Kenneth D. Merry" To: Shiva Bhanujan Cc: "freebsd-scsi@freebsd.org" Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Message-ID: <20171017023126.GA6559@mithlond.kdm.org> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 16 Oct 2017 22:31:27 -0400 (EDT) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mithlond.kdm.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Oct 2017 02:31:30 -0000 On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: > Hi Ken, > > I've attached the output of dmesg. Here's the SCSI CDB for a sample drive, da3. > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've tried this w/ 11.1, w/ the same results. > Yes, the mpr(4) driver is in all recent FreeBSD releases. In looking at the dmesg, this is telling: (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete (da0:mpr0:0:8:0): Error 16, Unretryable error If the drives are in the process of formatting, I guess it may make sense for them to reject read commands. Otherwise, it makes no sense for a hard drive to reject reads. Are you able to check the status of the format? You should be able to send a test unit ready and figure out how far along the format is: camcontrol tur da0 -v And so on for each of the drives. Ken > ________________________________ > From: Kenneth D. Merry [ken@FreeBSD.ORG] > Sent: Monday, October 16, 2017 7:42 AM > To: Shiva Bhanujan > Cc: freebsd-scsi@freebsd.org > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 > > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > > Hello, > > > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the following: > > > > > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code) > > > > The error message above is missing the SCSI CDB. We need that in order to > figure out what command the drive is complaining about. > > The error message means that FreeBSD is sending a SCSI command that the > drive doesn't support. That can be benign, or it can cause a problem. > > So, what error does gpart give you when you have this problem? > > > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also has the same result. > > > > > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > > > > > > Is there a different version of the mpr driver I should be using? I haven't checked the differences between the mpr driver in 10.2 vs 10.3 and 11.0. I do see that there are others who have experienced these issues. Can somebody please provide me some pointers > > as to why this is occurring? Or if there are some driver changes that I might be able to incorporate? > > > > In general, the latest mpr(4) driver is the best one. The driver itself > generally doesn't send SCSI commands (there are a few exceptions), but > rather passes them through from the upper layers of CAM. > > > Please note, that I have gone through the mail titled "scsi error at SEAGATE ST1200MM0088 TT31" and have started sg_format on all the SEAGATE disks. Having said that, I still need to figure out what would happen, if the disks were written to using FreeBSD 10.2, which doesn't seem to have SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated. > > > > Send the full error messages, and we may be able to figure out what's going > on. > > Ken > -- > Kenneth Merry > ken@FreeBSD.ORG > ________________________________ -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@freebsd.org Tue Oct 17 13:08:27 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 864BBE3A0B4 for ; Tue, 17 Oct 2017 13:08:27 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-35.reflexion.net [208.70.210.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 448AB6C2A2 for ; Tue, 17 Oct 2017 13:08:26 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 23109 invoked from network); 17 Oct 2017 13:08:25 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 17 Oct 2017 13:08:25 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 17 Oct 2017 09:08:25 -0400 (EDT) Received: (qmail 7157 invoked from network); 17 Oct 2017 13:08:24 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 17 Oct 2017 13:08:24 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Tue, 17 Oct 2017 06:08:23 -0700 From: Shiva Bhanujan To: "Kenneth D. Merry" CC: "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uIAAihwAgAA6+II= Date: Tue, 17 Oct 2017 13:08:22 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local>, <20171017023126.GA6559@mithlond.kdm.org> In-Reply-To: <20171017023126.GA6559@mithlond.kdm.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Oct 2017 13:08:27 -0000 Since I started having the SCSI errors, I ended up running sg_format to = format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete once the format is done, I can successfully format and partition using = gpart. The errors that show up when I try to run gpart for the first time are as = follows: (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da9:mpr0:0:17:0): CAM status: SCSI Status Error (da9:mpr0:0:17:0): SCSI status: Check Condition (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) (da9:mpr0:0:17:0): Error 22, Unretryable error It seems that it's the read that is failing, and is being tagged as an = illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG=5D Sent: Monday, October 16, 2017 7:31 PM To: Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: > Hi Ken, > > I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've = tried this w/ 11.1, w/ the same results. > Yes, the mpr(4) driver is in all recent FreeBSD releases. In looking at the dmesg, this is telling: (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete (da0:mpr0:0:8:0): Error 16, Unretryable error If the drives are in the process of formatting, I guess it may make sense for them to reject read commands. Otherwise, it makes no sense for a hard drive to reject reads. Are you able to check the status of the format? You should be able to send a test unit ready and figure out how far along the format is: camcontrol tur da0 -v And so on for each of the drives. Ken > ________________________________ > From: Kenneth D. Merry =5Bken=40FreeBSD.ORG=5D > Sent: Monday, October 16, 2017 7:42 AM > To: Shiva Bhanujan > Cc: freebsd-scsi=40freebsd.org > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 > > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > > Hello, > > > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the = following: > > > > > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) > > > > The error message above is missing the SCSI CDB. We need that in order to > figure out what command the drive is complaining about. > > The error message means that FreeBSD is sending a SCSI command that the > drive doesn't support. That can be benign, or it can cause a problem. > > So, what error does gpart give you when you have this problem? > > > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also = has the same result. > > > > > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 = doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > > > > > > Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 and = 11.0. I do see that there are others who have experienced these issues. = Can somebody please provide me some pointers > > as to why this is occurring? Or if there are some driver changes that = I might be able to incorporate? > > > > In general, the latest mpr(4) driver is the best one. The driver itself > generally doesn't send SCSI commands (there are a few exceptions), but > rather passes them through from the upper layers of CAM. > > > Please note, that I have gone through the mail titled =22scsi error at = SEAGATE ST1200MM0088 TT31=22 and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, if = the disks were written to using FreeBSD 10.2, which doesn't seem to have = SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated. > > > > Send the full error messages, and we may be able to figure out what's = going > on. > > Ken > -- > Kenneth Merry > ken=40FreeBSD.ORG > ________________________________ -- Kenneth Merry ken=40FreeBSD.ORG ________________________________ From owner-freebsd-scsi@freebsd.org Wed Oct 18 01:54:46 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B60BE4CDFD for ; Wed, 18 Oct 2017 01:54:46 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-49.reflexion.net [208.70.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 466A165EA1 for ; Wed, 18 Oct 2017 01:54:45 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 29630 invoked from network); 18 Oct 2017 01:54:44 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 18 Oct 2017 01:54:44 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 17 Oct 2017 21:54:44 -0400 (EDT) Received: (qmail 3505 invoked from network); 18 Oct 2017 01:54:44 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 18 Oct 2017 01:54:44 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Tue, 17 Oct 2017 18:54:43 -0700 From: Shiva Bhanujan To: "Kenneth D. Merry" CC: "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uIAAihwAgAA6+IKAANcq6YAAAGPJ Date: Wed, 18 Oct 2017 01:54:42 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local>, <20171017023126.GA6559@mithlond.kdm.org>, <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local>, <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Oct 2017 01:54:46 -0000 Sorry. perhaps I have failed to mention, the SCSI errors are only w/ the = seagate drives. These are of the model ST4000NM0023. ________________________________ From: Shiva Bhanujan Sent: Tuesday, October 17, 2017 6:53 PM To: Kenneth D. Merry Cc: freebsd-scsi=40freebsd.org Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? ________________________________ From: owner-freebsd-scsi=40freebsd.org = =5Bowner-freebsd-scsi=40freebsd.org=5D on behalf of Shiva Bhanujan = =5Bshiva.bhanujan=40quorum.net=5D Sent: Tuesday, October 17, 2017 6:08 AM To: Kenneth D. Merry Cc: freebsd-scsi=40freebsd.org Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Since I started having the SCSI errors, I ended up running sg_format to = format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete once the format is done, I can successfully format and partition using = gpart. The errors that show up when I try to run gpart for the first time are as = follows: (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da9:mpr0:0:17:0): CAM status: SCSI Status Error (da9:mpr0:0:17:0): SCSI status: Check Condition (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) (da9:mpr0:0:17:0): Error 22, Unretryable error It seems that it's the read that is failing, and is being tagged as an = illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG=5D Sent: Monday, October 16, 2017 7:31 PM To: Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: > Hi Ken, > > I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've = tried this w/ 11.1, w/ the same results. > Yes, the mpr(4) driver is in all recent FreeBSD releases. In looking at the dmesg, this is telling: (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete (da0:mpr0:0:8:0): Error 16, Unretryable error If the drives are in the process of formatting, I guess it may make sense for them to reject read commands. Otherwise, it makes no sense for a hard drive to reject reads. Are you able to check the status of the format? You should be able to send a test unit ready and figure out how far along the format is: camcontrol tur da0 -v And so on for each of the drives. Ken > ________________________________ > From: Kenneth D. Merry = =5Bken=40FreeBSD.ORG=5D > Sent: Monday, October 16, 2017 7:42 AM > To: Shiva Bhanujan > Cc: = freebsd-scsi=40freebsd.org > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 > > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > > Hello, > > > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the = following: > > > > > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) > > > > The error message above is missing the SCSI CDB. We need that in order to > figure out what command the drive is complaining about. > > The error message means that FreeBSD is sending a SCSI command that the > drive doesn't support. That can be benign, or it can cause a problem. > > So, what error does gpart give you when you have this problem? > > > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also = has the same result. > > > > > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 = doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > > > > > > Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 and = 11.0. I do see that there are others who have experienced these issues. = Can somebody please provide me some pointers > > as to why this is occurring? Or if there are some driver changes that = I might be able to incorporate? > > > > In general, the latest mpr(4) driver is the best one. The driver itself > generally doesn't send SCSI commands (there are a few exceptions), but > rather passes them through from the upper layers of CAM. > > > Please note, that I have gone through the mail titled =22scsi error at = SEAGATE ST1200MM0088 TT31=22 and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, if = the disks were written to using FreeBSD 10.2, which doesn't seem to have = SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated. > > > > Send the full error messages, and we may be able to figure out what's = going > on. > > Ken > -- > Kenneth Merry > = ken=40FreeBSD.ORG > ________________________________ -- Kenneth Merry ken=40FreeBSD.ORG ________________________________ _______________________________________________ freebsd-scsi=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-scsi To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org=22 ________________________________ From owner-freebsd-scsi@freebsd.org Wed Oct 18 01:59:50 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2C04E4CE76 for ; Wed, 18 Oct 2017 01:59:49 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-49.reflexion.net [208.70.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2E0865EFF for ; Wed, 18 Oct 2017 01:59:48 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 31050 invoked from network); 18 Oct 2017 01:53:08 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 18 Oct 2017 01:53:08 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Tue, 17 Oct 2017 21:53:08 -0400 (EDT) Received: (qmail 6380 invoked from network); 18 Oct 2017 01:53:07 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 18 Oct 2017 01:53:07 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Tue, 17 Oct 2017 18:53:06 -0700 From: Shiva Bhanujan To: "Kenneth D. Merry" CC: "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uIAAihwAgAA6+IKAANcq6Q== Date: Wed, 18 Oct 2017 01:53:05 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local>, <20171017023126.GA6559@mithlond.kdm.org>, <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Oct 2017 01:59:50 -0000 Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? ________________________________ From: owner-freebsd-scsi=40freebsd.org = =5Bowner-freebsd-scsi=40freebsd.org=5D on behalf of Shiva Bhanujan = =5Bshiva.bhanujan=40quorum.net=5D Sent: Tuesday, October 17, 2017 6:08 AM To: Kenneth D. Merry Cc: freebsd-scsi=40freebsd.org Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Since I started having the SCSI errors, I ended up running sg_format to = format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete once the format is done, I can successfully format and partition using = gpart. The errors that show up when I try to run gpart for the first time are as = follows: (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da9:mpr0:0:17:0): CAM status: SCSI Status Error (da9:mpr0:0:17:0): SCSI status: Check Condition (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) (da9:mpr0:0:17:0): Error 22, Unretryable error It seems that it's the read that is failing, and is being tagged as an = illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG=5D Sent: Monday, October 16, 2017 7:31 PM To: Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: > Hi Ken, > > I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've = tried this w/ 11.1, w/ the same results. > Yes, the mpr(4) driver is in all recent FreeBSD releases. In looking at the dmesg, this is telling: (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete (da0:mpr0:0:8:0): Error 16, Unretryable error If the drives are in the process of formatting, I guess it may make sense for them to reject read commands. Otherwise, it makes no sense for a hard drive to reject reads. Are you able to check the status of the format? You should be able to send a test unit ready and figure out how far along the format is: camcontrol tur da0 -v And so on for each of the drives. Ken > ________________________________ > From: Kenneth D. Merry = =5Bken=40FreeBSD.ORG=5D > Sent: Monday, October 16, 2017 7:42 AM > To: Shiva Bhanujan > Cc: = freebsd-scsi=40freebsd.org > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 > > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > > Hello, > > > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the = following: > > > > > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) > > > > The error message above is missing the SCSI CDB. We need that in order to > figure out what command the drive is complaining about. > > The error message means that FreeBSD is sending a SCSI command that the > drive doesn't support. That can be benign, or it can cause a problem. > > So, what error does gpart give you when you have this problem? > > > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also = has the same result. > > > > > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 = doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > > > > > > Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 and = 11.0. I do see that there are others who have experienced these issues. = Can somebody please provide me some pointers > > as to why this is occurring? Or if there are some driver changes that = I might be able to incorporate? > > > > In general, the latest mpr(4) driver is the best one. The driver itself > generally doesn't send SCSI commands (there are a few exceptions), but > rather passes them through from the upper layers of CAM. > > > Please note, that I have gone through the mail titled =22scsi error at = SEAGATE ST1200MM0088 TT31=22 and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, if = the disks were written to using FreeBSD 10.2, which doesn't seem to have = SCSI errors, and when I try to upgrade to 10.3. Any help is appreciated. > > > > Send the full error messages, and we may be able to figure out what's = going > on. > > Ken > -- > Kenneth Merry > = ken=40FreeBSD.ORG > ________________________________ -- Kenneth Merry ken=40FreeBSD.ORG ________________________________ _______________________________________________ freebsd-scsi=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-scsi To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org=22 ________________________________ From owner-freebsd-scsi@freebsd.org Thu Oct 19 05:56:22 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFD67E2B60B for ; Thu, 19 Oct 2017 05:56:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE33C7FDF5 for ; Thu, 19 Oct 2017 05:56:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9J5uMEd051495 for ; Thu, 19 Oct 2017 05:56:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 222898] [iscsi]: panic: page fault - system crashes while working as iSCSI target Date: Thu, 19 Oct 2017 05:56:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: emz@norma.perm.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-scsi@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Oct 2017 05:56:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222898 --- Comment #10 from emz@norma.perm.ru --- Is there something more needed from me ? Or may be my last answer was total= ly discouraging, and installing r315520 wasn't the expected action ? Do I need= to test this patch instead ? Because as far as I understood I was given the pa= tch 'cause "Okay, since you claiming that you cannot install r315520 - then take this patch". Did I understand this wrong ? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-scsi@freebsd.org Thu Oct 19 21:15:54 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABF7EE47AAB for ; Thu, 19 Oct 2017 21:15:54 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 68B0281225; Thu, 19 Oct 2017 21:15:54 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from [10.0.0.26] (mbp2013.int.kdm.org [10.0.0.26]) (authenticated bits=0) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPSA id v9JLFkDA069655 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Oct 2017 17:15:47 -0400 (EDT) (envelope-from ken@freebsd.org) From: Ken Merry Message-Id: <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> Mime-Version: 1.0 (Mac OS X Mail 11.0 \(3445.1.7\)) Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Date: Thu, 19 Oct 2017 17:15:45 -0400 In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> Cc: "freebsd-scsi@freebsd.org" , slm@freebsd.org To: Shiva Bhanujan References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> X-Mailer: Apple Mail (2.3445.1.7) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [96.89.93.250]); Thu, 19 Oct 2017 17:15:47 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Oct 2017 21:15:54 -0000 What does sg_readcap =E2=80=9416 show for these drives? If it has type 2 protection turned on, check to see what the DCIPZ value = is in the control mode page: camcontrol modepage daX -v -m 10 If that is set to 0, add a -e to the above command line and see if you = can set it to 1. That may or may not help anything. The mpr(4) and mps(4) drivers try to support protection information if = it is turned on on the drive. So, they set the protection information = if protection information is turned on in the drive. For that reason, = setting the DCIPZ bit may not fix it. There could be a problem with how that is implemented that=E2=80=99s = causing the drives to reject the command, but I=E2=80=99m not sure. If it is, Steve (CCed) can help us debug it. Ken =E2=80=94=20 Ken Merry ken@FreeBSD.ORG > On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan = wrote: >=20 > Sorry. perhaps I have failed to mention, the SCSI errors are only w/ = the seagate drives. These are of the model ST4000NM0023. >=20 >=20 > From: Shiva Bhanujan > Sent: Tuesday, October 17, 2017 6:53 PM > To: Kenneth D. Merry > Cc: freebsd-scsi@freebsd.org > Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 > Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? >=20 >=20 >=20 > From: owner-freebsd-scsi@freebsd.org = [owner-freebsd-scsi@freebsd.org = ] on behalf of Shiva Bhanujan = [shiva.bhanujan@quorum.net ] > Sent: Tuesday, October 17, 2017 6:08 AM > To: Kenneth D. Merry > Cc: freebsd-scsi@freebsd.org > Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 > Since I started having the SCSI errors, I ended up running sg_format = to format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. >=20 > (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not = ready, format in progress) > (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >=20 > once the format is done, I can successfully format and partition using = gpart. >=20 >=20 > The errors that show up when I try to run gpart for the first time are = as follows: >=20 > (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da9:mpr0:0:17:0): CAM status: SCSI Status Error > (da9:mpr0:0:17:0): SCSI status: Check Condition > (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) > (da9:mpr0:0:17:0): Error 22, Unretryable error >=20 >=20 > It seems that it's the read that is failing, and is being tagged as an = illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? >=20 >=20 >=20 >=20 > ________________________________ > From: Kenneth D. Merry [ken@FreeBSD.ORG ] > Sent: Monday, October 16, 2017 7:31 PM > To: Shiva Bhanujan > Cc: freebsd-scsi@freebsd.org > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 > On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: > > Hi Ken, > > > > I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. > > > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 = 00 01 00 00 00 > > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 = 00 01 00 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 = 00 01 00 00 00 > > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 = 00 01 00 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 = 00 01 00 00 00 > > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 = 00 01 00 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 = 00 01 00 00 00 > > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 = 00 01 00 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > > > > > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? = I've tried this w/ 11.1, w/ the same results. > > >=20 > Yes, the mpr(4) driver is in all recent FreeBSD releases. >=20 > In looking at the dmesg, this is telling: >=20 > (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 > (da0:mpr0:0:8:0): CAM status: SCSI Status Error > (da0:mpr0:0:8:0): SCSI status: Check Condition > (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not = ready, format in progress) > (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete > (da0:mpr0:0:8:0): Error 16, Unretryable error >=20 > If the drives are in the process of formatting, I guess it may make = sense > for them to reject read commands. Otherwise, it makes no sense for a = hard > drive to reject reads. >=20 > Are you able to check the status of the format? You should be able to = send > a test unit ready and figure out how far along the format is: >=20 > camcontrol tur da0 -v >=20 > And so on for each of the drives. >=20 > Ken >=20 > > ________________________________ > > From: Kenneth D. Merry [ken@FreeBSD.ORG = = >] > > Sent: Monday, October 16, 2017 7:42 AM > > To: Shiva Bhanujan > > Cc: freebsd-scsi@freebsd.org = = > > > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic = SAS3008 PCI-Express Fusion-MPT SAS-3 > > > > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > > > Hello, > > > > > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The = HBA330 SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. = When I try to access the disks (/dev/da0...) using gpart, I get SCSI = errors, like the following: > > > > > > > > > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > > > > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > > > > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) > > > > > > > The error message above is missing the SCSI CDB. We need that in = order to > > figure out what command the drive is complaining about. > > > > The error message means that FreeBSD is sending a SCSI command that = the > > drive doesn't support. That can be benign, or it can cause a = problem. > > > > So, what error does gpart give you when you have this problem? > > > > > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively = also has the same result. > > > > > > > > > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD = 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > > > > > > > > > > > Is there a different version of the mpr driver I should be using? = I haven't checked the differences between the mpr driver in 10.2 vs 10.3 = and 11.0. I do see that there are others who have experienced these = issues. Can somebody please provide me some pointers > > > as to why this is occurring? Or if there are some driver changes = that I might be able to incorporate? > > > > > > > In general, the latest mpr(4) driver is the best one. The driver = itself > > generally doesn't send SCSI commands (there are a few exceptions), = but > > rather passes them through from the upper layers of CAM. > > > > > Please note, that I have gone through the mail titled "scsi error = at SEAGATE ST1200MM0088 TT31" and have started sg_format on all the = SEAGATE disks. Having said that, I still need to figure out what would = happen, if the disks were written to using FreeBSD 10.2, which doesn't = seem to have SCSI errors, and when I try to upgrade to 10.3. Any help is = appreciated. > > > > > > > Send the full error messages, and we may be able to figure out = what's going > > on. > > > > Ken > > -- > > Kenneth Merry > > ken@FreeBSD.ORG = > = > > > ________________________________ >=20 >=20 > -- > Kenneth Merry > ken@FreeBSD.ORG = > > ________________________________ > _______________________________________________ > freebsd-scsi@freebsd.org mailing = list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi = > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org = " From owner-freebsd-scsi@freebsd.org Thu Oct 19 21:17:03 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F3AFE47B12 for ; Thu, 19 Oct 2017 21:17:03 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C3A6381277 for ; Thu, 19 Oct 2017 21:17:02 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from [10.0.0.26] (mbp2013.int.kdm.org [10.0.0.26]) (authenticated bits=0) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPSA id v9JLH1xP069670 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Oct 2017 17:17:01 -0400 (EDT) (envelope-from ken@freebsd.org) From: Ken Merry Message-Id: <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> Mime-Version: 1.0 (Mac OS X Mail 11.0 \(3445.1.7\)) Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Date: Thu, 19 Oct 2017 17:17:00 -0400 In-Reply-To: <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> Cc: "freebsd-scsi@freebsd.org" To: Shiva Bhanujan References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> X-Mailer: Apple Mail (2.3445.1.7) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [96.89.93.250]); Thu, 19 Oct 2017 17:17:01 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Oct 2017 21:17:03 -0000 By the way, the message you referenced is here: = https://lists.freebsd.org/pipermail/freebsd-scsi/2017-January/007237.html = And there is more here: https://bugs.freenas.org/issues/14517 = Ken =E2=80=94=20 Ken Merry ken@FreeBSD.ORG > On Oct 19, 2017, at 5:15 PM, Ken Merry wrote: >=20 > What does sg_readcap =E2=80=9416 show for these drives? >=20 > If it has type 2 protection turned on, check to see what the DCIPZ = value is in the control mode page: >=20 > camcontrol modepage daX -v -m 10 >=20 > If that is set to 0, add a -e to the above command line and see if you = can set it to 1. That may or may not help anything. >=20 > The mpr(4) and mps(4) drivers try to support protection information if = it is turned on on the drive. So, they set the protection information = if protection information is turned on in the drive. For that reason, = setting the DCIPZ bit may not fix it. >=20 > There could be a problem with how that is implemented that=E2=80=99s = causing the drives to reject the command, but I=E2=80=99m not sure. >=20 > If it is, Steve (CCed) can help us debug it. >=20 > Ken > =E2=80=94=20 > Ken Merry > ken@FreeBSD.ORG >=20 >=20 >=20 >> On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan = wrote: >>=20 >> Sorry. perhaps I have failed to mention, the SCSI errors are only w/ = the seagate drives. These are of the model ST4000NM0023. >>=20 >>=20 >> From: Shiva Bhanujan >> Sent: Tuesday, October 17, 2017 6:53 PM >> To: Kenneth D. Merry >> Cc: freebsd-scsi@freebsd.org >> Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >>=20 >> Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? >>=20 >>=20 >>=20 >> From: owner-freebsd-scsi@freebsd.org = [owner-freebsd-scsi@freebsd.org = ] on behalf of Shiva Bhanujan = [shiva.bhanujan@quorum.net ] >> Sent: Tuesday, October 17, 2017 6:08 AM >> To: Kenneth D. Merry >> Cc: freebsd-scsi@freebsd.org >> Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >>=20 >> Since I started having the SCSI errors, I ended up running sg_format = to format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. >>=20 >> (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not = ready, format in progress) >> (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >>=20 >> once the format is done, I can successfully format and partition = using gpart. >>=20 >>=20 >> The errors that show up when I try to run gpart for the first time = are as follows: >>=20 >> (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >> (da9:mpr0:0:17:0): CAM status: SCSI Status Error >> (da9:mpr0:0:17:0): SCSI status: Check Condition >> (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) >> (da9:mpr0:0:17:0): Error 22, Unretryable error >>=20 >>=20 >> It seems that it's the read that is failing, and is being tagged as = an illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? >>=20 >>=20 >>=20 >>=20 >> ________________________________ >> From: Kenneth D. Merry [ken@FreeBSD.ORG ] >> Sent: Monday, October 16, 2017 7:31 PM >> To: Shiva Bhanujan >> Cc: freebsd-scsi@freebsd.org >> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >>=20 >> On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: >>> Hi Ken, >>>=20 >>> I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. >>>=20 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 = 00 01 00 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >>> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >>>=20 >>>=20 >>> My understanding is that FreeBSD 11.1 contains the mpr(4) driver? = I've tried this w/ 11.1, w/ the same results. >>>=20 >>=20 >> Yes, the mpr(4) driver is in all recent FreeBSD releases. >>=20 >> In looking at the dmesg, this is telling: >>=20 >> (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 >> (da0:mpr0:0:8:0): CAM status: SCSI Status Error >> (da0:mpr0:0:8:0): SCSI status: Check Condition >> (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not = ready, format in progress) >> (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >> (da0:mpr0:0:8:0): Error 16, Unretryable error >>=20 >> If the drives are in the process of formatting, I guess it may make = sense >> for them to reject read commands. Otherwise, it makes no sense for a = hard >> drive to reject reads. >>=20 >> Are you able to check the status of the format? You should be able to = send >> a test unit ready and figure out how far along the format is: >>=20 >> camcontrol tur da0 -v >>=20 >> And so on for each of the drives. >>=20 >> Ken >>=20 >>> ________________________________ >>> From: Kenneth D. Merry [ken@FreeBSD.ORG = = >] >>> Sent: Monday, October 16, 2017 7:42 AM >>> To: Shiva Bhanujan >>> Cc: freebsd-scsi@freebsd.org = = > >>> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic = SAS3008 PCI-Express Fusion-MPT SAS-3 >>>=20 >>> On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: >>>> Hello, >>>>=20 >>>> I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like = the following: >>>>=20 >>>>=20 >>>>=20 >>>> (da0:mpr0:0:0:0): CAM status: SCSI Status Error >>>>=20 >>>> (da0:mpr0:0:0:0): SCSI status: Check Condition >>>>=20 >>>> (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) >>>>=20 >>>=20 >>> The error message above is missing the SCSI CDB. We need that in = order to >>> figure out what command the drive is complaining about. >>>=20 >>> The error message means that FreeBSD is sending a SCSI command that = the >>> drive doesn't support. That can be benign, or it can cause a = problem. >>>=20 >>> So, what error does gpart give you when you have this problem? >>>=20 >>>> I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively = also has the same result. >>>>=20 >>>=20 >>>=20 >>>> Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD = 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either. >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 = and 11.0. I do see that there are others who have experienced these = issues. Can somebody please provide me some pointers >>>> as to why this is occurring? Or if there are some driver changes = that I might be able to incorporate? >>>>=20 >>>=20 >>> In general, the latest mpr(4) driver is the best one. The driver = itself >>> generally doesn't send SCSI commands (there are a few exceptions), = but >>> rather passes them through from the upper layers of CAM. >>>=20 >>>> Please note, that I have gone through the mail titled "scsi error = at SEAGATE ST1200MM0088 TT31" and have started sg_format on all the = SEAGATE disks. Having said that, I still need to figure out what would = happen, if the disks were written to using FreeBSD 10.2, which doesn't = seem to have SCSI errors, and when I try to upgrade to 10.3. Any help is = appreciated. >>>>=20 >>>=20 >>> Send the full error messages, and we may be able to figure out = what's going >>> on. >>>=20 >>> Ken >>> -- >>> Kenneth Merry >>> ken@FreeBSD.ORG = > = > >>> ________________________________ >>=20 >>=20 >> -- >> Kenneth Merry >> ken@FreeBSD.ORG = > >> ________________________________ >> _______________________________________________ >> freebsd-scsi@freebsd.org mailing = list >> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi = >> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org = " >=20 > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Thu Oct 19 21:43:39 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28B17E4842C for ; Thu, 19 Oct 2017 21:43:39 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-80.reflexion.net [208.70.210.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DDB8C821FC for ; Thu, 19 Oct 2017 21:43:38 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 17792 invoked from network); 19 Oct 2017 21:43:36 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 19 Oct 2017 21:43:36 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Thu, 19 Oct 2017 17:43:37 -0400 (EDT) Received: (qmail 27068 invoked from network); 19 Oct 2017 21:43:36 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 19 Oct 2017 21:43:36 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Thu, 19 Oct 2017 14:43:35 -0700 From: Shiva Bhanujan To: Ken Merry CC: "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uIAAihwAgAA6+IKAANcq6YAAAGPJgANMRoCAAABZAP//kIxp Date: Thu, 19 Oct 2017 21:43:34 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org>, <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> In-Reply-To: <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.20.7.66] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Oct 2017 21:43:39 -0000 Here's the output of sg_readcap. =5Broot=40Filer-20-241 =7E=5D=23 sg_readcap --16 da1 Read Capacity results: Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 =5Btype 2 = protection=5D Logical block provisioning: lbpme=3D0, lbprz=3D0 Last logical block address=3D7814037167 (0x1d1c0beaf), Number of = logical blocks=3D7814037168 Logical block length=3D512 bytes Logical blocks per physical block exponent=3D0 Lowest aligned logical block address=3D0 Hence: Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB =5Broot=40Filer-20-241 =7E=5D=23 =5Broot=40Filer-20-241 =7E=5D=23 camcontrol modepage da1 -v -m 10 =7C grep = DPICZ DPICZ: 1 =5Broot=40Filer-20-241 =7E=5D=23 I did toggle the DPICZ on the drive from 1 to 0 and back. The sg_readcap = still shows 'type 2 protection', and that gpart still shows the SCSI = errors. I've narrowed this down to the Seagate ST4000NM0005, w/ a DOM of = 03/2016. We also have Constellation ES.3 drives model ST4000NM0023 that = don't exhibit this issue. I did go through the URLs that you have mentioned and that's how I found = that using sg_format did address this issue. Again, that works only for = new drives and we have appliances that already have data, for which = sg_format isn't an option. Is this boiling down to the 'type 2 protection' tag that we see in = sg_readcap? if so, would there be a way to turn it off? From: Ken Merry =5Bken=40freebsd.org=5D Sent: Thursday, October 19, 2017 2:17 PM To: Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 By the way, the message you referenced is here: https://lists.freebsd.org/pipermail/freebsd-scsi/2017-January/007237.html And there is more here: https://bugs.freenas.org/issues/14517 Ken =E2=80=94=20 Ken Merry ken=40FreeBSD.ORG On Oct 19, 2017, at 5:15 PM, Ken Merry wrote: What does sg_readcap =E2=80=9416 show for these drives? If it has type 2 protection turned on, check to see what the DCIPZ value = is in the control mode page: camcontrol modepage daX -v -m 10 If that is set to 0, add a -e to the above command line and see if you can = set it to 1. That may or may not help anything. The mpr(4) and mps(4) drivers try to support protection information if it = is turned on on the drive. So, they set the protection information if = protection information is turned on in the drive. For that reason, = setting the DCIPZ bit may not fix it. There could be a problem with how that is implemented that=E2=80=99s = causing the drives to reject the command, but I=E2=80=99m not sure. If it is, Steve (CCed) can help us debug it. Ken =E2=80=94=20 Ken Merry ken=40FreeBSD.ORG On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan = wrote: Sorry. perhaps I have failed to mention, the SCSI errors are only w/ the = seagate drives. These are of the model ST4000NM0023. From: Shiva Bhanujan Sent: Tuesday, October 17, 2017 6:53 PM To: Kenneth D. Merry Cc: freebsd-scsi=40freebsd.org Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? From: owner-freebsd-scsi=40freebsd.org = = =5Bowner-freebsd-scsi=40freebsd.org = =5D on behalf of Shiva Bhanujan = =5Bshiva.bhanujan=40quorum.net =5D Sent: Tuesday, October 17, 2017 6:08 AM To: Kenneth D. Merry Cc: freebsd-scsi=40freebsd.org Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Since I started having the SCSI errors, I ended up running sg_format to = format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete once the format is done, I can successfully format and partition using = gpart. The errors that show up when I try to run gpart for the first time are as = follows: (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da9:mpr0:0:17:0): CAM status: SCSI Status Error (da9:mpr0:0:17:0): SCSI status: Check Condition (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) (da9:mpr0:0:17:0): Error 22, Unretryable error It seems that it's the read that is failing, and is being tagged as an = illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG =5D Sent: Monday, October 16, 2017 7:31 PM To: Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: Hi Ken, I've attached the output of dmesg. Here's the SCSI CDB for a sample drive, = da3. (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've = tried this w/ 11.1, w/ the same results. Yes, the mpr(4) driver is in all recent FreeBSD releases. In looking at the dmesg, this is telling: (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete (da0:mpr0:0:8:0): Error 16, Unretryable error If the drives are in the process of formatting, I guess it may make sense for them to reject read commands. Otherwise, it makes no sense for a hard drive to reject reads. Are you able to check the status of the format? You should be able to send a test unit ready and figure out how far along the format is: camcontrol tur da0 -v And so on for each of the drives. Ken ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG = = >=5D Sent: Monday, October 16, 2017 7:42 AM To: Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org = = > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: Hello, I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 = controller is in pcipassthrough mode to the FreeBSD VM. When I try to = access the disks (/dev/da0...) using gpart, I get SCSI errors, like the = following: (da0:mpr0:0:0:0): CAM status: SCSI Status Error (da0:mpr0:0:0:0): SCSI status: Check Condition (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) The error message above is missing the SCSI CDB. We need that in order to figure out what command the drive is complaining about. The error message means that FreeBSD is sending a SCSI command that the drive doesn't support. That can be benign, or it can cause a problem. So, what error does gpart give you when you have this problem? I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also has = the same result. Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 = doesn't have any SCSI errors on the HBA330 SAS-3 controller either. Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 and = 11.0. I do see that there are others who have experienced these issues. = Can somebody please provide me some pointers as to why this is occurring? Or if there are some driver changes that I = might be able to incorporate? In general, the latest mpr(4) driver is the best one. The driver itself generally doesn't send SCSI commands (there are a few exceptions), but rather passes them through from the upper layers of CAM. Please note, that I have gone through the mail titled =22scsi error at = SEAGATE ST1200MM0088 TT31=22 and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, if = the disks were written to using FreeBSD 10.2, which doesn't seem to have SCSI = errors, and when I try to upgrade to 10.3. Any help is appreciated. Send the full error messages, and we may be able to figure out what's going on. Ken -- Kenneth Merry ken=40FreeBSD.ORG = > = > ________________________________ -- Kenneth Merry ken=40FreeBSD.ORG = > ________________________________ _______________________________________________ freebsd-scsi=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-scsi = To unsubscribe, send any mail to =22freebsd-scsi-unsubscribe=40freebsd.org = =22 _______________________________________________ freebsd-scsi=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-scsi To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org=22 From owner-freebsd-scsi@freebsd.org Thu Oct 19 22:23:41 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05403E48F36 for ; Thu, 19 Oct 2017 22:23:41 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-lf0-x231.google.com (mail-lf0-x231.google.com [IPv6:2a00:1450:4010:c07::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47B0B833C1 for ; Thu, 19 Oct 2017 22:23:40 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by mail-lf0-x231.google.com with SMTP id d10so11218018lfg.11 for ; Thu, 19 Oct 2017 15:23:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=z8ff/QD3jlbhDPxk36rZP8o59C0NvuwRuo8gX24kJd4=; b=U9HeeRgI8AYD9wkMLE+WPxrR3VLOauBDZb5gxEm/0c0Lq6BcbN9l8eKJvOWbaUNbji Cwp58GKpPoq1Pouc4icHMHa/GiWUBKDtcL2k/qoLmoI3gdPPqODvoFnyZW1YQf6WCgWK GCcs/cWHgStebh1e3v7HNGmbEomlUS7EX2b48DbribxZBQfXFdiJvmFYl9HTPapJ+BuB Mx11vQyDLXNGn396ktqEaZcUBIWP8L1tg4PeW3Qf8kG19m+5nG8DtCbCIEa/Hs+ZvJhL /TdnbJI9IpPt6/O9ljMp3DUrsT3Kow/sPU9jp/0C2udwan9SZzhuXRunigBpkmeIS0Lb VckQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=z8ff/QD3jlbhDPxk36rZP8o59C0NvuwRuo8gX24kJd4=; b=r7OFL0tp5E4v90JuIEYc8GtXxZoZn9bVhnY3KSAktu3bRgCk3+cqbggYOU1FQAG1tL xmkr1HFmNFcz7UYMQR8KDWMaZk6VvqeiYYPoqnsLfgafAhBxdcFWnaYj1VRG0e9m0yiC ruB6A//bKPit7ouKurWIYLjfSPwPqVpgHcGtMylmDAPfKmYagovUSjDUbGwCjNGjwk7b l3HhizzQNyLFTiC6cATyDT1LJuh7bZIkEK10k/hY2jpUwMuSgYHnQn7J3nTwmlHeyeOR H8ydpSfwUE75Oy/ujVJrNEoKVfwnEccfzR/aP1qLbRWtwEfB3dT7N0vatbfxIjvoeJlS eYNQ== X-Gm-Message-State: AMCzsaWIntCwKIH4HaBHirzUoGim6BoQGHz1SzQUVwdGMBCGoqEwP3eE LhYKZyCHX+tJJ+IYNfuky6VJOHVxmeH19aOMthXRwr7q X-Google-Smtp-Source: ABhQp+RRSkth+UVD684AshISEtMn0u3G2ZUopN7L5c80rgGZ4x2THaDL1LEVZuIamPgzxNOV5arROSIHuQRrbdRW5dw= X-Received: by 10.25.83.71 with SMTP id h68mr1039639lfb.241.1508451817505; Thu, 19 Oct 2017 15:23:37 -0700 (PDT) MIME-Version: 1.0 References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local> In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local> From: Steven Hartland Date: Thu, 19 Oct 2017 22:23:26 +0000 Message-ID: Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 To: Ken Merry , Shiva Bhanujan Cc: "freebsd-scsi@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Oct 2017 22:23:41 -0000 With type 2 protection the ref tag has to match the LBA + N Some info about it is here https://www.usenix.org/legacy/event/lsf07/tech/petersen.pdf https://www.seagate.com/files/staticfiles/docs/pdf/whitepaper/safeguarding-= data-from-corruption-technology-paper-tp621us.pdf >From reading the seagate paper the only way to change the protection level is to format. On Thu, 19 Oct 2017 at 22:43, Shiva Bhanujan wrote: > Here's the output of sg_readcap. > > [root@Filer-20-241 ~]# sg_readcap --16 da1 > Read Capacity results: > Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 [type 2 protecti= on] > Logical block provisioning: lbpme=3D0, lbprz=3D0 > Last logical block address=3D7814037167 (0x1d1c0beaf), Number of logic= al > blocks=3D7814037168 > Logical block length=3D512 bytes > Logical blocks per physical block exponent=3D0 > Lowest aligned logical block address=3D0 > Hence: > Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB > [root@Filer-20-241 ~]# > > [root@Filer-20-241 ~]# camcontrol modepage da1 -v -m 10 | grep DPICZ > DPICZ: 1 > [root@Filer-20-241 ~]# > > I did toggle the DPICZ on the drive from 1 to 0 and back. The sg_readcap > still shows 'type 2 protection', and that gpart still shows the SCSI > errors. I've narrowed this down to the Seagate ST4000NM0005, w/ a DOM of > 03/2016. We also have Constellation ES.3 drives model ST4000NM0023 that > don't exhibit this issue. > > I did go through the URLs that you have mentioned and that's how I found > that using sg_format did address this issue. Again, that works only for > new drives and we have appliances that already have data, for which > sg_format isn't an option. > > Is this boiling down to the 'type 2 protection' tag that we see in > sg_readcap? if so, would there be a way to turn it off? > > > > > > From: Ken Merry [ken@freebsd.org] > > Sent: Thursday, October 19, 2017 2:17 PM > > To: Shiva Bhanujan > > Cc: freebsd-scsi@freebsd.org > > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 > PCI-Express Fusion-MPT SAS-3 > > > > > > > > By the way, the message you referenced is here: > > > > > https://lists.freebsd.org/pipermail/freebsd-scsi/2017-January/007237.html > > > > > And there is more here: > > > > > https://bugs.freenas.org/issues/14517 > > > > > Ken > > > > =E2=80=94 > > Ken Merry > > ken@FreeBSD.ORG > > > > > > > > > > On Oct 19, 2017, at 5:15 PM, Ken Merry wrote: > > > > > What does sg_readcap =E2=80=9416 show for these drives? > > If it has type 2 protection turned on, check to see what the DCIPZ value > is in the control mode page: > > camcontrol modepage daX -v -m 10 > > If that is set to 0, add a -e to the above command line and see if you ca= n > set it to 1. That may or may not help anything. > > The mpr(4) and mps(4) drivers try to support protection information if it > is turned on on the drive. So, they set the protection information if > protection information is turned on in the drive. For that reason, setti= ng > the DCIPZ bit may not fix it. > > There could be a problem with how that is implemented that=E2=80=99s caus= ing the > drives to reject the command, but I=E2=80=99m not sure. > > If it is, Steve (CCed) can help us debug it. > > Ken > =E2=80=94 > Ken Merry > ken@FreeBSD.ORG > > > > On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan > wrote: > > Sorry. perhaps I have failed to mention, the SCSI errors are only w/ the > seagate drives. These are of the model ST4000NM0023. > > > From: Shiva Bhanujan > Sent: Tuesday, October 17, 2017 6:53 PM > To: Kenneth D. Merry > Cc: freebsd-scsi@freebsd.org > Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 > PCI-Express Fusion-MPT SAS-3 > > Please note, that this isn't an issue w/ Toshiba drives. is this a > firmware issue by any chance? > > > > From: owner-freebsd-scsi@freebsd.org owner-freebsd-scsi@freebsd.org> [owner-freebsd-scsi@freebsd.org owner-freebsd-scsi@freebsd.org>] on behalf of Shiva Bhanujan [ > shiva.bhanujan@quorum.net ] > Sent: Tuesday, October 17, 2017 6:08 AM > To: Kenneth D. Merry > Cc: freebsd-scsi@freebsd.org > Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 > PCI-Express Fusion-MPT SAS-3 > > Since I started having the SCSI errors, I ended up running sg_format to > format the disks. I've found that once the disks are formatted using > sg_format, there are no SCSI errors. The errors that show up during the > format are towards the end of the dmesg output. > > (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, > format in progress) > (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete > > once the format is done, I can successfully format and partition using > gpart. > > > The errors that show up when I try to run gpart for the first time are as > follows: > > (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da9:mpr0:0:17:0): CAM status: SCSI Status Error > (da9:mpr0:0:17:0): SCSI status: Check Condition > (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command > operation code) > (da9:mpr0:0:17:0): Error 22, Unretryable error > > > It seems that it's the read that is failing, and is being tagged as an > illegal request. While sg_format will address the issue at hand, this isn= 't > an option for us, because there are appliances that were formatted using > FreeBSD 10.2, and an upgrade to 10.3 > or 11.x might be an issue? > > > > > ________________________________ > From: Kenneth D. Merry [ken@FreeBSD.ORG ] > Sent: Monday, October 16, 2017 7:31 PM > To: Shiva Bhanujan > Cc: freebsd-scsi@freebsd.org > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 > PCI-Express Fusion-MPT SAS-3 > > On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: > Hi Ken, > > I've attached the output of dmesg. Here's the SCSI CDB for a sample drive= , > da3. > > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 > 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 > 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 > 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 > 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 > 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 > 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 > 00 00 00 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 > 00 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 > > > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've > tried this w/ 11.1, w/ the same results. > > > > Yes, the mpr(4) driver is in all recent FreeBSD releases. > > In looking at the dmesg, this is telling: > > (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 > (da0:mpr0:0:8:0): CAM status: SCSI Status Error > (da0:mpr0:0:8:0): SCSI status: Check Condition > (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, > format in progress) > (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete > (da0:mpr0:0:8:0): Error 16, Unretryable error > > If the drives are in the process of formatting, I guess it may make sense > for them to reject read commands. Otherwise, it makes no sense for a hard > drive to reject reads. > > Are you able to check the status of the format? You should be able to sen= d > a test unit ready and figure out how far along the format is: > > camcontrol tur da0 -v > > And so on for each of the drives. > > Ken > > ________________________________ > From: Kenneth D. Merry [ken@FreeBSD.ORG ken@FreeBSD.ORG> >] > Sent: Monday, October 16, 2017 7:42 AM > To: Shiva Bhanujan > Cc: freebsd-scsi@freebsd.org freebsd-scsi@freebsd.org> > > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 > PCI-Express Fusion-MPT SAS-3 > > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: > Hello, > > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 > controller is in pcipassthrough mode to the FreeBSD VM. When I try to > access the disks (/dev/da0...) using gpart, I get SCSI errors, like the > following: > > > > (da0:mpr0:0:0:0): CAM status: SCSI Status Error > > (da0:mpr0:0:0:0): SCSI status: Check Condition > > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command > operation code) > > > > The error message above is missing the SCSI CDB. We need that in order to > figure out what command the drive is complaining about. > > The error message means that FreeBSD is sending a SCSI command that the > drive doesn't support. That can be benign, or it can cause a problem. > > So, what error does gpart give you when you have this problem? > > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also ha= s > the same result. > > > > > Please note, that these errors don't show up on a Fusion-MPT SAS-2 > controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 > doesn't have any SCSI errors on the HBA330 SAS-3 controller either. > > > > > Is there a different version of the mpr driver I should be using? I > haven't checked the differences between the mpr driver in 10.2 vs 10.3 an= d > 11.0. I do see that there are others who have experienced these issues. C= an > somebody please provide me some pointers > as to why this is occurring? Or if there are some driver changes that I > might be able to incorporate? > > > > In general, the latest mpr(4) driver is the best one. The driver itself > generally doesn't send SCSI commands (there are a few exceptions), but > rather passes them through from the upper layers of CAM. > > Please note, that I have gone through the mail titled "scsi error at > SEAGATE ST1200MM0088 TT31" and have started sg_format on all the SEAGATE > disks. Having said that, I still need to figure out what would happen, if > the disks > were written to using FreeBSD 10.2, which doesn't seem to have SCSI > errors, and when I try to upgrade to 10.3. Any help is appreciated. > > > > Send the full error messages, and we may be able to figure out what's goi= ng > on. > > Ken > -- > Kenneth Merry > ken@FreeBSD.ORG ken@FreeBSD.ORG>> > > ________________________________ > > > > -- > Kenneth Merry > ken@FreeBSD.ORG ken@FreeBSD.ORG>> > ________________________________ > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi < > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi> > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org > " > > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > > > > > > > > > > > > > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Fri Oct 20 00:51:19 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93010E4B960 for ; Fri, 20 Oct 2017 00:51:19 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-80.reflexion.net [208.70.210.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 43AAD2523 for ; Fri, 20 Oct 2017 00:51:18 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 20303 invoked from network); 20 Oct 2017 00:51:16 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 20 Oct 2017 00:51:16 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Thu, 19 Oct 2017 20:51:16 -0400 (EDT) Received: (qmail 30816 invoked from network); 20 Oct 2017 00:51:16 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 20 Oct 2017 00:51:16 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Thu, 19 Oct 2017 17:51:14 -0700 From: Shiva Bhanujan To: Steven Hartland , Ken Merry CC: "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uIAAihwAgAA6+IKAANcq6YAAAGPJgANMRoCAAABZAP//kIxpgACCBAD//7I10w== Date: Fri, 20 Oct 2017 00:51:14 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C36666F5@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Oct 2017 00:51:19 -0000 would it possible to by default read SCSI disks w/ type 2 protection in = 10.3 and above? Please note, that the issue that I'm facing, is that the = partitions are created in FreeBSD 10.2, and for the Seagate drives = ST4000NM0005, they are created w/ type 2 partition. read/write to these = disks in FreeBSD 10.2 works just fine. However, if I upgrade to FreeBSD = 10.3 and above, I get SCSI errors for only these disks. Would it be = possible, that the SCSI reads default to reading disks that have type 2 = partition? In such a case, removing type 2 partition would not be needed. all other params in the output of sg_readcap look the same for the = ST4000NM0023, where this issue isn't seen. root=40Filer:=7E =23 root=40Filer:=7E =23 camcontrol devlist =7C grep da0 at scbus2 target 0 lun 0 (pass1,da0) root=40Filer:=7E =23 root=40Filer:=7E =23 camcontrol devlist =7C grep da2 at scbus2 target 2 lun 0 (pass3,da2) root=40Filer:=7E =23 root=40Filer:=7E =23 sg_readcap --16 da0 Read Capacity results: Protection: prot_en=3D0, p_type=3D0, p_i_exponent=3D0 Logical block provisioning: lbpme=3D0, lbprz=3D0 Last logical block address=3D7814037167 (0x1d1c0beaf), Number of = logical blocks=3D7814037168 Logical block length=3D512 bytes Logical blocks per physical block exponent=3D0 Lowest aligned logical block address=3D0 Hence: Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB root=40Filer:=7E =23 root=40Filer:=7E =23 sg_readcap --16 da2 Read Capacity results: Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 =5Btype 2 = protection=5D Logical block provisioning: lbpme=3D0, lbprz=3D0 Last logical block address=3D7814037167 (0x1d1c0beaf), Number of = logical blocks=3D7814037168 Logical block length=3D512 bytes Logical blocks per physical block exponent=3D0 Lowest aligned logical block address=3D0 Hence: Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB root=40Filer:=7E =23 is there some default SCSI read that is configurable in FreeBSD 10.3 and = above? From: Steven Hartland =5Bkilling=40multiplay.co.uk=5D Sent: Thursday, October 19, 2017 3:23 PM To: Ken Merry; Shiva Bhanujan Cc: freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 With type 2 protection the ref tag has to match the LBA + N=20 Some info about it is here https://www.usenix.org/legacy/event/lsf07/tech/petersen.pdf https://www.seagate.com/files/staticfiles/docs/pdf/whitepaper/safeguarding-= data-from-corruption-technology-paper-tp621us.pdf From=20reading the seagate paper the only way to change the protection = level is to format. On Thu, 19 Oct 2017 at 22:43, Shiva Bhanujan = wrote: Here's the output of sg_readcap. =5Broot=40Filer-20-241 =7E=5D=23 sg_readcap --16 da1 Read Capacity results: Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 =5Btype 2 = protection=5D Logical block provisioning: lbpme=3D0, lbprz=3D0 Last logical block address=3D7814037167 (0x1d1c0beaf), Number of = logical blocks=3D7814037168 Logical block length=3D512 bytes Logical blocks per physical block exponent=3D0 Lowest aligned logical block address=3D0 Hence: Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB =5Broot=40Filer-20-241 =7E=5D=23 =5Broot=40Filer-20-241 =7E=5D=23 camcontrol modepage da1 -v -m 10 =7C grep = DPICZ DPICZ: 1 =5Broot=40Filer-20-241 =7E=5D=23 I did toggle the DPICZ on the drive from 1 to 0 and back. The sg_readcap = still shows 'type 2 protection', and that gpart still shows the SCSI = errors. I've narrowed this down to the Seagate ST4000NM0005, w/ a DOM of = 03/2016. We also have Constellation ES.3 drives model ST4000NM0023 that don't exhibit this issue. I did go through the URLs that you have mentioned and that's how I found = that using sg_format did address this issue. Again, that works only for = new drives and we have appliances that already have data, for which = sg_format isn't an option. Is this boiling down to the 'type 2 protection' tag that we see in = sg_readcap? if so, would there be a way to turn it off? From: Ken Merry =5Bken=40freebsd.org=5D Sent: Thursday, October 19, 2017 2:17 PM To: Shiva Bhanujan Cc:=20 freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 By the way, the message you referenced is here: https://lists.freebsd.org/pipermail/freebsd-scsi/2017-January/007237.html And there is more here: https://bugs.freenas.org/issues/14517 Ken =E2=80=94 Ken Merry ken=40FreeBSD.ORG On Oct 19, 2017, at 5:15 PM, Ken Merry wrote: What does sg_readcap =E2=80=9416 show for these drives? If it has type 2 protection turned on, check to see what the DCIPZ value = is in the control mode page: camcontrol modepage daX -v -m 10 If that is set to 0, add a -e to the above command line and see if you can = set it to 1. That may or may not help anything. The mpr(4) and mps(4) drivers try to support protection information if it = is turned on on the drive. So, they set the protection information if = protection information is turned on in the drive. For that reason, = setting the DCIPZ bit may not fix it. There could be a problem with how that is implemented that=E2=80=99s = causing the drives to reject the command, but I=E2=80=99m not sure. If it is, Steve (CCed) can help us debug it. Ken =E2=80=94 Ken Merry ken=40FreeBSD.ORG On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan = wrote: Sorry. perhaps I have failed to mention, the SCSI errors are only w/ the = seagate drives. These are of the model ST4000NM0023. From: Shiva Bhanujan Sent: Tuesday, October 17, 2017 6:53 PM To: Kenneth D. Merry Cc:=20 freebsd-scsi=40freebsd.org Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? From:=20 owner-freebsd-scsi=40freebsd.org = =5Bowner-freebsd-scsi=40freebsd.org =5D on behalf of Shiva Bhanujan = =5Bshiva.bhanujan=40quorum.net =5D Sent: Tuesday, October 17, 2017 6:08 AM To: Kenneth D. Merry Cc:=20 freebsd-scsi=40freebsd.org Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Since I started having the SCSI errors, I ended up running sg_format to = format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete once the format is done, I can successfully format and partition using = gpart. The errors that show up when I try to run gpart for the first time are as = follows: (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da9:mpr0:0:17:0): CAM status: SCSI Status Error (da9:mpr0:0:17:0): SCSI status: Check Condition (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) (da9:mpr0:0:17:0): Error 22, Unretryable error It seems that it's the read that is failing, and is being tagged as an = illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 or 11.x might be an issue? ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG =5D Sent: Monday, October 16, 2017 7:31 PM To: Shiva Bhanujan Cc:=20 freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: Hi Ken, I've attached the output of dmesg. Here's the SCSI CDB for a sample drive, = da3. (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've = tried this w/ 11.1, w/ the same results. Yes, the mpr(4) driver is in all recent FreeBSD releases. In looking at the dmesg, this is telling: (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete (da0:mpr0:0:8:0): Error 16, Unretryable error If the drives are in the process of formatting, I guess it may make sense for them to reject read commands. Otherwise, it makes no sense for a hard drive to reject reads. Are you able to check the status of the format? You should be able to send a test unit ready and figure out how far along the format is: camcontrol tur da0 -v And so on for each of the drives. Ken ________________________________ From: Kenneth D. Merry =5Bken=40FreeBSD.ORG = >=5D Sent: Monday, October 16, 2017 7:42 AM To: Shiva Bhanujan Cc:=20 freebsd-scsi=40freebsd.org = > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: Hello, I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 = controller is in pcipassthrough mode to the FreeBSD VM. When I try to = access the disks (/dev/da0...) using gpart, I get SCSI errors, like the = following: (da0:mpr0:0:0:0): CAM status: SCSI Status Error (da0:mpr0:0:0:0): SCSI status: Check Condition (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) The error message above is missing the SCSI CDB. We need that in order to figure out what command the drive is complaining about. The error message means that FreeBSD is sending a SCSI command that the drive doesn't support. That can be benign, or it can cause a problem. So, what error does gpart give you when you have this problem? I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also has = the same result. Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 = doesn't have any SCSI errors on the HBA330 SAS-3 controller either. Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 and = 11.0. I do see that there are others who have experienced these issues. = Can somebody please provide me some pointers as to why this is occurring? Or if there are some driver changes that I = might be able to incorporate? In general, the latest mpr(4) driver is the best one. The driver itself generally doesn't send SCSI commands (there are a few exceptions), but rather passes them through from the upper layers of CAM. Please note, that I have gone through the mail titled =22scsi error at = SEAGATE ST1200MM0088 TT31=22 and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, if = the disks were written to using FreeBSD 10.2, which doesn't seem to have SCSI = errors, and when I try to upgrade to 10.3. Any help is appreciated. Send the full error messages, and we may be able to figure out what's going on. Ken -- Kenneth Merry ken=40FreeBSD.ORG > > ________________________________ -- Kenneth Merry ken=40FreeBSD.ORG > ________________________________ _______________________________________________ freebsd-scsi=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-scsi = To unsubscribe, send any mail to =22freebsd-scsi-unsubscribe=40freebsd.org = =22 _______________________________________________ freebsd-scsi=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-scsi To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org=22 _______________________________________________ freebsd-scsi=40freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-scsi To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org=22 From owner-freebsd-scsi@freebsd.org Fri Oct 20 02:09:28 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3068E4CC83 for ; Fri, 20 Oct 2017 02:09:28 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B0BD633F0 for ; Fri, 20 Oct 2017 02:09:27 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from [10.0.0.114] ([10.0.0.114]) (authenticated bits=0) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPSA id v9K29Pmj074042 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Oct 2017 22:09:26 -0400 (EDT) (envelope-from ken@freebsd.org) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 From: Ken Merry X-Mailer: iPhone Mail (15A432) In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C36666F5@QLEXC01.Quorum.local> Date: Thu, 19 Oct 2017 22:09:25 -0400 Cc: Steven Hartland , "freebsd-scsi@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <96BAD947-4AB0-4EAC-9DA8-4B1F10253287@freebsd.org> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C36666F5@QLEXC01.Quorum.local> To: Shiva Bhanujan X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [96.89.93.250]); Thu, 19 Oct 2017 22:09:26 -0400 (EDT) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Oct 2017 02:09:28 -0000 There are probably 3 solutions to get things fixed for you: 1. Track down why the mpr driver is sending requests that the drive doesn=E2= =80=99t like and fix it. 2. Turn on DPICZ in the drive and disable setting protection information in t= he mpr driver. This will take commenting out the right code and recompiling t= he kernel. 3. Use an older version of FreeBSD to read the data and the reformat and wri= te it back. We need to do #1 for the sake of the users who will run into this. #2 hopefu= lly will work and won=E2=80=99t require running an old OS. #3 is essentially= giving up. For #2, If you feel comfortable modifying the mpr driver, just look for the e= edp code and turn it off. Otherwise I can try to come up with something tomo= rrow. Ken --=20 Ken Merry ken@FreeBSD.ORG > On Oct 19, 2017, at 20:51, Shiva Bhanujan wrot= e: >=20 > would it possible to by default read SCSI disks w/ type 2 protection in 10= .3 and above? Please note, that the issue that I'm facing, is that the part= itions are created in FreeBSD 10.2, and for the Seagate drives ST4000NM0005,= they are created w/ type 2 partition. read/write to these disks in FreeBSD= 10.2 works just fine. However, if I upgrade to FreeBSD 10.3 and above, I g= et SCSI errors for only these disks. Would it be possible, that the SCSI re= ads default to reading disks that have type 2 partition? In such a case, re= moving type 2 partition would not be needed. >=20 > all other params in the output of sg_readcap look the same for the ST4000N= M0023, where this issue isn't seen. >=20 > root@Filer:~ # > root@Filer:~ # camcontrol devlist | grep da0 > at scbus2 target 0 lun 0 (pass1,da0) > root@Filer:~ # > root@Filer:~ # camcontrol devlist | grep da2 > at scbus2 target 2 lun 0 (pass3,da2) > root@Filer:~ # > root@Filer:~ # sg_readcap --16 da0 > Read Capacity results: > Protection: prot_en=3D0, p_type=3D0, p_i_exponent=3D0 > Logical block provisioning: lbpme=3D0, lbprz=3D0 > Last logical block address=3D7814037167 (0x1d1c0beaf), Number of logical= blocks=3D7814037168 > Logical block length=3D512 bytes > Logical blocks per physical block exponent=3D0 > Lowest aligned logical block address=3D0 > Hence: > Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB > root@Filer:~ # > root@Filer:~ # sg_readcap --16 da2 > Read Capacity results: > Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 [type 2 protection= ] > Logical block provisioning: lbpme=3D0, lbprz=3D0 > Last logical block address=3D7814037167 (0x1d1c0beaf), Number of logical= blocks=3D7814037168 > Logical block length=3D512 bytes > Logical blocks per physical block exponent=3D0 > Lowest aligned logical block address=3D0 > Hence: > Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB > root@Filer:~ # >=20 >=20 > is there some default SCSI read that is configurable in FreeBSD 10.3 and a= bove? >=20 >=20 >=20 >=20 >=20 >=20 >=20 > From: Steven Hartland [killing@multiplay.co.uk] >=20 > Sent: Thursday, October 19, 2017 3:23 PM >=20 > To: Ken Merry; Shiva Bhanujan >=20 > Cc: freebsd-scsi@freebsd.org >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-= Express Fusion-MPT SAS-3 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > With type 2 protection the ref tag has to match the LBA + N=20 >=20 > Some info about it is here >=20 > https://www.usenix.org/legacy/event/lsf07/tech/petersen.pdf >=20 >=20 > https://www.seagate.com/files/staticfiles/docs/pdf/whitepaper/safeguarding= -data-from-corruption-technology-paper-tp621us.pdf >=20 > =46rom reading the seagate paper the only way to change the protection lev= el is to format. >=20 >=20 >=20 >=20 > On Thu, 19 Oct 2017 at 22:43, Shiva Bhanujan w= rote: >=20 >=20 >=20 > Here's the output of sg_readcap. >=20 >=20 >=20 > [root@Filer-20-241 ~]# sg_readcap --16 da1 >=20 > Read Capacity results: >=20 > Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 [type 2 protection= ] >=20 > Logical block provisioning: lbpme=3D0, lbprz=3D0 >=20 > Last logical block address=3D7814037167 (0x1d1c0beaf), Number of logical= blocks=3D7814037168 >=20 > Logical block length=3D512 bytes >=20 > Logical blocks per physical block exponent=3D0 >=20 > Lowest aligned logical block address=3D0 >=20 > Hence: >=20 > Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB >=20 > [root@Filer-20-241 ~]# >=20 >=20 >=20 > [root@Filer-20-241 ~]# camcontrol modepage da1 -v -m 10 | grep DPICZ >=20 > DPICZ: 1 >=20 > [root@Filer-20-241 ~]# >=20 >=20 >=20 > I did toggle the DPICZ on the drive from 1 to 0 and back. The sg_readcap s= till shows 'type 2 protection', and that gpart still shows the SCSI errors. = I've narrowed this down to the Seagate ST4000NM0005, w/ a DOM of 03/2016. W= e also have Constellation ES.3 > drives model ST4000NM0023 that don't exhibit this issue. >=20 >=20 >=20 > I did go through the URLs that you have mentioned and that's how I found t= hat using sg_format did address this issue. Again, that works only for new d= rives and we have appliances that already have data, for which sg_format isn= 't an option. >=20 >=20 >=20 > Is this boiling down to the 'type 2 protection' tag that we see in sg_read= cap? if so, would there be a way to turn it off? >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > From: Ken Merry [ken@freebsd.org] >=20 >=20 >=20 > Sent: Thursday, October 19, 2017 2:17 PM >=20 >=20 >=20 > To: Shiva Bhanujan >=20 >=20 >=20 > Cc:=20 > freebsd-scsi@freebsd.org >=20 >=20 >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-= Express Fusion-MPT SAS-3 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > By the way, the message you referenced is here: >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > https://lists.freebsd.org/pipermail/freebsd-scsi/2017-January/007237.html >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > And there is more here: >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > https://bugs.freenas.org/issues/14517 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Ken >=20 >=20 >=20 >=20 >=20 >=20 >=20 > =E2=80=94 >=20 >=20 >=20 > Ken Merry >=20 >=20 >=20 > ken@FreeBSD.ORG >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > On Oct 19, 2017, at 5:15 PM, Ken Merry wrote: >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > What does sg_readcap =E2=80=9416 show for these drives? >=20 >=20 >=20 > If it has type 2 protection turned on, check to see what the DCIPZ value i= s in the control mode page: >=20 >=20 >=20 > camcontrol modepage daX -v -m 10 >=20 >=20 >=20 > If that is set to 0, add a -e to the above command line and see if you can= set it to 1. That may or may not help anything. >=20 >=20 >=20 > The mpr(4) and mps(4) drivers try to support protection information if it i= s turned on on the drive. So, they set the protection information if protec= tion information is turned on in the drive. For that reason, setting the DC= IPZ bit may not fix it. >=20 >=20 >=20 > There could be a problem with how that is implemented that=E2=80=99s causi= ng the drives to reject the command, but I=E2=80=99m not sure. >=20 >=20 >=20 > If it is, Steve (CCed) can help us debug it. >=20 >=20 >=20 > Ken >=20 > =E2=80=94 >=20 > Ken Merry >=20 > ken@FreeBSD.ORG >=20 >=20 >=20 >=20 >=20 >=20 >=20 > On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan wr= ote: >=20 >=20 >=20 > Sorry. perhaps I have failed to mention, the SCSI errors are only w/ the s= eagate drives. These are of the model ST4000NM0023. >=20 >=20 >=20 >=20 >=20 > From: Shiva Bhanujan >=20 > Sent: Tuesday, October 17, 2017 6:53 PM >=20 > To: Kenneth D. Merry >=20 > Cc:=20 > freebsd-scsi@freebsd.org >=20 > Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-= Express Fusion-MPT SAS-3 >=20 >=20 >=20 > Please note, that this isn't an issue w/ Toshiba drives. is this a firmwa= re issue by any chance? >=20 >=20 >=20 >=20 >=20 >=20 >=20 > From:=20 > owner-freebsd-scsi@freebsd.org [ow= ner-freebsd-scsi@freebsd.org > ] on behalf of Shiva Bhanujan [shiv= a.bhanujan@quorum.net > ] >=20 > Sent: Tuesday, October 17, 2017 6:08 AM >=20 > To: Kenneth D. Merry >=20 > Cc:=20 > freebsd-scsi@freebsd.org >=20 > Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-= Express Fusion-MPT SAS-3 >=20 >=20 >=20 > Since I started having the SCSI errors, I ended up running sg_format to fo= rmat the disks. I've found that once the disks are formatted using sg_format= , there are no SCSI errors. The errors that show up during the format are to= wards the end of the dmesg output. >=20 >=20 >=20 > (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, f= ormat in progress) >=20 > (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >=20 >=20 >=20 > once the format is done, I can successfully format and partition using gpa= rt. >=20 >=20 >=20 >=20 >=20 > The errors that show up when I try to run gpart for the first time are as f= ollows: >=20 >=20 >=20 > (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 > (da9:mpr0:0:17:0): CAM status: SCSI Status Error >=20 > (da9:mpr0:0:17:0): SCSI status: Check Condition >=20 > (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command o= peration code) >=20 > (da9:mpr0:0:17:0): Error 22, Unretryable error >=20 >=20 >=20 >=20 >=20 > It seems that it's the read that is failing, and is being tagged as an ill= egal request. While sg_format will address the issue at hand, this isn't an o= ption for us, because there are appliances that were formatted using FreeBSD= 10.2, and an upgrade to 10.3 >=20 > or 11.x might be an issue? >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > ________________________________ >=20 > From: Kenneth D. Merry [ken@FreeBSD.ORG ] >=20 > Sent: Monday, October 16, 2017 7:31 PM >=20 > To: Shiva Bhanujan >=20 > Cc:=20 > freebsd-scsi@freebsd.org >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-= Express Fusion-MPT SAS-3 >=20 >=20 >=20 > On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: >=20 > Hi Ken, >=20 >=20 >=20 > I've attached the output of dmesg. Here's the SCSI CDB for a sample drive,= da3. >=20 >=20 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 0= 0 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 0= 0 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 0= 0 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 0= 0 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 0= 0 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 0= 0 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 0= 0 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 0= 0 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >=20 >=20 >=20 >=20 >=20 > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've tri= ed this w/ 11.1, w/ the same results. >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Yes, the mpr(4) driver is in all recent FreeBSD releases. >=20 >=20 >=20 > In looking at the dmesg, this is telling: >=20 >=20 >=20 > (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 >=20 > (da0:mpr0:0:8:0): CAM status: SCSI Status Error >=20 > (da0:mpr0:0:8:0): SCSI status: Check Condition >=20 > (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, f= ormat in progress) >=20 > (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >=20 > (da0:mpr0:0:8:0): Error 16, Unretryable error >=20 >=20 >=20 > If the drives are in the process of formatting, I guess it may make sense >=20 > for them to reject read commands. Otherwise, it makes no sense for a hard >=20 > drive to reject reads. >=20 >=20 >=20 > Are you able to check the status of the format? You should be able to send= >=20 > a test unit ready and figure out how far along the format is: >=20 >=20 >=20 > camcontrol tur da0 -v >=20 >=20 >=20 > And so on for each of the drives. >=20 >=20 >=20 > Ken >=20 >=20 >=20 > ________________________________ >=20 > From: Kenneth D. Merry [ken@FreeBSD.ORG > >] >=20 > Sent: Monday, October 16, 2017 7:42 AM >=20 > To: Shiva Bhanujan >=20 > Cc:=20 > freebsd-scsi@freebsd.org > > >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-= Express Fusion-MPT SAS-3 >=20 >=20 >=20 > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: >=20 > Hello, >=20 >=20 >=20 > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 SAS-3 c= ontroller is in pcipassthrough mode to the FreeBSD VM. When I try to access t= he disks (/dev/da0...) using gpart, I get SCSI errors, like the following: >=20 >=20 >=20 >=20 >=20 >=20 >=20 > (da0:mpr0:0:0:0): CAM status: SCSI Status Error >=20 >=20 >=20 > (da0:mpr0:0:0:0): SCSI status: Check Condition >=20 >=20 >=20 > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command op= eration code) >=20 >=20 >=20 >=20 >=20 >=20 >=20 > The error message above is missing the SCSI CDB. We need that in order to >=20 > figure out what command the drive is complaining about. >=20 >=20 >=20 > The error message means that FreeBSD is sending a SCSI command that the >=20 > drive doesn't support. That can be benign, or it can cause a problem. >=20 >=20 >=20 > So, what error does gpart give you when you have this problem? >=20 >=20 >=20 > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also has= the same result. >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Please note, that these errors don't show up on a Fusion-MPT SAS-2 control= ler, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 doesn't h= ave any SCSI errors on the HBA330 SAS-3 controller either. >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Is there a different version of the mpr driver I should be using? I haven'= t checked the differences between the mpr driver in 10.2 vs 10.3 and 11.0. I= do see that there are others who have experienced these issues. Can somebod= y please provide me some pointers >=20 > as to why this is occurring? Or if there are some driver changes that I mi= ght be able to incorporate? >=20 >=20 >=20 >=20 >=20 >=20 >=20 > In general, the latest mpr(4) driver is the best one. The driver itself >=20 > generally doesn't send SCSI commands (there are a few exceptions), but >=20 > rather passes them through from the upper layers of CAM. >=20 >=20 >=20 > Please note, that I have gone through the mail titled "scsi error at SEAGA= TE ST1200MM0088 TT31" and have started sg_format on all the SEAGATE disks. H= aving said that, I still need to figure out what would happen, if the disks >=20 > were written to using FreeBSD 10.2, which doesn't seem to have SCSI errors= , and when I try to upgrade to 10.3. Any help is appreciated. >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Send the full error messages, and we may be able to figure out what's goin= g >=20 > on. >=20 >=20 >=20 > Ken >=20 > -- >=20 > Kenneth Merry >=20 > ken@FreeBSD.ORG > > > > >=20 > ________________________________ >=20 >=20 >=20 >=20 >=20 >=20 >=20 > -- >=20 > Kenneth Merry >=20 > ken@FreeBSD.ORG > > >=20 > ________________________________ >=20 > _______________________________________________ >=20 > freebsd-scsi@freebsd.org > mailing list >=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >=20 > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org " >=20 >=20 >=20 >=20 >=20 > _______________________________________________ >=20 > freebsd-scsi@freebsd.org mailing list >=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >=20 > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > _______________________________________________ >=20 > freebsd-scsi@freebsd.org mailing list >=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >=20 > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 From owner-freebsd-scsi@freebsd.org Fri Oct 20 13:26:44 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CFA6E34690 for ; Fri, 20 Oct 2017 13:26:44 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-210-80.reflexion.net [208.70.210.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B8ED2760B9 for ; Fri, 20 Oct 2017 13:26:42 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 12970 invoked from network); 20 Oct 2017 13:26:41 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 20 Oct 2017 13:26:41 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Fri, 20 Oct 2017 09:26:41 -0400 (EDT) Received: (qmail 20394 invoked from network); 20 Oct 2017 13:26:40 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 20 Oct 2017 13:26:40 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Fri, 20 Oct 2017 06:26:39 -0700 From: Shiva Bhanujan To: Ken Merry CC: Steven Hartland , "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uIAAihwAgAA6+IKAANcq6YAAAGPJgANMRoCAAABZAP//kIxpgACCBAD//7I104AAjO6AgABG8C8= Date: Fri, 20 Oct 2017 13:26:38 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C366674D@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C36666F5@QLEXC01.Quorum.local>, <96BAD947-4AB0-4EAC-9DA8-4B1F10253287@freebsd.org> In-Reply-To: <96BAD947-4AB0-4EAC-9DA8-4B1F10253287@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Oct 2017 13:26:44 -0000 =5B3=5D isn't an option, because we're specifically moving to FreeBSD 10.3 = or above to get the 'restartable zfs send/receive' feature. I believe for = =5B2=5D, this would require not setting the 'type 2 protection' when = configured formatting the disks? We have appliances that are already = formatted, so we'll need to figure out how to address that part. I'm not familiar w/ the mpr driver source code. I can start looking =40 = it, but I'm afraid, I'm not going to make much progress. If you could = help out w/ addressing this issue in 10.3 w/ the protection bit set for = SCSI disks, that'd be great. From: Ken Merry =5Bken=40freebsd.org=5D Sent: Thursday, October 19, 2017 7:09 PM To: Shiva Bhanujan Cc: Steven Hartland; freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 There are probably 3 solutions to get things fixed for you: 1. Track down why the mpr driver is sending requests that the drive = doesn=E2=80=99t like and fix it. 2. Turn on DPICZ in the drive and disable setting protection information = in the mpr driver. This will take commenting out the right code and = recompiling the kernel. 3. Use an older version of FreeBSD to read the data and the reformat and = write it back. We need to do =231 for the sake of the users who will run into this. =232 = hopefully will work and won=E2=80=99t require running an old OS. =233 is = essentially giving up. For =232, If you feel comfortable modifying the mpr driver, just look for = the eedp code and turn it off. Otherwise I can try to come up with = something tomorrow. Ken --=20 Ken Merry ken=40FreeBSD.ORG > On Oct 19, 2017, at 20:51, Shiva Bhanujan = wrote: >=20 > would it possible to by default read SCSI disks w/ type 2 protection in = 10.3 and above? Please note, that the issue that I'm facing, is that the = partitions are created in FreeBSD 10.2, and for the Seagate drives = ST4000NM0005, they are created w/ type 2 partition. read/write to these disks in FreeBSD 10.2 works just fine. However, if I = upgrade to FreeBSD 10.3 and above, I get SCSI errors for only these disks. = Would it be possible, that the SCSI reads default to reading disks that = have type 2 partition? In such a case, removing type 2 partition would not be needed. >=20 > all other params in the output of sg_readcap look the same for the = ST4000NM0023, where this issue isn't seen. >=20 >=20 root=40Filer:=7E =23 >=20 root=40Filer:=7E =23 camcontrol devlist =7C grep da0 > at scbus2 target 0 lun 0 (pass1,da0) >=20 root=40Filer:=7E =23 >=20 root=40Filer:=7E =23 camcontrol devlist =7C grep da2 > at scbus2 target 2 lun 0 (pass3,da2) >=20 root=40Filer:=7E =23 >=20 root=40Filer:=7E =23 sg_readcap --16 da0 > Read Capacity results: > Protection: prot_en=3D0, p_type=3D0, p_i_exponent=3D0 > Logical block provisioning: lbpme=3D0, lbprz=3D0 > Last logical block address=3D7814037167 (0x1d1c0beaf), Number of logical = blocks=3D7814037168 > Logical block length=3D512 bytes > Logical blocks per physical block exponent=3D0 > Lowest aligned logical block address=3D0 > Hence: > Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB >=20 root=40Filer:=7E =23 >=20 root=40Filer:=7E =23 sg_readcap --16 da2 > Read Capacity results: > Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 =5Btype 2 = protection=5D > Logical block provisioning: lbpme=3D0, lbprz=3D0 > Last logical block address=3D7814037167 (0x1d1c0beaf), Number of logical = blocks=3D7814037168 > Logical block length=3D512 bytes > Logical blocks per physical block exponent=3D0 > Lowest aligned logical block address=3D0 > Hence: > Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB >=20 root=40Filer:=7E =23 >=20 >=20 > is there some default SCSI read that is configurable in FreeBSD 10.3 and = above? >=20 >=20 >=20 >=20 >=20 >=20 >=20 > From: Steven Hartland =5Bkilling=40multiplay.co.uk=5D >=20 > Sent: Thursday, October 19, 2017 3:23 PM >=20 > To: Ken Merry; Shiva Bhanujan >=20 > Cc:=20 freebsd-scsi=40freebsd.org >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > With type 2 protection the ref tag has to match the LBA + N=20 >=20 > Some info about it is here >=20 >=20 https://www.usenix.org/legacy/event/lsf07/tech/petersen.pdf >=20 >=20 >=20 https://www.seagate.com/files/staticfiles/docs/pdf/whitepaper/safeguarding-= data-from-corruption-technology-paper-tp621us.pdf >=20 > From reading the seagate paper the only way to change the protection = level is to format. >=20 >=20 >=20 >=20 > On Thu, 19 Oct 2017 at 22:43, Shiva Bhanujan = wrote: >=20 >=20 >=20 > Here's the output of sg_readcap. >=20 >=20 >=20 > =5Broot=40Filer-20-241 =7E=5D=23 sg_readcap --16 da1 >=20 > Read Capacity results: >=20 > Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 =5Btype 2 = protection=5D >=20 > Logical block provisioning: lbpme=3D0, lbprz=3D0 >=20 > Last logical block address=3D7814037167 (0x1d1c0beaf), Number of logical = blocks=3D7814037168 >=20 > Logical block length=3D512 bytes >=20 > Logical blocks per physical block exponent=3D0 >=20 > Lowest aligned logical block address=3D0 >=20 > Hence: >=20 > Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB >=20 > =5Broot=40Filer-20-241 =7E=5D=23 >=20 >=20 >=20 > =5Broot=40Filer-20-241 =7E=5D=23 camcontrol modepage da1 -v -m 10 =7C = grep DPICZ >=20 > DPICZ: 1 >=20 > =5Broot=40Filer-20-241 =7E=5D=23 >=20 >=20 >=20 > I did toggle the DPICZ on the drive from 1 to 0 and back. The sg_readcap = still shows 'type 2 protection', and that gpart still shows the SCSI = errors. I've narrowed this down to the Seagate ST4000NM0005, w/ a DOM of = 03/2016. We also have Constellation ES.3 > drives model ST4000NM0023 that don't exhibit this issue. >=20 >=20 >=20 > I did go through the URLs that you have mentioned and that's how I found = that using sg_format did address this issue. Again, that works only for = new drives and we have appliances that already have data, for which = sg_format isn't an option. >=20 >=20 >=20 > Is this boiling down to the 'type 2 protection' tag that we see in = sg_readcap? if so, would there be a way to turn it off? >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > From: Ken Merry =5Bken=40freebsd.org=5D >=20 >=20 >=20 > Sent: Thursday, October 19, 2017 2:17 PM >=20 >=20 >=20 > To: Shiva Bhanujan >=20 >=20 >=20 > Cc:=20 >=20 freebsd-scsi=40freebsd.org >=20 >=20 >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > By the way, the message you referenced is here: >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 https://lists.freebsd.org/pipermail/freebsd-scsi/2017-January/007237.html >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > And there is more here: >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 https://bugs.freenas.org/issues/14517 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Ken >=20 >=20 >=20 >=20 >=20 >=20 >=20 > =E2=80=94 >=20 >=20 >=20 > Ken Merry >=20 >=20 >=20 >=20 ken=40FreeBSD.ORG >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > On Oct 19, 2017, at 5:15 PM, Ken Merry wrote: >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > What does sg_readcap =E2=80=9416 show for these drives? >=20 >=20 >=20 > If it has type 2 protection turned on, check to see what the DCIPZ value = is in the control mode page: >=20 >=20 >=20 > camcontrol modepage daX -v -m 10 >=20 >=20 >=20 > If that is set to 0, add a -e to the above command line and see if you = can set it to 1. That may or may not help anything. >=20 >=20 >=20 > The mpr(4) and mps(4) drivers try to support protection information if = it is turned on on the drive. So, they set the protection information if = protection information is turned on in the drive. For that reason, setting = the DCIPZ bit may not fix it. >=20 >=20 >=20 > There could be a problem with how that is implemented that=E2=80=99s = causing the drives to reject the command, but I=E2=80=99m not sure. >=20 >=20 >=20 > If it is, Steve (CCed) can help us debug it. >=20 >=20 >=20 > Ken >=20 > =E2=80=94 >=20 > Ken Merry >=20 >=20 ken=40FreeBSD.ORG >=20 >=20 >=20 >=20 >=20 >=20 >=20 > On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan = wrote: >=20 >=20 >=20 > Sorry. perhaps I have failed to mention, the SCSI errors are only w/ the = seagate drives. These are of the model ST4000NM0023. >=20 >=20 >=20 >=20 >=20 > From: Shiva Bhanujan >=20 > Sent: Tuesday, October 17, 2017 6:53 PM >=20 > To: Kenneth D. Merry >=20 > Cc:=20 >=20 freebsd-scsi=40freebsd.org >=20 > Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >=20 >=20 > Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? >=20 >=20 >=20 >=20 >=20 >=20 >=20 > From:=20 >=20 owner-freebsd-scsi=40freebsd.org = =5Bowner-freebsd-scsi=40freebsd.org > =5D on behalf of Shiva Bhanujan = =5Bshiva.bhanujan=40quorum.net > =5D >=20 > Sent: Tuesday, October 17, 2017 6:08 AM >=20 > To: Kenneth D. Merry >=20 > Cc:=20 >=20 freebsd-scsi=40freebsd.org >=20 > Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >=20 >=20 > Since I started having the SCSI errors, I ended up running sg_format to = format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. >=20 >=20 >=20 > (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) >=20 > (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >=20 >=20 >=20 > once the format is done, I can successfully format and partition using = gpart. >=20 >=20 >=20 >=20 >=20 > The errors that show up when I try to run gpart for the first time are = as follows: >=20 >=20 >=20 > (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 > (da9:mpr0:0:17:0): CAM status: SCSI Status Error >=20 > (da9:mpr0:0:17:0): SCSI status: Check Condition >=20 > (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) >=20 > (da9:mpr0:0:17:0): Error 22, Unretryable error >=20 >=20 >=20 >=20 >=20 > It seems that it's the read that is failing, and is being tagged as an = illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 >=20 > or 11.x might be an issue? >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > ________________________________ >=20 > From: Kenneth D. Merry =5Bken=40FreeBSD.ORG =5D >=20 > Sent: Monday, October 16, 2017 7:31 PM >=20 > To: Shiva Bhanujan >=20 > Cc:=20 >=20 freebsd-scsi=40freebsd.org >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >=20 >=20 > On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: >=20 > Hi Ken, >=20 >=20 >=20 > I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. >=20 >=20 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 01 = 00 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 01 = 00 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 01 = 00 00 00 >=20 > (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 01 = 00 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >=20 > (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >=20 >=20 >=20 >=20 >=20 > My understanding is that FreeBSD 11.1 contains the mpr(4) driver? I've = tried this w/ 11.1, w/ the same results. >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Yes, the mpr(4) driver is in all recent FreeBSD releases. >=20 >=20 >=20 > In looking at the dmesg, this is telling: >=20 >=20 >=20 > (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 >=20 > (da0:mpr0:0:8:0): CAM status: SCSI Status Error >=20 > (da0:mpr0:0:8:0): SCSI status: Check Condition >=20 > (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not ready, = format in progress) >=20 > (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >=20 > (da0:mpr0:0:8:0): Error 16, Unretryable error >=20 >=20 >=20 > If the drives are in the process of formatting, I guess it may make sense >=20 > for them to reject read commands. Otherwise, it makes no sense for a hard >=20 > drive to reject reads. >=20 >=20 >=20 > Are you able to check the status of the format? You should be able to send >=20 > a test unit ready and figure out how far along the format is: >=20 >=20 >=20 > camcontrol tur da0 -v >=20 >=20 >=20 > And so on for each of the drives. >=20 >=20 >=20 > Ken >=20 >=20 >=20 > ________________________________ >=20 > From: Kenneth D. Merry =5Bken=40FreeBSD.ORG = > >=5D >=20 > Sent: Monday, October 16, 2017 7:42 AM >=20 > To: Shiva Bhanujan >=20 > Cc:=20 >=20 freebsd-scsi=40freebsd.org = > > >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >=20 >=20 > On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: >=20 > Hello, >=20 >=20 >=20 > I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like the = following: >=20 >=20 >=20 >=20 >=20 >=20 >=20 > (da0:mpr0:0:0:0): CAM status: SCSI Status Error >=20 >=20 >=20 > (da0:mpr0:0:0:0): SCSI status: Check Condition >=20 >=20 >=20 > (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command = operation code) >=20 >=20 >=20 >=20 >=20 >=20 >=20 > The error message above is missing the SCSI CDB. We need that in order to >=20 > figure out what command the drive is complaining about. >=20 >=20 >=20 > The error message means that FreeBSD is sending a SCSI command that the >=20 > drive doesn't support. That can be benign, or it can cause a problem. >=20 >=20 >=20 > So, what error does gpart give you when you have this problem? >=20 >=20 >=20 > I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively also = has the same result. >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD 10.2 = doesn't have any SCSI errors on the HBA330 SAS-3 controller either. >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 and = 11.0. I do see that there are others who have experienced these issues. = Can somebody please provide me some pointers >=20 > as to why this is occurring? Or if there are some driver changes that I = might be able to incorporate? >=20 >=20 >=20 >=20 >=20 >=20 >=20 > In general, the latest mpr(4) driver is the best one. The driver itself >=20 > generally doesn't send SCSI commands (there are a few exceptions), but >=20 > rather passes them through from the upper layers of CAM. >=20 >=20 >=20 > Please note, that I have gone through the mail titled =22scsi error at = SEAGATE ST1200MM0088 TT31=22 and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, if = the disks >=20 > were written to using FreeBSD 10.2, which doesn't seem to have SCSI = errors, and when I try to upgrade to 10.3. Any help is appreciated. >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Send the full error messages, and we may be able to figure out what's = going >=20 > on. >=20 >=20 >=20 > Ken >=20 > -- >=20 > Kenneth Merry >=20 >=20 ken=40FreeBSD.ORG > > > > >=20 > ________________________________ >=20 >=20 >=20 >=20 >=20 >=20 >=20 > -- >=20 > Kenneth Merry >=20 >=20 ken=40FreeBSD.ORG > > >=20 > ________________________________ >=20 > _______________________________________________ >=20 >=20 freebsd-scsi=40freebsd.org > mailing list >=20 >=20 https://lists.freebsd.org/mailman/listinfo/freebsd-scsi = >=20 > To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org = =22 >=20 >=20 >=20 >=20 >=20 > _______________________________________________ >=20 >=20 freebsd-scsi=40freebsd.org mailing list >=20 >=20 https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >=20 > To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org=22 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > _______________________________________________ >=20 >=20 freebsd-scsi=40freebsd.org mailing list >=20 >=20 https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >=20 > To unsubscribe, send any mail to = =22freebsd-scsi-unsubscribe=40freebsd.org=22 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 From owner-freebsd-scsi@freebsd.org Fri Oct 20 14:34:58 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2F32E374E1 for ; Fri, 20 Oct 2017 14:34:58 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 789F47D183 for ; Fri, 20 Oct 2017 14:34:58 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from [10.0.0.26] (mbp2013.int.kdm.org [10.0.0.26]) (authenticated bits=0) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPSA id v9KEYts5085628 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 20 Oct 2017 10:34:56 -0400 (EDT) (envelope-from ken@freebsd.org) From: Ken Merry Message-Id: <95A66B26-548C-4BF5-9527-EE30F9C01D42@freebsd.org> Content-Type: multipart/mixed; boundary="Apple-Mail=_C43579B3-638F-4C3F-B4A7-22EDAD9BDBE9" Mime-Version: 1.0 (Mac OS X Mail 11.0 \(3445.1.7\)) Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Date: Fri, 20 Oct 2017 10:34:54 -0400 In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C366674D@QLEXC01.Quorum.local> Cc: Steven Hartland , "freebsd-scsi@freebsd.org" To: Shiva Bhanujan References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C36666F5@QLEXC01.Quorum.local> <96BAD947-4AB0-4EAC-9DA8-4B1F10253287@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366674D@QLEXC01.Quorum.local> X-Mailer: Apple Mail (2.3445.1.7) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [96.89.93.250]); Fri, 20 Oct 2017 10:34:56 -0400 (EDT) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Oct 2017 14:34:58 -0000 --Apple-Mail=_C43579B3-638F-4C3F-B4A7-22EDAD9BDBE9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Ok. Yes, for #2, in theory we can disable EEDP / protection information = in the mpr(4) driver and if DPICZ is set, the drive won=E2=80=99t = require setting protection information on read and write commands. That should let you access the disks normally. That said, I=E2=80=99ve never played with protection information before, = so I don=E2=80=99t know for sure. I have a drive that supports it and = I=E2=80=99m formatting it now to turn on type 2 protection I may be = able to debug things once I get that done. In the mean time, I=E2=80=99ve attached a patch against the stable/10 = version of the mpr(4) driver. Apply this patch, and rebuild and = reinstall your kernel. Then you=E2=80=99ll be able to disable EEDP in = the driver three different ways: 1. Set hw.mpr.disable_eedp=3D1 in /boot/loader.conf. That will disable = EEDP / Protection Information for all mpr instances. 2. Set dev.mpr.0.disable_eedp=3D1 in /boot/loader.conf. That will = disable EEDP for mpr0. 3. sysctl dev.mpr.0.disable_eedp=3D1 That will disable EEDP on the = fly for mpr0. Let me know what happens. Ken =E2=80=94=20 Ken Merry ken@FreeBSD.ORG --Apple-Mail=_C43579B3-638F-4C3F-B4A7-22EDAD9BDBE9 Content-Disposition: attachment; filename=mpr_eedp_diffs.stable10.20171020.txt Content-Type: text/plain; x-unix-mode=0600; name="mpr_eedp_diffs.stable10.20171020.txt" Content-Transfer-Encoding: quoted-printable =3D=3D=3D=3D //depot/users/kenm/FreeBSD-stable/10/sys/dev/mpr/mpr.c#5 - = /usr/home/kenm/perforce4/kenm/FreeBSD-stable/10/sys/dev/mpr/mpr.c =3D=3D=3D= =3D *** /tmp/tmp.44697.83 Fri Oct 20 08:17:31 2017 --- /usr/home/kenm/perforce4/kenm/FreeBSD-stable/10/sys/dev/mpr/mpr.c = Fri Oct 20 07:46:02 2017 *************** *** 1513,1518 **** --- 1513,1519 ---- TUNABLE_INT_FETCH("hw.mpr.enable_ssu", &sc->enable_ssu); TUNABLE_INT_FETCH("hw.mpr.spinup_wait_time", = &sc->spinup_wait_time); TUNABLE_INT_FETCH("hw.mpr.use_phy_num", &sc->use_phynum); + TUNABLE_INT_FETCH("hw.mpr.disable_eedp", &sc->disable_eedp); =20 /* Grab the unit-instance variables */ snprintf(tmpstr, sizeof(tmpstr), "dev.mpr.%d.debug_level", *************** *** 1551,1556 **** --- 1552,1561 ---- snprintf(tmpstr, sizeof(tmpstr), "dev.mpr.%d.use_phy_num", device_get_unit(sc->mpr_dev)); TUNABLE_INT_FETCH(tmpstr, &sc->use_phynum); +=20 + snprintf(tmpstr, sizeof(tmpstr), "dev.mpr.%d.disable_eedp", + device_get_unit(sc->mpr_dev)); + TUNABLE_INT_FETCH(tmpstr, &sc->disable_eedp); } =20 static void *************** *** 1646,1651 **** --- 1651,1660 ---- "Use the phy number for enumeration"); =20 SYSCTL_ADD_INT(sysctl_ctx, SYSCTL_CHILDREN(sysctl_tree), + OID_AUTO, "dsiable_eedp", CTLFLAG_RD, &sc->disable_eedp, 0, + "Disable Protection Info / EEDP"); +=20 + SYSCTL_ADD_INT(sysctl_ctx, SYSCTL_CHILDREN(sysctl_tree), OID_AUTO, "prp_pages_free", CTLFLAG_RD, &sc->prp_pages_free, 0, "number of free PRP pages"); =20 =3D=3D=3D=3D = //depot/users/kenm/FreeBSD-stable/10/sys/dev/mpr/mpr_sas.c#8 - = /usr/home/kenm/perforce4/kenm/FreeBSD-stable/10/sys/dev/mpr/mpr_sas.c = =3D=3D=3D=3D *** /tmp/tmp.44697.181 Fri Oct 20 08:17:31 2017 --- = /usr/home/kenm/perforce4/kenm/FreeBSD-stable/10/sys/dev/mpr/mpr_sas.c = Fri Oct 20 07:48:13 2017 *************** *** 2071,2077 **** * for EEDP transfer. */ eedp_flags =3D op_code_prot[req->CDB.CDB32[0]]; ! if (sc->eedp_enabled && eedp_flags) { SLIST_FOREACH(lun, &targ->luns, lun_link) { if (lun->lun_id =3D=3D csio->ccb_h.target_lun) { break; --- 2071,2077 ---- * for EEDP transfer. */ eedp_flags =3D op_code_prot[req->CDB.CDB32[0]]; ! if (sc->eedp_enabled && sc->disable_eedp =3D=3D 0 && eedp_flags) = { SLIST_FOREACH(lun, &targ->luns, lun_link) { if (lun->lun_id =3D=3D csio->ccb_h.target_lun) { break; =3D=3D=3D=3D //depot/users/kenm/FreeBSD-stable/10/sys/dev/mpr/mprvar.h#3 = - /usr/home/kenm/perforce4/kenm/FreeBSD-stable/10/sys/dev/mpr/mprvar.h = =3D=3D=3D=3D *** /tmp/tmp.44697.281 Fri Oct 20 08:17:31 2017 --- /usr/home/kenm/perforce4/kenm/FreeBSD-stable/10/sys/dev/mpr/mprvar.h = Fri Oct 20 07:44:26 2017 *************** *** 297,302 **** --- 297,303 ---- u_int enable_ssu; int spinup_wait_time; int use_phynum; + int disable_eedp; uint64_t chain_alloc_fail; uint64_t prp_page_alloc_fail; struct sysctl_ctx_list sysctl_ctx; --Apple-Mail=_C43579B3-638F-4C3F-B4A7-22EDAD9BDBE9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Oct 20, 2017, at 9:26 AM, Shiva Bhanujan = wrote: >=20 > [3] isn't an option, because we're specifically moving to FreeBSD 10.3 = or above to get the 'restartable zfs send/receive' feature. I believe = for [2], this would require not setting the 'type 2 protection' when = configured formatting the disks? We have appliances that are already = formatted, so we'll need to figure out how to address that part. >=20 > I'm not familiar w/ the mpr driver source code. I can start looking @ = it, but I'm afraid, I'm not going to make much progress. If you could = help out w/ addressing this issue in 10.3 w/ the protection bit set for = SCSI disks, that'd be great. >=20 >=20 >=20 >=20 >=20 > From: Ken Merry [ken@freebsd.org] >=20 > Sent: Thursday, October 19, 2017 7:09 PM >=20 > To: Shiva Bhanujan >=20 > Cc: Steven Hartland; freebsd-scsi@freebsd.org >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > There are probably 3 solutions to get things fixed for you: >=20 >=20 >=20 > 1. Track down why the mpr driver is sending requests that the drive = doesn=E2=80=99t like and fix it. >=20 >=20 >=20 > 2. Turn on DPICZ in the drive and disable setting protection = information in the mpr driver. This will take commenting out the right = code and recompiling the kernel. >=20 >=20 >=20 > 3. Use an older version of FreeBSD to read the data and the reformat = and write it back. >=20 >=20 >=20 > We need to do #1 for the sake of the users who will run into this. #2 = hopefully will work and won=E2=80=99t require running an old OS. #3 is = essentially giving up. >=20 >=20 >=20 > For #2, If you feel comfortable modifying the mpr driver, just look = for the eedp code and turn it off. Otherwise I can try to come up with = something tomorrow. >=20 >=20 >=20 > Ken >=20 > --=20 >=20 > Ken Merry >=20 > ken@FreeBSD.ORG >=20 >=20 >=20 >> On Oct 19, 2017, at 20:51, Shiva Bhanujan = wrote: >=20 >>=20 >=20 >> would it possible to by default read SCSI disks w/ type 2 protection = in 10.3 and above? Please note, that the issue that I'm facing, is that = the partitions are created in FreeBSD 10.2, and for the Seagate drives = ST4000NM0005, they are created w/ type 2 partition. > read/write to these disks in FreeBSD 10.2 works just fine. However, if = I upgrade to FreeBSD 10.3 and above, I get SCSI errors for only these = disks. Would it be possible, that the SCSI reads default to reading = disks that have type 2 partition? In such a case, > removing type 2 partition would not be needed. >=20 >>=20 >=20 >> all other params in the output of sg_readcap look the same for the = ST4000NM0023, where this issue isn't seen. >=20 >>=20 >=20 >>=20 > root@Filer:~ # >=20 >>=20 > root@Filer:~ # camcontrol devlist | grep da0 >=20 >> at scbus2 target 0 lun 0 (pass1,da0) >=20 >>=20 > root@Filer:~ # >=20 >>=20 > root@Filer:~ # camcontrol devlist | grep da2 >=20 >> at scbus2 target 2 lun 0 (pass3,da2) >=20 >>=20 > root@Filer:~ # >=20 >>=20 > root@Filer:~ # sg_readcap --16 da0 >=20 >> Read Capacity results: >=20 >> Protection: prot_en=3D0, p_type=3D0, p_i_exponent=3D0 >=20 >> Logical block provisioning: lbpme=3D0, lbprz=3D0 >=20 >> Last logical block address=3D7814037167 (0x1d1c0beaf), Number of = logical blocks=3D7814037168 >=20 >> Logical block length=3D512 bytes >=20 >> Logical blocks per physical block exponent=3D0 >=20 >> Lowest aligned logical block address=3D0 >=20 >> Hence: >=20 >> Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB >=20 >>=20 > root@Filer:~ # >=20 >>=20 > root@Filer:~ # sg_readcap --16 da2 >=20 >> Read Capacity results: >=20 >> Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 [type 2 = protection] >=20 >> Logical block provisioning: lbpme=3D0, lbprz=3D0 >=20 >> Last logical block address=3D7814037167 (0x1d1c0beaf), Number of = logical blocks=3D7814037168 >=20 >> Logical block length=3D512 bytes >=20 >> Logical blocks per physical block exponent=3D0 >=20 >> Lowest aligned logical block address=3D0 >=20 >> Hence: >=20 >> Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB >=20 >>=20 > root@Filer:~ # >=20 >>=20 >=20 >>=20 >=20 >> is there some default SCSI read that is configurable in FreeBSD 10.3 = and above? >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> From: Steven Hartland [killing@multiplay.co.uk] >=20 >>=20 >=20 >> Sent: Thursday, October 19, 2017 3:23 PM >=20 >>=20 >=20 >> To: Ken Merry; Shiva Bhanujan >=20 >>=20 >=20 >> Cc:=20 > freebsd-scsi@freebsd.org >=20 >>=20 >=20 >> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> With type 2 protection the ref tag has to match the LBA + N=20 >=20 >>=20 >=20 >> Some info about it is here >=20 >>=20 >=20 >>=20 > https://www.usenix.org/legacy/event/lsf07/tech/petersen.pdf >=20 >>=20 >=20 >>=20 >=20 >>=20 > = https://www.seagate.com/files/staticfiles/docs/pdf/whitepaper/safeguarding= -data-from-corruption-technology-paper-tp621us.pdf >=20 >>=20 >=20 >> =46rom reading the seagate paper the only way to change the = protection level is to format. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> On Thu, 19 Oct 2017 at 22:43, Shiva Bhanujan = wrote: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Here's the output of sg_readcap. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> [root@Filer-20-241 ~]# sg_readcap --16 da1 >=20 >>=20 >=20 >> Read Capacity results: >=20 >>=20 >=20 >> Protection: prot_en=3D1, p_type=3D1, p_i_exponent=3D0 [type 2 = protection] >=20 >>=20 >=20 >> Logical block provisioning: lbpme=3D0, lbprz=3D0 >=20 >>=20 >=20 >> Last logical block address=3D7814037167 (0x1d1c0beaf), Number of = logical blocks=3D7814037168 >=20 >>=20 >=20 >> Logical block length=3D512 bytes >=20 >>=20 >=20 >> Logical blocks per physical block exponent=3D0 >=20 >>=20 >=20 >> Lowest aligned logical block address=3D0 >=20 >>=20 >=20 >> Hence: >=20 >>=20 >=20 >> Device size: 4000787030016 bytes, 3815447.8 MiB, 4000.79 GB >=20 >>=20 >=20 >> [root@Filer-20-241 ~]# >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> [root@Filer-20-241 ~]# camcontrol modepage da1 -v -m 10 | grep DPICZ >=20 >>=20 >=20 >> DPICZ: 1 >=20 >>=20 >=20 >> [root@Filer-20-241 ~]# >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> I did toggle the DPICZ on the drive from 1 to 0 and back. The = sg_readcap still shows 'type 2 protection', and that gpart still shows = the SCSI errors. I've narrowed this down to the Seagate ST4000NM0005, w/ = a DOM of 03/2016. We also have Constellation ES.3 >=20 >> drives model ST4000NM0023 that don't exhibit this issue. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> I did go through the URLs that you have mentioned and that's how I = found that using sg_format did address this issue. Again, that works = only for new drives and we have appliances that already have data, for = which sg_format isn't an option. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Is this boiling down to the 'type 2 protection' tag that we see in = sg_readcap? if so, would there be a way to turn it off? >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> From: Ken Merry [ken@freebsd.org] >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Sent: Thursday, October 19, 2017 2:17 PM >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> To: Shiva Bhanujan >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Cc:=20 >=20 >>=20 > freebsd-scsi@freebsd.org >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> By the way, the message you referenced is here: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 > = https://lists.freebsd.org/pipermail/freebsd-scsi/2017-January/007237.html >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> And there is more here: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 > https://bugs.freenas.org/issues/14517 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Ken >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> =E2=80=94 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Ken Merry >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 > ken@FreeBSD.ORG >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> On Oct 19, 2017, at 5:15 PM, Ken Merry wrote: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> What does sg_readcap =E2=80=9416 show for these drives? >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> If it has type 2 protection turned on, check to see what the DCIPZ = value is in the control mode page: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> camcontrol modepage daX -v -m 10 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> If that is set to 0, add a -e to the above command line and see if = you can set it to 1. That may or may not help anything. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> The mpr(4) and mps(4) drivers try to support protection information = if it is turned on on the drive. So, they set the protection information = if protection information is turned on in the drive. For that reason, = setting the DCIPZ bit may not fix it. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> There could be a problem with how that is implemented that=E2=80=99s = causing the drives to reject the command, but I=E2=80=99m not sure. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> If it is, Steve (CCed) can help us debug it. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Ken >=20 >>=20 >=20 >> =E2=80=94 >=20 >>=20 >=20 >> Ken Merry >=20 >>=20 >=20 >>=20 > ken@FreeBSD.ORG >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> On Oct 17, 2017, at 9:54 PM, Shiva Bhanujan = wrote: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Sorry. perhaps I have failed to mention, the SCSI errors are only w/ = the seagate drives. These are of the model ST4000NM0023. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> From: Shiva Bhanujan >=20 >>=20 >=20 >> Sent: Tuesday, October 17, 2017 6:53 PM >=20 >>=20 >=20 >> To: Kenneth D. Merry >=20 >>=20 >=20 >> Cc:=20 >=20 >>=20 > freebsd-scsi@freebsd.org >=20 >>=20 >=20 >> Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Please note, that this isn't an issue w/ Toshiba drives. is this a = firmware issue by any chance? >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> From:=20 >=20 >>=20 > owner-freebsd-scsi@freebsd.org = [owner-freebsd-scsi@freebsd.org >=20 >> ] on behalf of Shiva Bhanujan = [shiva.bhanujan@quorum.net >=20 >> ] >=20 >>=20 >=20 >> Sent: Tuesday, October 17, 2017 6:08 AM >=20 >>=20 >=20 >> To: Kenneth D. Merry >=20 >>=20 >=20 >> Cc:=20 >=20 >>=20 > freebsd-scsi@freebsd.org >=20 >>=20 >=20 >> Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Since I started having the SCSI errors, I ended up running sg_format = to format the disks. I've found that once the disks are formatted using = sg_format, there are no SCSI errors. The errors that show up during the = format are towards the end of the dmesg output. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not = ready, format in progress) >=20 >>=20 >=20 >> (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> once the format is done, I can successfully format and partition = using gpart. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> The errors that show up when I try to run gpart for the first time = are as follows: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> (da9:mpr0:0:17:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 >>=20 >=20 >> (da9:mpr0:0:17:0): CAM status: SCSI Status Error >=20 >>=20 >=20 >> (da9:mpr0:0:17:0): SCSI status: Check Condition >=20 >>=20 >=20 >> (da9:mpr0:0:17:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) >=20 >>=20 >=20 >> (da9:mpr0:0:17:0): Error 22, Unretryable error >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> It seems that it's the read that is failing, and is being tagged as = an illegal request. While sg_format will address the issue at hand, this = isn't an option for us, because there are appliances that were formatted = using FreeBSD 10.2, and an upgrade to 10.3 >=20 >>=20 >=20 >> or 11.x might be an issue? >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> ________________________________ >=20 >>=20 >=20 >> From: Kenneth D. Merry [ken@FreeBSD.ORG ] >=20 >>=20 >=20 >> Sent: Monday, October 16, 2017 7:31 PM >=20 >>=20 >=20 >> To: Shiva Bhanujan >=20 >>=20 >=20 >> Cc:=20 >=20 >>=20 > freebsd-scsi@freebsd.org >=20 >>=20 >=20 >> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> On Tue, Oct 17, 2017 at 01:19:27 +0000, Shiva Bhanujan wrote: >=20 >>=20 >=20 >> Hi Ken, >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> I've attached the output of dmesg. Here's the SCSI CDB for a sample = drive, da3. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 = 01 00 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 = 01 00 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 = 01 00 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 = 01 00 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 00 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 00 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 00 00 00 = 01 00 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 00 00 00 = 01 00 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 00 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 a0 02 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 ba 22 00 00 = 01 00 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 d1 c0 bc 22 00 00 = 01 00 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 00 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 00 02 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fc 22 00 01 00 00 >=20 >>=20 >=20 >> (da3:mpr0:0:11:0): READ(10). CDB: 28 00 00 9f fe 22 00 01 00 00 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> My understanding is that FreeBSD 11.1 contains the mpr(4) driver? = I've tried this w/ 11.1, w/ the same results. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Yes, the mpr(4) driver is in all recent FreeBSD releases. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> In looking at the dmesg, this is telling: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> (da0:mpr0:0:8:0): WRITE(6). CDB: 0a 00 00 00 01 00 >=20 >>=20 >=20 >> (da0:mpr0:0:8:0): CAM status: SCSI Status Error >=20 >>=20 >=20 >> (da0:mpr0:0:8:0): SCSI status: Check Condition >=20 >>=20 >=20 >> (da0:mpr0:0:8:0): SCSI sense: NOT READY asc:4,4 (Logical unit not = ready, format in progress) >=20 >>=20 >=20 >> (da0:mpr0:0:8:0): Progress: 9% (6256/65536) complete >=20 >>=20 >=20 >> (da0:mpr0:0:8:0): Error 16, Unretryable error >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> If the drives are in the process of formatting, I guess it may make = sense >=20 >>=20 >=20 >> for them to reject read commands. Otherwise, it makes no sense for a = hard >=20 >>=20 >=20 >> drive to reject reads. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Are you able to check the status of the format? You should be able to = send >=20 >>=20 >=20 >> a test unit ready and figure out how far along the format is: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> camcontrol tur da0 -v >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> And so on for each of the drives. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Ken >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> ________________________________ >=20 >>=20 >=20 >> From: Kenneth D. Merry [ken@FreeBSD.ORG = >=20 >> >] >=20 >>=20 >=20 >> Sent: Monday, October 16, 2017 7:42 AM >=20 >>=20 >=20 >> To: Shiva Bhanujan >=20 >>=20 >=20 >> Cc:=20 >=20 >>=20 > freebsd-scsi@freebsd.org = >=20 >> > >=20 >>=20 >=20 >> Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> On Fri, Oct 13, 2017 at 20:12:02 +0000, Shiva Bhanujan wrote: >=20 >>=20 >=20 >> Hello, >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> I have a FreeBSD 10.3 install in a HVM on XenServer 6.5. The HBA330 = SAS-3 controller is in pcipassthrough mode to the FreeBSD VM. When I try = to access the disks (/dev/da0...) using gpart, I get SCSI errors, like = the following: >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> (da0:mpr0:0:0:0): CAM status: SCSI Status Error >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> (da0:mpr0:0:0:0): SCSI status: Check Condition >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> (da0:mpr0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid = command operation code) >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> The error message above is missing the SCSI CDB. We need that in = order to >=20 >>=20 >=20 >> figure out what command the drive is complaining about. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> The error message means that FreeBSD is sending a SCSI command that = the >=20 >>=20 >=20 >> drive doesn't support. That can be benign, or it can cause a problem. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> So, what error does gpart give you when you have this problem? >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> I get the same errors w/ FreeBSD 11.0 also. Running 10.3 natively = also has the same result. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Please note, that these errors don't show up on a Fusion-MPT SAS-2 = controller, or a MegaRAID SAS 2208 controller. Additionally, FreeBSD = 10.2 doesn't have any SCSI errors on the HBA330 SAS-3 controller either. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Is there a different version of the mpr driver I should be using? I = haven't checked the differences between the mpr driver in 10.2 vs 10.3 = and 11.0. I do see that there are others who have experienced these = issues. Can somebody please provide me some pointers >=20 >>=20 >=20 >> as to why this is occurring? Or if there are some driver changes that = I might be able to incorporate? >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> In general, the latest mpr(4) driver is the best one. The driver = itself >=20 >>=20 >=20 >> generally doesn't send SCSI commands (there are a few exceptions), = but >=20 >>=20 >=20 >> rather passes them through from the upper layers of CAM. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Please note, that I have gone through the mail titled "scsi error at = SEAGATE ST1200MM0088 TT31" and have started sg_format on all the SEAGATE = disks. Having said that, I still need to figure out what would happen, = if the disks >=20 >>=20 >=20 >> were written to using FreeBSD 10.2, which doesn't seem to have SCSI = errors, and when I try to upgrade to 10.3. Any help is appreciated. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Send the full error messages, and we may be able to figure out what's = going >=20 >>=20 >=20 >> on. >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> Ken >=20 >>=20 >=20 >> -- >=20 >>=20 >=20 >> Kenneth Merry >=20 >>=20 >=20 >>=20 > ken@FreeBSD.ORG >=20 >> > >=20 >> > >=20 >>=20 >=20 >> ________________________________ >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> -- >=20 >>=20 >=20 >> Kenneth Merry >=20 >>=20 >=20 >>=20 > ken@FreeBSD.ORG >=20 >> > >=20 >>=20 >=20 >> ________________________________ >=20 >>=20 >=20 >> _______________________________________________ >=20 >>=20 >=20 >>=20 > freebsd-scsi@freebsd.org >=20 >> mailing list >=20 >>=20 >=20 >>=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi = >=20 >>=20 >=20 >> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org = " >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> _______________________________________________ >=20 >>=20 >=20 >>=20 > freebsd-scsi@freebsd.org mailing list >=20 >>=20 >=20 >>=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >=20 >>=20 >=20 >> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >> _______________________________________________ >=20 >>=20 >=20 >>=20 > freebsd-scsi@freebsd.org mailing list >=20 >>=20 >=20 >>=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >=20 >>=20 >=20 >> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >>=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 --Apple-Mail=_C43579B3-638F-4C3F-B4A7-22EDAD9BDBE9-- From owner-freebsd-scsi@freebsd.org Fri Oct 20 18:02:35 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B37E0E3C364 for ; Fri, 20 Oct 2017 18:02:35 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: from asp.reflexion.net (outbound-mail-211-161.reflexion.net [208.70.211.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A7D8E3A for ; Fri, 20 Oct 2017 18:02:35 +0000 (UTC) (envelope-from shiva.bhanujan@quorum.com) Received: (qmail 5659 invoked from network); 20 Oct 2017 17:55:54 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 20 Oct 2017 17:55:53 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Fri, 20 Oct 2017 13:55:53 -0400 (EDT) Received: (qmail 19031 invoked from network); 20 Oct 2017 17:55:53 -0000 Received: from unknown (HELO mail.quorum.net) (64.74.133.216) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 20 Oct 2017 17:55:53 -0000 Received: from QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e]) by QLEXC01.Quorum.local ([fe80::edb6:63d5:778f:2f0e%14]) with mapi id 14.03.0351.000; Fri, 20 Oct 2017 10:55:52 -0700 From: Shiva Bhanujan To: Ken Merry CC: Steven Hartland , "freebsd-scsi@freebsd.org" Subject: RE: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Topic: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 Thread-Index: AQHTQ80AKMPHqs1A+0yLHlbMiv8g5KLiNr7zgATQ/oCAADv2uIAAihwAgAA6+IKAANcq6YAAAGPJgANMRoCAAABZAP//kIxpgACCBAD//7I104AAjO6AgABG8C+AAIlaAP//wmiN Date: Fri, 20 Oct 2017 17:55:51 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701C36667DC@QLEXC01.Quorum.local> References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C36666F5@QLEXC01.Quorum.local> <96BAD947-4AB0-4EAC-9DA8-4B1F10253287@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366674D@QLEXC01.Quorum.local>, <95A66B26-548C-4BF5-9527-EE30F9C01D42@freebsd.org> In-Reply-To: <95A66B26-548C-4BF5-9527-EE30F9C01D42@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.20.7.63] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Oct 2017 18:02:35 -0000 Thanks for the patch. I applied the patch on FreeBSD 10.3 and it has = addressed the issue. I've added the following to /boot/loader.conf. hw.mpr.disable_eedp=3D=221=22 ev.mpr.0.disable_eedp=3D=221=22 I'm going to apply the patch in FreeBSD 11.1 also, just to make sure that = works. Is this going to be checked into stable/10 and/or stable/11? From: Ken Merry =5Bken=40freebsd.org=5D Sent: Friday, October 20, 2017 7:34 AM To: Shiva Bhanujan Cc: Steven Hartland; freebsd-scsi=40freebsd.org Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 Ok. Yes, for =232, in theory we can disable EEDP / protection information = in the mpr(4) driver and if DPICZ is set, the drive won=E2=80=99t require = setting protection information on read and write commands. That should let you access the disks normally. That said, I=E2=80=99ve never played with protection information before, = so I don=E2=80=99t know for sure. I have a drive that supports it and = I=E2=80=99m formatting it now to turn on type 2 protection I may be able = to debug things once I get that done. In the mean time, I=E2=80=99ve attached a patch against the stable/10 = version of the mpr(4) driver. Apply this patch, and rebuild and reinstall = your kernel. Then you=E2=80=99ll be able to disable EEDP in the driver = three different ways: 1. Set hw.mpr.disable_eedp=3D1 in /boot/loader.conf. That will disable = EEDP / Protection Information for all mpr instances. 2. Set dev.mpr.0.disable_eedp=3D1 in /boot/loader.conf. That will disable = EEDP for mpr0. 3. sysctl dev.mpr.0.disable_eedp=3D1 That will disable EEDP on the fly for = mpr0. Let me know what happens. Ken =E2=80=94=20 Ken Merry ken=40FreeBSD.ORG From owner-freebsd-scsi@freebsd.org Fri Oct 20 19:08:12 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4DF11E3D8BD for ; Fri, 20 Oct 2017 19:08:12 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1DFBC330A for ; Fri, 20 Oct 2017 19:08:11 +0000 (UTC) (envelope-from ken@freebsd.org) Received: from [10.0.0.26] (mbp2013.int.kdm.org [10.0.0.26]) (authenticated bits=0) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPSA id v9KJ895A090024 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 20 Oct 2017 15:08:09 -0400 (EDT) (envelope-from ken@freebsd.org) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.0 \(3445.1.7\)) Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 From: Ken Merry In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701C36667DC@QLEXC01.Quorum.local> Date: Fri, 20 Oct 2017 15:08:07 -0400 Cc: "freebsd-scsi@freebsd.org" , Stephen Mcconnell Content-Transfer-Encoding: quoted-printable Message-Id: References: <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665D5D@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3665E8B@QLEXC01.Quorum.local> <20171016144231.GA94858@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366610E@QLEXC01.Quorum.local> <20171017023126.GA6559@mithlond.kdm.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666143@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666345@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666352@QLEXC01.Quorum.local> <3E746990-8C6D-4CA1-BD79-B5566CFB07F4@freebsd.org> <32C157CE-A122-435F-8430-9531BEEB5914@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C3666647@QLEXC01.Quorum.local> <3A5A10BE32AC9E45B4A22F89FC90EC0701C36666F5@QLEXC01.Quorum.local> <96BAD947-4AB0-4EAC-9DA8-4B1F10253287@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C366674D@QLEXC01.Quorum.local> <95A66B26-548C-4BF5-9527-EE30F9C01D42@freebsd.org> <3A5A10BE32AC9E45B4A22F89FC90EC0701C36667DC@QLEXC01.Quorum.local> To: Shiva Bhanujan X-Mailer: Apple Mail (2.3445.1.7) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [96.89.93.250]); Fri, 20 Oct 2017 15:08:09 -0400 (EDT) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Oct 2017 19:08:12 -0000 Great! I=E2=80=99m glad that fixed it. It would be better to fix the problem with EEDP / Protection Information = in the mpr(4) driver rather than having to have people disable the = check. Ken =E2=80=94=20 Ken Merry ken@FreeBSD.ORG > On Oct 20, 2017, at 1:55 PM, Shiva Bhanujan = wrote: >=20 > Thanks for the patch. I applied the patch on FreeBSD 10.3 and it has = addressed the issue. I've added the following to /boot/loader.conf. >=20 > hw.mpr.disable_eedp=3D"1" > ev.mpr.0.disable_eedp=3D"1" >=20 >=20 > I'm going to apply the patch in FreeBSD 11.1 also, just to make sure = that works. Is this going to be checked into stable/10 and/or = stable/11? >=20 >=20 >=20 >=20 >=20 > From: Ken Merry [ken@freebsd.org] >=20 > Sent: Friday, October 20, 2017 7:34 AM >=20 > To: Shiva Bhanujan >=20 > Cc: Steven Hartland; freebsd-scsi@freebsd.org >=20 > Subject: Re: FreeBSD 10.3/11.0 SCSI errors with Symbios Logic SAS3008 = PCI-Express Fusion-MPT SAS-3 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Ok. Yes, for #2, in theory we can disable EEDP / protection = information in the mpr(4) driver and if DPICZ is set, the drive won=E2=80=99= t require setting protection information on read and write commands. >=20 >=20 >=20 > That should let you access the disks normally. >=20 >=20 >=20 > That said, I=E2=80=99ve never played with protection information = before, so I don=E2=80=99t know for sure. I have a drive that supports = it and I=E2=80=99m formatting it now to turn on type 2 protection I may = be able to debug things once I get that done. >=20 >=20 >=20 > In the mean time, I=E2=80=99ve attached a patch against the stable/10 = version of the mpr(4) driver. Apply this patch, and rebuild and = reinstall your kernel. Then you=E2=80=99ll be able to disable EEDP in = the driver three different ways: >=20 >=20 >=20 > 1. Set hw.mpr.disable_eedp=3D1 in /boot/loader.conf. That will disable = EEDP / Protection Information for all mpr instances. >=20 > 2. Set dev.mpr.0.disable_eedp=3D1 in /boot/loader.conf. That will = disable EEDP for mpr0. >=20 > 3. sysctl dev.mpr.0.disable_eedp=3D1 That will disable EEDP on the fly = for mpr0. >=20 >=20 >=20 > Let me know what happens. >=20 >=20 >=20 > Ken >=20 > =E2=80=94=20 >=20 > Ken Merry >=20 > ken@FreeBSD.ORG >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org"