From owner-freebsd-hardware@freebsd.org Thu Oct 22 18:14:00 2015 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C780A1C4DA; Thu, 22 Oct 2015 18:14:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A6751ACA; Thu, 22 Oct 2015 18:14:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 27748B9BB; Thu, 22 Oct 2015 14:13:58 -0400 (EDT) From: John Baldwin To: freebsd-hardware@freebsd.org Cc: Dieter BSD , freebsd-hackers@freebsd.org Subject: Re: ECC support Date: Thu, 22 Oct 2015 11:09:50 -0700 Message-ID: <1492434.22kxSKhHEJ@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-PRERELEASE; KDE/4.14.3; amd64; ; ) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Oct 2015 14:13:59 -0400 (EDT) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 18:14:00 -0000 On Wednesday, September 16, 2015 10:56:52 AM Dieter BSD wrote: > Chris: > > MCA: Bank 1, Status 0x9400000000000151 > > MCA: Global Cap 0x0000000000000106, Status 0x0000000000000000 > > MCA: Vendor "AuthenticAMD", ID 0x100f52, APIC ID 2 > > > > MCA: Address 0x81cc0e9f0 > > > > Kind of freaky. I've never had this error on this board before. > > On others tho. > > > > Try a search for MCA instead. > > Is there a decoder ring for those messages? I don't recall seeing > messages like that, although I wasn't looking for them, and they > don't leap out at you screaming ERROR! ERROR! Digital Unix had its > problems, but at least the error messages were fairly clear. > Something like "single bit memory error at address 0x12345..." > A simple edit to sys/x86/x86/mca.c > s/printf("UNCOR ");/printf("Uncorrectable ");/ > s/printf("COR ");/printf("Correctable ");/ > would make the messages at least slightly more meaningful to a viewer > who isn't intimently(sp) familiar with the mca. Which most people aren't. The problem is that there are other fields to decode and you can only fit so much in one line. Also, there is not a CPU-independent way to know the address of an ECC error. On Intel Core i3/5/7 (anything with QPI) you can identify the individual DIMM at least, but the label that the motherboard manufacturer uses varies by manufacturer. (You can maybe scrape that text from the SMBIOS tables, but only if they aren't wrong which they sometimes are, and good luck knowing if they are wrong or right.) Digital UNIX had the luxury of running on hardware built by the same company, not on a random assortment of boards built by various vendors. FreeBSD does not. sysutils/mcelog does some more verbose decoding of MCA records, but I find it to be equally gibberish for anyone not intimately familiar with a specific CPU. I wrote a tool for a previous employer that was able to do some simple parsing of MCA errors for Supermicro X7-X10 boards (Intel CPUs) and give a short summary that was used in a nagios check. However, it only handles a narrow set of systems. https://github.com/freebsd/freebsd/compare/master...bsdjhb:ecc -- John Baldwin From owner-freebsd-hardware@freebsd.org Thu Oct 22 18:57:41 2015 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95046A1CED2; Thu, 22 Oct 2015 18:57:41 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3ED9411E3; Thu, 22 Oct 2015 18:57:41 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from [194.32.164.24] ([194.32.164.24]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id t9MInDTL087303; Thu, 22 Oct 2015 19:49:13 +0100 (BST) (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: ECC support From: Bob Bishop In-Reply-To: <1492434.22kxSKhHEJ@ralph.baldwin.cx> Date: Thu, 22 Oct 2015 19:49:13 +0100 Cc: freebsd-hardware@freebsd.org, freebsd-hackers@freebsd.org, Dieter BSD Content-Transfer-Encoding: quoted-printable Message-Id: <74705089-408A-4FD3-899E-CA677390F855@gid.co.uk> References: <1492434.22kxSKhHEJ@ralph.baldwin.cx> To: John Baldwin X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 18:57:41 -0000 HI, > On 22 Oct 2015, at 19:09, John Baldwin wrote: >=20 > On Wednesday, September 16, 2015 10:56:52 AM Dieter BSD wrote: >> Chris: >>> MCA: Bank 1, Status 0x9400000000000151 >>> MCA: Global Cap 0x0000000000000106, Status 0x0000000000000000 >>> MCA: Vendor "AuthenticAMD", ID 0x100f52, APIC ID 2 >>>=20 >>> MCA: Address 0x81cc0e9f0 >>>=20 >>> Kind of freaky. I've never had this error on this board before. >>> On others tho. >>>=20 >>> Try a search for MCA instead. >>=20 >> Is there a decoder ring for those messages? I don't recall seeing >> messages like that, although I wasn't looking for them, and they >> don't leap out at you screaming ERROR! ERROR! Digital Unix had its >> problems, but at least the error messages were fairly clear. >> Something like "single bit memory error at address 0x12345..." >> A simple edit to sys/x86/x86/mca.c >> s/printf("UNCOR ");/printf("Uncorrectable ");/ >> s/printf("COR ");/printf("Correctable ");/ >> would make the messages at least slightly more meaningful to a viewer >> who isn't intimently(sp) familiar with the mca. Which most people = aren't. >=20 > The problem is that there are other fields to decode and you can only = fit so > much in one line. Also, there is not a CPU-independent way to know = the > address of an ECC error. [etc] On server-class hardware, the platform management (BMC or whatever) is = probably decoding this stuff for event logs and can be interrogated via = IPMI (or whatever). -- Bob Bishop rb@gid.co.uk From owner-freebsd-hardware@freebsd.org Thu Oct 22 21:17:24 2015 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2136FA1CB25; Thu, 22 Oct 2015 21:17:24 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F20E4804; Thu, 22 Oct 2015 21:17:23 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5362FB94F; Thu, 22 Oct 2015 17:17:22 -0400 (EDT) From: John Baldwin To: Bob Bishop Cc: freebsd-hardware@freebsd.org, freebsd-hackers@freebsd.org, Dieter BSD Subject: Re: ECC support Date: Thu, 22 Oct 2015 14:17:07 -0700 Message-ID: <1483396.WZc3qgD2yz@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-PRERELEASE; KDE/4.14.3; amd64; ; ) In-Reply-To: <74705089-408A-4FD3-899E-CA677390F855@gid.co.uk> References: <1492434.22kxSKhHEJ@ralph.baldwin.cx> <74705089-408A-4FD3-899E-CA677390F855@gid.co.uk> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Oct 2015 17:17:22 -0400 (EDT) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 21:17:24 -0000 On Thursday, October 22, 2015 07:49:13 PM Bob Bishop wrote: > HI, > > > On 22 Oct 2015, at 19:09, John Baldwin wrote: > > > > On Wednesday, September 16, 2015 10:56:52 AM Dieter BSD wrote: > >> Chris: > >>> MCA: Bank 1, Status 0x9400000000000151 > >>> MCA: Global Cap 0x0000000000000106, Status 0x0000000000000000 > >>> MCA: Vendor "AuthenticAMD", ID 0x100f52, APIC ID 2 > >>> > >>> MCA: Address 0x81cc0e9f0 > >>> > >>> Kind of freaky. I've never had this error on this board before. > >>> On others tho. > >>> > >>> Try a search for MCA instead. > >> > >> Is there a decoder ring for those messages? I don't recall seeing > >> messages like that, although I wasn't looking for them, and they > >> don't leap out at you screaming ERROR! ERROR! Digital Unix had its > >> problems, but at least the error messages were fairly clear. > >> Something like "single bit memory error at address 0x12345..." > >> A simple edit to sys/x86/x86/mca.c > >> s/printf("UNCOR ");/printf("Uncorrectable ");/ > >> s/printf("COR ");/printf("Correctable ");/ > >> would make the messages at least slightly more meaningful to a viewer > >> who isn't intimently(sp) familiar with the mca. Which most people aren't. > > > > The problem is that there are other fields to decode and you can only fit so > > much in one line. Also, there is not a CPU-independent way to know the > > address of an ECC error. [etc] > > On server-class hardware, the platform management (BMC or whatever) is probably decoding this stuff for event logs and can be interrogated via IPMI (or whatever). Not always well and not always with side effects you want. On Core 2 and Nehalem i7 class hardware I measured that it took on the order of 400 milliseconds (not micro) in SMM (system management mode, so your entire OS is halted) to write out each log entry to NVRAM. At least one place I worked at turned the BIOS ECC logging off because that delay was too costly. Also, even though your BMC may log it, the format for doing so isn't standard. The details such as the affected DIMM are in the OEM bits of the log record, so not something you can easily extract from, say, ipmitool sel elist. You'd have to log into the BIOS itself (or the BMC's web UI) to see which DIMM is affected. Neither of those are really great for automated reporting. -- John Baldwin From owner-freebsd-hardware@freebsd.org Fri Oct 23 11:37:34 2015 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5995A1C97E; Fri, 23 Oct 2015 11:37:34 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6869D7F0; Fri, 23 Oct 2015 11:37:33 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from [194.32.164.28] ([194.32.164.28]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id t9NBbVjl080406; Fri, 23 Oct 2015 12:37:31 +0100 (BST) (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: ECC support From: Bob Bishop In-Reply-To: <1483396.WZc3qgD2yz@ralph.baldwin.cx> Date: Fri, 23 Oct 2015 12:37:31 +0100 Cc: freebsd-hackers@freebsd.org, Dieter BSD , freebsd-hardware@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <97482413-D2AA-4C32-AEFF-EB65D5D8542B@gid.co.uk> References: <1492434.22kxSKhHEJ@ralph.baldwin.cx> <74705089-408A-4FD3-899E-CA677390F855@gid.co.uk> <1483396.WZc3qgD2yz@ralph.baldwin.cx> To: John Baldwin X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 11:37:34 -0000 Hi, > On 22 Oct 2015, at 22:17, John Baldwin wrote: >=20 > On Thursday, October 22, 2015 07:49:13 PM Bob Bishop wrote: >> HI, >>=20 >>> On 22 Oct 2015, at 19:09, John Baldwin wrote: >>>=20 >>> On Wednesday, September 16, 2015 10:56:52 AM Dieter BSD wrote: >>>> Chris: >>>>> MCA: Bank 1, Status 0x9400000000000151 >>>>> MCA: Global Cap 0x0000000000000106, Status 0x0000000000000000 >>>>> MCA: Vendor "AuthenticAMD", ID 0x100f52, APIC ID 2 >>>>>=20 >>>>> MCA: Address 0x81cc0e9f0 >>>>>=20 >>>>> Kind of freaky. I've never had this error on this board before. >>>>> On others tho. >>>>>=20 >>>>> Try a search for MCA instead. >>>>=20 >>>> Is there a decoder ring for those messages? I don't recall seeing >>>> messages like that, although I wasn't looking for them, and they >>>> don't leap out at you screaming ERROR! ERROR! Digital Unix had its >>>> problems, but at least the error messages were fairly clear. >>>> Something like "single bit memory error at address 0x12345..." >>>> A simple edit to sys/x86/x86/mca.c >>>> s/printf("UNCOR ");/printf("Uncorrectable ");/ >>>> s/printf("COR ");/printf("Correctable ");/ >>>> would make the messages at least slightly more meaningful to a = viewer >>>> who isn't intimently(sp) familiar with the mca. Which most people = aren't. >>>=20 >>> The problem is that there are other fields to decode and you can = only fit so >>> much in one line. Also, there is not a CPU-independent way to know = the >>> address of an ECC error. [etc] >>=20 >> On server-class hardware, the platform management (BMC or whatever) = is probably decoding this stuff for event logs and can be interrogated = via IPMI (or whatever). >=20 > Not always well and not always with side effects you want. On Core 2 = and > Nehalem i7 class hardware I measured that it took on the order of 400 > milliseconds (not micro) in SMM (system management mode, so your = entire > OS is halted) to write out each log entry to NVRAM. At least one = place I > worked at turned the BIOS ECC logging off because that delay was too = costly. >=20 > Also, even though your BMC may log it, the format for doing so isn't > standard. The details such as the affected DIMM are in the OEM bits = of > the log record, so not something you can easily extract from, say, > ipmitool sel elist. You'd have to log into the BIOS itself (or the = BMC's > web UI) to see which DIMM is affected. Neither of those are really = great > for automated reporting. All agreed. I was just flagging up the existence of another possible = channel to get at ECC logging. > --=20 > John Baldwin -- Bob Bishop rb@gid.co.uk From owner-freebsd-hardware@freebsd.org Fri Oct 23 15:23:05 2015 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2697EA1D465 for ; Fri, 23 Oct 2015 15:23:05 +0000 (UTC) (envelope-from rpokala@panasas.com) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0061.outbound.protection.outlook.com [157.56.111.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9BAC19FC for ; Fri, 23 Oct 2015 15:23:03 +0000 (UTC) (envelope-from rpokala@panasas.com) Received: from CY1PR08MB1803.namprd08.prod.outlook.com (10.162.218.25) by CY1PR08MB1803.namprd08.prod.outlook.com (10.162.218.25) with Microsoft SMTP Server (TLS) id 15.1.306.13; Fri, 23 Oct 2015 15:22:56 +0000 Received: from CY1PR08MB1803.namprd08.prod.outlook.com ([10.162.218.25]) by CY1PR08MB1803.namprd08.prod.outlook.com ([10.162.218.25]) with mapi id 15.01.0306.003; Fri, 23 Oct 2015 15:22:56 +0000 From: "Pokala, Ravi" To: "freebsd-hardware@freebsd.org" Subject: Re: ECC support Thread-Topic: ECC support Thread-Index: AQHRDaawPphF5ZX7akCd4F5PFOh6ug== Date: Fri, 23 Oct 2015 15:22:54 +0000 Message-ID: <1917A1AA-B9AB-4612-A4E3-18FF4C909FC3@panasas.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/0.0.0.151008 authentication-results: spf=none (sender IP is ) smtp.mailfrom=rpokala@panasas.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [24.6.178.251] x-microsoft-exchange-diagnostics: 1; CY1PR08MB1803; 5:JPuM6hWxBmsp+Bijk1Xz6OcMnmHdzBIGKs6Uo1H0KUU0KI3q0o7qlfTo9IlvUFXtHz2zwiUxT4oCXjRUfaPyCbfNS2w+JUmU915+9etjyf+rWbfWTXyde3vDkDAY8zjmJFLwuv1rymSizdnhWgSM2A==; 24:7P5NplXbnr4uFxBXcumCoCXXQCY2Y75xyJ8R9v9Pp0QsUQ94jvVBQulf8DwBv66yOo8NQo1HIqrm+tjdceu8NiUyeo3L01o9yCI7ROI1CzY=; 20:mnCmBO6Z8oXlHmqRZnyUBh0sy0KwUmDiRWU6zW+rSOQSnTiQj8aaC9VWdY5rmuYUBDtCDYkvax0QlTnCyebgaA== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR08MB1803; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(520078)(5005006)(8121501046)(3002001)(102215026); SRVR:CY1PR08MB1803; BCL:0; PCL:0; RULEID:; SRVR:CY1PR08MB1803; x-forefront-prvs: 0738AF4208 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(13464003)(40224003)(199003)(189002)(164054003)(86362001)(575784001)(54356999)(83506001)(82746002)(50986999)(83716003)(87936001)(512954002)(110136002)(5001960100002)(99286002)(107886002)(106356001)(2351001)(189998001)(106116001)(105586002)(122556002)(5008740100001)(2900100001)(5007970100001)(5004730100002)(77096005)(40100003)(15975445007)(11100500001)(102836002)(450100001)(5002640100001)(10400500002)(92566002)(97736004)(66066001)(221733001)(101416001)(81156007)(33656002)(4001350100001)(5001920100001)(2501003)(19580405001)(36756003)(19580395003)(104396002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR08MB1803; H:CY1PR08MB1803.namprd08.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: panasas.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Oct 2015 15:22:54.8287 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR08MB1803 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 15:23:05 -0000 LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCg0KDQo+RGF0ZTogVGh1LCAyMiBPY3QgMjAxNSAx MTowOTo1MCAtMDcwMA0KPkZyb206IEpvaG4gQmFsZHdpbiA8amhiQGZyZWVic2Qub3JnPg0KPlRv OiBmcmVlYnNkLWhhcmR3YXJlQGZyZWVic2Qub3JnDQo+Q2M6IERpZXRlciBCU0QgPGRpZXRlcmJz ZEBnbWFpbC5jb20+LCBmcmVlYnNkLWhhY2tlcnNAZnJlZWJzZC5vcmcNCj5TdWJqZWN0OiBSZTog RUNDIHN1cHBvcnQNCj5NZXNzYWdlLUlEOiA8MTQ5MjQzNC4yMmt4U0toSEVKQHJhbHBoLmJhbGR3 aW4uY3g+DQo+Q29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0PSJ1cy1hc2NpaSINCj4N Cj5UaGUgcHJvYmxlbSBpcyB0aGF0IHRoZXJlIGFyZSBvdGhlciBmaWVsZHMgdG8gZGVjb2RlIGFu ZCB5b3UgY2FuIG9ubHkgZml0IHNvIG11Y2ggaW4gb25lIGxpbmUuDQoNCkF0IFBhbmFzYXMsIHdl IGRpZCBpbi1rZXJuZWwgcGFyc2luZyBhbmQgZ290IGl0IGRvd24gdG8gYSBvbmUtbGluZXIgbGlr ZSB0aGlzOg0KDQogICAgRGV0ZWN0ZWQgSFcgRXJyIChDTUMpIC0gQ29ycmVjdGFibGUgRUNDIGVy cm9yIENoYW5uZWw6MDsgRGltbTowOyBTeW5kcm9tZToyMTUxNjg2MTYwDQoNCg0KQnV0IHRoYXQg d2FzIG9ubHkgZm9yIG1haW4tbWVtb3J5IGNvcnJlY3RlZCBFQ0NzOyBmb3IgYWxsIG90aGVyIE1D QXMsIGl0IHdhcyBhIG11bHRpLWxpbmUgZm9ybWF0ICh3aGljaCBJIHRoaW5rIHdlIGdvdCBmcm9t IGJhY2twb3J0aW5nIE1DQSBzdXBwb3J0IGZyb20gKDgtU1RBQkxFPykpOg0KDQogICAgTUNBOiBC YW5rIDgsIFN0YXR1cyAweGIyMDAwMDAwMDAwNDAwOGYNCiAgICBNQ0E6IEdsb2JhbCBDYXAgMHgw MDAwMDAwMDAwMDAxYzA5LCBTdGF0dXMgMHgwMDAwMDAwMDAwMDAwMDA0DQogICAgTUNBOiBWZW5k b3IgIkdlbnVpbmVJbnRlbCIsIElEIDB4MTA2ZTQsIEFQSUMgSUQgMA0KICAgIE1DQTogQ1BVIDAg VU5DT1IgUENDIEdFTiBjaGFubmVsID8/IG1lbW9yeSBlcnJvcg0KDQoNCj5BbHNvLCB0aGVyZSBp cyBub3QgYSBDUFUtaW5kZXBlbmRlbnQgd2F5IHRvIGtub3cgdGhlIGFkZHJlc3Mgb2YgYW4gRUND IGVycm9yLiAgT24gSW50ZWwgQ29yZSBpMy81LzcgKGFueXRoaW5nIHdpdGggUVBJKSB5b3UgY2Fu IGlkZW50aWZ5IHRoZSBpbmRpdmlkdWFsIERJTU0gYXQgbGVhc3QsIGJ1dCB0aGUgbGFiZWwgdGhh dCB0aGUgbW90aGVyYm9hcmQgbWFudWZhY3R1cmVyIHVzZXMgdmFyaWVzIGJ5IG1hbnVmYWN0dXJl ci4gIChZb3UgY2FuIG1heWJlIHNjcmFwZSB0aGF0IHRleHQgZnJvbSB0aGUgU01CSU9TIHRhYmxl cywNCg0KVGhhdCdzIGV4YWN0bHkgd2hhdCB3ZSBkaWQgd2hlbiB1c2luZyBvZmYtdGhlLXNoZWxm IG1vdGhlcmJvYXJkcy4gV2Ugd2VyZSBhYmxlIHRvIGV4dHJhY3QgdGhlIG5hbWUgb2YgdGhlIERJ TU0gc2xvdCwgYXMgZGVmaW5lZCBpbiBTTUJJT1MsIGFzIHdlbGwgYXMgdGhlIHBhcnQgYW5kIHNl cmlhbCBudW1iZXJzIG9mIHRoZSBESU1NLCBhbmQgdGhlIHBoeXNpY2FsIGFkZHJlc3MgcmFuZ2Ug b2YgdGhlIERJTU0uIEZvciBleGFtcGxlOg0KDQogICAgaHcubWVtLmRpbW0uczogbG9jYXRvciAg IHNlcmlhbCMgIHBhcnQjICAgICAgICAgICAgICBiYW5rICAgICAgICAgICAgICAgICAgICAgICBz aXplICAgICBhZGRyMCAgICAgICAgIGFkZHJODQogICAgaHcubWVtLmRpbW0uMDogRElNTV9BMSAg IERDOTE3QUVGIDM2S0RaUzJHNzJQWi0xRzREMSBbTk9ERSAwIENIQU5ORUwgMCBESU1NIDBdICAx NjM4NE1CICAweDAwMDAwMDAwMDAwIDB4MDAzRkZGRkZGRkYNCiAgICBody5tZW0uZGltbS4xOiBE SU1NX0IxICAgRERBMEM3OTMgMzZLRFpTMkc3MlBaLTFHNEQxIFtOT0RFIDAgQ0hBTk5FTCAxIERJ TU0gMF0gIDE2Mzg0TUIgIDB4MDA0MDAwMDAwMDAgMHgwMDdGRkZGRkZGRg0KICAgIGh3Lm1lbS5k aW1tLjI6IERJTU1fQzEgICBEREEwQzdCNiAzNktEWlMyRzcyUFotMUc0RDEgW05PREUgMCBDSEFO TkVMIDIgRElNTSAwXSAgMTYzODRNQiAgMHgwMDgwMDAwMDAwMCAweDAwQkZGRkZGRkZGDQogICAg aHcubWVtLmRpbW0uMzogRElNTV9EMSAgIEREQTBDN0RFIDM2S0RaUzJHNzJQWi0xRzREMSBbTk9E RSAwIENIQU5ORUwgMyBESU1NIDBdICAxNjM4NE1CICAweDAwQzAwMDAwMDAwIDB4MDBGRkZGRkZG RkYNCg0KDQpSZS13aGFja2luZyB0aGF0IGNvZGUgZm9yIC1DVVJSRU5UIGFuZCBnZXR0aW5nIGl0 IHVwc3RyZWFtIGhhcyBiZWVuIG9uIG15IHRvLWRvIGxpc3QgZm9yIGEgZGVwcmVzc2luZ2x5IGxv bmcgdGltZTsgaXQga2VlcHMgZ2V0dGluZyBwcmUtZW1wdGVkLiA6LVMNCg0KDQo+YnV0IG9ubHkg aWYgdGhleSBhcmVuJ3Qgd3Jvbmcgd2hpY2ggdGhleSBzb21ldGltZXMgYXJlLCBhbmQgZ29vZCBs dWNrIGtub3dpbmcgaWYgdGhleSBhcmUgd3Jvbmcgb3IgcmlnaHQuKQ0KDQpNYWtpbmcgc3VyZSB0 aGUgU01CSU9TIGlkZW50aWZpZXIgbWF0Y2hlcyB0aGUgbGFiZWwgb24gdGhlIG1vdGhlcmJvYXJk IGlzIHBhcnQgb2YgdGhlIHByb2Nlc3Mgb2YgdmFsaWRhdGluZyB0aGUgbW90aGVyYm9hcmQgYXMg dXNhYmxlIGJ5IHVzLiA6LSkNCg0KPkRpZ2l0YWwgVU5JWCBoYWQgdGhlIGx1eHVyeSBvZiBydW5u aW5nIG9uIGhhcmR3YXJlIGJ1aWx0IGJ5IHRoZSBzYW1lIGNvbXBhbnksIG5vdCBvbiBhIHJhbmRv bSBhc3NvcnRtZW50IG9mIGJvYXJkcyBidWlsdCBieSB2YXJpb3VzIHZlbmRvcnMuICBGcmVlQlNE IGRvZXMgbm90Lg0KDQpZZWFoLiBMaWtlIEkgc2FpZCwgd2Ugc2NyYXBwZWQgU01CSU9TICpmb3Ig b2ZmLXRoZS1zaGVsZiBtb3RoZXJib2FyZHMqLiBGb3Igb3VyIGluLWhvdXNlIGRlc2lnbnMsIHdl IGhhcmRjb2RlZCB0aGUgQ2hhbm5lbC9ESU1NIG1hcHBpbmcgaW50byBhbiB1bmFtYmlndW91cyBm b3JtIGluc2lkZSB0aGUgZHJpdmVyIGl0c2VsZi4NCg0KPnN5c3V0aWxzL21jZWxvZyBkb2VzIHNv bWUgbW9yZSB2ZXJib3NlIGRlY29kaW5nIG9mIE1DQSByZWNvcmRzLCBidXQgSSBmaW5kIGl0IHRv IGJlIGVxdWFsbHkgZ2liYmVyaXNoIGZvciBhbnlvbmUgbm90IGludGltYXRlbHkgZmFtaWxpYXIg d2l0aCBhIHNwZWNpZmljIENQVS4NCj4NCj5JIHdyb3RlIGEgdG9vbCBmb3IgYSBwcmV2aW91cyBl bXBsb3llciB0aGF0IHdhcyBhYmxlIHRvIGRvIHNvbWUgc2ltcGxlIHBhcnNpbmcgb2YgTUNBIGVy cm9ycyBmb3IgU3VwZXJtaWNybyBYNy1YMTAgYm9hcmRzIChJbnRlbCBDUFVzKSBhbmQgZ2l2ZSBh IHNob3J0IHN1bW1hcnkgdGhhdCB3YXMgdXNlZCBpbiBhIG5hZ2lvcyBjaGVjay4gIEhvd2V2ZXIs IGl0IG9ubHkgaGFuZGxlcyBhIG5hcnJvdyBzZXQgb2Ygc3lzdGVtcy4NCj4NCj5odHRwczovL2dp dGh1Yi5jb20vZnJlZWJzZC9mcmVlYnNkL2NvbXBhcmUvbWFzdGVyLi4uYnNkamhiOmVjYw0KDQpP b29vLCB0aGF0IGxvb2tzIG5pY2UhIElzIHRoaXMgc29tZXRoaW5nIHRoYXQgY2FuIGJlIGNvbW1p dHRlZCB0byB0aGUgbWFpbiB0cmVlPyBJZiBub3RoaW5nIGVsc2UsIEknbGwgbmVlZCB0byBtYWtl IGEgbm90ZSBvZiB0aGUgd2F5IHlvdSdyZSBnZXR0aW5nIHRoZSBNQ0EgcmVjb3JkcyBpbnRvIHVz ZXJsYW5kLg0KDQpUaGFua3MsDQoNClJhdmkNCg0KPi0tIA0KPkpvaG4gQmFsZHdpbg0K