From owner-freebsd-virtualization@freebsd.org Sun Mar 15 06:17:01 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4F3EC259A29 for ; Sun, 15 Mar 2020 06:17:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48g8ND6G1gz3Pkj for ; Sun, 15 Mar 2020 06:17:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id D0049259A28; Sun, 15 Mar 2020 06:17:00 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CBC2C259A27 for ; Sun, 15 Mar 2020 06:17:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48g8ND3FbNz3Pjg for ; Sun, 15 Mar 2020 06:17:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 539C23711 for ; Sun, 15 Mar 2020 06:17:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02F6H0K4005161 for ; Sun, 15 Mar 2020 06:17:00 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02F6H0I6005156 for virtualization@FreeBSD.org; Sun, 15 Mar 2020 06:17:00 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236989] AWS EC2 lockups "Missing interrupt" Date: Sun, 15 Mar 2020 06:16:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: freebsd@darkain.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2020 06:17:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236989 Vincent Milum Jr changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |freebsd@darkain.com --- Comment #14 from Vincent Milum Jr --- I just ran into the "nvmeX: Missing interrupt" on FreeNAS under VMWare when testing something out tonight. It happened right at boot, and repeated quite often (to the point I couldn't even fully boot the OS after ~20 minutes). FreeNAS is still on 11.3-RELEASE. But this is a homelab system that I can easily build test cases with. nvme drives are Intel Optane M10 16GB. These are cheap on eBay right now. No data on them at all yet (empty partition table). I was using PCIe passthrou= gh on ESXi-6.7.0-20190802001-standard. I'll start testing various 12.x and 13 kernels to see if conditions are repeatable on any of those with this config too (if it is, I'll post more details) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sun Mar 15 14:20:59 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 374C8266D61 for ; Sun, 15 Mar 2020 14:20:59 +0000 (UTC) (envelope-from crowston@protonmail.com) Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48gM6d0F8Lz41pv; Sun, 15 Mar 2020 14:20:56 +0000 (UTC) (envelope-from crowston@protonmail.com) Date: Sun, 15 Mar 2020 14:20:51 +0000 To: Alex Erley From: Robert Crowston Cc: Peter Grehan , "freebsd-virtualization@freebsd.org" Reply-To: Robert Crowston Subject: Re: [GPU pass-through] no compatible bridge window for claimed BAR Message-ID: In-Reply-To: References: <07921dcf-11d5-f440-a42f-d7ec950cab10@freebsd.org> Feedback-ID: 2OVbcR1yHYpdkD8cgQllkFwcuMVZg_LiVMMPvptooFDfHD_03MuQO4ZaF626jWHZYFEhNR2cmIbZ53j4QGWMBQ==:Ext:ProtonMail MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=7.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mail.protonmail.ch X-Rspamd-Queue-Id: 48gM6d0F8Lz41pv X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.09 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; HAS_REPLYTO(0.00)[crowston@protonmail.com]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24:c]; DKIM_TRACE(0.00)[protonmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.80), ipnet: 185.70.40.0/24(-4.89), asn: 62371(-3.91), country: CH(0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[131.40.70.185.list.dnswl.org : 127.0.5.1]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.988,0]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=default]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2020 14:20:59 -0000 I suggest you map the BAR into the 32 bit address space, unless you have so= many PCI devices that this is not feasible. Just raise the limit of the sp= ecial 64 bit handling to 1 GB or something big. - Many/most(?) consumer BIOS/UEFIs map 64 bit bars into the 32 bit address = space by default, so this configuration is much more tested for device driv= ers and guest operating systems. - Passthrough doesn't work for me at all above the 4GB memory window on my = recent AMD system. It exits back to the hypervisor, which then crashes with= a failed assert because it doesn't expect to handle this. With this tweak it is possible to use the open source nVidia driver on Linu= x. However, your next problem---if you want to use proprietary drivers---is th= at access to the ROM BAR is not supported or emulated. If that could be fix= ed, it would be a big step forwards. =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original Me= ssage =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 On Saturday, 14 March 2020 12:50, Alex Erley wrote: > Hello, > > Some new findings to share. > > 1. Changing PCI_EMUL_MEMBASE64 from 0xD000000000 to any value > below 0x0440000000 makes bhyve fail when starting VM with message: > bhyve: failed to initialize BARs for PCI 1/0/0 > device emulation initialization error: Cannot allocate memory > > 2. Having PCI_EMUL_MEMBASE64 set to 0x0440000000 (or above) guest VM > can not configure BARs of pass-through device properly. > > =3D=3D (a) =3D=3D > On BHyve host ppt device is: > > > > devinfo -rv > > ... > pci0 > hostb0 at slot=3D0 function=3D0 dbsf=3Dpci0:0:0:0 > pcib1 at slot=3D1 function=3D0 dbsf=3Dpci0:0:1:0 handle=3D\SB.PCI0.P0P2 > I/O ports: 0xe000-0xefff > I/O memory addresses: > 0x00c0000000-0x00d30fffff <-- covers all child mem windows > pci1 > ppt0 at slot=3D0 function=3D0 dbsf=3Dpci0:1:0:0 > pcib1 I/O port window: 0xe000-0xe07f > pcib1 memory window: > 0x00c0000000-0x00cfffffff <-- 256M > 0x00d0000000-0x00d1ffffff <-- 32M > 0x00d2000000-0x00d2ffffff <-- 16M > ppt1 at slot=3D0 function=3D1 dbsf=3Dpci0:1:0:1 > pcib1 memory window: > 0xd3080000-0xd3083fff <-- 16K > ... > and there is no other device attached to pci1. > > =3D=3D (b) =3D=3D > On guest VM dmesg shows (timestamps are removed): > ... > BIOS-provided physical RAM map: > BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable > BIOS-e820: [mem 0x0000000000100000-0x00000000bea95fff] usable > BIOS-e820: [mem 0x00000000bea96000-0x00000000bea97fff] reserved > BIOS-e820: [mem 0x00000000bea98000-0x00000000bea99fff] ACPI data > BIOS-e820: [mem 0x00000000bea9a000-0x00000000beaa8fff] reserved > BIOS-e820: [mem 0x00000000beaa9000-0x00000000bfb28fff] usable > BIOS-e820: [mem 0x00000000bfb29000-0x00000000bfb58fff] type 20 > BIOS-e820: [mem 0x00000000bfb59000-0x00000000bfb7cfff] reserved > BIOS-e820: [mem 0x00000000bfb7d000-0x00000000bfb81fff] usable > BIOS-e820: [mem 0x00000000bfb82000-0x00000000bfb88fff] ACPI data > BIOS-e820: [mem 0x00000000bfb89000-0x00000000bfb8cfff] ACPI NVS > BIOS-e820: [mem 0x00000000bfb8d000-0x00000000bffcffff] usable > BIOS-e820: [mem 0x00000000bffd0000-0x00000000bffeffff] reserved > BIOS-e820: [mem 0x00000000bfff0000-0x00000000bfffffff] usable > BIOS-e820: [mem 0x0000000100000000-0x000000043fffffff] usable > ^^^-upper limit for adressable memory > ... > PM: Registered nosave memory: [mem 0xc0000000-0xffffffff] > [mem 0xc0000000-0xffffffff] available for PCI devices > ... > pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7] > pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] > pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] > ^-- 128K > pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff window] > ^-- 512M > pci_bus 0000:00: root bus resource [mem 0xf0000000-0xf07fffff window] > ^-- 8M > pci_bus 0000:00: root bus resource [bus 00-ff] > > =3D=3D (c) =3D=3D > Until now all runs OK. > > Guest Linux then allocates memory regions for devices. > Allocation is done from lower reg (0x10) to higher (0x30) > for each device (i.e. from 00.0 to 1f.0) on PCI bus. > > Here I reordered dmesg output to groups to show continuous RAM regions: > (pass-through device is marked with ) > pci 0000:00:01.0: reg 0x24: [io 0x2000-0x207f] > pci 0000:00:02.0: reg 0x10: [io 0x2080-0x209f] > pci 0000:00:03.0: reg 0x10: [io 0x20c0-0x20ff] > ... > pci 0000:00:00.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > pci 0000:00:02.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > pci 0000:00:03.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > pci 0000:00:1d.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > pci 0000:00:1e.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > pci 0000:00:1f.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > ... > pci 0000:00:01.0: reg 0x10:[mem 0xc0000000-0xc0ffffff] 16M... 0xc1000000-= 0xc1ffffff 16M gap > pci 0000:00:01.0: reg 0x1c:[mem 0xc2000000-0xc3ffffff 64bit pref] 32M > pci 0000:00:01.1: reg 0x10:[mem 0xc4000000-0xc4003fff]pci 0000:00:02.0: r= eg 0x14: [mem 0xc4004000-0xc4005fff] > pci 0000:00:03.0: reg 0x14: [mem 0xc4006000-0xc4007fff] > pci 0000:00:1d.0: reg 0x10: [mem 0xc4008000-0xc400807f] > ... 0xc4008080-0xc4ffffff <16M gap > pci 0000:00:1d.0: reg 0x14: [mem 0xc5000000-0xc5ffffff] 16M > pci 0000:00:1e.0: reg 0x10: [mem 0xc6000000-0xc6000fff] > ... 0xc6001000-0xd2ffffff <208M gap > pci 0000:00:01.0: reg 0x30:[mem 0xd3000000-0xd307ffff pref] 512K > 0xd3080000-0xdfffffff <208M gap > pci 0000:00:01.0: reg0x14:[mem 0x440000000-0x44fffffff 64bit pref] 256M^^= ^- this value is outside allowed range > > =3D=3D (d) =3D=3D > So, there is no window for 256M BAR, although there are 2 big gapes > of 208M in 512M space provided for BAR allocation by PCI bus. > > So, BAR reg 0x14 of size 256M for device 01.0 must be inside provisioned > 512M region 0xc0000000-0xdfffffff. > But refering to (1) above, setting base address to any value below > 0x440000000 breaks bhyve on start. > According to (b), this value corresponds to upper addressable memory > limit in guest VM. > > So I'm blocked here at the moment: > > - Guest VM requires a value which BHyve doesn't like. > - Guest VM allocates BARs with huge gapes. > > I have little knowledge about PCI bus internals, although I already r= ead > some articles on internet. > Could it be some ACPI trick to do? > I'd be happy to hear any ideas... > > PS > I suspect that if I take other OS as a guest VM or other pass-through > GPU model, it would probably allocate BARs properly. > But this is not what I want for this config. > There should be a way to allocate 256M BAR in guest Linux. > > Have a nice day, > Alex > > > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@freebsd.org Sun Mar 15 15:57:07 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4489526F640 for ; Sun, 15 Mar 2020 15:57:07 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48gPFZ5X3Tz3yJ5 for ; Sun, 15 Mar 2020 15:57:06 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: by mailman.nyi.freebsd.org (Postfix) id B899E26F637; Sun, 15 Mar 2020 15:57:06 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B841D26F636; Sun, 15 Mar 2020 15:57:06 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48gPFX6Lc9z3yHg; Sun, 15 Mar 2020 15:57:04 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 7195B14F9B; Sun, 15 Mar 2020 16:56:55 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8oDTxIEBuVSh; Sun, 15 Mar 2020 16:56:54 +0100 (CET) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id C2A1014F8C; Sun, 15 Mar 2020 16:56:54 +0100 (CET) Subject: Re: [RFC] Adding a Rados block driver to bhyve To: Alan Somers Cc: "freebsd-virtualization@freebsd.org" , FreeBSD Hackers References: <9c7a8dea-ac8a-4d17-ed33-b6c4e882add8@digiware.nl> <936ed7c2-99d2-5df8-de3f-f64f28d2ba6f@digiware.nl> From: Willem Jan Withagen Message-ID: <64ff960a-0ea9-60aa-a3b9-bd7f3e1a9d35@digiware.nl> Date: Sun, 15 Mar 2020 16:56:54 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Content-Language: nl X-Rspamd-Queue-Id: 48gPFX6Lc9z3yHg X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wjw@digiware.nl designates 2001:4cb8:90:ffff::3 as permitted sender) smtp.mailfrom=wjw@digiware.nl X-Spamd-Result: default: False [-5.25 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.85)[-0.848,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[digiware.nl]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-3.10)[ip: (-9.51), ipnet: 2001:4cb8::/29(-4.68), asn: 28878(-1.35), country: NL(0.03)]; TO_DN_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:28878, ipnet:2001:4cb8::/29, country:NL]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2020 15:57:07 -0000 On 10-3-2020 17:48, Alan Somers wrote: > On Tue, Mar 10, 2020 at 10:28 AM Willem Jan Withagen > wrote: > >> On 10-3-2020 17:21, Alan Somers wrote: >> >> On Tue, Mar 10, 2020 at 9:41 AM Willem Jan Withagen >> wrote: >> >>> On 10-3-2020 16:15, Alan Somers wrote: >>> >>> On Tue, Mar 10, 2020 at 3:59 AM Willem Jan Withagen >>> wrote: >>> >>>> On 9-3-2020 14:46, Alan Somers wrote: >>>> >>>> On Mon, Mar 9, 2020 at 4:32 AM Willem Jan Withagen >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> And sorry for crosspoing three groups, but the answer can/could be a mix >>>>> of things to do in these three areas. >>>>> >>>>> I have a prototype of bhyve running on Rados/Ceph working: >>>>> https://github.com/freebsd/freebsd/pull/426 >>>>> >>>>> .......... >>>> 4) Create a bhyve-blockrbd port. >>>>> This is much like 3) but instead of building a bhyve-rbd >>>>> executable, >>>>> it delivers a libblockrbd.so that is dynamically loadable by the >>>>> standaard bhyve that comes with base. >>>>> >>>> ............ >>>> >>>>> Great work! I also agree that option 4 sounds like the best. There's >>>> precedent for ports that >>>>> require the FreeBSD Sources. For example, see devel/py-libzfs or >>>> emulators/virtualbox-ose. >>>>> You just need to define the SRC_BASE variable. >>>> Hi Alan, >>>> >>>> Thanx for the hint, and it made me check what is actually available >>>> within the poudriere jail >>>> And that does have full source, so the Makefile code is mainly for those >>>> that build in a different way. >>>> >>>> I've got a proto version working when compiling stuff with `make >>>> buildworld`, but run in the >>>> problem that libblock_rbd.so is stripped in such a way that the symbol I >>>> need is removed. >>>> Using the unstripped version does work. >>>> >>>> Is there an incantation for the SRC Makefiles that builds a dynamical >>>> loadable lib?? >>>> And I'm still looking for a PORTS example of building a dynamical >>>> loadable lib. >>>> Or is there no generic code for that in the PORTS Mk files? >>>> >>>> --WjW >>>> >>>> BTW: Still haven't worked in your AIO code :( >>>> >>> There are plenty of dynamic libraries built with the SRC makefiles. For >>> example, >>> https://svnweb.freebsd.org/base/head/lib/libbsdstat/Makefile?view=markup >>> . >>> >>> >>> That looks dangerously close to what I have for libblock_rbd. >>> === >>>> cat Makefile-librbd >>> # >>> # $FreeBSD$ >>> # >>> >>> PACKAGE=lib${LIB} >>> >>> .include >>> >>> LIB= block_rbd >>> SHLIB_MAJOR= 1 >>> >>> SRCS= block_rbd.c >>> >>> CFLAGS+=-I${SRCTOP}/sys >>> CFLAGS+=-g -O0 -fPIC -rdynamic >>> LDFLAGS+=-Wl,-export-dynamic,-Bdynamic >>> CFLAGS+=-DWITHOUT_CAPSICUM >>> >>> LOCALBASE?= /usr/local >>> CFLAGS+= -I${LOCALBASE}/include >>> LDFLAGS+= -L${LOCALBASE}/lib -lrados -lrbd >>> >>> WARNS?= 2 >>> >>> === >>> >>> This is the code that mk.lib.bsd runs: >>> objcopy --only-keep-debug libblock_rbd.so.1.full libblock_rbd.so.1.debug >>> objcopy --strip-debug --add-gnu-debuglink=libblock_rbd.so.1.debug >>> libblock_rbd.so.1.full libblock_rbd.so.1 >>> >>> So still I get a stripped lib in /usr/lib. And then the one and only >>> symbol I need to load >>> is not found. Copying libblock_rbd.so.1.full actually works for me. >>> >>> So either I'm doing it the wrong way, like special options on the symbols >>> oid. >>> Or mk.lib.bsd cannot deliver dlopen/dlsym-able files? >>> >>> And there are plenty of ports that build shared libraries too, just look >>> at /usr/local/lib/*.so. However, the ports framework doesn't have much >>> special code just to support building libraries. Instead the hard work is >>> always done by the ports themselves. Some use autotools, some cmake, etc >>> etc. The simplest port I can find that uses both SRC_BASE and INSTALL_LIB >>> is this one: https://svnweb.freebsd.org/ports/head/devel/linux_libusb/ . >>> >>> Oke thanx, I'll have a look at it, and given that I can see most of the >>> compile build stuff >>> in the SRC_BASE version I'll get it to work. >>> >>> --WjW >>> >> Try setting "STRIP= " in your makefile. That should prevent the >> stripping. However, I think there's something wrong with your library, >> too. The library should be usable even if it's stripped. >> >> I checked with objdump, and the symbol that I need is definitly not >> present in the stripped version. >> And it does not really matter if I declare it static or not. >> But I'll give it a few more itterations to try it out. >> Including 'STRIP= ' >> >> Thanx, >> --WjW >> > What does "nm --dynamic libblock_rbd.so.1" show? Don't know what has changed, but I just rebuild everything. And now it seems to work.... ;-) One thing I'm wondering if th DATA_SET(block_backend_set, ....) gets updated when dyn-loading the library it is in? Or if not, how do I add an entry to this DATA_SET, once we are running the program? I'm dlsym() looking for block_backend_rbd: w _Jv_RegisterClasses U __assert w __cxa_finalize U __error U __stack_chk_fail U __stack_chk_guard 0000000000006098 D __start_set_block_backend_set U __stderrp 00000000000060a0 D __stop_set_block_backend_set 000000000000475c T _fini 000000000000474c T _init 0000000000006008 D block_backend_rbd 0000000000006270 B blocklocal_backend 0000000000002170 T blockrbd_cleanup 0000000000006250 B blockrbd_debug 0000000000002080 T blockrbd_init Which is the variable holding the block_if:jumptable, and gives me access to all that I need to work with the driver. ============== block_backend_t block_backend_rbd = { .bb_name = "rbd", .bb_init = blockrbd_init, .bb_cleanup = blockrbd_cleanup, .bb_open = blockrbd_open, .bb_size = blockrbd_size, .bb_chs = blockrbd_chs, .bb_sectsz = blockrbd_sectsz, .bb_psectsz = blockrbd_psectsz, .bb_queuesz = blockrbd_queuesz, .bb_is_ro = blockrbd_is_ro, .bb_candelete = blockrbd_candelete, .bb_read = blockrbd_read, .bb_write = blockrbd_write, .bb_flush = blockrbd_flush, .bb_delete = blockrbd_delete, .bb_cancel = blockrbd_cancel, .bb_close = blockrbd_close, .bb_opaque = (void*)&block_backend_rbd, }; DATA_SET(block_backend_set, block_backend_rbd); ============== This is the Makefile: # # $FreeBSD$ # PACKAGE= lib${LIB} LIB= block_rbd .include SHLIB_MAJOR= 1 SRCS= block_rbd.c CFLAGS+= -I${SRCTOP}/sys CFLAGS+= -g -O0 -fPIC -rdynamic LDFLAGS+= -Wl,-export-dynamic,-Bdynamic CFLAGS+= -DWITHOUT_CAPSICUM LOCALBASE?= /usr/local CFLAGS+= -I${LOCALBASE}/include LDFLAGS+= -L${LOCALBASE}/lib -lrados -lrbd WARNS?= 2 .include --WjW From owner-freebsd-virtualization@freebsd.org Sun Mar 15 16:46:21 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9BE5627301F for ; Sun, 15 Mar 2020 16:46:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48gQLP3G7vz4Sb6 for ; Sun, 15 Mar 2020 16:46:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 0B35727301E; Sun, 15 Mar 2020 16:46:21 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0AD3E27301D for ; Sun, 15 Mar 2020 16:46:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48gQLN55rwz4SYr for ; Sun, 15 Mar 2020 16:46:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 85B45AA45 for ; Sun, 15 Mar 2020 16:46:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02FGkKt3059213 for ; Sun, 15 Mar 2020 16:46:20 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02FGkKEw059212 for virtualization@FreeBSD.org; Sun, 15 Mar 2020 16:46:20 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236989] AWS EC2 lockups "Missing interrupt" Date: Sun, 15 Mar 2020 16:46:20 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cperciva@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2020 16:46:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236989 --- Comment #15 from Colin Percival --- Vincent: There are known issues with the nvme driver on 11.3 and 12.0. If = you can reproduce this on 12.1 I'll be *very* interested but I think it's very likely the problem will go away when you upgrade. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sun Mar 15 21:01:14 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 136CE2787CE for ; Sun, 15 Mar 2020 21:01:14 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48gX0T61hGz3KPG for ; Sun, 15 Mar 2020 21:01:13 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id C16F52787C0; Sun, 15 Mar 2020 21:01:13 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C0E692787BF for ; Sun, 15 Mar 2020 21:01:13 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48gX0T4MWSz3KP1 for ; Sun, 15 Mar 2020 21:01:13 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8A045DBB8 for ; Sun, 15 Mar 2020 21:01:13 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02FL1Dvt028233 for ; Sun, 15 Mar 2020 21:01:13 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02FL1Dc0028232 for virtualization@FreeBSD.org; Sun, 15 Mar 2020 21:01:13 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202003152101.02FL1Dc0028232@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 15 Mar 2020 21:01:13 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2020 21:01:14 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 240945 | [hyper-v] [netvsc] hn network driver incorrectly 1 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Mon Mar 16 02:45:15 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C357327F6CD for ; Mon, 16 Mar 2020 02:45:15 +0000 (UTC) (envelope-from stryqx@gmail.com) Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48ggdQ4GKCz3Gdf for ; Mon, 16 Mar 2020 02:45:14 +0000 (UTC) (envelope-from stryqx@gmail.com) Received: by mail-io1-xd34.google.com with SMTP id c19so14545065ioo.6 for ; Sun, 15 Mar 2020 19:45:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=YXxHmIEhLXRWOvrvOLdtLVWkvGWVgI9wgvhKwT9l1ig=; b=oA9MSAXcA2wKHH/h4lgxn0Y+OuLMu6qHRRyvxfTiJrLhK8PTdEVzAAHjsAqU2r9DcI 4l3ttkitzP6/aZTgQ99czNYrh6zDPLoJgILriIvBmKiuEFmo2kzT+bvFEv21JqZlgYVq FurRAlSudF42RwPd2apOvBOzy+a6+chQ39JGoXqFWyOqhKO7+f0LJQyX+EHYh5fIgP5t 9rB7BdibSItwTnNBaX7l8ajrhpm/6QK6tXr36GGFpLNa2M2Lz/AVDpL41HGrL2xapUFt NdRPOYo6ax9QKK39mqu0Ts8t7c8kvnF8Ljk2hMxjhFlGI5b4CkriwOG0UjrrdBeLG+ZK eYuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=YXxHmIEhLXRWOvrvOLdtLVWkvGWVgI9wgvhKwT9l1ig=; b=MrPMnUQwbItvtDQtPRTON972wOj0Q0BjVsyJaNUmf/S7s/NoH/UK0f4JJ1ZgO1++WY MLTsn58w/eJ8ZA6m6k2iK4HXEZsDaxy72QgH1WF/q3+yErx9nXSR5gmDHb+xJ0UTZP+k MfJMgP8Ny2xpLuLHdZhCjGlhDG7ZqwK6hk3lStFgYJNWaSJtzSH6iG5ssRyQ7M59fxj9 XsDCgUkS6OaoEri5lA85VF+MDJZs1RTZcBSZK1sycM9Lh04Ml6FrQLSYvx3rwugmNMfZ 6ZuEr+3iquBnFKbbnA/jIg5W3sKay7hcFOuhU9KorH5IcfJYPYbGogJjV2Ny9HTXtY62 GkmA== X-Gm-Message-State: ANhLgQ1PBVRnwPH3qT+NzKUKut7PDlYSFSBat/6sVymtCjG9QodwX3Rj +odoEB9DNoGShKRuptLbYsXYFRsiSRIU8BxbwzpCLR4nQcA= X-Google-Smtp-Source: ADFU+vuulPL7GtohoXC4BMrsTbHx0nqeM/bjv6kdBYtSazfziC9wo+QtS9q31LQQc94HfHpjZO2Welt2OP9SkTAl5NU= X-Received: by 2002:a02:2384:: with SMTP id u126mr2556073jau.64.1584326712870; Sun, 15 Mar 2020 19:45:12 -0700 (PDT) MIME-Version: 1.0 References: <20200312190322.GG58645@bastion.zyxst.net> In-Reply-To: <20200312190322.GG58645@bastion.zyxst.net> From: Chris Knight Date: Mon, 16 Mar 2020 13:44:36 +1100 Message-ID: Subject: Re: freebsd vm on azure - how to backup the vm as an image To: "freebsd-virtualization@freebsd.org" X-Rspamd-Queue-Id: 48ggdQ4GKCz3Gdf X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=oA9MSAXc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of stryqx@gmail.com designates 2607:f8b0:4864:20::d34 as permitted sender) smtp.mailfrom=stryqx@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; URI_COUNT_ODD(1.00)[9]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ip: (-5.22), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 02:45:15 -0000 Howdy, Not tried it, but if the FreeBSD VM has the Azure Agent installed on it, then you should be able to get crash-consistent image backups (and maybe application-consistent image backups if the Azure Agent + the integration components support the application-consistent backup framework) using Azure Backup. More info here: https://docs.microsoft.com/en-us/azure/backup/backup-support-matrix-iaas https://docs.microsoft.com/en-us/azure/backup/backup-azure-linux-app-consistent https://docs.microsoft.com/en-au/azure/virtual-machines/extensions/agent-linux On Fri, 13 Mar 2020 at 06:03, tech-lists wrote: > Hi, > > Is there a way (yet) of backing up a freebsd-12 vm hosted on azure? Of > course > on bhyve it's simple, turn off the vm and download it. But on Azure? > > thanks, > -- > J. > -- Regards, Chris Knight From owner-freebsd-virtualization@freebsd.org Mon Mar 16 05:33:45 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DFF3B25A85C for ; Mon, 16 Mar 2020 05:33:45 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48glMr16Pqz3PZ1 for ; Mon, 16 Mar 2020 05:33:43 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-qt1-x842.google.com with SMTP id z8so9771361qto.12 for ; Sun, 15 Mar 2020 22:33:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=pnSGKMvg7WjzEetpE1+w63qs7Rg4uCDJqbMf0mpRMMo=; b=ZrWPY9HCdZRSGIq4/YY2uFRuugpNXh4eQMUrE2UVoN+N8T+JZzKPZ4WV4kMfM1tzqT rdQvOKOWG2/MFOgjaP/Tw5Vao+11A324mdc4fFrbDaBNDBwzdd8VUGdS6Q1edyOyMh2t QB1DuABcAM7+kNYSKoXPbBWz1w2lckvuLN3a4FGaR3STCznBqcsmeMV+0P0jI7AyByME GgzUMLycbtS4HFcCtbvRx88ChCOqZqaa3kE5wrsb/Rat13FHCpKf5p6VT1isbfUb002H oDKK41+r2hEtpv6LMXKbqNpoPnvPeLMrmDWZewWnLenWSjeUednsZtJajmQ7wvaxOEcI IiDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=pnSGKMvg7WjzEetpE1+w63qs7Rg4uCDJqbMf0mpRMMo=; b=QZJxsEVWumyOrnbWofFp2BZhUfVH1ZUrJJBJyRZjCQNpID6F4D53GoXgzhLjOTKVQO 7QWHw8esiMDWkcfuwQc6Xg8LBEjr8nbn1evPsDPt9rpDAdFSnTb85VXSvdnuZH4wKQLS kI3RYN6VFGsZ+lLtp9heNC6Dzps6qqoG9R9hrAxK0gfZHh6XUt5uoTrBxwtr0jZBjEgc pzge1oHfKZUB9613hSNG+6rUV6rch5IxJjkXxsVUj5TuShzN9Z7WEVcSqUgJrICOzKLl 6bBuQ0d7jkHcDVY6TiDC0KAR5zEPH2tAKAli39AymPzWmA8u/xlnVETi56l5Cunj3wpv mX6Q== X-Gm-Message-State: ANhLgQ1nQ9mcG56CSYtuZ9LC8bMxpamCdkc30OMfPV+2/YpZUoXP8QcN yiAHd2UJFVXWfxRdf9vxrAOTQbmwwjHfuzgBvLm7joRPYl0= X-Google-Smtp-Source: ADFU+vuHW02Ri/Jk8x9T/gMgUCOOVaFHmAvDIn+BVEmRSb3bI/ID60Kn3F7GwTujrbb3drZSgLYRFCg4pzkmqk62kp8= X-Received: by 2002:ac8:1bf5:: with SMTP id m50mr12542756qtk.200.1584336822159; Sun, 15 Mar 2020 22:33:42 -0700 (PDT) MIME-Version: 1.0 From: Wanpeng Qian Date: Mon, 16 Mar 2020 14:33:30 +0900 Message-ID: Subject: bhyve: Passthru second port of i350-T2 to Windows cause system crash. To: FreeBSD virtualization Content-Type: multipart/mixed; boundary="0000000000007dff2a05a0f22920" X-Rspamd-Queue-Id: 48glMr16Pqz3PZ1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=ZrWPY9HC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of wanpengqian@gmail.com designates 2607:f8b0:4864:20::842 as permitted sender) smtp.mailfrom=wanpengqian@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE_FREEMAIL(0.00)[]; IP_SCORE(0.00)[ip: (1.75), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 05:33:46 -0000 --0000000000007dff2a05a0f22920 Content-Type: text/plain; charset="UTF-8" Hello all I am using FreeBSD 12.1R + patch https://reviews.freebsd.org/D20531 Without D20531 patch, I can passthru this first port of i350-T2 to a Linux vm. and with the Patch, I can passthough the second port to A windows VM. it works. about several minutes later, system crash. I reset the sever via IPMI. the last dmesg keep. see the attached file. I sometimes get such error duing the passthrough in FreeBSD 12.1 without passthrough(with/without patch), the system is stable. and I run ESXi with passthough also. pretty stable for months without reboot. any idea? am I doing something wrong? I think I am separate the iommu group to different vm? how to check the iommu group in FreeBSD? Thanks. --0000000000007dff2a05a0f22920 Content-Type: application/octet-stream; name="demsg20200316.log" Content-Disposition: attachment; filename="demsg20200316.log" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_k7u1300l0 bnZtZTI6IFdSSVRFIHNxaWQ6MTMgY2lkOjEyMSBuc2lkOjEgbGJhOjY0OTcyMTg4MCBsZW46MzIK bnZtZTI6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTMgY2lkOjEyMSBjZHcwOjAKbnZtZTA6 IFdSSVRFIHNxaWQ6MyBjaWQ6NjggbnNpZDoxIGxiYTo2NDk3MjE4ODggbGVuOjMyCm52bWUwOiBX UklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjMgY2lkOjY4IGNkdzA6MApudm1lMjogYXN5bmMgZXZl bnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IGFzeW5j IGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUxOiBX UklURSBzcWlkOjE0IGNpZDo1MiBuc2lkOjEgbGJhOjIxODA2NTk0MDggbGVuOjMyCm52bWUxOiBX UklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE0IGNpZDo1MiBjZHcwOjAKbnZtZTI6IFdSSVRFIHNx aWQ6MTQgY2lkOjExMiBuc2lkOjEgbGJhOjIxODA2NTk0MDggbGVuOjMyCm52bWUyOiBXUklURSBG QVVMVFMgKDAyLzgwKSBzcWlkOjE0IGNpZDoxMTIgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBv Y2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMjogYXN5bmMgZXZl bnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTI6IFdSSVRF IHNxaWQ6MTIgY2lkOjExNiBuc2lkOjEgbGJhOjIxODA2NjE2ODAgbGVuOjMyCm52bWUyOiBXUklU RSBGQVVMVFMgKDAyLzgwKSBzcWlkOjEyIGNpZDoxMTYgY2R3MDowCm52bWUwOiBXUklURSBzcWlk OjEyIGNpZDoxMjQgbnNpZDoxIGxiYToyMTgwNjYxNjgwIGxlbjozMgpudm1lMDogV1JJVEUgRkFV TFRTICgwMi84MCkgc3FpZDoxMiBjaWQ6MTI0IGNkdzA6MApudm1lMjogYXN5bmMgZXZlbnQgb2Nj dXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IGFzeW5jIGV2ZW50 IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCihhZGEzOmFoY2ljaDM6 MDowOjApOiBXUklURV9GUERNQV9RVUVVRUQuIEFDQjogNjEgNDAgYTAgOTAgMWEgNDAgMDYgMDAg MDAgMDAgMDAgMDAKKGFkYTM6YWhjaWNoMzowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0 IHdhcyBpbnZhbGlkCihhZGEzOmFoY2ljaDM6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKbnZtZTE6IFdSSVRFIHNxaWQ6MTMgY2lkOjEwMyBuc2lkOjEgbGJhOjkwNDA2OTcyMCBs ZW46MzIKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTMgY2lkOjEwMyBjZHcwOjAK bnZtZTE6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4 MDEpCm52bWUyOiBXUklURSBzcWlkOjQgY2lkOjc0IG5zaWQ6MSBsYmE6OTA0MDcxNjY0IGxlbjox Ngpudm1lMjogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo0IGNpZDo3NCBjZHcwOjAKbnZtZTI6 IFdSSVRFIHNxaWQ6NCBjaWQ6MTI3IG5zaWQ6MSBsYmE6OTA0MDcxNjgwIGxlbjo0OApudm1lMjog V1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo0IGNpZDoxMjcgY2R3MDowCm52bWUyOiBhc3luYyBl dmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMTogV1JJ VEUgc3FpZDoyIGNpZDo3OCBuc2lkOjEgbGJhOjY0OTcyMjI4MCBsZW46MzIKbnZtZTE6IFdSSVRF IEZBVUxUUyAoMDIvODApIHNxaWQ6MiBjaWQ6NzggY2R3MDowCm52bWUwOiBXUklURSBzcWlkOjcg Y2lkOjkwIG5zaWQ6MSBsYmE6NjQ5NzIyMjgwIGxlbjozMgpudm1lMDogV1JJVEUgRkFVTFRTICgw Mi84MCkgc3FpZDo3IGNpZDo5MCBjZHcwOjAKbnZtZTE6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0 eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUwOiBhc3luYyBldmVudCBvY2N1cnJl ZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMjogV1JJVEUgc3FpZDoxNCBj aWQ6NzkgbnNpZDoxIGxiYTo5MDQxMDMyMDggbGVuOjMyCm52bWUyOiBXUklURSBGQVVMVFMgKDAy LzgwKSBzcWlkOjE0IGNpZDo3OSBjZHcwOjAKbnZtZTA6IFdSSVRFIHNxaWQ6MyBjaWQ6NjQgbnNp ZDoxIGxiYTo5MDQxMDMyMTYgbGVuOjMyCm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlk OjMgY2lkOjY0IGNkdzA6MApudm1lMDogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBp bmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTI6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4 MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUxOiBXUklURSBzcWlkOjEyIGNpZDoxMTEgbnNp ZDoxIGxiYTo4MjgyNzE2MjQgbGVuOjY0Cm52bWUxOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlk OjEyIGNpZDoxMTEgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAs IGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMjogV1JJVEUgc3FpZDoxNCBjaWQ6NzkgbnNpZDox IGxiYTo5MDQxMDcxMzYgbGVuOjY0Cm52bWUyOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE0 IGNpZDo3OSBjZHcwOjAKbnZtZTI6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5m byAweDA0LCBwYWdlIDB4MDEpCm52bWUxOiBXUklURSBzcWlkOjE0IGNpZDo1MiBuc2lkOjEgbGJh OjI1OTQ4OTg1MDQgbGVuOjY0Cm52bWUxOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE0IGNp ZDo1MiBjZHcwOjAKbnZtZTE6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAw eDA0LCBwYWdlIDB4MDEpCm52bWUxOiBXUklURSBzcWlkOjE0IGNpZDo1OCBuc2lkOjEgbGJhOjkw NDEwNzYwMCBsZW46NjQKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTQgY2lkOjU4 IGNkdzA6MApudm1lMTogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQs IHBhZ2UgMHgwMSkKbnZtZTE6IFdSSVRFIHNxaWQ6MTQgY2lkOjUyIG5zaWQ6MSBsYmE6NjI4NDQy OTI4IGxlbjo2NApudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxNCBjaWQ6NTIgY2R3 MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFn ZSAweDAxKQpTR0wgZm9yIGNvbW1hbmQgMHhmZmZmZmUwMTdhYzJhMjkwCjAwMDAgICAwYSAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAyOCBlZSA0MSAwMyAgfC4uLi4uLi4uLi4uLigu QS58CjAwMTAgICAwMCAwMCBmYyAwMCAxOCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAxMCAwMCAw MCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMjAgICAwMCAwMCAwMCAwMCAwYSAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMzAgICAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMSAgfC4uLi4uLi4uLi4uLi4uLi58 CjAwNDAgICAyYSAwMCAxYyAyNiA0OSBhMCAwMCAwMCAwOCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAg fCouLiZJLi4uLi4uLi4uLi58CjAwNTAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNjAgICAwMCAxMCAwMCBkNyAw MCA5MCA3MCBjZSAyMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLnAuIC4uLi4uLi58CjAw NzAgICAwMCAwMCAwMCA0NiAzZSAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4u LkY+Li4uLi4uLi4uLi58CnNlZzAgZmxhZ3M9MHhkNyBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAw MDIwY2U3MDkwMDAKU0dMIGZvciBjb21tYW5kIDB4ZmZmZmZlMDE3YWM4ZDQzMAowMDAwICAgMGEg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgYjggNzcgNDYgMDMgIHwuLi4uLi4uLi4u Li4ud0YufAowMDEwICAgMDAgMDAgZmMoZGExOm1wczA6MDo2NTowKTogV1JJVEUoMTApLiBDREI6 IDJhIDAwIDFjIDI2IDQ5IGEwIDAwIDAwIDA4IDAwIAogMDAoZGExOm1wczA6MDo2NTowKTogQ0FN IHN0YXR1czogVW5yZWNvdmVyYWJsZSBIb3N0IEJ1cyBBZGFwdGVyIEVycm9yCiAxOChkYTE6bXBz MDowOjY1OjApOiBSZXRyeWluZyBjb21tYW5kLCAzIG1vcmUgdHJpZXMgcmVtYWluCiAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCA4MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMjAgICAwMCAw MCAwMCAwMCAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4u Li4uLi58CjAwMzAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMSAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNDAgICA4YSAwMCAwMCAwMCAwMCAwMSAwYSBmZCAx NSBmOCAwMCAwMCAwMCA0MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi5ALi58CjAwNTAgICAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4u Li58CjAwNjAgICAwMCAxMCAwMCAxNiAwMCAxMCAyMCAwNyAyMiAwMCAwMCAwMCAwMCAxMCAwMCA5 NiAgfC4uLi4uLiAuIi4uLi4uLi58CjAwNzAgICAwMCAxMCA5YiAxNCAyMiAwMCAwMCAwMCA4MCAw MCAwMCAzMCAwMCBmNCAyYyAwMyAgfC4uLi4iLi4uLi4uMC4uLC58CnNlZzAgZmxhZ3M9MHgxNiBs ZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyMDcyMDEwMDAKc2VnMSBmbGFncz0weDk2IGxlbj0w eDAwMTAwMCBhZGRyPTB4MDAwMDAwMjIxNDliMTAwMApjaGFpbiBmbGFncz0weDMwIGxlbj0weDgw IE9mZnNldD0weDAgQWRkcmVzcz0weDMyY2Y0MDAKMDAwMCAgIDAwIDEwIDAwIDE2IDAwIDQwIGIz IGNjIDM5IDAwIDAwIDAwIDAwIDEwIDAwIDE2ICB8Li4uLi5ALi45Li4uLi4uLnwKMDAxMCAgIDAw IDEwIDAwIGE1IDIyIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIGIwIDI5IDNmICB8Li4uLiIuLi4u Li4uLi4pP3wKMDAyMCAgIDIxIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIDgwIDBjIGUyIDIxIDAw IDAwIDAwICB8IS4uLi4uLi4uLi4uIS4uLnwKMDAzMCAgIDAwIDEwIDAwIDE2IDAwIDMwIDQ4IDA1 IDIxIDAwIDAwIDAwIDAwIDEwIDAwIGQ3ICB8Li4uLi4wSC4hLi4uLi4uLnwKMDA0MCAgIDAwIDkw IGQwIDYxIDIxIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uYSEuLi4uLi4u Li4uLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA3MCAgIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4u LnwKc2VnMiBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMzljY2IzNDAwMApz ZWczIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmE1MDAxMDAwCnNlZzQg ZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIxM2YyOWIwMDAKc2VnNSBmbGFn cz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjFlMjBjODAwMApzZWc2IGZsYWdzPTB4 MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMTA1NDgzMDAwCnNlZzcgZmxhZ3M9MHhkNyBs ZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIxNjFkMDkwMDAKKGRhMTptcHMwOjA6NjU6MCk6IFdS SVRFKDE2KS4gQ0RCOiA4YSAwMCAwMCAwMCAwMCAwMSAwYSBmZCAxNSBmOCAwMCAwMCAwMCA0MCAw MCAwMCAKKGRhMTptcHMwOjA6NjU6MCk6IENBTSBzdGF0dXM6IFVucmVjb3ZlcmFibGUgSG9zdCBC dXMgQWRhcHRlciBFcnJvcgooZGExOm1wczA6MDo2NTowKTogUmV0cnlpbmcgY29tbWFuZCwgMyBt b3JlIHRyaWVzIHJlbWFpbgooZGExOm1wczA6MDo2NTowKTogV1JJVEUoMTApLiBDREI6IDJhIDAw IDFjIDI2IDQ5IGEwIDAwIDAwIDA4IDAwIAooZGExOm1wczA6MDo2NTowKTogQ0FNIHN0YXR1czog U0NTSSBTdGF0dXMgRXJyb3IKKGRhMTptcHMwOjA6NjU6MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBD b25kaXRpb24KKGRhMTptcHMwOjA6NjU6MCk6IFNDU0kgc2Vuc2U6IFVOSVQgQVRURU5USU9OIGFz YzoyOSwwIChQb3dlciBvbiwgcmVzZXQsIG9yIGJ1cyBkZXZpY2UgcmVzZXQgb2NjdXJyZWQpCihk YTE6bXBzMDowOjY1OjApOiBSZXRyeWluZyBjb21tYW5kIChwZXIgc2Vuc2UgZGF0YSkKU0dMIGZv ciBjb21tYW5kIDB4ZmZmZmZlMDE3YWM2YzJjMAowMDAwICAgMGEgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgZTAgZjMgNDQgMDMgIHwuLi4uLi4uLi4uLi4uLkQufAowMDEwICAgMDAg MDAgZmMgMDAgMTggMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMTAgMDAgMDAgIHwuLi4uLi4uLi4u Li4uLi4ufAowMDIwICAgMDAgMDAgMDAgMDAgMGEgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDMwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDEgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDQwICAgMmEgMDAg MWMgMjYgNDkgYTAgMDAgMDAgMDggMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwqLi4mSS4uLi4uLi4u Li4ufAowMDUwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDYwICAgMDAgMTAgMDAgZDcgMDAgOTAgNzAgY2UgMjAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi5wLiAuLi4uLi4ufAowMDcwICAgMDAgMDAgYzAg ZDIgM2MgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uPC4uLi4uLi4uLi4u fApzZWcwIGZsYWdzPTB4ZDcgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMGNlNzA5MDAwClNH TCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjYjM4ZjAKMDAwMCAgIDBhIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDk4IDM4IDQ4IDAzICB8Li4uLi4uLi4uLi4uLjhILnwKMDAxMCAg IDAwIDAwIGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDgwIDAwIDAwICB8Li4uLi4u Li4uLi4uLi4uLnwKMDAyMCAgIDAwIDAwIDAwIDAwIDEwIDAwKGRhMTptcHMwOjA6NjU6MCk6IFdS SVRFKDEwKS4gQ0RCOiAyYSAwMCAxYyAyNiA0OSBhMCAwMCAwMCAwOCAwMCAKIDAwKGRhMTptcHMw OjA6NjU6MCk6IENBTSBzdGF0dXM6IFVucmVjb3ZlcmFibGUgSG9zdCBCdXMgQWRhcHRlciBFcnJv cgooZGExOm1wczA6MDo2NTowKTogUmV0cnlpbmcgY29tbWFuZCwgMSBtb3JlIHRyaWVzIHJlbWFp bgogMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDMwICAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDEgIHwuLi4uLi4u Li4uLi4uLi4ufAowMDQwICAgOGEgMDAgMDAgMDAgMDAgMDEgMGEgZmQgMTUgZjggMDAgMDAgMDAg NDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uQC4ufAowMDUwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDYwICAgMDAg MTAgMDAgMTYgMDAgMTAgMjAgMDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgOTYgIHwuLi4uLi4gLiIu Li4uLi4ufAowMDcwICAgMDAgMTAgOWIgMTQgMjIgMDAgMDAgMDAgODAgMDAgMDAgMzAgODAgZjQg MmMgMDMgIHwuLi4uIi4uLi4uLjAuLiwufApzZWcwIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFk ZHI9MHgwMDAwMDAyMjA3MjAxMDAwCnNlZzEgZmxhZ3M9MHg5NiBsZW49MHgwMDEwMDAgYWRkcj0w eDAwMDAwMDIyMTQ5YjEwMDAKY2hhaW4gZmxhZ3M9MHgzMCBsZW49MHg4MCBPZmZzZXQ9MHgwIEFk ZHJlc3M9MHgzMmNmNDgwCjAwMDAgICAwMCAxMCAwMCAxNiAwMCA0MCBiMyBjYyAzOSAwMCAwMCAw MCAwMCAxMCAwMCAxNiAgfC4uLi4uQC4uOS4uLi4uLi58CjAwMTAgICAwMCAxMCAwMCBhNSAyMiAw MCAwMCAwMCAwMCAxMCAwMCAxNiAwMCBiMCAyOSAzZiAgfC4uLi4iLi4uLi4uLi4uKT98CjAwMjAg ICAyMSAwMCAwMCAwMCAwMCAxMCAwMCAxNiAwMCA4MCAwYyBlMiAyMSAwMCAwMCAwMCAgfCEuLi4u Li4uLi4uLiEuLi58CjAwMzAgICAwMCAxMCAwMCAxNiAwMCAzMCA0OCAwNSAyMSAwMCAwMCAwMCAw MCAxMCAwMCBkNyAgfC4uLi4uMEguIS4uLi4uLi58CjAwNDAgICAwMCA5MCBkMCA2MSAyMSAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLmEhLi4uLi4uLi4uLi58CjAwNTAgICAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4u Li4uLi4uLi58CjAwNjAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNzAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CnNlZzIgZmxhZ3M9 MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDM5Y2NiMzQwMDAKc2VnMyBmbGFncz0weDE2 IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjJhNTAwMTAwMApzZWc0IGZsYWdzPTB4MTYgbGVu PTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMTNmMjliMDAwCnNlZzUgZmxhZ3M9MHgxNiBsZW49MHgw MDEwMDAgYWRkcj0weDAwMDAwMDIxZTIwYzgwMDAKc2VnNiBmbGFncz0weDE2IGxlbj0weDAwMTAw MCBhZGRyPTB4MDAwMDAwMjEwNTQ4MzAwMApzZWc3IGZsYWdzPTB4ZDcgbGVuPTB4MDAxMDAwIGFk ZHI9MHgwMDAwMDAyMTYxZDA5MDAwCihkYTE6bXBzMDowOjY1OjApOiBXUklURSgxNikuIENEQjog OGEgMDAgMDAgMDAgMDAgMDEgMGEgZmQgMTUgZjggMDAgMDAgMDAgNDAgMDAgMDAgCihkYTE6bXBz MDowOjY1OjApOiBDQU0gc3RhdHVzOiBVbnJlY292ZXJhYmxlIEhvc3QgQnVzIEFkYXB0ZXIgRXJy b3IKKGRhMTptcHMwOjA6NjU6MCk6IFJldHJ5aW5nIGNvbW1hbmQsIDIgbW9yZSB0cmllcyByZW1h aW4KKGRhMTptcHMwOjA6NjU6MCk6IFdSSVRFKDEwKS4gQ0RCOiAyYSAwMCAxYyAyNiA0OSBhMCAw MCAwMCAwOCAwMCAKKGRhMTptcHMwOjA6NjU6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVy cm9yCihkYTE6bXBzMDowOjY1OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTE6 bXBzMDowOjY1OjApOiBTQ1NJIHNlbnNlOiBVTklUIEFUVEVOVElPTiBhc2M6MjksMCAoUG93ZXIg b24sIHJlc2V0LCBvciBidXMgZGV2aWNlIHJlc2V0IG9jY3VycmVkKQooZGExOm1wczA6MDo2NTow KTogUmV0cnlpbmcgY29tbWFuZCAocGVyIHNlbnNlIGRhdGEpClNHTCBmb3IgY29tbWFuZCAweGZm ZmZmZTAxN2FjM2U1MTAKMDAwMCAgIDBhIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDY4IGRhIDQyIDAzICB8Li4uLi4uLi4uLi4uaC5CLnwKMDAxMCAgIDAwIDAwIGZjIDAwIDE4IDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDEwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAyMCAg IDAwIDAwIDAwIDAwIDBhIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4u Li4uLi4uLi4uLnwKMDAzMCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAxICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA0MCAgIDJhIDAwIDFjIDI2IDQ5IGEwIDAw IDAwIDA4IDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Ki4uJkkuLi4uLi4uLi4uLnwKMDA1MCAgIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4u Li4uLi4uLnwKMDA2MCAgIDAwIDEwIDAwIGQ3IDAwIDkwIDcwIGNlIDIwIDAwIDAwIDAwIDAwIDAw IDAwIDAwICB8Li4uLi4ucC4gLi4uLi4uLnwKMDA3MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKc2VnMCBmbGFncz0w eGQ3IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjBjZTcwOTAwMApTR0wgZm9yIGNvbW1hbmQg MHhmZmZmZmUwMTdhYzZjMDEwCjAwMDAgICAwYSAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCBlOCBmMSA0NCAwMyAgfC4uLi4uLi4uLi4uLi4uRC58CjAwMTAgICAwMCAwMCBmYyAwMCAx OCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCA4MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAw MjAgICAwMCAwMCAwMCAwMCAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4u Li4uLihkYTE6bXBzMDowOjY1OjApOiBXUklURSgxMCkuIENEQjogMmEgMDAgMWMgMjYgNDkgYTAg MDAgMDAgMDggMDAgCi4oZGExOm1wczA6MDo2NTowKTogQ0FNIHN0YXR1czogVW5yZWNvdmVyYWJs ZSBIb3N0IEJ1cyBBZGFwdGVyIEVycm9yCihkYTE6bXBzMDowOjY1OjApOiBFcnJvciA1LCBSZXRy aWVzIGV4aGF1c3RlZAouLi4uLi4uLi58CjAwMzAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMSAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNDAgICA4YSAwMCAw MCAwMCAwMCAwMSAwYSBmZCAxNSBmOCAwMCAwMCAwMCA0MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi5A Li58CjAwNTAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNjAgICAwMCAxMCAwMCAxNiAwMCAxMCAyMCAwNyAyMiAw MCAwMCAwMCAwMCAxMCAwMCA5NiAgfC4uLi4uLiAuIi4uLi4uLi58CjAwNzAgICAwMCAxMCA5YiAx NCAyMiAwMCAwMCAwMCA4MCAwMCAwMCAzMCAwMCBmNSAyYyAwMyAgfC4uLi4iLi4uLi4uMC4uLC58 CnNlZzAgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyMDcyMDEwMDAKc2Vn MSBmbGFncz0weDk2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjIxNDliMTAwMApjaGFpbiBm bGFncz0weDMwIGxlbj0weDgwIE9mZnNldD0weDAgQWRkcmVzcz0weDMyY2Y1MDAKMDAwMCAgIDAw IDEwIDAwIDE2IDAwIDQwIGIzIGNjIDM5IDAwIDAwIDAwIDAwIDEwIDAwIDE2ICB8Li4uLi5ALi45 Li4uLi4uLnwKMDAxMCAgIDAwIDEwIDAwIGE1IDIyIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIGIw IDI5IDNmICB8Li4uLiIuLi4uLi4uLi4pP3wKMDAyMCAgIDIxIDAwIDAwIDAwIDAwIDEwIDAwIDE2 IDAwIDgwIDBjIGUyIDIxIDAwIDAwIDAwICB8IS4uLi4uLi4uLi4uIS4uLnwKMDAzMCAgIDAwIDEw IDAwIDE2IDAwIDMwIDQ4IDA1IDIxIDAwIDAwIDAwIDAwIDEwIDAwIGQ3ICB8Li4uLi4wSC4hLi4u Li4uLnwKMDA0MCAgIDAwIDkwIGQwIDYxIDIxIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwICB8Li4uYSEuLi4uLi4uLi4uLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4u LnwKMDA3MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw ICB8Li4uLi4uLi4uLi4uLi4uLnwKc2VnMiBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4 MDAwMDAwMzljY2IzNDAwMApzZWczIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAw MDAyMmE1MDAxMDAwCnNlZzQgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIx M2YyOWIwMDAKc2VnNSBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjFlMjBj ODAwMApzZWc2IGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMTA1NDgzMDAw CnNlZzcgZmxhZ3M9MHhkNyBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIxNjFkMDkwMDAKKGRh MTptcHMwOjA6NjU6MCk6IFdSSVRFKDE2KS4gQ0RCOiA4YSAwMCAwMCAwMCAwMCAwMSAwYSBmZCAx NSBmOCAwMCAwMCAwMCA0MCAwMCAwMCAKKGRhMTptcHMwOjA6NjU6MCk6IENBTSBzdGF0dXM6IFVu cmVjb3ZlcmFibGUgSG9zdCBCdXMgQWRhcHRlciBFcnJvcgooZGExOm1wczA6MDo2NTowKTogUmV0 cnlpbmcgY29tbWFuZCwgMSBtb3JlIHRyaWVzIHJlbWFpbgooZGExOm1wczA6MDo2NTowKTogUkVB RCgxNikuIENEQjogODggMDAgMDAgMDAgMDAgMDMgYTMgODEgMjggMTAgMDAgMDAgMDAgMTAgMDAg MDAgCihkYTE6bXBzMDowOjY1OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBFcnJvcgooZGEx Om1wczA6MDo2NTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGExOm1wczA6MDo2 NTowKTogU0NTSSBzZW5zZTogVU5JVCBBVFRFTlRJT04gYXNjOjI5LDAgKFBvd2VyIG9uLCByZXNl dCwgb3IgYnVzIGRldmljZSByZXNldCBvY2N1cnJlZCkKKGRhMTptcHMwOjA6NjU6MCk6IFJldHJ5 aW5nIGNvbW1hbmQgKHBlciBzZW5zZSBkYXRhKQpTR0wgZm9yIGNvbW1hbmQgMHhmZmZmZmUwMTdh YzgwMTM4CjAwMDAgICAwYSAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAyYyBkZCA0 NSAwMyAgfC4uLi4uLi4uLi4uLiwuRS58CjAwMTAgICAwMCAwMCBmYyAwMCAxOCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCA4MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMjAgICAwMCAwMCAw MCAwMCAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4u Li58CjAwMzAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MSAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNDAgICA4YSAwMCAwMCAwMCAwMCAwMSAwYSBmZCAxNSBm OCAwMCAwMCAwMCA0MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi5ALi58CjAwNTAgICAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58 CjAwNjAgICAwMCAxMCAwMCAxNiAwMCAxMCAyMCAwNyAyMiAwMCAwMCAwMCAwMCAxMCAwMCA5NiAg fC4uLi4uLiAuIi4uLi4uLi58CjAwNzAgICAwMCAxMCA5YiAxNCAyMiAwMCAwMCAwMCA4MCAwMCAw MCAzMCA4MCBmNSAyYyAwMyAgfC4uLi4iLi4uLi4uMC4uLC58CnNlZzAgZmxhZ3M9MHgxNiBsZW49 MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyMDcyMDEwMDAKc2VnMSBmbGFncz0weDk2IGxlbj0weDAw MTAwMCBhZGRyPTB4MDAwMDAwMjIxNDliMTAwMApjaGFpbiBmbGFncz0weDMwIGxlbj0weDgwIE9m ZnNldD0weDAgQWRkcmVzcz0weDMyY2Y1ODAKMDAwMCAgIDAwIDEwIDAwIDE2IDAwIDQwIGIzIGNj IDM5IDAwIDAwIDAwIDAwIDEwIDAwIDE2ICB8Li4uLi5ALi45Li4uLi4uLnwKMDAxMCAgIDAwIDEw IDAwIGE1IDIyIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIGIwIDI5IDNmICB8Li4uLiIuLi4uLi4u Li4pP3wKMDAyMCAgIDIxIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIDgwIDBjIGUyIDIxIDAwIDAw IDAwICB8IS4uLi4uLi4uLi4uIS4uLnwKMDAzMCAgIDAwIDEwIDAwIDE2IDAwIDMwIDQ4IDA1IDIx IDAwIDAwIDAwIDAwIDEwIDAwIGQ3ICB8Li4uLi4wSC4hLi4uLi4uLnwKMDA0MCAgIDAwIDkwIGQw IDYxIDIxIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uYSEuLi4uLi4uLi4u LnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw ICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA3MCAgIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwK c2VnMiBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMzljY2IzNDAwMApzZWcz IGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmE1MDAxMDAwCnNlZzQgZmxh Z3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIxM2YyOWIwMDAKc2VnNSBmbGFncz0w eDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjFlMjBjODAwMApzZWc2IGZsYWdzPTB4MTYg bGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMTA1NDgzMDAwCnNlZzcgZmxhZ3M9MHhkNyBsZW49 MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIxNjFkMDkwMDAKU0dMIGZvciBjb21tYW5kIDB4ZmZmZmZl MDE3YWNhNjhhOAowMDAwICAgMGEgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDQg YTAgNDcgMDMgIHwuLi4uLi4uLi4uLi4uLihkYTE6bXBzMDowOjY1OjApOiBXUklURSgxNikuIENE QjogOGEgMDAgMDAgMDAgMDAgMDEgMGEgZmQgMTUgZjggMDAgMDAgMDAgNDAgMDAgMDAgCkcoZGEx Om1wczA6MDo2NTowKTogQ0FNIHN0YXR1czogVW5yZWNvdmVyYWJsZSBIb3N0IEJ1cyBBZGFwdGVy IEVycm9yCihkYTE6bXBzMDowOjY1OjApOiBSZXRyeWluZyBjb21tYW5kLCAwIG1vcmUgdHJpZXMg cmVtYWluCi58CjAwMTAgICAwMCAwMCBmYyAwMCAxOCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAy MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4gLi58CjAwMjAgICAwMCAwMCAwMCAwMCAwNiAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMzAgICAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMiAgfC4uLi4uLi4uLi4u Li4uLi58CjAwNDAgICAwOCAwMCAwYSAxMCAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNTAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNjAgICAwMCAyMCAw MCBkMyAwMCBhMCA5NiAwNiAxMiAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4gLi4uLi4uLi4uLi4u Li58CjAwNzAgICAwMCAwMCAwMCBlOCAzZCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAgfC4uLi49Li4uLi4uLi4uLi58CnNlZzAgZmxhZ3M9MHhkMyBsZW49MHgwMDIwMDAgYWRkcj0w eDAwMDAwMDEyMDY5NmEwMDAKU0dMIGZvciBjb21tYW5kIDB4ZmZmZmZlMDE3YWM3M2JiMAowMDAw ICAgMGEgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgNzggNGMgNDUgMDMgIHwuLi4u Li4uLi4uLi54TEUufAowMDEwICAgMDAgMDAgZmMgMDAgMTggMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMjAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uIC4ufAowMDIwICAgMDAgMDAgMDAgMDAgMTAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDMwICAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDIgIHwuLi4uLi4u Li4uLi4uLi4ufAowMDQwICAgODggMDAgMDAgMDAgMDAgMDMgYTMgODEgMjYgMTAgMDAgMDAgMDAg MTAgMDAgMDAgIHwuLi4uLi4uLiYuLi4uLi4ufAowMDUwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDYwICAgMDAg MjAgMDAgZDMgMDAgOTAgNTkgMDkgMDYgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuIC4uLi5ZLi4u Li4uLi4ufAowMDcwICAgMDAgMDAgMDAgYWMgMTAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufApzZWcwIGZsYWdzPTB4ZDMgbGVuPTB4MDAyMDAwIGFk ZHI9MHgwMDAwMDAwNjA5NTk5MDAwClNHTCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjN2NlMjgK MDAwMCAgIDBhIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIGM0IGI3IDQ1IDAzICB8 Li4uLi4uLi4uLi4uLi5FLnwKMDAxMCAgIDAwIDAwIGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDIwIDAwIDAwICB8Li4uLi4uLi4uLi4uLiAuLnwKMDAyMCAgIDAwIDAwIDAwIDAwIDEw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAz MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAyICB8Li4u Li4uLi4uLi4uLi4uLnwKMDA0MCAgIDg4IDAwIDAwIDAwIDAwIDAzIGEzIDgxIDI4IDEwIDAwIDAw IDAwIDEwIDAwIDAwICB8Li4uLi4uLi4oLi4uLi4uLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAg IDAwIDIwIDAwIGQzIDAwIGMwIDdmIGI4IDAzIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8LiAuLi4u Li4uLi4uLi4uLnwKMDA3MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKc2VnMCBmbGFncz0weGQzIGxlbj0weDAwMjAw MCBhZGRyPTB4MDAwMDAwMDNiODdmYzAwMAooZGExOm1wczA6MDo2NTowKTogUkVBRCg2KS4gQ0RC OiAwOCAwMCAwYSAxMCAxMCAwMCAKKGRhMTptcHMwOjA6NjU6MCk6IENBTSBzdGF0dXM6IFVucmVj b3ZlcmFibGUgSG9zdCBCdXMgQWRhcHRlciBFcnJvcgooZGExOm1wczA6MDo2NTowKTogUmV0cnlp bmcgY29tbWFuZCwgMyBtb3JlIHRyaWVzIHJlbWFpbgooZGExOm1wczA6MDo2NTowKTogUkVBRCgx NikuIENEQjogODggMDAgMDAgMDAgMDAgMDMgYTMgODEgMjYgMTAgMDAgMDAgMDAgMTAgMDAgMDAg CihkYTE6bXBzMDowOjY1OjApOiBDQU0gc3RhdHVzOiBVbnJlY292ZXJhYmxlIEhvc3QgQnVzIEFk YXB0ZXIgRXJyb3IKKGRhMTptcHMwOjA6NjU6MCk6IFJldHJ5aW5nIGNvbW1hbmQsIDMgbW9yZSB0 cmllcyByZW1haW4KKGRhMTptcHMwOjA6NjU6MCk6IFJFQUQoMTYpLiBDREI6IDg4IDAwIDAwIDAw IDAwIDAzIGEzIDgxIDI4IDEwIDAwIDAwIDAwIDEwIDAwIDAwIAooZGExOm1wczA6MDo2NTowKTog Q0FNIHN0YXR1czogVW5yZWNvdmVyYWJsZSBIb3N0IEJ1cyBBZGFwdGVyIEVycm9yCihkYTE6bXBz MDowOjY1OjApOiBSZXRyeWluZyBjb21tYW5kLCAyIG1vcmUgdHJpZXMgcmVtYWluCihkYTE6bXBz MDowOjY1OjApOiBXUklURSgxNikuIENEQjogOGEgMDAgMDAgMDAgMDAgMDEgMGEgZmQgMTUgZjgg MDAgMDAgMDAgNDAgMDAgMDAgCihkYTE6bXBzMDowOjY1OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0 YXR1cyBFcnJvcgooZGExOm1wczA6MDo2NTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlv bgooZGExOm1wczA6MDo2NTowKTogU0NTSSBzZW5zZTogVU5JVCBBVFRFTlRJT04gYXNjOjI5LDAg KFBvd2VyIG9uLCByZXNldCwgb3IgYnVzIGRldmljZSByZXNldCBvY2N1cnJlZCkKKGRhMTptcHMw OjA6NjU6MCk6IEVycm9yIDYsIFJldHJpZXMgZXhoYXVzdGVkCihkYTE6bXBzMDowOjY1OjApOiBJ bnZhbGlkYXRpbmcgcGFjawpudm1lMTogV1JJVEUgc3FpZDoxNiBjaWQ6NzcgbnNpZDoxIGxiYToy MTgwNDA4MDcyIGxlbjo2NApudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxNiBjaWQ6 NzcgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgw NCwgcGFnZSAweDAxKQpTR0wgZm9yIGNvbW1hbmQgMHhmZmZmZmUwMTdhYzRkOWEwCjAwMDAgICAw YyAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCA5MCA4ZCA0MyAwMyAgfC4uLi4uLi4u Li4uLi4uQy58CjAwMTAgICAwMCAwMCBmYyAwMCAxOCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAx MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMjAgICAwMCAwMCAwMCAwMCAwYSAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMzAgICAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMSAgfC4uLi4uLi4uLi4u Li4uLi58CjAwNDAgICAyYSAwMCAxYyAyNiA1ZCAzMCAwMCAwMCAwOCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAgfCouLiZdMC4uLi4uLi4uLi58CjAwNTAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNjAgICAwMCAxMCAw MCBkNyAwMCAwMCBkNCA1ZCAwOCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi5dLi4uLi4u Li58CjAwNzAgICAwMCAwMCAwMCA0NiAzZSAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAgfC4uLkY+Li4uLi4uLi4uLi58CnNlZzAgZmxhZ3M9MHhkNyBsZW49MHgwMDEwMDAgYWRkcj0w eDAwMDAwMDA4NWRkNDAwMDAKU0dMIGZvciBjb21tYW5kIDB4ZmZmZmZlMDE3YWMyOWQzMAowMDAw ICAgMGMgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMzggZWEgNDEgMDMgIHwuLi4u Li4uLi4uLi44LkEufAowMDEwICAgMDAgMDAgZmMgMDAgMTggMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgODAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDIwICAgMDAgMDAgMDAgMDAgMTAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLihkYTM6bXBzMDowOjY3OjApOiBX UklURSgxMCkuIENEQjogMmEgMDAgMWMgMjYgNWQgMzAgMDAgMDAgMDggMDAgCi4oZGEzOm1wczA6 MDo2NzowKTogQ0FNIHN0YXR1czogVW5yZWNvdmVyYWJsZSBIb3N0IEJ1cyBBZGFwdGVyIEVycm9y CihkYTM6bXBzMDowOjY3OjApOiBSZXRyeWluZyBjb21tYW5kLCAzIG1vcmUgdHJpZXMgcmVtYWlu Ci4uLi4uLi4uLi58CjAwMzAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMSAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNDAgICA4YSAwMCAwMCAwMCAwMCAwMSAw YSBmZCAxYSBjOCAwMCAwMCAwMCA0MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi5ALi58CjAwNTAgICAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4u Li4uLi4uLi58CjAwNjAgICAwMCAxMCAwMCAxNiAwMCAxMCA2MCAwNiAyMiAwMCAwMCAwMCAwMCAx MCAwMCA5NiAgfC4uLi4uLmAuIi4uLi4uLi58CjAwNzAgICAwMCBjMCA1YSBkNyAyMiAwMCAwMCAw MCA4MCAwMCAwMCAzMCA4MCBmYSAyYyAwMyAgfC4uWi4iLi4uLi4uMC4uLC58CnNlZzAgZmxhZ3M9 MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyMDY2MDEwMDAKc2VnMSBmbGFncz0weDk2 IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjJkNzVhYzAwMApjaGFpbiBmbGFncz0weDMwIGxl bj0weDgwIE9mZnNldD0weDAgQWRkcmVzcz0weDMyY2ZhODAKMDAwMCAgIDAwIDEwIDAwIDE2IDAw IDEwIDIwIDA4IDIyIDAwIDAwIDAwIDAwIDEwIDAwIDE2ICB8Li4uLi4uIC4iLi4uLi4uLnwKMDAx MCAgIDAwIDQwIDZiIGQ3IDIyIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIGQwIDc0IDZlICB8LkBr LiIuLi4uLi4uLi50bnwKMDAyMCAgIDIxIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIDEwIGMwIDBh IDIyIDAwIDAwIDAwICB8IS4uLi4uLi4uLi4uIi4uLnwKMDAzMCAgIDAwIDEwIDAwIDE2IDAwIDEw IDYwIDYwIDIyIDAwIDAwIDAwIDAwIDEwIDAwIGQ3ICB8Li4uLi4uYGAiLi4uLi4uLnwKMDA0MCAg IDAwIGMwIDE4IGMyIDIyIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLiIu Li4uLi4uLi4uLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA3MCAgIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4u Li4uLi4uLnwKc2VnMiBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjIwODIw MTAwMApzZWczIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmQ3NmI0MDAw CnNlZzQgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIxNmU3NGQwMDAKc2Vn NSBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjIwYWMwMTAwMApzZWc2IGZs YWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMjYwNjAxMDAwCnNlZzcgZmxhZ3M9 MHhkNyBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyYzIxOGMwMDAKKGRhMzptcHMwOjA6Njc6 MCk6IFdSSVRFKDE2KS4gQ0RCOiA4YSAwMCAwMCAwMCAwMCAwMSAwYSBmZCAxYSBjOCAwMCAwMCAw MCA0MCAwMCAwMCAKKGRhMzptcHMwOjA6Njc6MCk6IENBTSBzdGF0dXM6IFVucmVjb3ZlcmFibGUg SG9zdCBCdXMgQWRhcHRlciBFcnJvcgooZGEzOm1wczA6MDo2NzowKTogUmV0cnlpbmcgY29tbWFu ZCwgMyBtb3JlIHRyaWVzIHJlbWFpbgooZGEzOm1wczA6MDo2NzowKTogV1JJVEUoMTYpLiBDREI6 IDhhIDAwIDAwIDAwIDAwIDAxIDBhIGZkIDFhIGM4IDAwIDAwIDAwIDQwIDAwIDAwIAooZGEzOm1w czA6MDo2NzowKTogQ0FNIHN0YXR1czogU0NTSSBTdGF0dXMgRXJyb3IKKGRhMzptcHMwOjA6Njc6 MCk6IFNDU0kgc3RhdHVzOiBDaGVjayBDb25kaXRpb24KKGRhMzptcHMwOjA6Njc6MCk6IFNDU0kg c2Vuc2U6IFVOSVQgQVRURU5USU9OIGFzYzoyOSwwIChQb3dlciBvbiwgcmVzZXQsIG9yIGJ1cyBk ZXZpY2UgcmVzZXQgb2NjdXJyZWQpCihkYTM6bXBzMDowOjY3OjApOiBSZXRyeWluZyBjb21tYW5k IChwZXIgc2Vuc2UgZGF0YSkKU0dMIGZvciBjb21tYW5kIDB4ZmZmZmZlMDE3YWM0OTExMAowMDAw ICAgMGMgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgNjggNTggNDMgMDMgIHwuLi4u Li4uLi4uLi5oWEMufAowMDEwICAgMDAgMDAgZmMgMDAgMTggMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDIwICAgMDAgMDAgMDAgMDAgMGEgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDMwICAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4u Li4uLi4uLi4ufAowMDQwICAgMzUgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgIHw1Li4uLi4uLi4uLi4uLi4ufAowMDUwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDYwICAgMDAg MDAgMDAgZDcgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4u Li4uLi4ufAowMDcwICAgMDAgMDAgYzAgZDIgM2MgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgIHwuLi4uPC4uLi4uLi4uLi4ufApzZWcwIGZsYWdzPTB4ZDcgbGVuPTB4MDAwMDAwIGFk ZHI9MHgwMDAwMDAwMDAwMDAwMDAwClNHTCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjOWQ3ODgK MDAwMCAgIDBjIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIGI0IDM1IDQ3IDAzICB8 Li4uLi4uLi4uLi4uLjVHLnwKMDAxMCAgIDAwIDAwIGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDgwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAyMCAgIDAwIDAwIDAwIDAwIDEw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLihkYTM6bXBzMDowOjY3 OjApOiBTWU5DSFJPTklaRSBDQUNIRSgxMCkuIENEQjogMzUgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgCi4oZGEzOm1wczA6MDo2NzowKTogQ0FNIHN0YXR1czogVW5yZWNvdmVyYWJsZSBIb3N0 IEJ1cyBBZGFwdGVyIEVycm9yCihkYTM6bXBzMDowOjY3OjApOiBSZXRyeWluZyBjb21tYW5kLCAw IG1vcmUgdHJpZXMgcmVtYWluCi4uLi4uLi4ufAowMDMwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDEgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDQwICAgOGEg MDAgMDAgMDAgMDAgMDEgMGEgZmQgMWEgYzggMDAgMDAgMDAgNDAgMDAgMDAgIHwuLi4uLi4uLi4u Li4uQC4ufAowMDUwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDYwICAgMDAgMTAgMDAgMTYgMDAgMTAgNjAgMDYg MjIgMDAgMDAgMDAgMDAgMTAgMDAgOTYgIHwuLi4uLi5gLiIuLi4uLi4ufAowMDcwICAgMDAgYzAg NWEgZDcgMjIgMDAgMDAgMDAgODAgMDAgMDAgMzAgODAgZmIgMmMgMDMgIHwuLlouIi4uLi4uLjAu LiwufApzZWcwIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMjA2NjAxMDAw CnNlZzEgZmxhZ3M9MHg5NiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyZDc1YWMwMDAKY2hh aW4gZmxhZ3M9MHgzMCBsZW49MHg4MCBPZmZzZXQ9MHgwIEFkZHJlc3M9MHgzMmNmYjgwCjAwMDAg ICAwMCAxMCAwMCAxNiAwMCAxMCAyMCAwOCAyMiAwMCAwMCAwMCAwMCAxMCAwMCAxNiAgfC4uLi4u LiAuIi4uLi4uLi58CjAwMTAgICAwMCA0MCA2YiBkNyAyMiAwMCAwMCAwMCAwMCAxMCAwMCAxNiAw MCBkMCA3NCA2ZSAgfC5Aay4iLi4uLi4uLi4udG58CjAwMjAgICAyMSAwMCAwMCAwMCAwMCAxMCAw MCAxNiAwMCAxMCBjMCAwYSAyMiAwMCAwMCAwMCAgfCEuLi4uLi4uLi4uLiIuLi58CjAwMzAgICAw MCAxMCAwMCAxNiAwMCAxMCA2MCA2MCAyMiAwMCAwMCAwMCAwMCAxMCAwMCBkNyAgfC4uLi4uLmBg Ii4uLi4uLi58CjAwNDAgICAwMCBjMCAxOCBjMiAyMiAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAgfC4uLi4iLi4uLi4uLi4uLi58CjAwNTAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNjAgICAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4u Li4uLi58CjAwNzAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CnNlZzIgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRk cj0weDAwMDAwMDIyMDgyMDEwMDAKc2VnMyBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4 MDAwMDAwMjJkNzZiNDAwMApzZWc0IGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAw MDAyMTZlNzRkMDAwCnNlZzUgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIy MGFjMDEwMDAKc2VnNiBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjI2MDYw MTAwMApzZWc3IGZsYWdzPTB4ZDcgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmMyMThjMDAw CihkYTM6bXBzMDowOjY3OjApOiBXUklURSgxNikuIENEQjogOGEgMDAgMDAgMDAgMDAgMDEgMGEg ZmQgMWEgYzggMDAgMDAgMDAgNDAgMDAgMDAgCihkYTM6bXBzMDowOjY3OjApOiBDQU0gc3RhdHVz OiBVbnJlY292ZXJhYmxlIEhvc3QgQnVzIEFkYXB0ZXIgRXJyb3IKKGRhMzptcHMwOjA6Njc6MCk6 IFJldHJ5aW5nIGNvbW1hbmQsIDEgbW9yZSB0cmllcyByZW1haW4KKGRhMzptcHMwOjA6Njc6MCk6 IFNZTkNIUk9OSVpFIENBQ0hFKDEwKS4gQ0RCOiAzNSAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAKKGRhMzptcHMwOjA6Njc6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTM6 bXBzMDowOjY3OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTM6bXBzMDowOjY3 OjApOiBTQ1NJIHNlbnNlOiBVTklUIEFUVEVOVElPTiBhc2M6MjksMCAoUG93ZXIgb24sIHJlc2V0 LCBvciBidXMgZGV2aWNlIHJlc2V0IG9jY3VycmVkKQooZGEzOm1wczA6MDo2NzowKTogRXJyb3Ig NiwgUmV0cmllcyBleGhhdXN0ZWQKKGRhMzptcHMwOjA6Njc6MCk6IEludmFsaWRhdGluZyBwYWNr ClNHTCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjMjlmZTAKMDAwMCAgIDBjIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDMwIGVjIDQxIDAzICB8Li4uLi4uLi4uLi4uMC5BLnwKMDAx MCAgIDAwIDAwIGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDgwIDAwIDAwICB8Li4u Li4uLi4uLi4uLi4uLnwKMDAyMCAgIDAwIDAwIDAwIDAwIDEwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAzMCAgIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAxICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA0MCAg IDhhIDAwIDAwIDAwIDAwIDAxIDBhIGZkIDFhIGM4IDAwIDAwIDAwIDQwIDAwIDAwICB8Li4uLi4u Li4uLi4uLkAuLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDEwIDAwIDE2IDAwIDEwIDYw IDA2IDIyIDAwIDAwIDAwIDAwIDEwIDAwIDk2ICB8Li4uLi4uYC4iLi4uLi4uLnwKMDA3MCAgIDAw IGMwIDVhIGQ3IDIyIDAwIDAwIDAwIDgwIDAwIDAwIDMwIDAwIGZjIDJjIDAzICB8Li5aLiIuLi4u Li4wLi4sLnwKc2VnMCBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjIwNjYw MTAwMApzZWcxIGZsYWdzPTB4OTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmQ3NWFjMDAw CmNoYWluIGZsYWdzPTB4MzAgbGVuPTB4ODAgT2Zmc2V0PTB4MCBBZGRyZXNzPTB4MzJjZmMwMAow MDAwICAgMDAgMTAgMDAgMTYgMDAgMTAgMjAgMDggMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYgIHwu Li4uLi4gLiIuLi4uLi4ufAowMDEwICAgMDAgNDAgNmIgZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAg MTYgMDAgZDAgNzQgNmUgIHwuQGsuIi4uLi4uLi4uLnRufAowMDIwICAgMjEgMDAgMDAgMDAgMDAg MTAgMDAgMTYgMDAgMTAgYzAgMGEgMjIgMDAgMDAgMDAgIHwhLi4uLi4uLi4uLi4iLi4ufAowMDMw ICAgMDAgMTAgMDAgMTYgMDAgMTAgNjAgNjAgMjIgMDAgMDAgMDAgMDAgMTAgMDAgZDcgIHwuLi4u Li5gYCIuLi4uLi4ufAowMDQwICAgMDAgYzAgMTggYzIgMjIgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgIHwuLi4uIi4uLi4uLi4uLi4ufAowMDUwICAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDYwICAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4u Li4uLi4uLi4ufAowMDcwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufApzZWcyIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAw IGFkZHI9MHgwMDAwMDAyMjA4MjAxMDAwCnNlZzMgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRk cj0weDAwMDAwMDIyZDc2YjQwMDAKc2VnNCBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4 MDAwMDAwMjE2ZTc0ZDAwMApzZWc1IGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAw MDAyMjBhYzAxMDAwCnNlZzYgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIy NjA2MDEwMDAKc2VnNyBmbGFncz0weGQ3IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjJjMjE4 YzAwMAooZGEzOm1wczA6MDo2NzowKTogV1JJVEUoMTYpLiBDREI6IDhhIDAwIDAwIDAwIDAwIDAx IDBhIGZkIDFhIGM4IDAwIDAwIDAwIDQwIDAwIDAwIAooZGEzOm1wczA6MDo2NzowKTogQ0FNIHN0 YXR1czogVW5yZWNvdmVyYWJsZSBIb3N0IEJ1cyBBZGFwdGVyIEVycm9yCihkYTM6bXBzMDowOjY3 OjApOiBSZXRyeWluZyBjb21tYW5kLCAwIG1vcmUgdHJpZXMgcmVtYWluCihkYTM6bXBzMDowOjY3 OjApOiBXUklURSgxNikuIENEQjogOGEgMDAgMDAgMDAgMDAgMDEgMGEgZmQgMWEgYzggMDAgMDAg MDAgNDAgMDAgMDAgCihkYTM6bXBzMDowOjY3OjApOiBDQU0gc3RhdHVzOiBTQ1NJIFN0YXR1cyBF cnJvcgooZGEzOm1wczA6MDo2NzowKTogU0NTSSBzdGF0dXM6IENoZWNrIENvbmRpdGlvbgooZGEz Om1wczA6MDo2NzowKTogU0NTSSBzZW5zZTogVU5JVCBBVFRFTlRJT04gYXNjOjI5LDAgKFBvd2Vy IG9uLCByZXNldCwgb3IgYnVzIGRldmljZSByZXNldCBvY2N1cnJlZCkKKGRhMzptcHMwOjA6Njc6 MCk6IEVycm9yIDYsIFJldHJpZXMgZXhoYXVzdGVkCm52bWUwOiBXUklURSBzcWlkOjQgY2lkOjkx IG5zaWQ6MSBsYmE6OTA0MTM4NzYwIGxlbjo2NApudm1lMDogV1JJVEUgRkFVTFRTICgwMi84MCkg c3FpZDo0IGNpZDo5MSBjZHcwOjAKbnZtZTA6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4 MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUxOiBXUklURSBzcWlkOjUgY2lkOjkzIG5zaWQ6 MSBsYmE6MjE4MDczMTc2OCBsZW46NjQKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6 NSBjaWQ6OTMgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGlu Zm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMTogV1JJVEUgc3FpZDoxNiBjaWQ6MTA3IG5zaWQ6MSBs YmE6NjUwODgxNTM2IGxlbjo4Cm52bWUxOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE2IGNp ZDoxMDcgY2R3MDowCm52bWUxOiBXUklURSBzcWlkOjE2IGNpZDo4OSBuc2lkOjEgbGJhOjY1MDg4 MTQ4MCBsZW46NTYKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTYgY2lkOjg5IGNk dzA6MApudm1lMTogV1JJVEUgc3FpZDoxMiBjaWQ6ODUgbnNpZDoxIGxiYToyMTgwNzk5Nzg0IGxl bjo4MApudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMiBjaWQ6ODUgY2R3MDowCm52 bWUwOiBXUklURSBzcWlkOjEyIGNpZDoxMjAgbnNpZDoxIGxiYToyMTgwNzk5NzkyIGxlbjo3Mgpu dm1lMDogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMiBjaWQ6MTIwIGNkdzA6MApudm1lMDog V1JJVEUgc3FpZDoxMiBjaWQ6MTIwIG5zaWQ6MSBsYmE6MjE4MDgwMDkxMiBsZW46ODAKbnZtZTA6 IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTIgY2lkOjEyMCBjZHcwOjAKbnZtZTE6IGFzeW5j IGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUwOiBh c3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1l MTogV1JJVEUgc3FpZDoxMCBjaWQ6OTEgbnNpZDoxIGxiYToyMTgwODAxNzg0IGxlbjo4Cm52bWUx OiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjEwIGNpZDo5MSBjZHcwOjAKbnZtZTE6IFdSSVRF IHNxaWQ6MTAgY2lkOjExMyBuc2lkOjEgbGJhOjIxODA4MDE3OTIgbGVuOjcyCm52bWUxOiBXUklU RSBGQVVMVFMgKDAyLzgwKSBzcWlkOjEwIGNpZDoxMTMgY2R3MDowCm52bWUwOiBXUklURSBzcWlk OjEwIGNpZDo4MiBuc2lkOjEgbGJhOjIxODA4MDM1MDQgbGVuOjQwCm52bWUwOiBXUklURSBGQVVM VFMgKDAyLzgwKSBzcWlkOjEwIGNpZDo4MiBjZHcwOjAKbnZtZTI6IFdSSVRFIHNxaWQ6MTAgY2lk OjExOCBuc2lkOjEgbGJhOjIxODA4MDM1MDQgbGVuOjMyCm52bWUyOiBXUklURSBGQVVMVFMgKDAy LzgwKSBzcWlkOjEwIGNpZDoxMTggY2R3MDowCm52bWUwOiBXUklURSBzcWlkOjEyIGNpZDoxMTQg bnNpZDoxIGxiYToyMTgwODAzNjQ4IGxlbjo2NApudm1lMDogV1JJVEUgRkFVTFRTICgwMi84MCkg c3FpZDoxMiBjaWQ6MTE0IGNkdzA6MApudm1lMTogV1JJVEUgc3FpZDoxMiBjaWQ6ODUgbnNpZDox IGxiYToyMTgwODAzNjQwIGxlbjo3Mgpudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDox MiBjaWQ6ODUgY2R3MDowCm52bWUyOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGlu Zm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMTogV1JJVEUgc3FpZDo2IGNpZDo3OSBuc2lkOjEgbGJh Ojg5OTA5NzI4OCBsZW46NTYKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6NiBjaWQ6 NzkgY2R3MDowCm52bWUxOiBXUklURSBzcWlkOjYgY2lkOjExOCBuc2lkOjEgbGJhOjg5OTA5NzM0 NCBsZW46MjQKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6NiBjaWQ6MTE4IGNkdzA6 MApudm1lMTogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2Ug MHgwMSkKbnZtZTA6IFdSSVRFIHNxaWQ6MTIgY2lkOjExNCBuc2lkOjEgbGJhOjkwNDE2MjM2OCBs ZW46MTQ0Cm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjEyIGNpZDoxMTQgY2R3MDow Cm52bWUwOiBXUklURSBzcWlkOjggY2lkOjY3IG5zaWQ6MSBsYmE6NjI3Mzc4NDMyIGxlbjozMgpu dm1lMDogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo4IGNpZDo2NyBjZHcwOjAKbnZtZTI6IFdS SVRFIHNxaWQ6OCBjaWQ6ODYgbnNpZDoxIGxiYTo4MjgyNzQ2ODggbGVuOjEyOApudm1lMjogV1JJ VEUgRkFVTFRTICgwMi84MCkgc3FpZDo4IGNpZDo4NiBjZHcwOjAKbnZtZTA6IFdSSVRFIHNxaWQ6 OCBjaWQ6OTkgbnNpZDoxIGxiYTo2MjczNzgzMDQgbGVuOjEyOApudm1lMDogV1JJVEUgRkFVTFRT ICgwMi84MCkgc3FpZDo4IGNpZDo5OSBjZHcwOjAKbnZtZTA6IGFzeW5jIGV2ZW50IG9jY3VycmVk ICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUyOiBhc3luYyBldmVudCBvY2N1 cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQooYWRhMTphaGNpY2gxOjA6MDow KTogV1JJVEVfRlBETUFfUVVFVUVELiBBQ0I6IDYxIGEwIGQwIGUxIDEyIDQwIDA4IDAwIDAwIDAw IDAwIDAwCihhZGExOmFoY2ljaDE6MDowOjApOiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMg aW52YWxpZAooYWRhMTphaGNpY2gxOjA6MDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y Cm52bWUyOiBXUklURSBzcWlkOjYgY2lkOjc4IG5zaWQ6MSBsYmE6ODUxODE1NzQ0IGxlbjo0MApu dm1lMjogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo2IGNpZDo3OCBjZHcwOjAKbnZtZTE6IFdS SVRFIHNxaWQ6NiBjaWQ6NzkgbnNpZDoxIGxiYTo4NTE4MTU3NTIgbGVuOjMyCm52bWUxOiBXUklU RSBGQVVMVFMgKDAyLzgwKSBzcWlkOjYgY2lkOjc5IGNkdzA6MApudm1lMjogV1JJVEUgc3FpZDox NCBjaWQ6OTcgbnNpZDoxIGxiYTo4NTE4MTU3NDQgbGVuOjQwCm52bWUyOiBXUklURSBGQVVMVFMg KDAyLzgwKSBzcWlkOjE0IGNpZDo5NyBjZHcwOjAKbnZtZTE6IFdSSVRFIHNxaWQ6MTQgY2lkOjU2 IG5zaWQ6MSBsYmE6ODUxODE1NzUyIGxlbjozMgpudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkg c3FpZDoxNCBjaWQ6NTYgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAw eDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMjogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5 cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IFdSSVRFIHNxaWQ6OSBjaWQ6MTAw IG5zaWQ6MSBsYmE6MjE4MDgwNzQyNCBsZW46NDAKbnZtZTA6IFdSSVRFIEZBVUxUUyAoMDIvODAp IHNxaWQ6OSBjaWQ6MTAwIGNkdzA6MApudm1lMDogV1JJVEUgc3FpZDo5IGNpZDo3NiBuc2lkOjEg bGJhOjIxODA4MDczOTIgbGVuOjMyCm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjkg Y2lkOjc2IGNkdzA6MApudm1lMTogV1JJVEUgc3FpZDo5IGNpZDo4NyBuc2lkOjEgbGJhOjkwNDE2 NzgxNiBsZW46ODAKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6OSBjaWQ6ODcgY2R3 MDowCm52bWUyOiBXUklURSBzcWlkOjkgY2lkOjgxIG5zaWQ6MSBsYmE6NjA1MjY3MjMyIGxlbjox NDQKbnZtZTI6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6OSBjaWQ6ODEgY2R3MDowCm52bWUw OiBXUklURSBzcWlkOjExIGNpZDo3NSBuc2lkOjEgbGJhOjkwNDE2ODUyOCBsZW46ODAKbnZtZTA6 IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTEgY2lkOjc1IGNkdzA6MApudm1lMTogYXN5bmMg ZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTI6IGFz eW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUw OiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpu dm1lMjogV1JJVEUgc3FpZDoxMCBjaWQ6MTA3IG5zaWQ6MSBsYmE6OTAzODg1NTkyIGxlbjo4MApu dm1lMjogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMCBjaWQ6MTA3IGNkdzA6MApudm1lMjog V1JJVEUgc3FpZDoxMiBjaWQ6OTUgbnNpZDoxIGxiYToyMjM3MDM2MzM2IGxlbjo4MApudm1lMjog V1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMiBjaWQ6OTUgY2R3MDowCihhZGEzOmFoY2ljaDM6 MDowOjApOiBXUklURV9GUERNQV9RVUVVRUQuIEFDQjogNjEgYTAgNDAgOGIgNzIgNDAgMDYgMDAg MDAgMDAgMDAgMDAKKGFkYTM6YWhjaWNoMzowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0 IHdhcyBpbnZhbGlkCihhZGEzOmFoY2ljaDM6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUg ZXJyb3IKbnZtZTI6IFdSSVRFIHNxaWQ6MTYgY2lkOjExNiBuc2lkOjEgbGJhOjIxODA4MDgzNzYg bGVuOjQwCm52bWUyOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE2IGNpZDoxMTYgY2R3MDow Cm52bWUwOiBXUklURSBzcWlkOjE2IGNpZDo3NSBuc2lkOjEgbGJhOjIxODA4MDgzODQgbGVuOjMy Cm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE2IGNpZDo3NSBjZHcwOjAKbnZtZTA6 IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52 bWUyOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAx KQpudm1lMDogV1JJVEUgc3FpZDoxNiBjaWQ6NzUgbnNpZDoxIGxiYTo2MDQxMDcwNDAgbGVuOjgw Cm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE2IGNpZDo3NSBjZHcwOjAKbnZtZTA6 IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCihh ZGEzOmFoY2ljaDM6MDowOjApOiBXUklURV9GUERNQV9RVUVVRUQuIEFDQjogNjEgYTAgYjAgM2Ug ZGEgNDAgMDQgMDAgMDAgMDAgMDAgMDAKKGFkYTM6YWhjaWNoMzowOjA6MCk6IENBTSBzdGF0dXM6 IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlkCihhZGEzOmFoY2ljaDM6MDowOjApOiBFcnJvciAyMiwg VW5yZXRyeWFibGUgZXJyb3IKbnZtZTA6IFdSSVRFIHNxaWQ6NCBjaWQ6OTEgbnNpZDoxIGxiYToy MTgwNjg1ODE2IGxlbjo4Cm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjQgY2lkOjkx IGNkdzA6MApudm1lMDogV1JJVEUgc3FpZDo0IGNpZDo5NiBuc2lkOjEgbGJhOjIxODA2ODU4MjQg bGVuOjcyCm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjQgY2lkOjk2IGNkdzA6MApu dm1lMDogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgw MSkKbnZtZTI6IFdSSVRFIHNxaWQ6OSBjaWQ6ODEgbnNpZDoxIGxiYToyMTgwODExODY0IGxlbjo3 Mgpudm1lMjogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo5IGNpZDo4MSBjZHcwOjAKbnZtZTI6 IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52 bWUwOiBXUklURSBzcWlkOjEwIGNpZDoxMTUgbnNpZDoxIGxiYTo5MDQxNzMxMjAgbGVuOjE2MApu dm1lMDogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMCBjaWQ6MTE1IGNkdzA6MApudm1lMDog YXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKKGFk YTM6YWhjaWNoMzowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSA5MCA1MCA5MSA3 MiA0MCAwNiAwMCAwMCAwMCAwMCAwMAooYWRhMzphaGNpY2gzOjA6MDowKTogQ0FNIHN0YXR1czog Q0NCIHJlcXVlc3Qgd2FzIGludmFsaWQKKGFkYTM6YWhjaWNoMzowOjA6MCk6IEVycm9yIDIyLCBV bnJldHJ5YWJsZSBlcnJvcgpudm1lMTogV1JJVEUgc3FpZDoxNiBjaWQ6MTAwIG5zaWQ6MSBsYmE6 MjE4MDY4NjY2NCBsZW46NDAKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTYgY2lk OjEwMCBjZHcwOjAKbnZtZTA6IFdSSVRFIHNxaWQ6MTYgY2lkOjc1IG5zaWQ6MSBsYmE6MjE4MDY4 NjY2NCBsZW46NDAKbnZtZTA6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTYgY2lkOjc1IGNk dzA6MApudm1lMDogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBh Z2UgMHgwMSkKbnZtZTE6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0 LCBwYWdlIDB4MDEpClNHTCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjNDYyMDgKMDAwMCAgIDEw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIGY0IDM1IDQzIDAzICB8Li4uLi4uLi4u Li4uLjVDLnwKMDAxMCAgIDAwIDAwIGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDEw IDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAyMCAgIDAwIDAwIDAwIDAwIDBhIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAzMCAgIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAxICB8Li4uLi4uLi4uLi4u Li4uLnwKMDA0MCAgIDJhIDAwIDFjIDI2IDYwIDQ4IDAwIDAwIDA4IDAwIDAwIDAwIDAwIDAwIDAw IDAwICB8Ki4uJmBILi4uLi4uLi4uLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDEwIDAw IGQ3IDAwIGEwIDZhIGZmIDEwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uai4uLi4uLi4u LnwKMDA3MCAgIDAwIDAwIGMwIGQyIDNjIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw ICB8Li4uLjwuLi4uLi4uLi4uLnwKc2VnMCBmbGFncz0weGQ3IGxlbj0weDAwMTAwMCBhZGRyPTB4 MDAwMDAwMTBmZjZhYTAwMApTR0wgZm9yIGNvbW1hbmQgMHhmZmZmZmUwMTdhY2E3OGM4CjAwMDAg ICAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCBkNCBhYiA0NyAwMyAgfC4uLi4u Li4uLi4uLi4uRy58CjAwMTAgICAwMCAwMCBmYyAwMCAxOCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAxMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMjAgICAwMCAwMCAwMCAwMCAwYSAwMCAw MCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMzAgICAw MCAwMCAwMCAwMCAwMChkYTc6bXBzMDowOjcxOjApOiBXUklURSgxMCkuIENEQjogMmEgMDAgMWMg MjYgNjAgNDggMDAgMDAgMDggMDAgCiAwMChkYTc6bXBzMDowOjcxOjApOiBDQU0gc3RhdHVzOiBV bnJlY292ZXJhYmxlIEhvc3QgQnVzIEFkYXB0ZXIgRXJyb3IKKGRhNzptcHMwOjA6NzE6MCk6IFJl dHJ5aW5nIGNvbW1hbmQsIDMgbW9yZSB0cmllcyByZW1haW4KIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAxICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA0MCAgIDJhIDAwIDE0IGJmIDk4IDUwIDAw IDAwIDA4IDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Ki4uLi5QLi4uLi4uLi4uLnwKMDA1MCAgIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4u Li4uLi4uLnwKMDA2MCAgIDAwIDEwIDAwIGQ3IDAwIDgwIGI4IDIzIDA4IDAwIDAwIDAwIDAwIDAw IDAwIDAwICB8Li4uLi4uLiMuLi4uLi4uLnwKMDA3MCAgIDAwIDAwIGMwIGQyIDNjIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLjwuLi4uLi4uLi4uLnwKc2VnMCBmbGFncz0w eGQ3IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMDgyM2I4ODAwMApTR0wgZm9yIGNvbW1hbmQg MHhmZmZmZmUwMTdhYzc1N2U4CjAwMDAgICAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAyNCA2MSA0NSAwMyAgfC4uLi4uLi4uLi4uLiRhRS58CjAwMTAgICAwMCAwMCBmYyAwMCAx OCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCA5MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAw MjAgICAwMCAwMCAwMCAwMCAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4u Li4uLi4uLi4uLi4uLi58CjAwMzAgICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMSAgfC4uLi4uLi4uLi4uLi4uLi58CjAwNDAgICA4YSAwMCAwMCAwMCAwMCAw MSAwYSBmZCAxZiAzOCAwMCAwMCAwMCA0OCAwMCAwMCAgfC4uLi4uLi4uLjguLi5ILi58CjAwNTAg ICAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4u Li4uLi4uLi4uLi58CjAwNjAgICAwMCAxMCAwMCAxNiAwMCAxMCA0MCAzMyAyMiAwMCAwMCAwMCAw MCAxMCAwMCA5NiAgfC4uLi4uLkAzIi4uLi4uLi58CjAwNzAgICAwMCA5MCBmNSBkNyAyMiAwMCAw MCAwMCA4MCAwMCAwMCAzMCAwMCBmZCAyYyAwMyAgfC4uLi4iLi4uLi4uMC4uLC58CnNlZzAgZmxh Z3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyMzM0MDEwMDAKc2VnMSBmbGFncz0w eDk2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjJkN2Y1OTAwMApjaGFpbiBmbGFncz0weDMw IGxlbj0weDgwIE9mZnNldD0weDAgQWRkcmVzcz0weDMyY2ZkMDAKMDAwMCAgIDAwIDEwIDAwIDE2 IDAwIDkwIGY2IGQ3IDIyIDAwIDAwIDAwIDAwIDEwIDAwIDE2ICB8Li4uLi4uLi4iLi4uLi4uLnwK MDAxMCAgIDAwIGQwIGMyIDMzIDMxIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIDEwIGEwIDcxICB8 Li4uMzEuLi4uLi4uLi4ucXwKMDAyMCAgIDIyIDAwIDAwIDAwIDAwIDEwIDAwIDE2IDAwIDYwIDBk IDQ1IDMxIDAwIDAwIDAwICB8Ii4uLi4uLi4uYC5FMS4uLnwKMDAzMCAgIDAwIDEwIDAwIDE2IDAw IDMwIGZhIGQ3IDIyIDAwIDAwIDAwIDAwIDEwIDAwIDE2ICB8Li4uLi4wLi4iLi4uLi4uLnwKMDA0 MCAgIDAwIGYwIGY4IGQ3IDIyIDAwIDAwIDAwIDAwIDEwIDAwIGQ3IDAwIDMwIDg2IGQ2ICB8Li4u LiIuLi4uLi4uLjAuLnwKMDA1MCAgIDM4IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwICB8OC4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA3MCAg IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4u Li4uLi4uLi4uLnwKc2VnMiBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjJk N2Y2OTAwMApzZWczIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAzMTMzYzJk MDAwCnNlZzQgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyNzFhMDEwMDAK c2VnNSBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMzE0NTBkNjAwMApzZWc2 IGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmQ3ZmEzMDAwCnNlZzcgZmxh Z3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyZDdmOGYwMDAKc2VnOCBmbGFncz0w eGQ3IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMzhkNjg2MzAwMAooZGE3Om1wczA6MDo3MTow KTogV1JJVEUoMTApLiBDREI6IDJhIDAwIDE0IGJmIDk4IDUwIDAwIDAwIDA4IDAwIAooZGE3Om1w czA6MDo3MTowKTogQ0FNIHN0YXR1czogVW5yZWNvdmVyYWJsZSBIb3N0IEJ1cyBBZGFwdGVyIEVy cm9yCihkYTc6bXBzMDowOjcxOjApOiBSZXRyeWluZyBjb21tYW5kLCAzIG1vcmUgdHJpZXMgcmVt YWluCihkYTc6bXBzMDowOjcxOjApOiBXUklURSgxNikuIENEQjogOGEgMDAgMDAgMDAgMDAgMDEg MGEgZmQgMWYgMzggMDAgMDAgMDAgNDggMDAgMDAgCihkYTc6bXBzMDowOjcxOjApOiBDQU0gc3Rh dHVzOiBVbnJlY292ZXJhYmxlIEhvc3QgQnVzIEFkYXB0ZXIgRXJyb3IKKGRhNzptcHMwOjA6NzE6 MCk6IFJldHJ5aW5nIGNvbW1hbmQsIDMgbW9yZSB0cmllcyByZW1haW4KKGRhNzptcHMwOjA6NzE6 MCk6IFdSSVRFKDEwKS4gQ0RCOiAyYSAwMCAxYyAyNiA2MCA0OCAwMCAwMCAwOCAwMCAKKGRhNzpt cHMwOjA6NzE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTc6bXBzMDowOjcx OjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTc6bXBzMDowOjcxOjApOiBTQ1NJ IHNlbnNlOiBVTklUIEFUVEVOVElPTiBhc2M6MjksMCAoUG93ZXIgb24sIHJlc2V0LCBvciBidXMg ZGV2aWNlIHJlc2V0IG9jY3VycmVkKQooZGE3Om1wczA6MDo3MTowKTogUmV0cnlpbmcgY29tbWFu ZCAocGVyIHNlbnNlIGRhdGEpClNHTCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjNzIyMjgKMDAw MCAgIDEwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIGM0IDM5IDQ1IDAzICB8Li4u Li4uLi4uLi4uLjlFLnwKMDAxMCAgIDAwIDAwIGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDEwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAyMCAgIDAwIDAwIDAwIDAwIDBhIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAzMCAg IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAxICB8Li4uLi4u Li4uLi4uLi4uLnwKMDA0MCAgIDJhIDAwIDFjIDI2IDYwIDQ4IDAwIDAwIDA4IDAwIDAwIDAwIDAw IDAwIDAwIDAwICB8Ki4uJmBILi4uLi4uLi4uLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAw IDEwIDAwIGQ3IDAwIGEwIDZhIGZmIDEwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uai4u Li4uLi4uLnwKMDA3MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKc2VnMCBmbGFncz0weGQ3IGxlbj0weDAwMTAwMCBh ZGRyPTB4MDAwMDAwMTBmZjZhYTAwMApTR0wgZm9yIGNvbW1hbmQgMHhmZmZmZmUwMTdhYzdlOTA4 CjAwMDAgICAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCA3NCBjYiA0NSAwMyAg fC4uLi4uLi4uLi4uLnQuRS58CjAwMTAgICAwMCAwMCBmYyAwMCAxOCAwMCAwMCAwMCAwMCAwMCAw MCAwMCAwMCA5MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMjAgICAwMCAwMCAwMChkYTc6 bXBzMDowOjcxOjApOiBXUklURSgxMCkuIENEQjogMmEgMDAgMWMgMjYgNjAgNDggMDAgMDAgMDgg MDAgCiAwMChkYTc6bXBzMDowOjcxOjApOiBDQU0gc3RhdHVzOiBVbnJlY292ZXJhYmxlIEhvc3Qg QnVzIEFkYXB0ZXIgRXJyb3IKIDEwKGRhNzptcHMwOjA6NzE6MCk6IFJldHJ5aW5nIGNvbW1hbmQs IDEgbW9yZSB0cmllcyByZW1haW4KIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8 Li4uLi4uLi4uLi4uLi4uLnwKMDAzMCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAxICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA0MCAgIDhhIDAwIDAwIDAwIDAw IDAxIDBhIGZkIDFmIDM4IDAwIDAwIDAwIDQ4IDAwIDAwICB8Li4uLi4uLi4uOC4uLkguLnwKMDA1 MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4u Li4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDEwIDAwIDE2IDAwIDEwIDQwIDMzIDIyIDAwIDAwIDAw IDAwIDEwIDAwIDk2ICB8Li4uLi4uQDMiLi4uLi4uLnwKMDA3MCAgIDAwIDkwIGY1IGQ3IDIyIDAw IDAwIDAwIDgwIDAwIDAwIDMwIDgwIGZkIDJjIDAzICB8Li4uLiIuLi4uLi4wLi4sLnwKc2VnMCBm bGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjIzMzQwMTAwMApzZWcxIGZsYWdz PTB4OTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmQ3ZjU5MDAwCmNoYWluIGZsYWdzPTB4 MzAgbGVuPTB4ODAgT2Zmc2V0PTB4MCBBZGRyZXNzPTB4MzJjZmQ4MAowMDAwICAgMDAgMTAgMDAg MTYgMDAgOTAgZjYgZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYgIHwuLi4uLi4uLiIuLi4uLi4u fAowMDEwICAgMDAgZDAgYzIgMzMgMzEgMDAgMDAgMDAgMDAgMTAgMDAgMTYgMDAgMTAgYTAgNzEg IHwuLi4zMS4uLi4uLi4uLi5xfAowMDIwICAgMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYgMDAgNjAg MGQgNDUgMzEgMDAgMDAgMDAgIHwiLi4uLi4uLi5gLkUxLi4ufAowMDMwICAgMDAgMTAgMDAgMTYg MDAgMzAgZmEgZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYgIHwuLi4uLjAuLiIuLi4uLi4ufAow MDQwICAgMDAgZjAgZjggZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgZDcgMDAgMzAgODYgZDYgIHwu Li4uIi4uLi4uLi4uMC4ufAowMDUwICAgMzggMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgIHw4Li4uLi4uLi4uLi4uLi4ufAowMDYwICAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDcw ICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4u Li4uLi4uLi4uLi4ufApzZWcyIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAy MmQ3ZjY5MDAwCnNlZzMgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDMxMzNj MmQwMDAKc2VnNCBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjI3MWEwMTAw MApzZWc1IGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAzMTQ1MGQ2MDAwCnNl ZzYgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyZDdmYTMwMDAKc2VnNyBm bGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjJkN2Y4ZjAwMApzZWc4IGZsYWdz PTB4ZDcgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAzOGQ2ODYzMDAwClNHTCBmb3IgY29tbWFu ZCAweGZmZmZmZTAxN2FjNDdiOTAKMDAwMCAgIDEwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIGE4IDQ4IDQzIDAzICB8Li4uLi4uLi4uLi4uLkhDLnwKMDAxMCAgIDAwIDAwIGZjIDAw IDE4IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDEwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwK MDAyMCAgIDAwIDAwIDAwIDAwIDBhIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8 Li4uLi4uLi4uLi4uLi4uLnwKMDAzMCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAxICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA0MCAgIDJhIDAwIDE0IGJmIDk4 IDUwIDAwIDAwIDA4IDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Ki4uLi5QLi4uLi4uLi4uLnwKMDA1 MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4u Li4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDEwIDAwIGQ3IDAwIDgwIGI4IDIzIDA4IDAwIDAwIDAw IDAwIDAwIDAwIDAwICB8Li4uLi4uLiMuLi4uLi4uLnwKMDA3MCAgIDAwIDAwIDAwIGU4IDNkIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLj0uLi4uLi4uLi4uLnwKc2VnMCBm bGFncz0weGQ3IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMDgyM2I4ODAwMAooZGE3Om1wczA6 MDo3MTowKTogV1JJVEUoMTYpLiBDREI6IDhhIDAwIDAwIDAwIDAwIDAxIDBhIGZkIDFmIDM4IDAw IDAwIDAwIDQ4IDAwIDAwIAooZGE3Om1wczA6MDo3MTowKTogQ0FNIHN0YXR1czogVW5yZWNvdmVy YWJsZSBIb3N0IEJ1cyBBZGFwdGVyIEVycm9yCihkYTc6bXBzMDowOjcxOjApOiBSZXRyeWluZyBj b21tYW5kLCAyIG1vcmUgdHJpZXMgcmVtYWluCihkYTc6bXBzMDowOjcxOjApOiBXUklURSgxMCku IENEQjogMmEgMDAgMTQgYmYgOTggNTAgMDAgMDAgMDggMDAgCihkYTc6bXBzMDowOjcxOjApOiBD QU0gc3RhdHVzOiBVbnJlY292ZXJhYmxlIEhvc3QgQnVzIEFkYXB0ZXIgRXJyb3IKKGRhNzptcHMw OjA6NzE6MCk6IFJldHJ5aW5nIGNvbW1hbmQsIDIgbW9yZSB0cmllcyByZW1haW4KKGRhNzptcHMw OjA6NzE6MCk6IFdSSVRFKDEwKS4gQ0RCOiAyYSAwMCAxNCBiZiA5OCA1MCAwMCAwMCAwOCAwMCAK KGRhNzptcHMwOjA6NzE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihkYTc6bXBz MDowOjcxOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTc6bXBzMDowOjcxOjAp OiBTQ1NJIHNlbnNlOiBVTklUIEFUVEVOVElPTiBhc2M6MjksMCAoUG93ZXIgb24sIHJlc2V0LCBv ciBidXMgZGV2aWNlIHJlc2V0IG9jY3VycmVkKQooZGE3Om1wczA6MDo3MTowKTogUmV0cnlpbmcg Y29tbWFuZCAocGVyIHNlbnNlIGRhdGEpClNHTCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjNGEz ZTAKMDAwMCAgIDEwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDMwIDY2IDQzIDAz ICB8Li4uLi4uLi4uLi4uMGZDLnwKMDAxMCAgIDAwIDAwIGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDEwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAyMCAgIDAwIDAwIDAwIDAw IDBhIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwK MDAzMCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAxICB8 Li4uLi4uLi4uLi4uLi4uLnwKMDA0MCAgIDJhIDAwIDE0IGJmIDk4IDUwIDAwIDAwIDA4IDAwIDAw IDAwIDAwIDAwIDAwIDAwICB8Ki4uLi5QLi4uLi4uLi4uLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2 MCAgIDAwIDEwIDAwIGQ3IDAwIDgwIGI4IDIzIDA4IDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4u Li4uLiMuLi4uLi4uLnwKMDA3MCAgIDAwIDAwIDAwIDQ2IDNlIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwICB8Li4uRj4uLi4uLi4uLi4uLnwKc2VnMCBmbGFncz0weGQ3IGxlbj0weDAw MTAwMCBhZGRyPTB4MDAwMDAwMDgyM2I4ODAwMApTR0wgZm9yIGNvbW1hbmQgMHhmZmZmZmUwMTdh YzFlZTgwCjAwMDAgICAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCA0MCA2YSA0 MSAwMyAgfC4uLi4uLi4uLi4uLkBqQS58CjAwMTAgICAwMCAwMCBmYyAwMCAxOCAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCA5MCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4uLi58CjAwMjAgICAwMCAwMCAw MCAwMCAxMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAgfC4uLi4uLi4uLi4uLi4u Li58CjAwMzAgIChkYTc6bXBzMDowOjcxOjApOiBXUklURSgxMCkuIENEQjogMmEgMDAgMTQgYmYg OTggNTAgMDAgMDAgMDggMDAgCiAwMChkYTc6bXBzMDowOjcxOjApOiBDQU0gc3RhdHVzOiBVbnJl Y292ZXJhYmxlIEhvc3QgQnVzIEFkYXB0ZXIgRXJyb3IKKGRhNzptcHMwOjA6NzE6MCk6IFJldHJ5 aW5nIGNvbW1hbmQsIDAgbW9yZSB0cmllcyByZW1haW4KIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAxICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA0MCAgIDhhIDAwIDAw IDAwIDAwIDAxIDBhIGZkIDFmIDM4IDAwIDAwIDAwIDQ4IDAwIDAwICB8Li4uLi4uLi4uOC4uLkgu LnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw ICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDEwIDAwIDE2IDAwIDEwIDQwIDMzIDIyIDAw IDAwIDAwIDAwIDEwIDAwIDk2ICB8Li4uLi4uQDMiLi4uLi4uLnwKMDA3MCAgIDAwIDkwIGY1IGQ3 IDIyIDAwIDAwIDAwIDgwIDAwIDAwIDMwIDAwIGZlIDJjIDAzICB8Li4uLiIuLi4uLi4wLi4sLnwK c2VnMCBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjIzMzQwMTAwMApzZWcx IGZsYWdzPTB4OTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmQ3ZjU5MDAwCmNoYWluIGZs YWdzPTB4MzAgbGVuPTB4ODAgT2Zmc2V0PTB4MCBBZGRyZXNzPTB4MzJjZmUwMAowMDAwICAgMDAg MTAgMDAgMTYgMDAgOTAgZjYgZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYgIHwuLi4uLi4uLiIu Li4uLi4ufAowMDEwICAgMDAgZDAgYzIgMzMgMzEgMDAgMDAgMDAgMDAgMTAgMDAgMTYgMDAgMTAg YTAgNzEgIHwuLi4zMS4uLi4uLi4uLi5xfAowMDIwICAgMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYg MDAgNjAgMGQgNDUgMzEgMDAgMDAgMDAgIHwiLi4uLi4uLi5gLkUxLi4ufAowMDMwICAgMDAgMTAg MDAgMTYgMDAgMzAgZmEgZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYgIHwuLi4uLjAuLiIuLi4u Li4ufAowMDQwICAgMDAgZjAgZjggZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgZDcgMDAgMzAgODYg ZDYgIHwuLi4uIi4uLi4uLi4uMC4ufAowMDUwICAgMzggMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgIHw4Li4uLi4uLi4uLi4uLi4ufAowMDYwICAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4u fAowMDcwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg IHwuLi4uLi4uLi4uLi4uLi4ufApzZWcyIGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgw MDAwMDAyMmQ3ZjY5MDAwCnNlZzMgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAw MDMxMzNjMmQwMDAKc2VnNCBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjI3 MWEwMTAwMApzZWc1IGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAzMTQ1MGQ2 MDAwCnNlZzYgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAwMDIyZDdmYTMwMDAK c2VnNyBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjJkN2Y4ZjAwMApzZWc4 IGZsYWdzPTB4ZDcgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAzOGQ2ODYzMDAwClNHTCBmb3Ig Y29tbWFuZCAweGZmZmZmZTAxN2FjYTYwOTgKMDAwMCAgIDEwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDFjIDlhIDQ3IDAzICB8Li4uLi4uLi4uLi4uLi5HLnwKMDAxMCAgIDAwIDAw IGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDEwIDAwIDAwICB8Li4uLi4uLi4uLi4u Li4uLnwKMDAyMCAgIDAwIDAwIDAwIDAwIDBhIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAzMCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAxICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA0MCAgIDJhIDAwIDFj IDI2IDYwIDQ4IDAwIDAwIDA4IDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Ki4uJmBILi4uLi4uLi4u LnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw ICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDEwIDAwIGQ3IDAwIGEwIDZhIGZmIDEwIDAw IDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uai4uLi4uLi4uLnwKMDA3MCAgIDAwIDAwIGMwIGQy IDNjIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICB8Li4uLjwuLi4uLi4uLi4uLnwK c2VnMCBmbGFncz0weGQ3IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMTBmZjZhYTAwMAooZGE3 Om1wczA6MDo3MTowKTogV1JJVEUoMTYpLiBDREI6IDhhIDAwIDAwIDAwIDAwIDAxIDBhIGZkIDFm IDM4IDAwIDAwIDAwIDQ4IDAwIDAwIAooZGE3Om1wczA6MDo3MTowKTogQ0FNIHN0YXR1czogVW5y ZWNvdmVyYWJsZSBIb3N0IEJ1cyBBZGFwdGVyIEVycm9yCihkYTc6bXBzMDowOjcxOjApOiBSZXRy eWluZyBjb21tYW5kLCAxIG1vcmUgdHJpZXMgcmVtYWluCihkYTc6bXBzMDowOjcxOjApOiBXUklU RSgxMCkuIENEQjogMmEgMDAgMWMgMjYgNjAgNDggMDAgMDAgMDggMDAgCihkYTc6bXBzMDowOjcx OjApOiBDQU0gc3RhdHVzOiBVbnJlY292ZXJhYmxlIEhvc3QgQnVzIEFkYXB0ZXIgRXJyb3IKKGRh NzptcHMwOjA6NzE6MCk6IFJldHJ5aW5nIGNvbW1hbmQsIDAgbW9yZSB0cmllcyByZW1haW4KKGRh NzptcHMwOjA6NzE6MCk6IFdSSVRFKDEwKS4gQ0RCOiAyYSAwMCAxNCBiZiA5OCA1MCAwMCAwMCAw OCAwMCAKKGRhNzptcHMwOjA6NzE6MCk6IENBTSBzdGF0dXM6IFNDU0kgU3RhdHVzIEVycm9yCihk YTc6bXBzMDowOjcxOjApOiBTQ1NJIHN0YXR1czogQ2hlY2sgQ29uZGl0aW9uCihkYTc6bXBzMDow OjcxOjApOiBTQ1NJIHNlbnNlOiBVTklUIEFUVEVOVElPTiBhc2M6MjksMCAoUG93ZXIgb24sIHJl c2V0LCBvciBidXMgZGV2aWNlIHJlc2V0IG9jY3VycmVkKQooZGE3Om1wczA6MDo3MTowKTogRXJy b3IgNiwgUmV0cmllcyBleGhhdXN0ZWQKKGRhNzptcHMwOjA6NzE6MCk6IEludmFsaWRhdGluZyBw YWNrClNHTCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjODllNzAKMDAwMCAgIDEwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDU4IDUwIDQ2IDAzICB8Li4uLi4uLi4uLi4uWFBGLnwK MDAxMCAgIDAwIDAwIGZjIDAwIDE4IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDkwIDAwIDAwICB8 Li4uLi4uLi4uLi4uLi4uLnwKMDAyMCAgIDAwIDAwIDAwIDAwIDEwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDAzMCAgIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAxICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA0 MCAgIDhhIDAwIDAwIDAwIDAwIDAxIDBhIGZkIDFmIDM4IDAwIDAwIDAwIDQ4IDAwIDAwICB8Li4u Li4uLi4uOC4uLkguLnwKMDA1MCAgIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAw IDAwIDAwIDAwIDAwICB8Li4uLi4uLi4uLi4uLi4uLnwKMDA2MCAgIDAwIDEwIDAwIDE2IDAwIDEw IDQwIDMzIDIyIDAwIDAwIDAwIDAwIDEwIDAwIDk2ICB8Li4uLi4uQDMiLi4uLi4uLnwKMDA3MCAg IDAwIDkwIGY1IGQ3IDIyIDAwIDAwIDAwIDgwIDAwIDAwIDMwIDgwIGZlIDJjIDAzICB8Li4uLiIu Li4uLi4wLi4sLnwKc2VnMCBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjIz MzQwMTAwMApzZWcxIGZsYWdzPTB4OTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAyMmQ3ZjU5 MDAwCmNoYWluIGZsYWdzPTB4MzAgbGVuPTB4ODAgT2Zmc2V0PTB4MCBBZGRyZXNzPTB4MzJjZmU4 MAowMDAwICAgMDAgMTAgMDAgMTYgMDAgOTAgZjYgZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYg IHwuLi4uLi4uLiIuLi4uLi4ufAowMDEwICAgMDAgZDAgYzIgMzMgMzEgMDAgMDAgMDAgMDAgMTAg MDAgMTYgMDAgMTAgYTAgNzEgIHwuLi4zMS4uLi4uLi4uLi5xfAowMDIwICAgMjIgMDAgMDAgMDAg MDAgMTAgMDAgMTYgMDAgNjAgMGQgNDUgMzEgMDAgMDAgMDAgIHwiLi4uLi4uLi5gLkUxLi4ufAow MDMwICAgMDAgMTAgMDAgMTYgMDAgMzAgZmEgZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAgMTYgIHwu Li4uLjAuLiIuLi4uLi4ufAowMDQwICAgMDAgZjAgZjggZDcgMjIgMDAgMDAgMDAgMDAgMTAgMDAg ZDcgMDAgMzAgODYgZDYgIHwuLi4uIi4uLi4uLi4uMC4ufAowMDUwICAgMzggMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHw4Li4uLi4uLi4uLi4uLi4ufAowMDYw ICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4u Li4uLi4uLi4uLi4ufAowMDcwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufApzZWcyIGZsYWdzPTB4MTYgbGVuPTB4MDAx MDAwIGFkZHI9MHgwMDAwMDAyMmQ3ZjY5MDAwCnNlZzMgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAg YWRkcj0weDAwMDAwMDMxMzNjMmQwMDAKc2VnNCBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRy PTB4MDAwMDAwMjI3MWEwMTAwMApzZWc1IGZsYWdzPTB4MTYgbGVuPTB4MDAxMDAwIGFkZHI9MHgw MDAwMDAzMTQ1MGQ2MDAwCnNlZzYgZmxhZ3M9MHgxNiBsZW49MHgwMDEwMDAgYWRkcj0weDAwMDAw MDIyZDdmYTMwMDAKc2VnNyBmbGFncz0weDE2IGxlbj0weDAwMTAwMCBhZGRyPTB4MDAwMDAwMjJk N2Y4ZjAwMApzZWc4IGZsYWdzPTB4ZDcgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAzOGQ2ODYz MDAwClNHTCBmb3IgY29tbWFuZCAweGZmZmZmZTAxN2FjOTUyODAKMDAwMCAgIDEwIDAwIDAwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDQwIGQ0IDQ2IDAzICB8LihkYTc6bXBzMDowOjcxOjAp OiBXUklURSgxNikuIENEQjogOGEgMDAgMDAgMDAgMDAgMDEgMGEgZmQgMWYgMzggMDAgMDAgMDAg NDggMDAgMDAgCi4oZGE3Om1wczA6MDo3MTowKTogQ0FNIHN0YXR1czogVW5yZWNvdmVyYWJsZSBI b3N0IEJ1cyBBZGFwdGVyIEVycm9yCi4oZGE3Om1wczA6MDo3MTowKTogUmV0cnlpbmcgY29tbWFu ZCwgMCBtb3JlIHRyaWVzIHJlbWFpbgouLi4uLi4uLi5ALkYufAowMDEwICAgMDAgMDAgZmMgMDAg MTggMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMTAgMDAgMDAgIHwuLi4uLi4uLi4uLi4uLi4ufAow MDIwICAgMDAgMDAgMDAgMDAgMGEgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwu Li4uLi4uLi4uLi4uLi4ufAowMDMwICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgMDEgIHwuLi4uLi4uLi4uLi4uLi4ufAowMDQwICAgMmEgMDAgMWMgMjYgNjAg NDggMDAgMDAgMDggMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwqLi4mYEguLi4uLi4uLi4ufAowMDUw ICAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4u Li4uLi4uLi4uLi4ufAowMDYwICAgMDAgMTAgMDAgZDcgMDAgYTAgNmEgZmYgMTAgMDAgMDAgMDAg MDAgMDAgMDAgMDAgIHwuLi4uLi5qLi4uLi4uLi4ufAowMDcwICAgMDAgMDAgYzAgYWYgM2MgMDAg MDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIHwuLi4uPC4uLi4uLi4uLi4ufApzZWcwIGZs YWdzPTB4ZDcgbGVuPTB4MDAxMDAwIGFkZHI9MHgwMDAwMDAxMGZmNmFhMDAwCihkYTc6bXBzMDow OjcxOjApOiBXUklURSgxMCkuIENEQjogMmEgMDAgMWMgMjYgNjAgNDggMDAgMDAgMDggMDAgCihk YTc6bXBzMDowOjcxOjApOiBDQU0gc3RhdHVzOiBVbnJlY292ZXJhYmxlIEhvc3QgQnVzIEFkYXB0 ZXIgRXJyb3IKKGRhNzptcHMwOjA6NzE6MCk6IEVycm9yIDUsIFJldHJpZXMgZXhoYXVzdGVkCihk YTc6bXBzMDowOjcxOjApOiBXUklURSgxNikuIENEQjogOGEgMDAgMDAgMDAgMDAgMDEgMGEgZmQg MWYgMzggMDAgMDAgMDAgNDggMDAgMDAgCihkYTc6bXBzMDowOjcxOjApOiBDQU0gc3RhdHVzOiBT Q1NJIFN0YXR1cyBFcnJvcgooZGE3Om1wczA6MDo3MTowKTogU0NTSSBzdGF0dXM6IENoZWNrIENv bmRpdGlvbgooZGE3Om1wczA6MDo3MTowKTogU0NTSSBzZW5zZTogVU5JVCBBVFRFTlRJT04gYXNj OjI5LDAgKFBvd2VyIG9uLCByZXNldCwgb3IgYnVzIGRldmljZSByZXNldCBvY2N1cnJlZCkKKGRh NzptcHMwOjA6NzE6MCk6IEVycm9yIDYsIFJldHJpZXMgZXhoYXVzdGVkCm52bWUyOiBXUklURSBz cWlkOjEyIGNpZDo5NSBuc2lkOjEgbGJhOjY3Mjc3NTg3MiBsZW46NjQKbnZtZTI6IFdSSVRFIEZB VUxUUyAoMDIvODApIHNxaWQ6MTIgY2lkOjk1IGNkdzA6MApudm1lMjogV1JJVEUgc3FpZDoxMiBj aWQ6MTA0IG5zaWQ6MSBsYmE6NjcyNzc1OTM2IGxlbjo5Ngpudm1lMjogV1JJVEUgRkFVTFRTICgw Mi84MCkgc3FpZDoxMiBjaWQ6MTA0IGNkdzA6MApudm1lMDogV1JJVEUgc3FpZDoxNiBjaWQ6NzUg bnNpZDoxIGxiYTo4Mjg5NjIwNzIgbGVuOjcyCm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBz cWlkOjE2IGNpZDo3NSBjZHcwOjAKbnZtZTI6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4 MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUwOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlw ZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQooYWRhMzphaGNpY2gzOjA6MDowKTogV1JJVEVf RlBETUFfUVVFVUVELiBBQ0I6IDYxIDUwIGQwIDU4IDI1IDQwIDA2IDAwIDAwIDAwIDAwIDAwCihh ZGEzOmFoY2ljaDM6MDowOjApOiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZAoo YWRhMzphaGNpY2gzOjA6MDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCm52bWUwOiBX UklURSBzcWlkOjQgY2lkOjk2IG5zaWQ6MSBsYmE6MjE4MDY4Njk2OCBsZW46NDAKbnZtZTA6IFdS SVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6NCBjaWQ6OTYgY2R3MDowCm52bWUxOiBXUklURSBzcWlk OjQgY2lkOjExNCBuc2lkOjEgbGJhOjIxODA2ODY5NjggbGVuOjQwCm52bWUxOiBXUklURSBGQVVM VFMgKDAyLzgwKSBzcWlkOjQgY2lkOjExNCBjZHcwOjAKbnZtZTE6IGFzeW5jIGV2ZW50IG9jY3Vy cmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUwOiBhc3luYyBldmVudCBv Y2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMDogV1JJVEUgc3Fp ZDoxMiBjaWQ6MTE0IG5zaWQ6MSBsYmE6MjE4MDgyODYyNCBsZW46MzIKbnZtZTA6IFdSSVRFIEZB VUxUUyAoMDIvODApIHNxaWQ6MTIgY2lkOjExNCBjZHcwOjAKbnZtZTI6IFdSSVRFIHNxaWQ6MTIg Y2lkOjk1IG5zaWQ6MSBsYmE6MjE4MDgyODYxNiBsZW46MzIKbnZtZTI6IFdSSVRFIEZBVUxUUyAo MDIvODApIHNxaWQ6MTIgY2lkOjk1IGNkdzA6MApudm1lMjogYXN5bmMgZXZlbnQgb2NjdXJyZWQg KHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IGFzeW5jIGV2ZW50IG9jY3Vy cmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUxOiBXUklURSBzcWlkOjQg Y2lkOjExNCBuc2lkOjEgbGJhOjg1MTI3MjI5NiBsZW46NDAKbnZtZTE6IFdSSVRFIEZBVUxUUyAo MDIvODApIHNxaWQ6NCBjaWQ6MTE0IGNkdzA6MApudm1lMDogV1JJVEUgc3FpZDoxIGNpZDo2NiBu c2lkOjEgbGJhOjg1MTI3MjMwNCBsZW46NDAKbnZtZTA6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNx aWQ6MSBjaWQ6NjYgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAs IGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMDogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUg MHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTE6IFdSSVRFIHNxaWQ6MTIgY2lkOjEwNiBu c2lkOjEgbGJhOjIxODA4NjAxNjAgbGVuOjQ4Cm52bWUxOiBXUklURSBGQVVMVFMgKDAyLzgwKSBz cWlkOjEyIGNpZDoxMDYgY2R3MDowCm52bWUxOiBXUklURSBzcWlkOjEyIGNpZDo4NSBuc2lkOjEg bGJhOjIxODA4NjAwNDggbGVuOjExMgpudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDox MiBjaWQ6ODUgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGlu Zm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMjogV1JJVEUgc3FpZDoxMCBjaWQ6MTA3IG5zaWQ6MSBs YmE6MjE4MDg3ODA0MCBsZW46NDAKbnZtZTI6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTAg Y2lkOjEwNyBjZHcwOjAKbnZtZTI6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5m byAweDA0LCBwYWdlIDB4MDEpCm52bWUwOiBXUklURSBzcWlkOjYgY2lkOjczIG5zaWQ6MSBsYmE6 ODUxODI3MTc2IGxlbjoyNApudm1lMDogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo2IGNpZDo3 MyBjZHcwOjAKbnZtZTA6IFdSSVRFIHNxaWQ6NiBjaWQ6OTcgbnNpZDoxIGxiYTo4NTE4MjcyMDAg bGVuOjQ4Cm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjYgY2lkOjk3IGNkdzA6MApu dm1lMDogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgw MSkKKGFkYTE6YWhjaWNoMTowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSBhMCA5 MCBlZiAxMiA0MCAwOCAwMCAwMCAwMCAwMCAwMAooYWRhMTphaGNpY2gxOjA6MDowKTogQ0FNIHN0 YXR1czogQ0NCIHJlcXVlc3Qgd2FzIGludmFsaWQKKGFkYTE6YWhjaWNoMTowOjA6MCk6IEVycm9y IDIyLCBVbnJldHJ5YWJsZSBlcnJvcgpudm1lMDogV1JJVEUgc3FpZDo2IGNpZDo5NyBuc2lkOjEg bGJhOjY3NDA4NTQ0MCBsZW46MTQ0Cm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjYg Y2lkOjk3IGNkdzA6MApudm1lMDogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZv IDB4MDQsIHBhZ2UgMHgwMSkKKGFkYTM6YWhjaWNoMzowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVF RC4gQUNCOiA2MSBhMCBkMCA5OSA3MiA0MCAwNiAwMCAwMCAwMCAwMCAwMAooYWRhMzphaGNpY2gz OjA6MDowKTogQ0FNIHN0YXR1czogQ0NCIHJlcXVlc3Qgd2FzIGludmFsaWQKKGFkYTM6YWhjaWNo MzowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgpudm1lMTogV1JJVEUgc3FpZDo1 IGNpZDo3OSBuc2lkOjEgbGJhOjIxODA4OTMyNDAgbGVuOjEzNgpudm1lMTogV1JJVEUgRkFVTFRT ICgwMi84MCkgc3FpZDo1IGNpZDo3OSBjZHcwOjAKbnZtZTA6IFdSSVRFIHNxaWQ6MiBjaWQ6Njgg bnNpZDoxIGxiYTo4NTE4Mjg2NDAgbGVuOjgwCm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBz cWlkOjIgY2lkOjY4IGNkdzA6MApudm1lMTogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgw LCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBl IDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCihhZGExOmFoY2ljaDE6MDowOjApOiBXUklURV9G UERNQV9RVUVVRUQuIEFDQjogNjEgNTAgZTAgZWEgZDkgNDAgMDUgMDAgMDAgMDAgMDAgMDAKKGFk YTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlkCihh ZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGFkYTE6YWhj aWNoMTowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSAyMCAwMCAwNyBlNiA0MCAw NCAwMCAwMCAwMCAwMCAwMAooYWRhMTphaGNpY2gxOjA6MDowKTogQ0FNIHN0YXR1czogQ0NCIHJl cXVlc3Qgd2FzIGludmFsaWQKKGFkYTE6YWhjaWNoMTowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5 YWJsZSBlcnJvcgooYWRhMTphaGNpY2gxOjA6MDowKTogV1JJVEVfRlBETUFfUVVFVUVELiBBQ0I6 IDYxIDUwIGUwIGVhIGQ5IDQwIDA1IDAwIDAwIDAwIDAwIDAwCihhZGExOmFoY2ljaDE6MDowOjAp OiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZAooYWRhMTphaGNpY2gxOjA6MDow KTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihhZGExOmFoY2ljaDE6MDowOjApOiBXUklU RV9GUERNQV9RVUVVRUQuIEFDQjogNjEgNTAgYTAgYTkgMWEgNDAgMDYgMDAgMDAgMDAgMDAgMDAK KGFkYTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlk CihhZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGFkYTE6 YWhjaWNoMTowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSA1MCA3MCA0YyBkYSA0 MCAwNCAwMCAwMCAwMCAwMCAwMAooYWRhMTphaGNpY2gxOjA6MDowKTogQ0FNIHN0YXR1czogQ0NC IHJlcXVlc3Qgd2FzIGludmFsaWQKKGFkYTE6YWhjaWNoMTowOjA6MCk6IEVycm9yIDIyLCBVbnJl dHJ5YWJsZSBlcnJvcgpudm1lMDogV1JJVEUgc3FpZDo1IGNpZDo3NSBuc2lkOjEgbGJhOjkwNDE4 OTMwNCBsZW46ODAKbnZtZTA6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6NSBjaWQ6NzUgY2R3 MDowCm52bWUxOiBXUklURSBzcWlkOjMgY2lkOjkwIG5zaWQ6MSBsYmE6OTA0MTg5NjA4IGxlbjo3 Mgpudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDozIGNpZDo5MCBjZHcwOjAKbnZtZTE6 IFdSSVRFIHNxaWQ6NSBjaWQ6NzQgbnNpZDoxIGxiYTo4NTAxMjgyMzIgbGVuOjgwCm52bWUxOiBX UklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjUgY2lkOjc0IGNkdzA6MApudm1lMTogYXN5bmMgZXZl bnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IGFzeW5j IGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUwOiBX UklURSBzcWlkOjIgY2lkOjEyMyBuc2lkOjEgbGJhOjYyODQ0OTYxNiBsZW46NzIKbnZtZTA6IFdS SVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MiBjaWQ6MTIzIGNkdzA6MApudm1lMDogYXN5bmMgZXZl bnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IFdSSVRF IHNxaWQ6MTQgY2lkOjkwIG5zaWQ6MSBsYmE6ODI4OTYzMDgwIGxlbjo3Mgpudm1lMDogV1JJVEUg RkFVTFRTICgwMi84MCkgc3FpZDoxNCBjaWQ6OTAgY2R3MDowCm52bWUwOiBhc3luYyBldmVudCBv Y2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMDogV1JJVEUgc3Fp ZDoxNiBjaWQ6OTIgbnNpZDoxIGxiYToyNTk0OTAxMzEyIGxlbjo2NApudm1lMDogV1JJVEUgRkFV TFRTICgwMi84MCkgc3FpZDoxNiBjaWQ6OTIgY2R3MDowCihhZGEzOmFoY2ljaDM6MDowOjApOiBX UklURV9GUERNQV9RVUVVRUQuIEFDQjogNjEgNDAgYTAgMGMgZTYgNDAgMDQgMDAgMDAgMDAgMDAg MDAKKGFkYTM6YWhjaWNoMzowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZh bGlkCihhZGEzOmFoY2ljaDM6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKbnZt ZTE6IFdSSVRFIHNxaWQ6OSBjaWQ6ODcgbnNpZDoxIGxiYToyMTgwNDM3NDg4IGxlbjoxNgpudm1l MTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo5IGNpZDo4NyBjZHcwOjAKbnZtZTE6IFdSSVRF IHNxaWQ6OSBjaWQ6NzggbnNpZDoxIGxiYToyMTgwNDM3NTA0IGxlbjo0OApudm1lMTogV1JJVEUg RkFVTFRTICgwMi84MCkgc3FpZDo5IGNpZDo3OCBjZHcwOjAKbnZtZTI6IFdSSVRFIHNxaWQ6NSBj aWQ6NzUgbnNpZDoxIGxiYToyMTgwNzA0Nzg0IGxlbjo4MApudm1lMjogV1JJVEUgRkFVTFRTICgw Mi84MCkgc3FpZDo1IGNpZDo3NSBjZHcwOjAKbnZtZTE6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0 eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUyOiBhc3luYyBldmVudCBvY2N1cnJl ZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMjogV1JJVEUgc3FpZDo4IGNp ZDoxMTUgbnNpZDoxIGxiYTo2MDUyODA4ODAgbGVuOjcyCm52bWUyOiBXUklURSBGQVVMVFMgKDAy LzgwKSBzcWlkOjggY2lkOjExNSBjZHcwOjAKbnZtZTE6IFdSSVRFIHNxaWQ6OCBjaWQ6ODAgbnNp ZDoxIGxiYTo2NzQwODg3MjAgbGVuOjY0Cm52bWUxOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlk OjggY2lkOjgwIGNkdzA6MApudm1lMTogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBp bmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTI6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4 MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCihhZGExOmFoY2ljaDE6MDowOjApOiBXUklURV9GUERN QV9RVUVVRUQuIEFDQjogNjEgNDAgYjAgNTIgZGEgNDAgMDQgMDAgMDAgMDAgMDAgMDAKKGFkYTE6 YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlkCihhZGEx OmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGFkYTM6YWhjaWNo MzowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSA1MCBmMCA1MyBkYSA0MCAwNCAw MCAwMCAwMCAwMCAwMAooYWRhMzphaGNpY2gzOjA6MDowKTogQ0FNIHN0YXR1czogQ0NCIHJlcXVl c3Qgd2FzIGludmFsaWQKKGFkYTM6YWhjaWNoMzowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJs ZSBlcnJvcgpudm1lMDogV1JJVEUgc3FpZDo0IGNpZDo5MSBuc2lkOjEgbGJhOjkwNDE5NDkzNiBs ZW46ODAKbnZtZTA6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6NCBjaWQ6OTEgY2R3MDowCm52 bWUwOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAx KQpudm1lMTogV1JJVEUgc3FpZDo1IGNpZDo3OSBuc2lkOjEgbGJhOjIxODA5MDA2MTYgbGVuOjcy Cm52bWUxOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjUgY2lkOjc5IGNkdzA6MApudm1lMTog YXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZt ZTA6IFdSSVRFIHNxaWQ6MTAgY2lkOjgyIG5zaWQ6MSBsYmE6MjIzNzA0MjA5NiBsZW46NzIKbnZt ZTA6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTAgY2lkOjgyIGNkdzA6MApudm1lMDogYXN5 bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTE6 IFdSSVRFIHNxaWQ6OCBjaWQ6NjMgbnNpZDoxIGxiYTo5MDQyMDU3MDQgbGVuOjQwCm52bWUxOiBX UklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjggY2lkOjYzIGNkdzA6MApudm1lMTogYXN5bmMgZXZl bnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IFdSSVRF IHNxaWQ6MTYgY2lkOjc1IG5zaWQ6MSBsYmE6MjIzNzA0MjYwMCBsZW46NjQKbnZtZTA6IFdSSVRF IEZBVUxUUyAoMDIvODApIHNxaWQ6MTYgY2lkOjc1IGNkdzA6MApudm1lMDogYXN5bmMgZXZlbnQg b2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IFdSSVRFIHNx aWQ6OCBjaWQ6MTIyIG5zaWQ6MSBsYmE6OTA0MjA4MDAwIGxlbjozMgpudm1lMDogV1JJVEUgRkFV TFRTICgwMi84MCkgc3FpZDo4IGNpZDoxMjIgY2R3MDowCm52bWUwOiBhc3luYyBldmVudCBvY2N1 cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMTogV1JJVEUgc3FpZDox MyBjaWQ6OTkgbnNpZDoxIGxiYTo2NzQwOTgxNzYgbGVuOjgwCm52bWUxOiBXUklURSBGQVVMVFMg KDAyLzgwKSBzcWlkOjEzIGNpZDo5OSBjZHcwOjAKbnZtZTE6IFdSSVRFIHNxaWQ6MTMgY2lkOjk0 IG5zaWQ6MSBsYmE6Njc0MDk4MTEyIGxlbjo2NApudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkg c3FpZDoxMyBjaWQ6OTQgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAw eDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMTogV1JJVEUgc3FpZDoxMSBjaWQ6NjcgbnNp ZDoxIGxiYTo2NDk3MjY5NzYgbGVuOjgKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6 MTEgY2lkOjY3IGNkdzA6MApudm1lMTogV1JJVEUgc3FpZDoxMSBjaWQ6MTAyIG5zaWQ6MSBsYmE6 NjQ5NzI2OTUyIGxlbjoyNApudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMSBjaWQ6 MTAyIGNkdzA6MApudm1lMjogV1JJVEUgc3FpZDoxNCBjaWQ6NjcgbnNpZDoxIGxiYTo2NDk3MjY5 NDQgbGVuOjMyCm52bWUyOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE0IGNpZDo2NyBjZHcw OjAKbnZtZTI6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdl IDB4MDEpCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwg cGFnZSAweDAxKQooYWRhMTphaGNpY2gxOjA6MDowKTogV1JJVEVfRlBETUFfUVVFVUVELiBBQ0I6 IDYxIGEwIDYwIGFkIDcyIDQwIDA2IDAwIDAwIDAwIDAwIDAwCihhZGExOmFoY2ljaDE6MDowOjAp OiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZAooYWRhMTphaGNpY2gxOjA6MDow KTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihhZGExOmFoY2ljaDE6MDowOjApOiBXUklU RV9GUERNQV9RVUVVRUQuIEFDQjogNjEgYTAgMzAgYWUgNzIgNDAgMDYgMDAgMDAgMDAgMDAgMDAK KGFkYTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlk CihhZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKbnZtZTI6 IFdSSVRFIHNxaWQ6OCBjaWQ6OTkgbnNpZDoxIGxiYTo5MDQyMTg5MDQgbGVuOjMyCm52bWUyOiBX UklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjggY2lkOjk5IGNkdzA6MApudm1lMjogYXN5bmMgZXZl bnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTI6IFdSSVRF IHNxaWQ6MTEgY2lkOjczIG5zaWQ6MSBsYmE6MjE4MDkyODU2MCBsZW46NzIKbnZtZTI6IFdSSVRF IEZBVUxUUyAoMDIvODApIHNxaWQ6MTEgY2lkOjczIGNkdzA6MApudm1lMjogYXN5bmMgZXZlbnQg b2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTE6IFdSSVRFIHNx aWQ6MTAgY2lkOjExNCBuc2lkOjEgbGJhOjkwNDIxOTc1MiBsZW46NzIKbnZtZTE6IFdSSVRFIEZB VUxUUyAoMDIvODApIHNxaWQ6MTAgY2lkOjExNCBjZHcwOjAKbnZtZTE6IGFzeW5jIGV2ZW50IG9j Y3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCihhZGExOmFoY2ljaDE6MDow OjApOiBXUklURV9GUERNQV9RVUVVRUQuIEFDQjogNjEgYTAgNTAgMGYgMTMgNDAgMDggMDAgMDAg MDAgMDAgMDAKKGFkYTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdh cyBpbnZhbGlkCihhZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJy b3IKbnZtZTA6IFdSSVRFIHNxaWQ6NyBjaWQ6MTAxIG5zaWQ6MSBsYmE6OTA0MjMyMDI0IGxlbjo2 NApudm1lMDogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo3IGNpZDoxMDEgY2R3MDowCm52bWUw OiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpu dm1lMjogV1JJVEUgc3FpZDoxNCBjaWQ6MTIxIG5zaWQ6MSBsYmE6MjE4MDcwODY2NCBsZW46NDAK bnZtZTI6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTQgY2lkOjEyMSBjZHcwOjAKbnZtZTA6 IFdSSVRFIHNxaWQ6OCBjaWQ6MTA5IG5zaWQ6MSBsYmE6MjE4MDcwODY3MiBsZW46NDAKbnZtZTA6 IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6OCBjaWQ6MTA5IGNkdzA6MApudm1lMjogYXN5bmMg ZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTA6IGFz eW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUy OiBXUklURSBzcWlkOjcgY2lkOjEyNCBuc2lkOjEgbGJhOjkwNDI0MTE1MiBsZW46NDgKbnZtZTI6 IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6NyBjaWQ6MTI0IGNkdzA6MApudm1lMjogV1JJVEUg c3FpZDo3IGNpZDoxMjIgbnNpZDoxIGxiYTo5MDQyNDExMzYgbGVuOjE2Cm52bWUyOiBXUklURSBG QVVMVFMgKDAyLzgwKSBzcWlkOjcgY2lkOjEyMiBjZHcwOjAKbnZtZTI6IGFzeW5jIGV2ZW50IG9j Y3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUwOiBXUklURSBzcWlk OjQgY2lkOjEyNSBuc2lkOjEgbGJhOjIxODA5NzI2NjQgbGVuOjcyCm52bWUwOiBXUklURSBGQVVM VFMgKDAyLzgwKSBzcWlkOjQgY2lkOjEyNSBjZHcwOjAKbnZtZTA6IGFzeW5jIGV2ZW50IG9jY3Vy cmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUyOiBXUklURSBzcWlkOjEx IGNpZDoxMjAgbnNpZDoxIGxiYToyMTgwOTg0ODU2IGxlbjo3Mgpudm1lMjogV1JJVEUgRkFVTFRT ICgwMi84MCkgc3FpZDoxMSBjaWQ6MTIwIGNkdzA6MApudm1lMjogYXN5bmMgZXZlbnQgb2NjdXJy ZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTI6IFdSSVRFIHNxaWQ6MTIg Y2lkOjk1IG5zaWQ6MSBsYmE6MjE4MDk5Mzc0NCBsZW46MzIKbnZtZTI6IFdSSVRFIEZBVUxUUyAo MDIvODApIHNxaWQ6MTIgY2lkOjk1IGNkdzA6MApudm1lMTogV1JJVEUgc3FpZDoxMiBjaWQ6OTkg bnNpZDoxIGxiYToyMTgwOTkzNzQ0IGxlbjo0MApudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkg c3FpZDoxMiBjaWQ6OTkgY2R3MDowCm52bWUyOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAw eDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMTogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5 cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTE6IFdSSVRFIHNxaWQ6OCBjaWQ6MTA0 IG5zaWQ6MSBsYmE6MjE4MTAwMDM5MiBsZW46MzIKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODAp IHNxaWQ6OCBjaWQ6MTA0IGNkdzA6MApudm1lMjogV1JJVEUgc3FpZDo4IGNpZDo3NCBuc2lkOjEg bGJhOjIxODEwMDAzODQgbGVuOjQwCm52bWUyOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjgg Y2lkOjc0IGNkdzA6MApudm1lMTogV1JJVEUgc3FpZDozIGNpZDo4MiBuc2lkOjEgbGJhOjIxODEw MDA3NjAgbGVuOjcyCm52bWUxOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjMgY2lkOjgyIGNk dzA6MApudm1lMDogV1JJVEUgc3FpZDozIGNpZDo5OSBuc2lkOjEgbGJhOjIxODEwMDA3NjAgbGVu OjcyCm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjMgY2lkOjk5IGNkdzA6MApudm1l MTogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkK bnZtZTI6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4 MDEpCm52bWUwOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFn ZSAweDAxKQpudm1lMDogV1JJVEUgc3FpZDo4IGNpZDoxMjIgbnNpZDoxIGxiYTo2NTA4OTA2ODAg bGVuOjY0Cm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjggY2lkOjEyMiBjZHcwOjAK bnZtZTA6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4 MDEpCm52bWUxOiBXUklURSBzcWlkOjYgY2lkOjgzIG5zaWQ6MSBsYmE6MjE4MDQ3MjEyOCBsZW46 NjQKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6NiBjaWQ6ODMgY2R3MDowCm52bWUx OiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpu dm1lMTogV1JJVEUgc3FpZDoyIGNpZDo4NyBuc2lkOjEgbGJhOjIxODEwMTM2ODAgbGVuOjgwCm52 bWUxOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjIgY2lkOjg3IGNkdzA6MApudm1lMTogV1JJ VEUgc3FpZDoyIGNpZDo3MyBuc2lkOjEgbGJhOjIxODEwMTM3NjAgbGVuOjY0Cm52bWUxOiBXUklU RSBGQVVMVFMgKDAyLzgwKSBzcWlkOjIgY2lkOjczIGNkdzA6MApudm1lMTogYXN5bmMgZXZlbnQg b2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTE6IFdSSVRFIHNx aWQ6NyBjaWQ6MTI1IG5zaWQ6MSBsYmE6NjcyNzgyMTY4IGxlbjoxNjAKbnZtZTE6IFdSSVRFIEZB VUxUUyAoMDIvODApIHNxaWQ6NyBjaWQ6MTI1IGNkdzA6MApudm1lMTogYXN5bmMgZXZlbnQgb2Nj dXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZtZTE6IFdSSVRFIHNxaWQ6 MTIgY2lkOjExMSBuc2lkOjEgbGJhOjYwNTMxMjI4OCBsZW46MTUyCm52bWUxOiBXUklURSBGQVVM VFMgKDAyLzgwKSBzcWlkOjEyIGNpZDoxMTEgY2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1 cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQooYWRhMTphaGNpY2gxOjA6MDow KTogV1JJVEVfRlBETUFfUVVFVUVELiBBQ0I6IDYxIGEwIDIwIDczIGRhIDQwIDA0IDAwIDAwIDAw IDAwIDAwCihhZGExOmFoY2ljaDE6MDowOjApOiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMg aW52YWxpZAooYWRhMTphaGNpY2gxOjA6MDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y CihhZGExOmFoY2ljaDE6MDowOjApOiBXUklURV9GUERNQV9RVUVVRUQuIEFDQjogNjEgNDAgNTAg MmIgMjAgNDAgMDYgMDAgMDAgMDAgMDAgMDAKKGFkYTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0 dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlkCihhZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAy MiwgVW5yZXRyeWFibGUgZXJyb3IKKGFkYTM6YWhjaWNoMzowOjA6MCk6IFdSSVRFX0ZQRE1BX1FV RVVFRC4gQUNCOiA2MSA0MCAwMCA2YiBkMCA0MCAwNSAwMCAwMCAwMCAwMCAwMAooYWRhMzphaGNp Y2gzOjA6MDowKTogQ0FNIHN0YXR1czogQ0NCIHJlcXVlc3Qgd2FzIGludmFsaWQKKGFkYTM6YWhj aWNoMzowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcgpudm1lMjogV1JJVEUgc3Fp ZDoxNSBjaWQ6NjkgbnNpZDoxIGxiYTo5MDQzMDM2MTYgbGVuOjE2MApudm1lMjogV1JJVEUgRkFV TFRTICgwMi84MCkgc3FpZDoxNSBjaWQ6NjkgY2R3MDowCm52bWUyOiBhc3luYyBldmVudCBvY2N1 cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQooYWRhMzphaGNpY2gzOjA6MDow KTogV1JJVEVfRlBETUFfUVVFVUVELiBBQ0I6IDYxIDkwIDMwIDFlIDEzIDQwIDA4IDAwIDAwIDAw IDAwIDAwCihhZGEzOmFoY2ljaDM6MDowOjApOiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMg aW52YWxpZAooYWRhMzphaGNpY2gzOjA6MDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9y Cm52bWUyOiBXUklURSBzcWlkOjE2IGNpZDo5NCBuc2lkOjEgbGJhOjIxODEwMjkxMDQgbGVuOjE2 Cm52bWUyOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE2IGNpZDo5NCBjZHcwOjAKbnZtZTI6 IFdSSVRFIHNxaWQ6MTYgY2lkOjgwIG5zaWQ6MSBsYmE6MjE4MTAyOTEyMCBsZW46NTYKbnZtZTI6 IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTYgY2lkOjgwIGNkdzA6MApudm1lMjogYXN5bmMg ZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKKGFkYTM6YWhj aWNoMzowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSBhMCAwMCA3NyBkYSA0MCAw NCAwMCAwMCAwMCAwMCAwMAooYWRhMzphaGNpY2gzOjA6MDowKTogQ0FNIHN0YXR1czogQ0NCIHJl cXVlc3Qgd2FzIGludmFsaWQKKGFkYTM6YWhjaWNoMzowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5 YWJsZSBlcnJvcgpudm1lMTogV1JJVEUgc3FpZDo5IGNpZDo5MyBuc2lkOjEgbGJhOjIxODEwMzE1 NjggbGVuOjExMgpudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo5IGNpZDo5MyBjZHcw OjAKbnZtZTE6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdl IDB4MDEpCihhZGExOmFoY2ljaDE6MDowOjApOiBXUklURV9GUERNQV9RVUVVRUQuIEFDQjogNjEg NDAgZjAgMmEgZTYgNDAgMDQgMDAgMDAgMDAgMDAgMDAKKGFkYTE6YWhjaWNoMTowOjA6MCk6IENB TSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlkCihhZGExOmFoY2ljaDE6MDowOjApOiBF cnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKbnZtZTA6IFdSSVRFIHNxaWQ6MTAgY2lkOjkyIG5z aWQ6MSBsYmE6MjE4MTAzMjQ4MCBsZW46MTM2Cm52bWUwOiBXUklURSBGQVVMVFMgKDAyLzgwKSBz cWlkOjEwIGNpZDo5MiBjZHcwOjAKbnZtZTA6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0eXBlIDB4 MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUxOiBXUklURSBzcWlkOjkgY2lkOjk1IG5zaWQ6 MSBsYmE6MjE4MTAzNTI1NiBsZW46OApudm1lMTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo5 IGNpZDo5NSBjZHcwOjAKbnZtZTE6IFdSSVRFIHNxaWQ6OSBjaWQ6NzkgbnNpZDoxIGxiYToyMTgx MDM1MjY0IGxlbjoxNDQKbnZtZTE6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6OSBjaWQ6Nzkg Y2R3MDowCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwg cGFnZSAweDAxKQooYWRhMTphaGNpY2gxOjA6MDowKTogV1JJVEVfRlBETUFfUVVFVUVELiBBQ0I6 IDYxIDUwIGUwIDZmIDYzIDQwIDA3IDAwIDAwIDAwIDAwIDAwCihhZGExOmFoY2ljaDE6MDowOjAp OiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZAooYWRhMTphaGNpY2gxOjA6MDow KTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihhZGExOmFoY2ljaDE6MDowOjApOiBXUklU RV9GUERNQV9RVUVVRUQuIEFDQjogNjEgNTAgZjAgN2UgZGEgNDAgMDQgMDAgMDAgMDAgMDAgMDAK KGFkYTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlk CihhZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGFkYTE6 YWhjaWNoMTowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSA0MCAyMCAzMCBlNiA0 MCAwNCAwMCAwMCAwMCAwMCAwMAooYWRhMTphaGNpY2gxOjA6MDowKTogQ0FNIHN0YXR1czogQ0NC IHJlcXVlc3Qgd2FzIGludmFsaWQKKGFkYTE6YWhjaWNoMTowOjA6MCk6IEVycm9yIDIyLCBVbnJl dHJ5YWJsZSBlcnJvcgooYWRhMTphaGNpY2gxOjA6MDowKTogV1JJVEVfRlBETUFfUVVFVUVELiBB Q0I6IDYxIDUwIGUwIDBmIGRhIDQwIDA1IDAwIDAwIDAwIDAwIDAwCihhZGExOmFoY2ljaDE6MDow OjApOiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZAooYWRhMTphaGNpY2gxOjA6 MDowKTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihhZGExOmFoY2ljaDE6MDowOjApOiBX UklURV9GUERNQV9RVUVVRUQuIEFDQjogNjEgNDAgMjAgODQgMjUgNDAgMDYgMDAgMDAgMDAgMDAg MDAKKGFkYTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZh bGlkCihhZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKbnZt ZTI6IFdSSVRFIHNxaWQ6MTMgY2lkOjExOSBuc2lkOjEgbGJhOjIyMzcwNTI4MTYgbGVuOjcyCm52 bWUyOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjEzIGNpZDoxMTkgY2R3MDowCm52bWUyOiBh c3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1l MjogV1JJVEUgc3FpZDoxMiBjaWQ6NjggbnNpZDoxIGxiYTo5MDQzMjUyNTYgbGVuOjEyMApudm1l MjogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMiBjaWQ6NjggY2R3MDowCm52bWUyOiBhc3lu YyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMTog V1JJVEUgc3FpZDoxMCBjaWQ6MTE0IG5zaWQ6MSBsYmE6NjI4NDU1Nzc2IGxlbjo2NApudm1lMTog V1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMCBjaWQ6MTE0IGNkdzA6MApudm1lMTogYXN5bmMg ZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKKGFkYTE6YWhj aWNoMTowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSA0MCA5MCAzMyBlNiA0MCAw NCAwMCAwMCAwMCAwMCAwMAooYWRhMTphaGNpY2gxOjA6MDowKTogQ0FNIHN0YXR1czogQ0NCIHJl cXVlc3Qgd2FzIGludmFsaWQKKGFkYTE6YWhjaWNoMTowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5 YWJsZSBlcnJvcgooYWRhMzphaGNpY2gzOjA6MDowKTogV1JJVEVfRlBETUFfUVVFVUVELiBBQ0I6 IDYxIDUwIGIwIDE1IGRhIDQwIDA1IDAwIDAwIDAwIDAwIDAwCihhZGEzOmFoY2ljaDM6MDowOjAp OiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZAooYWRhMzphaGNpY2gzOjA6MDow KTogRXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihhZGExOmFoY2ljaDE6MDowOjApOiBXUklU RV9GUERNQV9RVUVVRUQuIEFDQjogNjEgYTAgYTAgZDQgNzIgNDAgMDYgMDAgMDAgMDAgMDAgMDAK KGFkYTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlk CihhZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKbnZtZTI6 IFdSSVRFIHNxaWQ6OCBjaWQ6ODIgbnNpZDoxIGxiYTo4NTE4NzIxMzYgbGVuOjQwCm52bWUyOiBX UklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjggY2lkOjgyIGNkdzA6MApudm1lMjogYXN5bmMgZXZl bnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKKGFkYTE6YWhjaWNo MTowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSA0MCA0MCAzOCBlNiA0MCAwNCAw MCAwMCAwMCAwMCAwMAooYWRhMTphaGNpY2gxOjA6MDowKTogQ0FNIHN0YXR1czogQ0NCIHJlcXVl c3Qgd2FzIGludmFsaWQKKGFkYTE6YWhjaWNoMTowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJs ZSBlcnJvcgooYWRhMTphaGNpY2gxOjA6MDowKTogV1JJVEVfRlBETUFfUVVFVUVELiBBQ0I6IDYx IDYwIGEwIDE3IGRhIDQwIDA1IDAwIDAwIDAwIDAwIDAwCihhZGExOmFoY2ljaDE6MDowOjApOiBD QU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZAooYWRhMTphaGNpY2gxOjA6MDowKTog RXJyb3IgMjIsIFVucmV0cnlhYmxlIGVycm9yCihhZGExOmFoY2ljaDE6MDowOjApOiBXUklURV9G UERNQV9RVUVVRUQuIEFDQjogNjEgNTAgODAgOGIgMjUgNDAgMDYgMDAgMDAgMDAgMDAgMDAKKGFk YTE6YWhjaWNoMTowOjA6MCk6IENBTSBzdGF0dXM6IENDQiByZXF1ZXN0IHdhcyBpbnZhbGlkCihh ZGExOmFoY2ljaDE6MDowOjApOiBFcnJvciAyMiwgVW5yZXRyeWFibGUgZXJyb3IKKGFkYTE6YWhj aWNoMTowOjA6MCk6IFdSSVRFX0ZQRE1BX1FVRVVFRC4gQUNCOiA2MSAzMCAyMCBkNSAxYSA0MCAw NiAwMCAwMCAwMCAwMCAwMAooYWRhMTphaGNpY2gxOjA6MDowKTogQ0FNIHN0YXR1czogQ0NCIHJl cXVlc3Qgd2FzIGludmFsaWQKKGFkYTE6YWhjaWNoMTowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5 YWJsZSBlcnJvcgpudm1lMjogV1JJVEUgc3FpZDo4IGNpZDo3NSBuc2lkOjEgbGJhOjIwNTgwMTk0 NTYgbGVuOjQwCm52bWUyOiBXUklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjggY2lkOjc1IGNkdzA6 MApudm1lMDogV1JJVEUgc3FpZDo4IGNpZDoxMTAgbnNpZDoxIGxiYToyMDU4MDE5NDU2IGxlbjo0 MApudm1lMDogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDo4IGNpZDoxMTAgY2R3MDowCm52bWUy OiBhc3luYyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpu dm1lMDogYXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgw MSkKbnZtZTI6IFdSSVRFIHNxaWQ6MTEgY2lkOjczIG5zaWQ6MSBsYmE6MjA1ODAyMjQ0OCBsZW46 NDAKbnZtZTI6IFdSSVRFIEZBVUxUUyAoMDIvODApIHNxaWQ6MTEgY2lkOjczIGNkdzA6MApudm1l MTogV1JJVEUgc3FpZDoxMSBjaWQ6ODIgbnNpZDoxIGxiYToyMDU4MDIyNDQ4IGxlbjo0MApudm1l MTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMSBjaWQ6ODIgY2R3MDowCm52bWUxOiBhc3lu YyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMjog YXN5bmMgZXZlbnQgb2NjdXJyZWQgKHR5cGUgMHgwLCBpbmZvIDB4MDQsIHBhZ2UgMHgwMSkKbnZt ZTE6IFdSSVRFIHNxaWQ6MTMgY2lkOjY0IG5zaWQ6MSBsYmE6ODc2NTg3NjU2IGxlbjo3Mgpudm1l MTogV1JJVEUgRkFVTFRTICgwMi84MCkgc3FpZDoxMyBjaWQ6NjQgY2R3MDowCm52bWUxOiBhc3lu YyBldmVudCBvY2N1cnJlZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQpudm1lMDog V1JJVEUgc3FpZDoxNCBjaWQ6NzMgbnNpZDoxIGxiYTo2MDQwODkzNDQgbGVuOjQwCm52bWUwOiBX UklURSBGQVVMVFMgKDAyLzgwKSBzcWlkOjE0IGNpZDo3MyBjZHcwOjAKbnZtZTE6IFdSSVRFIHNx aWQ6MTUgY2lkOjExNiBuc2lkOjEgbGJhOjYwNDA4OTMzNiBsZW46OApudm1lMTogV1JJVEUgRkFV TFRTICgwMi84MCkgc3FpZDoxNSBjaWQ6MTE2IGNkdzA6MApudm1lMTogV1JJVEUgc3FpZDoxNSBj aWQ6OTAgbnNpZDoxIGxiYTo2MDQwODkzNDQgbGVuOjMyCm52bWUxOiBXUklURSBGQVVMVFMgKDAy LzgwKSBzcWlkOjE1IGNpZDo5MCBjZHcwOjAKbnZtZTA6IGFzeW5jIGV2ZW50IG9jY3VycmVkICh0 eXBlIDB4MCwgaW5mbyAweDA0LCBwYWdlIDB4MDEpCm52bWUxOiBhc3luYyBldmVudCBvY2N1cnJl ZCAodHlwZSAweDAsIGluZm8gMHgwNCwgcGFnZSAweDAxKQotLS08PEJPT1Q+Pi0tLQpDb3B5cmln aHQgKGMpIDE5OTItMjAxOSBUaGUgRnJlZUJTRCBQcm9qZWN0LgpDb3B5cmlnaHQgKGMpIDE5Nzks IDE5ODAsIDE5ODMsIDE5ODYsIDE5ODgsIDE5ODksIDE5OTEsIDE5OTIsIDE5OTMsIDE5OTQKCVRo ZSBSZWdlbnRzIG9mIHRoZSBVbml2ZXJzaXR5IG9mIENhbGlmb3JuaWEuIEFsbCByaWdodHMgcmVz ZXJ2ZWQuCkZyZWVCU0QgaXMgYSByZWdpc3RlcmVkIHRyYWRlbWFyayBvZiBUaGUgRnJlZUJTRCBG b3VuZGF0aW9uLgpGcmVlQlNEIDEyLjEtUkVMRUFTRSByMzU0MjMzIEdFTkVSSUMgYW1kNjQKRnJl ZUJTRCBjbGFuZyB2ZXJzaW9uIDguMC4xICh0YWdzL1JFTEVBU0VfODAxL2ZpbmFsIDM2NjU4MSkg KGJhc2VkIG9uIExMVk0gOC4wLjEpClZUKHZnYSk6IHJlc29sdXRpb24gNjQweDQ4MApDUFU6IElu dGVsKFIpIFhlb24oUikgQ1BVIEU1LTI2ODdXIHYyIEAgMy40MEdIeiAoMzQwMC4wNy1NSHogSzgt Y2xhc3MgQ1BVKQogIE9yaWdpbj0iR2VudWluZUludGVsIiAgSWQ9MHgzMDZlNCAgRmFtaWx5PTB4 NiAgTW9kZWw9MHgzZSAgU3RlcHBpbmc9NAogIEZlYXR1cmVzPTB4YmZlYmZiZmY8RlBVLFZNRSxE RSxQU0UsVFNDLE1TUixQQUUsTUNFLENYOCxBUElDLFNFUCxNVFJSLFBHRSxNQ0EsQ01PVixQQVQs UFNFMzYsQ0xGTFVTSCxEVFMsQUNQSSxNTVgsRlhTUixTU0UsU1NFMixTUyxIVFQsVE0sUEJFPgog IEZlYXR1cmVzMj0weDdmYmVlM2ZmPFNTRTMsUENMTVVMUURRLERURVM2NCxNT04sRFNfQ1BMLFZN WCxTTVgsRVNULFRNMixTU1NFMyxDWDE2LHhUUFIsUERDTSxQQ0lELERDQSxTU0U0LjEsU1NFNC4y LHgyQVBJQyxQT1BDTlQsVFNDRExULEFFU05JLFhTQVZFLE9TWFNBVkUsQVZYLEYxNkMsUkRSQU5E PgogIEFNRCBGZWF0dXJlcz0weDJjMTAwODAwPFNZU0NBTEwsTlgsUGFnZTFHQixSRFRTQ1AsTE0+ CiAgQU1EIEZlYXR1cmVzMj0weDE8TEFIRj4KICBTdHJ1Y3R1cmVkIEV4dGVuZGVkIEZlYXR1cmVz PTB4MjgxPEZTR1NCQVNFLFNNRVAsRVJNUz4KICBTdHJ1Y3R1cmVkIEV4dGVuZGVkIEZlYXR1cmVz Mz0weDljMDAwMDAwPElCUEIsU1RJQlAsTDFERkwsU1NCRD4KICBYU0FWRSBGZWF0dXJlcz0weDE8 WFNBVkVPUFQ+CiAgVlQteDogUEFULEhMVCxNVEYsUEFVU0UsRVBULFVHLFZQSUQsVklELFBvc3RJ bnRyCiAgVFNDOiBQLXN0YXRlIGludmFyaWFudCwgcGVyZm9ybWFuY2Ugc3RhdGlzdGljcwpyZWFs IG1lbW9yeSAgPSAyNzQ4Nzc5MDY5NDQgKDI2MjE0NCBNQikKYXZhaWwgbWVtb3J5ID0gMjY3ODIx NjU0MDE2ICgyNTU0MTQgTUIpCkV2ZW50IHRpbWVyICJMQVBJQyIgcXVhbGl0eSA2MDAKQUNQSSBB UElDIFRhYmxlOiA8QUxBU0tBIEEgTSBJPgpGcmVlQlNEL1NNUDogTXVsdGlwcm9jZXNzb3IgU3lz dGVtIERldGVjdGVkOiAzMiBDUFVzCkZyZWVCU0QvU01QOiAyIHBhY2thZ2UocykgeCA4IGNvcmUo cykgeCAyIGhhcmR3YXJlIHRocmVhZHMKcmFuZG9tOiB1bmJsb2NraW5nIGRldmljZS4KaW9hcGlj MCA8VmVyc2lvbiAyLjA+IGlycXMgMC0yMyBvbiBtb3RoZXJib2FyZAppb2FwaWMxIDxWZXJzaW9u IDIuMD4gaXJxcyAyNC00NyBvbiBtb3RoZXJib2FyZAppb2FwaWMyIDxWZXJzaW9uIDIuMD4gaXJx cyA0OC03MSBvbiBtb3RoZXJib2FyZApMYXVuY2hpbmcgQVBzOiAzMSAxNiA4IDIxIDE1IDcgMTIg MTMgNCAyMyAxMCAyOSAyIDI1IDI3IDE3IDYgMSAyNiAyOCAxMSAxOCA1IDMgMjQgMjAgOSAxOSAz MCAxNCAyMgpUaW1lY291bnRlciAiVFNDLWxvdyIgZnJlcXVlbmN5IDE3MDAwMzQ4NTAgSHogcXVh bGl0eSAxMDAwCnJhbmRvbTogZW50cm9weSBkZXZpY2UgZXh0ZXJuYWwgaW50ZXJmYWNlCjAwMC4w MDAwMTcgWzQzMzVdIG5ldG1hcF9pbml0ICAgICAgICAgICAgICAgbmV0bWFwOiBsb2FkZWQgbW9k dWxlClthdGhfaGFsXSBsb2FkZWQKbW9kdWxlX3JlZ2lzdGVyX2luaXQ6IE1PRF9MT0FEICh2ZXNh LCAweGZmZmZmZmZmODExMmUwNTAsIDApIGVycm9yIDE5CnJhbmRvbTogcmVnaXN0ZXJpbmcgZmFz dCBzb3VyY2UgSW50ZWwgU2VjdXJlIEtleSBSTkcKcmFuZG9tOiBmYXN0IHByb3ZpZGVyOiAiSW50 ZWwgU2VjdXJlIEtleSBSTkciCmtiZDEgYXQga2JkbXV4MApuZXh1czAKdnR2Z2EwOiA8VlQgVkdB IGRyaXZlcj4gb24gbW90aGVyYm9hcmQKY3J5cHRvc29mdDA6IDxzb2Z0d2FyZSBjcnlwdG8+IG9u IG1vdGhlcmJvYXJkCmFjcGkwOiA8QUxBU0tBIEEgTSBJPiBvbiBtb3RoZXJib2FyZAphY3BpMDog UG93ZXIgQnV0dG9uIChmaXhlZCkKY3B1MDogPEFDUEkgQ1BVPiBudW1hLWRvbWFpbiAwIG9uIGFj cGkwCmF0dGltZXIwOiA8QVQgdGltZXI+IHBvcnQgMHg0MC0weDQzIGlycSAwIG9uIGFjcGkwClRp bWVjb3VudGVyICJpODI1NCIgZnJlcXVlbmN5IDExOTMxODIgSHogcXVhbGl0eSAwCkV2ZW50IHRp bWVyICJpODI1NCIgZnJlcXVlbmN5IDExOTMxODIgSHogcXVhbGl0eSAxMDAKYXRydGMwOiA8QVQg cmVhbHRpbWUgY2xvY2s+IHBvcnQgMHg3MC0weDcxIGlycSA4IG9uIGFjcGkwCmF0cnRjMDogcmVn aXN0ZXJlZCBhcyBhIHRpbWUtb2YtZGF5IGNsb2NrLCByZXNvbHV0aW9uIDEuMDAwMDAwcwpFdmVu dCB0aW1lciAiUlRDIiBmcmVxdWVuY3kgMzI3NjggSHogcXVhbGl0eSAwCmhwZXQwOiA8SGlnaCBQ cmVjaXNpb24gRXZlbnQgVGltZXI+IGlvbWVtIDB4ZmVkMDAwMDAtMHhmZWQwMDNmZiBvbiBhY3Bp MApUaW1lY291bnRlciAiSFBFVCIgZnJlcXVlbmN5IDE0MzE4MTgwIEh6IHF1YWxpdHkgOTUwCkV2 ZW50IHRpbWVyICJIUEVUIiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSAzNTAKRXZlbnQg dGltZXIgIkhQRVQxIiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSAzNDAKRXZlbnQgdGlt ZXIgIkhQRVQyIiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSAzNDAKRXZlbnQgdGltZXIg IkhQRVQzIiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSAzNDAKRXZlbnQgdGltZXIgIkhQ RVQ0IiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSAzNDAKRXZlbnQgdGltZXIgIkhQRVQ1 IiBmcmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSAzNDAKRXZlbnQgdGltZXIgIkhQRVQ2IiBm cmVxdWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSAzNDAKRXZlbnQgdGltZXIgIkhQRVQ3IiBmcmVx dWVuY3kgMTQzMTgxODAgSHogcXVhbGl0eSAzNDAKVGltZWNvdW50ZXIgIkFDUEktZmFzdCIgZnJl cXVlbmN5IDM1Nzk1NDUgSHogcXVhbGl0eSA5MDAKYWNwaV90aW1lcjA6IDwyNC1iaXQgdGltZXIg YXQgMy41Nzk1NDVNSHo+IHBvcnQgMHg0MDgtMHg0MGIgb24gYWNwaTAKcGNpYjA6IDxBQ1BJIEhv c3QtUENJIGJyaWRnZT4gcG9ydCAweGNmOC0weGNmZiBudW1hLWRvbWFpbiAwIG9uIGFjcGkwCnBj aTA6IDxBQ1BJIFBDSSBidXM+IG51bWEtZG9tYWluIDAgb24gcGNpYjAKcGNpYjE6IDxBQ1BJIFBD SS1QQ0kgYnJpZGdlPiBpcnEgMjYgYXQgZGV2aWNlIDEuMCBudW1hLWRvbWFpbiAwIG9uIHBjaTAK cGNpMTogPEFDUEkgUENJIGJ1cz4gbnVtYS1kb21haW4gMCBvbiBwY2liMQptcHMwOiA8QXZhZ28g VGVjaG5vbG9naWVzIChMU0kpIFNBUzIzMDg+IHBvcnQgMHg4MDAwLTB4ODBmZiBtZW0gMHhkZjA0 MDAwMC0weGRmMDRmZmZmLDB4ZGYwMDAwMDAtMHhkZjAzZmZmZiBpcnEgMTYgYXQgZGV2aWNlIDAu MCBudW1hLWRvbWFpbiAwIG9uIHBjaTEKbXBzMDogRmlybXdhcmU6IDIwLjAwLjA3LjAwLCBEcml2 ZXI6IDIxLjAyLjAwLjAwLWZic2QKbXBzMDogSU9DQ2FwYWJpbGl0aWVzOiA1YTg1YzxTY3NpVGFz a0Z1bGwsRGlhZ1RyYWNlLFNuYXBCdWYsRUVEUCxUcmFuc1JldHJ5LEV2ZW50UmVwbGF5LE1TSVhJ bmRleCxIb3N0RGlzYz4KcGNpYjI6IDxBQ1BJIFBDSS1QQ0kgYnJpZGdlPiBpcnEgMzIgYXQgZGV2 aWNlIDIuMCBudW1hLWRvbWFpbiAwIG9uIHBjaTAKcGNpMjogPEFDUEkgUENJIGJ1cz4gbnVtYS1k b21haW4gMCBvbiBwY2liMgp0NW5leDA6IDxDaGVsc2lvIFQ1MjAtQ1I+IG1lbSAweGRkMzAwMDAw LTB4ZGQzN2ZmZmYsMHhkYzAwMDAwMC0weGRjZmZmZmZmLDB4ZGRiMDQwMDAtMHhkZGIwNWZmZiBp cnEgMzIgYXQgZGV2aWNlIDAuNCBudW1hLWRvbWFpbiAwIG9uIHBjaTIKY3hsMDogPHBvcnQgMD4g bnVtYS1kb21haW4gMCBvbiB0NW5leDAKY3hsMDogRXRoZXJuZXQgYWRkcmVzczogMDA6MDc6NDM6 M2E6YWQ6MTAKY3hsMDogMTYgdHhxLCA4IHJ4cSAoTklDKTsgOCB0eHEgKFRPRSksIDIgcnhxIChU T0UpCmN4bDE6IDxwb3J0IDE+IG51bWEtZG9tYWluIDAgb24gdDVuZXgwCmN4bDE6IEV0aGVybmV0 IGFkZHJlc3M6IDAwOjA3OjQzOjNhOmFkOjE4CmN4bDE6IDE2IHR4cSwgOCByeHEgKE5JQyk7IDgg dHhxIChUT0UpLCAyIHJ4cSAoVE9FKQp0NW5leDA6IFBDSWUgZ2VuMyB4OCwgMiBwb3J0cywgMjIg TVNJLVggaW50ZXJydXB0cywgNzAgZXEsIDIxIGlxCnBjaTI6IDxtYXNzIHN0b3JhZ2UsIFNDU0k+ IGF0IGRldmljZSAwLjUgKG5vIGRyaXZlciBhdHRhY2hlZCkKcGNpMjogPHNlcmlhbCBidXMsIEZp YnJlIENoYW5uZWw+IGF0IGRldmljZSAwLjYgKG5vIGRyaXZlciBhdHRhY2hlZCkKcGNpYjM6IDxB Q1BJIFBDSS1QQ0kgYnJpZGdlPiBpcnEgMzIgYXQgZGV2aWNlIDIuMiBudW1hLWRvbWFpbiAwIG9u IHBjaTAKcGNpMzogPEFDUEkgUENJIGJ1cz4gbnVtYS1kb21haW4gMCBvbiBwY2liMwpwcHQwIG1l bSAweGRmMzAwMDAwLTB4ZGYzMDNmZmYsMHhkZjMwNDAwMC0weGRmMzA0MGZmIGlycSAzNCBhdCBk ZXZpY2UgMC4wIG51bWEtZG9tYWluIDAgb24gcGNpMwpwY2liNDogPEFDUEkgUENJLVBDSSBicmlk Z2U+IGlycSA0MCBhdCBkZXZpY2UgMy4wIG51bWEtZG9tYWluIDAgb24gcGNpMApwY2k0OiA8QUNQ SSBQQ0kgYnVzPiBudW1hLWRvbWFpbiAwIG9uIHBjaWI0CnBwdDEgbWVtIDB4ZGI4MDAwMDAtMHhk YjlmZmZmZiwweGRiYTA0MDAwLTB4ZGJhMDdmZmYgaXJxIDQ0IGF0IGRldmljZSAwLjAgbnVtYS1k b21haW4gMCBvbiBwY2k0CnBwdDIgbWVtIDB4ZGI2MDAwMDAtMHhkYjdmZmZmZiwweGRiYTAwMDAw LTB4ZGJhMDNmZmYgaXJxIDQwIGF0IGRldmljZSAwLjEgbnVtYS1kb21haW4gMCBvbiBwY2k0CnBj aWI1OiA8QUNQSSBQQ0ktUENJIGJyaWRnZT4gaXJxIDQwIGF0IGRldmljZSAzLjIgbnVtYS1kb21h aW4gMCBvbiBwY2kwCnBjaTU6IDxBQ1BJIFBDSSBidXM+IG51bWEtZG9tYWluIDAgb24gcGNpYjUK cHB0MyBwb3J0IDB4NzAyMC0weDcwM2YgbWVtIDB4ZGYyMjAwMDAtMHhkZjIzZmZmZiwweGRmMmM0 MDAwLTB4ZGYyYzdmZmYgaXJxIDQyIGF0IGRldmljZSAwLjAgbnVtYS1kb21haW4gMCBvbiBwY2k1 CnBwdDQgcG9ydCAweDcwMDAtMHg3MDFmIG1lbSAweGRmMjAwMDAwLTB4ZGYyMWZmZmYsMHhkZjJj MDAwMC0weGRmMmMzZmZmIGlycSA0NSBhdCBkZXZpY2UgMC4xIG51bWEtZG9tYWluIDAgb24gcGNp NQpwY2liNjogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGlycSAxNiBhdCBkZXZpY2UgMTcuMCBudW1h LWRvbWFpbiAwIG9uIHBjaTAKcGNpNjogPEFDUEkgUENJIGJ1cz4gbnVtYS1kb21haW4gMCBvbiBw Y2liNgppc2NpMDogPEludGVsKFIpIEM2MDAgU2VyaWVzIENoaXBzZXQgU0FTIENvbnRyb2xsZXIg KFNBVEEgbW9kZSk+IHBvcnQgMHg2MDAwLTB4NjBmZiBtZW0gMHhkYjQ3YzAwMC0weGRiNDdmZmZm LDB4ZGIwMDAwMDAtMHhkYjNmZmZmZiBpcnEgMTYgYXQgZGV2aWNlIDAuMCBudW1hLWRvbWFpbiAw IG9uIHBjaTYKcGNpMDogPHNpbXBsZSBjb21tcz4gYXQgZGV2aWNlIDIyLjAgKG5vIGRyaXZlciBh dHRhY2hlZCkKcGNpMDogPHNpbXBsZSBjb21tcz4gYXQgZGV2aWNlIDIyLjEgKG5vIGRyaXZlciBh dHRhY2hlZCkKZWhjaTA6IDxJbnRlbCBQYXRzYnVyZyBVU0IgMi4wIGNvbnRyb2xsZXI+IG1lbSAw eGRkYzI0MDAwLTB4ZGRjMjQzZmYgaXJxIDE2IGF0IGRldmljZSAyNi4wIG51bWEtZG9tYWluIDAg b24gcGNpMAp1c2J1czA6IEVIQ0kgdmVyc2lvbiAxLjAKdXNidXMwIG51bWEtZG9tYWluIDAgb24g ZWhjaTAKdXNidXMwOiA0ODBNYnBzIEhpZ2ggU3BlZWQgVVNCIHYyLjAKZWhjaTE6IDxJbnRlbCBQ YXRzYnVyZyBVU0IgMi4wIGNvbnRyb2xsZXI+IG1lbSAweGRkYzIzMDAwLTB4ZGRjMjMzZmYgaXJx IDIzIGF0IGRldmljZSAyOS4wIG51bWEtZG9tYWluIDAgb24gcGNpMAp1c2J1czE6IEVIQ0kgdmVy c2lvbiAxLjAKdXNidXMxIG51bWEtZG9tYWluIDAgb24gZWhjaTEKdXNidXMxOiA0ODBNYnBzIEhp Z2ggU3BlZWQgVVNCIHYyLjAKcGNpYjc6IDxBQ1BJIFBDSS1QQ0kgYnJpZGdlPiBhdCBkZXZpY2Ug MzAuMCBudW1hLWRvbWFpbiAwIG9uIHBjaTAKcGNpNzogPEFDUEkgUENJIGJ1cz4gbnVtYS1kb21h aW4gMCBvbiBwY2liNwp2Z2FwY2kwOiA8VkdBLWNvbXBhdGlibGUgZGlzcGxheT4gbWVtIDB4ZGEw MDAwMDAtMHhkYWZmZmZmZiwweGRlODAwMDAwLTB4ZGU4MDNmZmYsMHhkZTAwMDAwMC0weGRlN2Zm ZmZmIGlycSAxNiBhdCBkZXZpY2UgMy4wIG51bWEtZG9tYWluIDAgb24gcGNpNwp2Z2FwY2kwOiBC b290IHZpZGVvIGRldmljZQppc2FiMDogPFBDSS1JU0EgYnJpZGdlPiBhdCBkZXZpY2UgMzEuMCBu dW1hLWRvbWFpbiAwIG9uIHBjaTAKaXNhMDogPElTQSBidXM+IG51bWEtZG9tYWluIDAgb24gaXNh YjAKYWhjaTA6IDxJbnRlbCBQYXRzYnVyZyBBSENJIFNBVEEgY29udHJvbGxlcj4gcG9ydCAweDkw NzAtMHg5MDc3LDB4OTA2MC0weDkwNjMsMHg5MDUwLTB4OTA1NywweDkwNDAtMHg5MDQzLDB4OTAy MC0weDkwM2YgbWVtIDB4ZGRjMjIwMDAtMHhkZGMyMjdmZiBpcnEgMTkgYXQgZGV2aWNlIDMxLjIg bnVtYS1kb21haW4gMCBvbiBwY2kwCmFoY2kwOiBBSENJIHYxLjMwIHdpdGggNiA2R2JwcyBwb3J0 cywgUG9ydCBNdWx0aXBsaWVyIG5vdCBzdXBwb3J0ZWQKYWhjaWNoMDogPEFIQ0kgY2hhbm5lbD4g YXQgY2hhbm5lbCAwIG9uIGFoY2kwCmFoY2ljaDE6IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwg MSBvbiBhaGNpMAphaGNpY2gyOiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDIgb24gYWhjaTAK YWhjaWNoMzogPEFIQ0kgY2hhbm5lbD4gYXQgY2hhbm5lbCAzIG9uIGFoY2kwCmFoY2ljaDQ6IDxB SENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgNCBvbiBhaGNpMAphaGNpY2g1OiA8QUhDSSBjaGFubmVs PiBhdCBjaGFubmVsIDUgb24gYWhjaTAKYWhjaWVtMDogPEFIQ0kgZW5jbG9zdXJlIG1hbmFnZW1l bnQgYnJpZGdlPiBhdCBjaGFubmVsIDIxNDc0ODM2NDcgb24gYWhjaTAKZGV2aWNlX2F0dGFjaDog YWhjaWVtMCBhdHRhY2ggcmV0dXJuZWQgNgpwY2liODogPEFDUEkgSG9zdC1QQ0kgYnJpZGdlPiBu dW1hLWRvbWFpbiAwIG9uIGFjcGkwCnBjaTg6IDxBQ1BJIFBDSSBidXM+IG51bWEtZG9tYWluIDAg b24gcGNpYjgKcGNpODogPGRhc3AsIHBlcmZvcm1hbmNlIGNvdW50ZXJzPiBhdCBkZXZpY2UgMTQu MSAobm8gZHJpdmVyIGF0dGFjaGVkKQpwY2k4OiA8ZGFzcCwgcGVyZm9ybWFuY2UgY291bnRlcnM+ IGF0IGRldmljZSAxOS4xIChubyBkcml2ZXIgYXR0YWNoZWQpCnBjaTg6IDxkYXNwLCBwZXJmb3Jt YW5jZSBjb3VudGVycz4gYXQgZGV2aWNlIDE5LjUgKG5vIGRyaXZlciBhdHRhY2hlZCkKcGNpYjk6 IDxBQ1BJIEhvc3QtUENJIGJyaWRnZT4gbnVtYS1kb21haW4gMSBvbiBhY3BpMApwY2k5OiA8QUNQ SSBQQ0kgYnVzPiBudW1hLWRvbWFpbiAxIG9uIHBjaWI5CnBjaWIxMDogPEFDUEkgUENJLVBDSSBi cmlkZ2U+IGlycSA1MCBhdCBkZXZpY2UgMS4wIG51bWEtZG9tYWluIDEgb24gcGNpOQpwY2kxMDog PEFDUEkgUENJIGJ1cz4gbnVtYS1kb21haW4gMSBvbiBwY2liMTAKbnZtZTA6IDxHZW5lcmljIE5W TWUgRGV2aWNlPiBtZW0gMHhmYmUxMDAwMC0weGZiZTEzZmZmIGlycSA1MCBhdCBkZXZpY2UgMC4w IG51bWEtZG9tYWluIDEgb24gcGNpMTAKcGNpYjExOiA8QUNQSSBQQ0ktUENJIGJyaWRnZT4gaXJx IDU2IGF0IGRldmljZSAyLjAgbnVtYS1kb21haW4gMSBvbiBwY2k5CnBjaTExOiA8QUNQSSBQQ0kg YnVzPiBudW1hLWRvbWFpbiAxIG9uIHBjaWIxMQpudm1lMTogPEdlbmVyaWMgTlZNZSBEZXZpY2U+ IG1lbSAweGZiZDEwMDAwLTB4ZmJkMTNmZmYgaXJxIDU2IGF0IGRldmljZSAwLjAgbnVtYS1kb21h aW4gMSBvbiBwY2kxMQpwY2liMTI6IDxBQ1BJIFBDSS1QQ0kgYnJpZGdlPiBpcnEgNTYgYXQgZGV2 aWNlIDIuMiBudW1hLWRvbWFpbiAxIG9uIHBjaTkKcGNpMTI6IDxBQ1BJIFBDSSBidXM+IG51bWEt ZG9tYWluIDEgb24gcGNpYjEyCm52bWUyOiA8R2VuZXJpYyBOVk1lIERldmljZT4gbWVtIDB4ZmJj MTAwMDAtMHhmYmMxM2ZmZiBpcnEgNTggYXQgZGV2aWNlIDAuMCBudW1hLWRvbWFpbiAxIG9uIHBj aTEyCnBjaWIxMzogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGlycSA2NCBhdCBkZXZpY2UgMy4wIG51 bWEtZG9tYWluIDEgb24gcGNpOQpwY2kxMzogPEFDUEkgUENJIGJ1cz4gbnVtYS1kb21haW4gMSBv biBwY2liMTMKcHB0NSBwb3J0IDB4ZjA2MC0weGYwN2YgbWVtIDB4ZmJhODAwMDAtMHhmYmFmZmZm ZiwweGZiYjBjMDAwLTB4ZmJiMGZmZmYgaXJxIDY0IGF0IGRldmljZSAwLjAgbnVtYS1kb21haW4g MSBvbiBwY2kxMwpwcHQ2IHBvcnQgMHhmMDQwLTB4ZjA1ZiBtZW0gMHhmYmEwMDAwMC0weGZiYTdm ZmZmLDB4ZmJiMDgwMDAtMHhmYmIwYmZmZiBpcnEgNjggYXQgZGV2aWNlIDAuMSBudW1hLWRvbWFp biAxIG9uIHBjaTEzCnBwdDcgcG9ydCAweGYwMjAtMHhmMDNmIG1lbSAweGZiOTgwMDAwLTB4ZmI5 ZmZmZmYsMHhmYmIwNDAwMC0weGZiYjA3ZmZmIGlycSA2OSBhdCBkZXZpY2UgMC4yIG51bWEtZG9t YWluIDEgb24gcGNpMTMKcHB0OCBwb3J0IDB4ZjAwMC0weGYwMWYgbWVtIDB4ZmI5MDAwMDAtMHhm Yjk3ZmZmZiwweGZiYjAwMDAwLTB4ZmJiMDNmZmYgaXJxIDcwIGF0IGRldmljZSAwLjMgbnVtYS1k b21haW4gMSBvbiBwY2kxMwpwY2liMTQ6IDxBQ1BJIEhvc3QtUENJIGJyaWRnZT4gbnVtYS1kb21h aW4gMSBvbiBhY3BpMApwY2kxNDogPEFDUEkgUENJIGJ1cz4gbnVtYS1kb21haW4gMSBvbiBwY2li MTQKcGNpMTQ6IDxkYXNwLCBwZXJmb3JtYW5jZSBjb3VudGVycz4gYXQgZGV2aWNlIDE0LjEgKG5v IGRyaXZlciBhdHRhY2hlZCkKcGNpMTQ6IDxkYXNwLCBwZXJmb3JtYW5jZSBjb3VudGVycz4gYXQg ZGV2aWNlIDE5LjEgKG5vIGRyaXZlciBhdHRhY2hlZCkKcGNpMTQ6IDxkYXNwLCBwZXJmb3JtYW5j ZSBjb3VudGVycz4gYXQgZGV2aWNlIDE5LjUgKG5vIGRyaXZlciBhdHRhY2hlZCkKYWNwaV9idXR0 b24wOiA8UG93ZXIgQnV0dG9uPiBvbiBhY3BpMAphY3BpX3N5c2NvbnRhaW5lcjA6IDxTeXN0ZW0g Q29udGFpbmVyPiBvbiBhY3BpMAphY3BpX3N5c2NvbnRhaW5lcjE6IDxTeXN0ZW0gQ29udGFpbmVy PiBvbiBhY3BpMAp1YXJ0MDogPDE2NTUwIG9yIGNvbXBhdGlibGU+IHBvcnQgMHgzZjgtMHgzZmYg aXJxIDQgZmxhZ3MgMHgxMCBvbiBhY3BpMAp1YXJ0MTogPDE2NTUwIG9yIGNvbXBhdGlibGU+IHBv cnQgMHgyZjgtMHgyZmYgaXJxIDMgb24gYWNwaTAKaXBtaTA6IDxJUE1JIFN5c3RlbSBJbnRlcmZh Y2U+IHBvcnQgMHhjYTIsMHhjYTMgb24gYWNwaTAKaXBtaTA6IEtDUyBtb2RlIGZvdW5kIGF0IGlv IDB4Y2EyIG9uIGFjcGkKb3JtMDogPElTQSBPcHRpb24gUk9NPiBhdCBpb21lbSAweGMwMDAwLTB4 YzdmZmYgcG5waWQgT1JNMDAwMCBvbiBpc2EwCmNvcmV0ZW1wMDogPENQVSBPbi1EaWUgVGhlcm1h bCBTZW5zb3JzPiBudW1hLWRvbWFpbiAwIG9uIGNwdTAKZXN0MDogPEVuaGFuY2VkIFNwZWVkU3Rl cCBGcmVxdWVuY3kgQ29udHJvbD4gbnVtYS1kb21haW4gMCBvbiBjcHUwClpGUyBmaWxlc3lzdGVt IHZlcnNpb246IDUKWkZTIHN0b3JhZ2UgcG9vbCB2ZXJzaW9uOiBmZWF0dXJlcyBzdXBwb3J0ICg1 MDAwKQpUaW1lY291bnRlcnMgdGljayBldmVyeSAxLjAwMCBtc2VjCnVnZW4wLjE6IDxJbnRlbCBF SENJIHJvb3QgSFVCPiBhdCB1c2J1czAKdWdlbjEuMTogPEludGVsIEVIQ0kgcm9vdCBIVUI+IGF0 IHVzYnVzMQp1aHViMDogPEludGVsIEVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDIuMDAv MS4wMCwgYWRkciAxPiBvbiB1c2J1czAKdWh1YjE6IDxJbnRlbCBFSENJIHJvb3QgSFVCLCBjbGFz cyA5LzAsIHJldiAyLjAwLzEuMDAsIGFkZHIgMT4gb24gdXNidXMxCm52ZDA6IDxNTzE2MDBLRUZI UT4gTlZNZSBuYW1lc3BhY2UKbnZkMDogMTUyNjE4NU1CICgzMTI1NjI3NTY4IDUxMiBieXRlIHNl Y3RvcnMpCm52ZDE6IDxNTzE2MDBLRUZIUT4gTlZNZSBuYW1lc3BhY2UKbnZkMTogMTUyNjE4NU1C ICgzMTI1NjI3NTY4IDUxMiBieXRlIHNlY3RvcnMpCm52ZDI6IDxNTzE2MDBLRUZIUT4gTlZNZSBu YW1lc3BhY2UKbnZkMjogMTUyNjE4NU1CICgzMTI1NjI3NTY4IDUxMiBieXRlIHNlY3RvcnMpCmlw bWkwOiBJUE1JIGRldmljZSByZXYuIDEsIGZpcm13YXJlIHJldi4gMy41NSwgdmVyc2lvbiAyLjAs IGRldmljZSBzdXBwb3J0IG1hc2sgMHhiZgppcG1pMDogTnVtYmVyIG9mIGNoYW5uZWxzIDIKaXBt aTA6IEF0dGFjaGVkIHdhdGNoZG9nCmlwbWkwOiBFc3RhYmxpc2hpbmcgcG93ZXIgY3ljbGUgaGFu ZGxlcgp1aHViMDogMiBwb3J0cyB3aXRoIDIgcmVtb3ZhYmxlLCBzZWxmIHBvd2VyZWQKdWh1YjE6 IDIgcG9ydHMgd2l0aCAyIHJlbW92YWJsZSwgc2VsZiBwb3dlcmVkCnVnZW4xLjI6IDx2ZW5kb3Ig MHg4MDg3IHByb2R1Y3QgMHgwMDI0PiBhdCB1c2J1czEKdWh1YjIgbnVtYS1kb21haW4gMCBvbiB1 aHViMQp1aHViMjogPHZlbmRvciAweDgwODcgcHJvZHVjdCAweDAwMjQsIGNsYXNzIDkvMCwgcmV2 IDIuMDAvMC4wMCwgYWRkciAyPiBvbiB1c2J1czEKdWdlbjAuMjogPHZlbmRvciAweDgwODcgcHJv ZHVjdCAweDAwMjQ+IGF0IHVzYnVzMAp1aHViMyBudW1hLWRvbWFpbiAwIG9uIHVodWIwCnVodWIz OiA8dmVuZG9yIDB4ODA4NyBwcm9kdWN0IDB4MDAyNCwgY2xhc3MgOS8wLCByZXYgMi4wMC8wLjAw LCBhZGRyIDI+IG9uIHVzYnVzMAp1aHViMzogNiBwb3J0cyB3aXRoIDYgcmVtb3ZhYmxlLCBzZWxm IHBvd2VyZWQKdWh1YjI6IDggcG9ydHMgd2l0aCA4IHJlbW92YWJsZSwgc2VsZiBwb3dlcmVkCnVn ZW4wLjM6IDxXaW5ib25kIEVsZWN0cm9uaWNzIENvcnAgSGVybW9uIFVTQiBoaWRtb3VzZSBEZXZp Y2U+IGF0IHVzYnVzMAp1a2JkMCBudW1hLWRvbWFpbiAwIG9uIHVodWIzCnVrYmQwOiA8V2luYm9u ZCBFbGVjdHJvbmljcyBDb3JwIEhlcm1vbiBVU0IgaGlkbW91c2UgRGV2aWNlLCBjbGFzcyAwLzAs IHJldiAxLjEwLzAuMDEsIGFkZHIgMz4gb24gdXNidXMwCmtiZDIgYXQgdWtiZDAKYWRhMCBhdCBh aGNpY2gwIGJ1cyAwIHNjYnVzMiB0YXJnZXQgMCBsdW4gMAphZGEwOiA8SU5URUwgU1NEU0MyQkI2 MDBHNCBEMjAxMjM3MD4gQUNTLTIgQVRBIFNBVEEgMy54IGRldmljZQphZGEwOiBTZXJpYWwgTnVt YmVyIAphZGEwOiA2MDAuMDAwTUIvcyB0cmFuc2ZlcnMgKFNBVEEgMy54LCBVRE1BNiwgUElPIDUx MmJ5dGVzKQphZGEwOiBDb21tYW5kIFF1ZXVlaW5nIGVuYWJsZWQKYWRhMDogNTcyMzI1TUIgKDEx NzIxMjM1NjggNTEyIGJ5dGUgc2VjdG9ycykKYWRhMSBhdCBhaGNpY2gxIGJ1cyAwIHNjYnVzMyB0 YXJnZXQgMCBsdW4gMAphZGExOiA8U0FNU1VORyBNQ0NPRTFIRzVNWFAtMFZCRDMgVkJDM0FEM1E+ IEFUQTgtQUNTIFNBVEEgMi54IGRldmljZQphZGExOiBTZXJpYWwgTnVtYmVyIAphZGExOiAzMDAu MDAwTUIvcyB0cmFuc2ZlcnMgKFNBVEEgMi54LCBVRE1BNiwgUElPIDQwOTZieXRlcykKYWRhMTog Q29tbWFuZCBRdWV1ZWluZyBlbmFibGVkCmFkYTE6IDk1Mzk2TUIgKDE5NTM3MTU2OCA1MTIgYnl0 ZSBzZWN0b3JzKQphZGEyIGF0IGFoY2ljaDIgYnVzIDAgc2NidXM0IHRhcmdldCAwIGx1biAwCmFk YTI6IDxWSzA0ODBHREpYViA0SVdWSFBHMT4gQVRBOC1BQ1MgU0FUQSAzLnggZGV2aWNlCmFkYTI6 IFNlcmlhbCBOdW1iZXIgCmFkYTI6IDMwMC4wMDBNQi9zIHRyYW5zZmVycyAoU0FUQSAyLngsIFVE TUE1LCBQSU8gNTEyYnl0ZXMpCmFkYTI6IENvbW1hbmQgUXVldWVpbmcgZW5hYmxlZAphZGEyOiA0 NTc4NjJNQiAoOTM3NzAzMDg4IDUxMiBieXRlIHNlY3RvcnMpCmFkYTMgYXQgYWhjaWNoMyBidXMg MCBzY2J1czUgdGFyZ2V0IDAgbHVuIDAKYWRhMzogPFNBTVNVTkcgTUNDT0UxSEc1TVhQLTBWQkQz IFZCQzNBRDNRPiBBVEE4LUFDUyBTQVRBIDIueCBkZXZpY2UKYWRhMzogU2VyaWFsIE51bWJlciAK YWRhMzogMzAwLjAwME1CL3MgdHJhbnNmZXJzIChTQVRBIDIueCwgVURNQTYsIFBJTyA0MDk2Ynl0 ZXMpCmFkYTM6IENvbW1hbmQgUXVldWVpbmcgZW5hYmxlZAphZGEzOiA5NTM5Nk1CICgxOTUzNzE1 NjggNTEyIGJ5dGUgc2VjdG9ycykKZGExMCBhdCBpc2NpMCBidXMgMCBzY2J1czEgdGFyZ2V0IDIg bHVuIDAKZGExMDogPEFUQSBUT1NISUJBIE1EMDVBQ0E4IEdYMEI+IEZpeGVkIERpcmVjdCBBY2Nl c3MgU1BDLTMgU0NTSSBkZXZpY2UKZGExMDogU2VyaWFsIE51bWJlciAKZGExMDogMzAwLjAwME1C L3MgdHJhbnNmZXJzCmRhMTA6IENvbW1hbmQgUXVldWVpbmcgZW5hYmxlZApkYTEwOiA3NjMwODg1 TUIgKDE1NjI4MDUzMTY4IDUxMiBieXRlIHNlY3RvcnMpCmRhOCBhdCBpc2NpMCBidXMgMCBzY2J1 czEgdGFyZ2V0IDAgbHVuIDAKZGE4OiA8QVRBIFRPU0hJQkEgTUQwNUFDQTggR1gwQj4gRml4ZWQg RGlyZWN0IEFjY2VzcyBTUEMtMyBTQ1NJIGRldmljZQpkYTg6IFNlcmlhbCBOdW1iZXIgCmRhODog MzAwLjAwME1CL3MgdHJhbnNmZXJzCmRhODogQ29tbWFuZCBRdWV1ZWluZyBlbmFibGVkCmRhODog NzYzMDg4NU1CICgxNTYyODA1MzE2OCA1MTIgYnl0ZSBzZWN0b3JzKQpkYTExIGF0IGlzY2kwIGJ1 cyAwIHNjYnVzMSB0YXJnZXQgMyBsdW4gMApkYTExOiA8QVRBIFRPU0hJQkEgTUQwNUFDQTggR1gw Qj4gRml4ZWQgRGlyZWN0IEFjY2VzcyBTUEMtMyBTQ1NJIGRldmljZQpkYTExOiBTZXJpYWwgTnVt YmVyIApkYTExOiAzMDAuMDAwTUIvcyB0cmFuc2ZlcnMKZGExMTogQ29tbWFuZCBRdWV1ZWluZyBl bmFibGVkCmRhMTE6IDc2MzA4ODVNQiAoMTU2MjgwNTMxNjggNTEyIGJ5dGUgc2VjdG9ycykKZGE5 IGF0IGlzY2kwIGJ1cyAwIHNjYnVzMSB0YXJnZXQgMSBsdW4gMApkYTk6IDxBVEEgVE9TSElCQSBN RDA1QUNBOCBHWDBCPiBGaXhlZCBEaXJlY3QgQWNjZXNzIFNQQy0zIFNDU0kgZGV2aWNlCmRhOTog U2VyaWFsIE51bWJlciAKZGE5OiAzMDAuMDAwTUIvcyB0cmFuc2ZlcnMKZGE5OiBDb21tYW5kIFF1 ZXVlaW5nIGVuYWJsZWQKZGE5OiA3NjMwODg1TUIgKDE1NjI4MDUzMTY4IDUxMiBieXRlIHNlY3Rv cnMpCmRhNyBhdCBtcHMwIGJ1cyAwIHNjYnVzMCB0YXJnZXQgNzEgbHVuIDAKZGE3OiA8QVRBIFRP U0hJQkEgTUQwNUFDQTggR1gwQj4gRml4ZWQgRGlyZWN0IEFjY2VzcyBTUEMtNCBTQ1NJIGRldmlj ZQpkYTc6IFNlcmlhbCBOdW1iZXIgCmRhNzogNjAwLjAwME1CL3MgdHJhbnNmZXJzCmRhNzogQ29t bWFuZCBRdWV1ZWluZyBlbmFibGVkCmRhNzogNzYzMDg4NU1CICgxNTYyODA1MzE2OCA1MTIgYnl0 ZSBzZWN0b3JzKQpkYTQgYXQgbXBzMCBidXMgMCBzY2J1czAgdGFyZ2V0IDY4IGx1biAwCmRhNDog PEFUQSBUT1NISUJBIE1EMDVBQ0E4IEdYMEI+IEZpeGVkIERpcmVjdCBBY2Nlc3MgU1BDLTQgU0NT SSBkZXZpY2UKZGE0OiBTZXJpYWwgTnVtYmVyIApkYTQ6IDYwMC4wMDBNQi9zIHRyYW5zZmVycwpk YTQ6IENvbW1hbmQgUXVldWVpbmcgZW5hYmxlZApkYTQ6IDc2MzA4ODVNQiAoMTU2MjgwNTMxNjgg NTEyIGJ5dGUgc2VjdG9ycykKZGEzIGF0IG1wczAgYnVzIDAgc2NidXMwIHRhcmdldCA2NyBsdW4g MApkYTM6IDxBVEEgVE9TSElCQSBNRDA1QUNBOCBHWDBCPiBGaXhlZCBEaXJlY3QgQWNjZXNzIFNQ Qy00IFNDU0kgZGV2aWNlCmRhMzogU2VyaWFsIE51bWJlciAKZGEzOiA2MDAuMDAwTUIvcyB0cmFu c2ZlcnMKZGEzOiBDb21tYW5kIFF1ZXVlaW5nIGVuYWJsZWQKZGEzOiA3NjMwODg1TUIgKDE1NjI4 MDUzMTY4IDUxMiBieXRlIHNlY3RvcnMpCmRhNSBhdCBtcHMwIGJ1cyAwIHNjYnVzMCB0YXJnZXQg NjkgbHVuIDAKZGE1OiA8QVRBIFRPU0hJQkEgTUQwNUFDQTggR1gwQj4gRml4ZWQgRGlyZWN0IEFj Y2VzcyBTUEMtNCBTQ1NJIGRldmljZQpkYTU6IFNlcmlhbCBOdW1iZXIgCmRhNTogNjAwLjAwME1C L3MgdHJhbnNmZXJzCmRhNTogQ29tbWFuZCBRdWV1ZWluZyBlbmFibGVkCmRhNTogNzYzMDg4NU1C ICgxNTYyODA1MzE2OCA1MTIgYnl0ZSBzZWN0b3JzKQpkYTYgYXQgbXBzMCBidXMgMCBzY2J1czAg dGFyZ2V0IDcwIGx1biAwCmRhNjogPEFUQSBUT1NISUJBIE1EMDVBQ0E4IEdYMEI+IEZpeGVkIERp cmVjdCBBY2Nlc3MgU1BDLTQgU0NTSSBkZXZpY2UKZGE2OiBTZXJpYWwgTnVtYmVyIApkYTY6IDYw MC4wMDBNQi9zIHRyYW5zZmVycwpkYTY6IENvbW1hbmQgUXVldWVpbmcgZW5hYmxlZApkYTY6IDc2 MzA4ODVNQiAoMTU2MjgwNTMxNjggNTEyIGJ5dGUgc2VjdG9ycykKZGExIGF0IG1wczAgYnVzIDAg c2NidXMwIHRhcmdldCA2NSBsdW4gMApkYTE6IDxBVEEgVE9TSElCQSBNRDA1QUNBOCBHWDBCPiBG aXhlZCBEaXJlY3QgQWNjZXNzIFNQQy00IFNDU0kgZGV2aWNlCmRhMTogU2VyaWFsIE51bWJlciAK ZGExOiA2MDAuMDAwTUIvcyB0cmFuc2ZlcnMKZGExOiBDb21tYW5kIFF1ZXVlaW5nIGVuYWJsZWQK ZGExOiA3NjMwODg1TUIgKDE1NjI4MDUzMTY4IDUxMiBieXRlIHNlY3RvcnMpCmRhMCBhdCBtcHMw IGJ1cyAwIHNjYnVzMCB0YXJnZXQgNjQgbHVuIDAKZGEwOiA8QVRBIFRPU0hJQkEgTUQwNUFDQTgg R1gwQj4gRml4ZWQgRGlyZWN0IEFjY2VzcyBTUEMtNCBTQ1NJIGRldmljZQpkYTA6IFNlcmlhbCBO dW1iZXIgCmRhMDogNjAwLjAwME1CL3MgdHJhbnNmZXJzCmRhMDogQ29tbWFuZCBRdWV1ZWluZyBl bmFibGVkCmRhMDogNzYzMDg4NU1CICgxNTYyODA1MzE2OCA1MTIgYnl0ZSBzZWN0b3JzKQpkYTIg YXQgbXBzMCBidXMgMCBzY2J1czAgdGFyZ2V0IDY2IGx1biAwCmRhMjogPEFUQSBUT1NISUJBIE1E MDVBQ0E4IEdYMEI+IEZpeGVkIERpcmVjdCBBY2Nlc3MgU1BDLTQgU0NTSSBkZXZpY2UKZGEyOiBT ZXJpYWwgTnVtYmVyIApkYTI6IDYwMC4wMDBNQi9zIHRyYW5zZmVycwpkYTI6IENvbW1hbmQgUXVl dWVpbmcgZW5hYmxlZApkYTI6IDc2MzA4ODVNQiAoMTU2MjgwNTMxNjggNTEyIGJ5dGUgc2VjdG9y cykKVHJ5aW5nIHRvIG1vdW50IHJvb3QgZnJvbSB6ZnM6enJvb3QvUk9PVC9kZWZhdWx0IFtdLi4u ClNldHRpbmcgaG9zdHV1aWQ6IDAwMDAwMDAwLTAwMDAtMDAwMC0wMDAwLTAwMjU5MGU1MjNmNi4K U2V0dGluZyBob3N0aWQ6IDB4YzA3NDA5OWMuClN0YXJ0aW5nIGZpbGUgc3lzdGVtIGNoZWNrczoK TW91bnRpbmcgbG9jYWwgZmlsZXN5c3RlbXM6LgpFTEYgbGRjb25maWcgcGF0aDogL2xpYiAvdXNy L2xpYiAvdXNyL2xpYi9jb21wYXQgL3Vzci9sb2NhbC9saWIgL3Vzci9sb2NhbC9saWIvcGVybDUv NS4zMC9tYWNoL0NPUkUgL3Vzci9sb2NhbC9saWIvc2FtYmE0CjMyLWJpdCBjb21wYXRpYmlsaXR5 IGxkY29uZmlnIHBhdGg6IC91c3IvbGliMzIKU2V0dGluZyBob3N0bmFtZTogbmFzMjUzLgpTZXR0 aW5nIHVwIGhhcnZlc3Rpbmc6IFBVUkVfUkRSQU5ELFtVTUFdLFtGU19BVElNRV0sU1dJLElOVEVS UlVQVCxORVRfTkcsTkVUX0VUSEVSLE5FVF9UVU4sTU9VU0UsS0VZQk9BUkQsQVRUQUNILENBQ0hF RApGZWVkaW5nIGVudHJvcHk6IC4KbG8wOiBsaW5rIHN0YXRlIGNoYW5nZWQgdG8gVVAKY3hsMDog bGluayBzdGF0ZSBjaGFuZ2VkIHRvIFVQCmN4bDE6IGxpbmsgc3RhdGUgY2hhbmdlZCB0byBVUApT dGFydGluZyBOZXR3b3JrOiBsbzAgY3hsMCBjeGwxLgpsbzA6IGZsYWdzPTgwNDk8VVAsTE9PUEJB Q0ssUlVOTklORyxNVUxUSUNBU1Q+IG1ldHJpYyAwIG10dSAxNjM4NAoJb3B0aW9ucz02ODAwMDM8 UlhDU1VNLFRYQ1NVTSxMSU5LU1RBVEUsUlhDU1VNX0lQVjYsVFhDU1VNX0lQVjY+CglpbmV0NiA6 OjEgcHJlZml4bGVuIDEyOAoJaW5ldDYgZmU4MDo6MSVsbzAgcHJlZml4bGVuIDY0IHNjb3BlaWQg MHgzCglpbmV0IDEyNy4wLjAuMSBuZXRtYXNrIDB4ZmYwMDAwMDAKCWdyb3VwczogbG8KCW5kNiBv cHRpb25zPTIxPFBFUkZPUk1OVUQsQVVUT19MSU5LTE9DQUw+CmN4bDA6IGZsYWdzPTg4NDM8VVAs QlJPQURDQVNULFJVTk5JTkcsU0lNUExFWCxNVUxUSUNBU1Q+IG1ldHJpYyAwIG10dSA5MDAwCglv cHRpb25zPTJlYzA3YmI8UlhDU1VNLFRYQ1NVTSxWTEFOX01UVSxWTEFOX0hXVEFHR0lORyxKVU1C T19NVFUsVkxBTl9IV0NTVU0sVFNPNCxUU082LExSTyxWTEFOX0hXVFNPLExJTktTVEFURSxSWENT VU1fSVBWNixUWENTVU1fSVBWNixIV1JYVFNUTVA+CglldGhlciAwMDowNzo0MzozYTphZDoxMAoJ aW5ldCAxOTIuMTY4LjExLjI1MyBuZXRtYXNrIDB4ZmZmZmZmMDAgYnJvYWRjYXN0IDE5Mi4xNjgu MTEuMjU1CgltZWRpYTogRXRoZXJuZXQgMTBHYmFzZS1TUiA8ZnVsbC1kdXBsZXgscnhwYXVzZSx0 eHBhdXNlPgoJc3RhdHVzOiBhY3RpdmUKCW5kNiBvcHRpb25zPTI5PFBFUkZPUk1OVUQsSUZESVNB QkxFRCxBVVRPX0xJTktMT0NBTD4KY3hsMTogZmxhZ3M9ODg0MzxVUCxCUk9BRENBU1QsUlVOTklO RyxTSU1QTEVYLE1VTFRJQ0FTVD4gbWV0cmljIDAgbXR1IDE1MDAKCW9wdGlvbnM9MmVjMDRiYjxS WENTVU0sVFhDU1VNLFZMQU5fTVRVLFZMQU5fSFdUQUdHSU5HLEpVTUJPX01UVSxWTEFOX0hXQ1NV TSxMUk8sVkxBTl9IV1RTTyxMSU5LU1RBVEUsUlhDU1VNX0lQVjYsVFhDU1VNX0lQVjYsSFdSWFRT VE1QPgoJZXRoZXIgMDA6MDc6NDM6M2E6YWQ6MTgKCW1lZGlhOiBFdGhlcm5ldCAxMEdiYXNlLVNS IDxmdWxsLWR1cGxleCxyeHBhdXNlLHR4cGF1c2U+CglzdGF0dXM6IGFjdGl2ZQoJbmQ2IG9wdGlv bnM9Mjk8UEVSRk9STU5VRCxJRkRJU0FCTEVELEFVVE9fTElOS0xPQ0FMPgpTdGFydGluZyBkZXZk LgpjeGwxOiBsaW5rIHN0YXRlIGNoYW5nZWQgdG8gRE9XTgpjeGwxOiBsaW5rIHN0YXRlIGNoYW5n ZWQgdG8gVVAKY3hsMTogbGluayBzdGF0ZSBjaGFuZ2VkIHRvIERPV04KY3hsMTogbGluayBzdGF0 ZSBjaGFuZ2VkIHRvIFVQCmN4bDA6IGxpbmsgc3RhdGUgY2hhbmdlZCB0byBET1dOCkF1dG9sb2Fk aW5nIG1vZHVsZTogaW9hdC5rbwppb2F0MDogPElWQiBJT0FUIENoMD4gbWVtIDB4ZGRjMWMwMDAt MHhkZGMxZmZmZiBpcnEgMzEgYXQgZGV2aWNlIDQuMCBudW1hLWRvbWFpbiAwIG9uIHBjaTAKaW9h dDA6IENhcGFiaWxpdGllczogMmY3PFBRLEV4dGVuZGVkX0FQSUNfSUQsQmxvY2tfRmlsbCxNb3Zl X0NSQyxEQ0EsTWFya2VyX1NraXBwaW5nLENSQyxQYWdlX0JyZWFrPgppb2F0MTogPElWQiBJT0FU IENoMT4gbWVtIDB4ZGRjMTgwMDAtMHhkZGMxYmZmZiBpcnEgMzkgYXQgZGV2aWNlIDQuMSBudW1h LWRvbWFpbiAwIG9uIHBjaTAKaW9hdDE6IENhcGFiaWxpdGllczogMmY3PFBRLEV4dGVuZGVkX0FQ SUNfSUQsQmxvY2tfRmlsbCxNb3ZlX0NSQyxEQ0EsTWFya2VyX1NraXBwaW5nLENSQyxQYWdlX0Jy ZWFrPgppb2F0MjogPElWQiBJT0FUIENoMj4gbWVtIDB4ZGRjMTQwMDAtMHhkZGMxN2ZmZiBpcnEg MzEgYXQgZGV2aWNlIDQuMiBudW1hLWRvbWFpbiAwIG9uIHBjaTAKaW9hdDI6IENhcGFiaWxpdGll czogZjc8RXh0ZW5kZWRfQVBJQ19JRCxCbG9ja19GaWxsLE1vdmVfQ1JDLERDQSxNYXJrZXJfU2tp cHBpbmcsQ1JDLFBhZ2VfQnJlYWs+CmlvYXQzOiA8SVZCIElPQVQgQ2gzPiBtZW0gMHhkZGMxMDAw MC0weGRkYzEzZmZmIGlycSAzOSBhdCBkZXZpY2UgNC4zIG51bWEtZG9tYWluIDAgb24gcGNpMApp b2F0MzogQ2FwYWJpbGl0aWVzOiBmNzxFeHRlbmRlZF9BUElDX0lELEJsb2NrX0ZpbGwsTW92ZV9D UkMsRENBLE1hcmtlcl9Ta2lwcGluZyxDUkMsUGFnZV9CcmVhaz4KaW9hdDQ6IDxJVkIgSU9BVCBD aDQ+IG1lbSAweGRkYzBjMDAwLTB4ZGRjMGZmZmYgaXJxIDMxIGF0IGRldmljZSA0LjQgbnVtYS1k b21haW4gMCBvbiBwY2kwCmlvYXQ0OiBDYXBhYmlsaXRpZXM6IGY3PEV4dGVuZGVkX0FQSUNfSUQs QmxvY2tfRmlsbCxNb3ZlX0NSQyxEQ0EsTWFya2VyX1NraXBwaW5nLENSQyxQYWdlX0JyZWFrPgpp b2F0NTogPElWQiBJT0FUIENoNT4gbWVtIDB4ZGRjMDgwMDAtMHhkZGMwYmZmZiBpcnEgMzkgYXQg ZGV2aWNlIDQuNSBudW1hLWRvbWFpbiAwIG9uIHBjaTAKaW9hdDU6IENhcGFiaWxpdGllczogZjc8 RXh0ZW5kZWRfQVBJQ19JRCxCbG9ja19GaWxsLE1vdmVfQ1JDLERDQSxNYXJrZXJfU2tpcHBpbmcs Q1JDLFBhZ2VfQnJlYWs+CmlvYXQ2OiA8SVZCIElPQVQgQ2g2PiBtZW0gMHhkZGMwNDAwMC0weGRk YzA3ZmZmIGlycSAzMSBhdCBkZXZpY2UgNC42IG51bWEtZG9tYWluIDAgb24gcGNpMAppb2F0Njog Q2FwYWJpbGl0aWVzOiBmNzxFeHRlbmRlZF9BUElDX0lELEJsb2NrX0ZpbGwsTW92ZV9DUkMsRENB LE1hcmtlcl9Ta2lwcGluZyxDUkMsUGFnZV9CcmVhaz4KaW9hdDc6IDxJVkIgSU9BVCBDaDc+IG1l bSAweGRkYzAwMDAwLTB4ZGRjMDNmZmYgaXJxIDM5IGF0IGRldmljZSA0LjcgbnVtYS1kb21haW4g MCBvbiBwY2kwCmlvYXQ3OiBDYXBhYmlsaXRpZXM6IGY3PEV4dGVuZGVkX0FQSUNfSUQsQmxvY2tf RmlsbCxNb3ZlX0NSQyxEQ0EsTWFya2VyX1NraXBwaW5nLENSQyxQYWdlX0JyZWFrPgpjeGwwOiBs aW5rIHN0YXRlIGNoYW5nZWQgdG8gVVAKaW9hdDg6IDxJVkIgSU9BVCBDaDA+IG1lbSAweGZiZjFj MDAwLTB4ZmJmMWZmZmYgaXJxIDU1IGF0IGRldmljZSA0LjAgbnVtYS1kb21haW4gMSBvbiBwY2k5 CmlvYXQ4OiBDYXBhYmlsaXRpZXM6IDJmNzxQUSxFeHRlbmRlZF9BUElDX0lELEJsb2NrX0ZpbGws TW92ZV9DUkMsRENBLE1hcmtlcl9Ta2lwcGluZyxDUkMsUGFnZV9CcmVhaz4KaW9hdDk6IDxJVkIg SU9BVCBDaDE+IG1lbSAweGZiZjE4MDAwLTB4ZmJmMWJmZmYgaXJxIDYzIGF0IGRldmljZSA0LjEg bnVtYS1kb21haW4gMSBvbiBwY2k5CmlvYXQ5OiBDYXBhYmlsaXRpZXM6IDJmNzxQUSxFeHRlbmRl ZF9BUElDX0lELEJsb2NrX0ZpbGwsTW92ZV9DUkMsRENBLE1hcmtlcl9Ta2lwcGluZyxDUkMsUGFn ZV9CcmVhaz4KaW9hdDEwOiA8SVZCIElPQVQgQ2gyPiBtZW0gMHhmYmYxNDAwMC0weGZiZjE3ZmZm IGlycSA1NSBhdCBkZXZpY2UgNC4yIG51bWEtZG9tYWluIDEgb24gcGNpOQppb2F0MTA6IENhcGFi aWxpdGllczogZjc8RXh0ZW5kZWRfQVBJQ19JRCxCbG9ja19GaWxsLE1vdmVfQ1JDLERDQSxNYXJr ZXJfU2tpcHBpbmcsQ1JDLFBhZ2VfQnJlYWs+CmlvYXQxMTogPElWQiBJT0FUIENoMz4gbWVtIDB4 ZmJmMTAwMDAtMHhmYmYxM2ZmZiBpcnEgNjMgYXQgZGV2aWNlIDQuMyBudW1hLWRvbWFpbiAxIG9u IHBjaTkKaW9hdDExOiBDYXBhYmlsaXRpZXM6IGY3PEV4dGVuZGVkX0FQSUNfSUQsQmxvY2tfRmls bCxNb3ZlX0NSQyxEQ0EsTWFya2VyX1NraXBwaW5nLENSQyxQYWdlX0JyZWFrPgppb2F0MTI6IDxJ VkIgSU9BVCBDaDQ+IG1lbSAweGZiZjBjMDAwLTB4ZmJmMGZmZmYgaXJxIDU1IGF0IGRldmljZSA0 LjQgbnVtYS1kb21haW4gMSBvbiBwY2k5CmlvYXQxMjogQ2FwYWJpbGl0aWVzOiBmNzxFeHRlbmRl ZF9BUElDX0lELEJsb2NrX0ZpbGwsTW92ZV9DUkMsRENBLE1hcmtlcl9Ta2lwcGluZyxDUkMsUGFn ZV9CcmVhaz4KaW9hdDEzOiA8SVZCIElPQVQgQ2g1PiBtZW0gMHhmYmYwODAwMC0weGZiZjBiZmZm IGlycSA2MyBhdCBkZXZpY2UgNC41IG51bWEtZG9tYWluIDEgb24gcGNpOQppb2F0MTM6IENhcGFi aWxpdGllczogZjc8RXh0ZW5kZWRfQVBJQ19JRCxCbG9ja19GaWxsLE1vdmVfQ1JDLERDQSxNYXJr ZXJfU2tpcHBpbmcsQ1JDLFBhZ2VfQnJlYWs+CmlvYXQxNDogPElWQiBJT0FUIENoNj4gbWVtIDB4 ZmJmMDQwMDAtMHhmYmYwN2ZmZiBpcnEgNTUgYXQgZGV2aWNlIDQuNiBudW1hLWRvbWFpbiAxIG9u IHBjaTkKaW9hdDE0OiBDYXBhYmlsaXRpZXM6IGY3PEV4dGVuZGVkX0FQSUNfSUQsQmxvY2tfRmls bCxNb3ZlX0NSQyxEQ0EsTWFya2VyX1NraXBwaW5nLENSQyxQYWdlX0JyZWFrPgppb2F0MTU6IDxJ VkIgSU9BVCBDaDc+IG1lbSAweGZiZjAwMDAwLTB4ZmJmMDNmZmYgaXJxIDYzIGF0IGRldmljZSA0 LjcgbnVtYS1kb21haW4gMSBvbiBwY2k5CmlvYXQxNTogQ2FwYWJpbGl0aWVzOiBmNzxFeHRlbmRl ZF9BUElDX0lELEJsb2NrX0ZpbGwsTW92ZV9DUkMsRENBLE1hcmtlcl9Ta2lwcGluZyxDUkMsUGFn ZV9CcmVhaz4KQXV0b2xvYWRpbmcgbW9kdWxlOiBpb2F0LmtvCkF1dG9sb2FkaW5nIG1vZHVsZTog aW9hdC5rbwpBdXRvbG9hZGluZyBtb2R1bGU6IGlvYXQua28KQXV0b2xvYWRpbmcgbW9kdWxlOiBp b2F0LmtvCkF1dG9sb2FkaW5nIG1vZHVsZTogaW9hdC5rbwpBdXRvbG9hZGluZyBtb2R1bGU6IGlv YXQua28KQXV0b2xvYWRpbmcgbW9kdWxlOiBpb2F0LmtvCkF1dG9sb2FkaW5nIG1vZHVsZTogaW9h dC5rbwpBdXRvbG9hZGluZyBtb2R1bGU6IGlvYXQua28KQXV0b2xvYWRpbmcgbW9kdWxlOiBpb2F0 LmtvCkF1dG9sb2FkaW5nIG1vZHVsZTogaW9hdC5rbwpBdXRvbG9hZGluZyBtb2R1bGU6IGlvYXQu a28KQXV0b2xvYWRpbmcgbW9kdWxlOiBpb2F0LmtvCkF1dG9sb2FkaW5nIG1vZHVsZTogaW9hdC5r bwpBdXRvbG9hZGluZyBtb2R1bGU6IGlvYXQua28KQXV0b2xvYWRpbmcgbW9kdWxlOiB1aGlkLmtv CkF1dG9sb2FkaW5nIG1vZHVsZTogdW1zLmtvCnVtczAgbnVtYS1kb21haW4gMCBvbiB1aHViMwp1 bXMwOiA8V2luYm9uZCBFbGVjdHJvbmljcyBDb3JwIEhlcm1vbiBVU0IgaGlkbW91c2UgRGV2aWNl LCBjbGFzcyAwLzAsIHJldiAxLjEwLzAuMDEsIGFkZHIgMz4gb24gdXNidXMwCnVtczA6IDMgYnV0 dG9ucyBhbmQgW1pdIGNvb3JkaW5hdGVzIElEPTAKU3RhcnRpbmcgdW1zMCBtb3VzZWQuCmFkZCBo b3N0IDEyNy4wLjAuMTogZ2F0ZXdheSBsbzAgZmliIDA6IHJvdXRlIGFscmVhZHkgaW4gdGFibGUK YWRkIG5ldCBkZWZhdWx0OiBnYXRld2F5IDE5Mi4xNjguMTEuMjU0CmFkZCBob3N0IDo6MTogZ2F0 ZXdheSBsbzAgZmliIDA6IHJvdXRlIGFscmVhZHkgaW4gdGFibGUKYWRkIG5ldCBmZTgwOjo6IGdh dGV3YXkgOjoxCmFkZCBuZXQgZmYwMjo6OiBnYXRld2F5IDo6MQphZGQgbmV0IDo6ZmZmZjowLjAu MC4wOiBnYXRld2F5IDo6MQphZGQgbmV0IDo6MC4wLjAuMDogZ2F0ZXdheSA6OjEKQ3JlYXRpbmcg YW5kL29yIHRyaW1taW5nIGxvZyBmaWxlcy4KU3RhcnRpbmcgc3lzbG9nZC4KYnJpZGdlMDogRXRo ZXJuZXQgYWRkcmVzczogMDI6YzA6NzQ6MDk6OWM6MDAKYnJpZGdlMDogY2hhbmdpbmcgbmFtZSB0 byAndm0tbGFuJwp2bS1sYW46IGxpbmsgc3RhdGUgY2hhbmdlZCB0byBVUApjeGwxOiBwcm9taXNj dW91cyBtb2RlIGVuYWJsZWQKYnJpZGdlMTogRXRoZXJuZXQgYWRkcmVzczogMDI6YzA6NzQ6MDk6 OWM6MDEKYnJpZGdlMTogY2hhbmdpbmcgbmFtZSB0byAndm0tbGFuMjQ0Jwp2bGFuMDogY2hhbmdp bmcgbmFtZSB0byAnY3hsMS4yNDQnCmN4bDEuMjQ0OiBwcm9taXNjdW91cyBtb2RlIGVuYWJsZWQK dm0tbGFuMjQ0OiBsaW5rIHN0YXRlIGNoYW5nZWQgdG8gVVAKYnJpZGdlMjogRXRoZXJuZXQgYWRk cmVzczogMDI6YzA6NzQ6MDk6OWM6MDIKYnJpZGdlMjogY2hhbmdpbmcgbmFtZSB0byAndm0tdjZw bHVzJwp2bGFuMTogY2hhbmdpbmcgbmFtZSB0byAnY3hsMS42MDAnCmN4bDEuNjAwOiBwcm9taXNj dW91cyBtb2RlIGVuYWJsZWQKdm0tdjZwbHVzOiBsaW5rIHN0YXRlIGNoYW5nZWQgdG8gVVAKQ2xl YXJpbmcgL3RtcCAoWCByZWxhdGVkKS4KU3RhcnRpbmcgbG9jYWwgZGFlbW9uczpMb2NhdGluZyBz ZW5zb3IgcmVjb3JkICdGQU4xJy4uLgpTZXR0aW5nIHNlbnNvciAiRkFOMSIgTG93ZXIgTm9uLVJl Y292ZXJhYmxlIHRocmVzaG9sZCB0byA3NS4wMDAKU2V0dGluZyBzZW5zb3IgIkZBTjEiIExvd2Vy IENyaXRpY2FsIHRocmVzaG9sZCB0byAxNTAuMDAwClNldHRpbmcgc2Vuc29yICJGQU4xIiBMb3dl ciBOb24tQ3JpdGljYWwgdGhyZXNob2xkIHRvIDIyNS4wMDAKTG9jYXRpbmcgc2Vuc29yIHJlY29y ZCAnRkFOMicuLi4KU2V0dGluZyBzZW5zb3IgIkZBTjIiIExvd2VyIE5vbi1SZWNvdmVyYWJsZSB0 aHJlc2hvbGQgdG8gNzUuMDAwClNldHRpbmcgc2Vuc29yICJGQU4yIiBMb3dlciBDcml0aWNhbCB0 aHJlc2hvbGQgdG8gMTUwLjAwMApTZXR0aW5nIHNlbnNvciAiRkFOMiIgTG93ZXIgTm9uLUNyaXRp Y2FsIHRocmVzaG9sZCB0byAyMjUuMDAwCkxvY2F0aW5nIHNlbnNvciByZWNvcmQgJ0ZBTjMnLi4u ClNldHRpbmcgc2Vuc29yICJGQU4zIiBMb3dlciBOb24tUmVjb3ZlcmFibGUgdGhyZXNob2xkIHRv IDc1LjAwMApTZXR0aW5nIHNlbnNvciAiRkFOMyIgTG93ZXIgQ3JpdGljYWwgdGhyZXNob2xkIHRv IDE1MC4wMDAKU2V0dGluZyBzZW5zb3IgIkZBTjMiIExvd2VyIE5vbi1Dcml0aWNhbCB0aHJlc2hv bGQgdG8gMjI1LjAwMApMb2NhdGluZyBzZW5zb3IgcmVjb3JkICdGQU40Jy4uLgpTZXR0aW5nIHNl bnNvciAiRkFONCIgTG93ZXIgTm9uLVJlY292ZXJhYmxlIHRocmVzaG9sZCB0byA3NS4wMDAKU2V0 dGluZyBzZW5zb3IgIkZBTjQiIExvd2VyIENyaXRpY2FsIHRocmVzaG9sZCB0byAxNTAuMDAwClNl dHRpbmcgc2Vuc29yICJGQU40IiBMb3dlciBOb24tQ3JpdGljYWwgdGhyZXNob2xkIHRvIDIyNS4w MDAKTG9jYXRpbmcgc2Vuc29yIHJlY29yZCAnRkFONScuLi4KU2V0dGluZyBzZW5zb3IgIkZBTjUi IExvd2VyIE5vbi1SZWNvdmVyYWJsZSB0aHJlc2hvbGQgdG8gNzUuMDAwClNldHRpbmcgc2Vuc29y ICJGQU41IiBMb3dlciBDcml0aWNhbCB0aHJlc2hvbGQgdG8gMTUwLjAwMApTZXR0aW5nIHNlbnNv ciAiRkFONSIgTG93ZXIgTm9uLUNyaXRpY2FsIHRocmVzaG9sZCB0byAyMjUuMDAwCkxvY2F0aW5n IHNlbnNvciByZWNvcmQgJ0ZBTjYnLi4uClNldHRpbmcgc2Vuc29yICJGQU42IiBMb3dlciBOb24t UmVjb3ZlcmFibGUgdGhyZXNob2xkIHRvIDc1LjAwMApTZXR0aW5nIHNlbnNvciAiRkFONiIgTG93 ZXIgQ3JpdGljYWwgdGhyZXNob2xkIHRvIDE1MC4wMDAKU2V0dGluZyBzZW5zb3IgIkZBTjYiIExv d2VyIE5vbi1Dcml0aWNhbCB0aHJlc2hvbGQgdG8gMjI1LjAwMApMb2NhdGluZyBzZW5zb3IgcmVj b3JkICdGQU5BJy4uLgpTZXR0aW5nIHNlbnNvciAiRkFOQSIgTG93ZXIgTm9uLVJlY292ZXJhYmxl IHRocmVzaG9sZCB0byA3NS4wMDAKU2V0dGluZyBzZW5zb3IgIkZBTkEiIExvd2VyIENyaXRpY2Fs IHRocmVzaG9sZCB0byAxNTAuMDAwClNldHRpbmcgc2Vuc29yICJGQU5BIiBMb3dlciBOb24tQ3Jp dGljYWwgdGhyZXNob2xkIHRvIDIyNS4wMDAKTG9jYXRpbmcgc2Vuc29yIHJlY29yZCAnRkFOQicu Li4KU2V0dGluZyBzZW5zb3IgIkZBTkIiIExvd2VyIE5vbi1SZWNvdmVyYWJsZSB0aHJlc2hvbGQg dG8gNzUuMDAwClNldHRpbmcgc2Vuc29yICJGQU5CIiBMb3dlciBDcml0aWNhbCB0aHJlc2hvbGQg dG8gMTUwLjAwMApTZXR0aW5nIHNlbnNvciAiRkFOQiIgTG93ZXIgTm9uLUNyaXRpY2FsIHRocmVz aG9sZCB0byAyMjUuMDAwCi4KVXBkYXRpbmcgbW90ZDouCk1vdW50aW5nIGxhdGUgZmlsZXN5c3Rl bXM6LgpTZWN1cml0eSBwb2xpY3kgbG9hZGVkOiBNQUMvbnRwZCAobWFjX250cGQpClN0YXJ0aW5n IG50cGQuClN0YXJ0aW5nIHBvd2VyZC4KUGVyZm9ybWluZyBzYW5pdHkgY2hlY2sgb24gU2FtYmEg Y29uZmlndXJhdGlvbjogT0sKU3RhcnRpbmcgc21iZC4KQ29uZmlndXJpbmcgdnQ6IGJsYW5rdGlt ZS4KUGVyZm9ybWluZyBzYW5pdHkgY2hlY2sgb24gc3NoZCBjb25maWd1cmF0aW9uLgpTdGFydGlu ZyBzc2hkLgpTdGFydGluZyBjcm9uLgpTdGFydGluZyBiYWNrZ3JvdW5kIGZpbGUgc3lzdGVtIGNo ZWNrcyBpbiA2MCBzZWNvbmRzLgoK --0000000000007dff2a05a0f22920-- From owner-freebsd-virtualization@freebsd.org Mon Mar 16 07:33:59 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1F22525D219 for ; Mon, 16 Mar 2020 07:33:59 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48gp2X5Bwxz4dnC for ; Mon, 16 Mar 2020 07:33:56 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-qt1-x843.google.com with SMTP id i26so3546929qtq.8 for ; Mon, 16 Mar 2020 00:33:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=8Q1m8tvjW+6riATwEjp1llaIWUjwzh/x2pzUMAMQiOs=; b=iOXf416/+UcB1IIn2bjcqGc2CMKVF+9/2wE8Zb7KSAVe6toqjmmDgSdq26bQ6C4ZVp Nazbggd0B6MIzWuXfH5wBLYYxSk4zYvJVLdghs0sy9eVb15Bo4syh1WkS7KL90lV8g5J 4WfkxDrKPhJrn1bj1dJACo584a/uf8U3YjUCJKiyLyaRN2FzOOyX3RBj8gbm/aJPUjve 4P5UgP+ZPCkr7ITw7sXOVE6z5kboRBe/Vhat+znPZ7QDAbrFs0gDiCsq8x8lEH94NSj6 JZeACV+091fI1/PPrhS1cyPNHAAvSsESRJMS2xu+2orZ3svN9IYzlGVnOoKLLbGazp9Z ZynQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=8Q1m8tvjW+6riATwEjp1llaIWUjwzh/x2pzUMAMQiOs=; b=Wy3V/q8jLJRzErPShXvtLc719mYjtLdWma/by8cCFX14sAfXiPRoUWcgKSn5OaK9DT jPVYjVuvrrPN/RnXNhC0K5bqqgzt4ngDvsjGFJG8ZoWGycR4zlZSX6uYTNmsezttR+15 DOvt9gyNeowpD6tKPI7tfiZItRfPNAoDZdpqZs/Ow8SvXBI+c98sWixZu/lts9GsFbtj VXQS55x2wlY9FoawZJ+6MlhFvccvBppqm/gXrlanMIgaqYKgsBqTkSgbD0vcoFM4uCaC 6gHTEk8HZ3L1UxE1uojJ2ayJPiiuxtlO1bwgp9sqx7kWg/K2iUYwSYU40NRE9KwB263x Wdwg== X-Gm-Message-State: ANhLgQ0AOhRiKrQey60JUBOMYr1SdgNuJaCFmTX60Di818KZxiuKJv+M TP4xkiDn3uftusTWHncND/8VAGUkfXaG474jzeFa0JNcn+w= X-Google-Smtp-Source: ADFU+vuWMtCokzM/yhZB3CbPaUOy4wi1rF1+xVB7W+1h9ixfCdGpH25DNPuAkF2llHVUBsyyiNrE+0n0RVtutkKO1A0= X-Received: by 2002:aed:3244:: with SMTP id y62mr16667565qtd.242.1584344034851; Mon, 16 Mar 2020 00:33:54 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Wanpeng Qian Date: Mon, 16 Mar 2020 16:33:40 +0900 Message-ID: Subject: Re: bhyve: Passthru second port of i350-T2 to Windows cause system crash. To: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48gp2X5Bwxz4dnC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=iOXf416/; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of wanpengqian@gmail.com designates 2607:f8b0:4864:20::843 as permitted sender) smtp.mailfrom=wanpengqian@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ip: (1.92), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2020 07:33:59 -0000 I found a commit relate to VT-d, I will try this patch first. https://reviews.freebsd.org/rS349184 On Mon, Mar 16, 2020 at 2:33 PM Wanpeng Qian wrote: > > Hello all > > I am using FreeBSD 12.1R + patch > https://reviews.freebsd.org/D20531 > > Without D20531 patch, I can passthru this first port of i350-T2 > to a Linux vm. > and with the Patch, I can passthough the second port to A windows > VM. it works. > > about several minutes later, system crash. I reset the sever via IPMI. > the last dmesg keep. see the attached file. > > I sometimes get such error duing the passthrough in FreeBSD 12.1 > without passthrough(with/without patch), the system is stable. > and I run ESXi with passthough also. pretty stable for months without reboot. > > any idea? am I doing something wrong? > I think I am separate the iommu group to different vm? > how to check the iommu group in FreeBSD? > > Thanks. From owner-freebsd-virtualization@freebsd.org Tue Mar 17 00:56:45 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EEC64274F10 for ; Tue, 17 Mar 2020 00:56:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48hF9n5F1bz4DLR for ; Tue, 17 Mar 2020 00:56:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 83896274F0F; Tue, 17 Mar 2020 00:56:45 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 78C56274F0E for ; Tue, 17 Mar 2020 00:56:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48hF9n0Gcqz4DKj for ; Tue, 17 Mar 2020 00:56:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D8D8F123A for ; Tue, 17 Mar 2020 00:56:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02H0uin3011547 for ; Tue, 17 Mar 2020 00:56:44 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02H0uiwX011543 for virtualization@FreeBSD.org; Tue, 17 Mar 2020 00:56:44 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236989] AWS EC2 lockups "Missing interrupt" Date: Tue, 17 Mar 2020 00:56:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: freebsd@darkain.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Mar 2020 00:56:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236989 --- Comment #16 from Vincent Milum Jr --- I actually get a kernel panic in 12.1, but it works in 13 https://twitter.com/DarkainMX/status/1239715138027962368 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Mar 17 12:42:12 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F2FD264C44; Tue, 17 Mar 2020 12:42:12 +0000 (UTC) (envelope-from daniel.nicolescu10@gmail.com) Received: from mail-vk1-xa33.google.com (mail-vk1-xa33.google.com [IPv6:2607:f8b0:4864:20::a33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48hXql4wk0z3Qy0; Tue, 17 Mar 2020 12:42:11 +0000 (UTC) (envelope-from daniel.nicolescu10@gmail.com) Received: by mail-vk1-xa33.google.com with SMTP id k63so5900113vka.7; Tue, 17 Mar 2020 05:42:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=ZfVaZiTETHUimPDSX5rhf3miFggNXjxOvBuGrrXHtcs=; b=Ofzpr7YIr7D9DfXebPbLHr92ElpI48QkPbYUlFC4oiXG/pcYKYybB4FzotQwVuYEAt FFVdRRXUEQ8TPq8aC39brMHvXiqJMmkBi1lY+KKAOtfs5rDMgvf/4e1ezioXLud1tozo c7FwsI1TTaBE2PAlk098SX+fv2sYKmRXtoyseZErws5+Dn/oH6JuPzte7Rluae9EYMqU c+89pbSp02/Hdxg5q9wZLKslEIKxGtGWQ+1cmsqKEFFVUiCIT5xe9YEbtgUOK9jMYh6y Me6SMsZgXuGA2Nay5iPZp0g4YR4dlk5BdB/bPrvZpIt7jSwHAZJy3L7v2M+Liw0XcTUS zEMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=ZfVaZiTETHUimPDSX5rhf3miFggNXjxOvBuGrrXHtcs=; b=MGbSv7SRpcsAacKfWdD3H+Mj525cS6W/Aw5Dqnh8CmurHaAlqS595kRQbzv7mYdIRD fHgfiNsfCeNQDikiQRu3rihSoGPgTJp3UWkbRh5ZJeDeok1tIedKfGxOglKIn3gd0Gn2 Avd7PFLcP482oSZggK31j/zqqE/XQj+XR4Rd/psjmBioRC+MBEVfeJ+dq4qRy48NyE5Z OrNXzakNJevHQqOZpG0XbvaCTN53K8nzBdAqM1rcNhIru7GYbiXCkNclebwSOoOFGJ85 07iiKdCCXg8gcD+Qp0yAWNvaFfJywB50chpNRP84UEwrQkn1SkZRKMAKVqJyrSecmXlX 3qzg== X-Gm-Message-State: ANhLgQ0XO9YiCprkjhZTlC11ubDSpWOTZH5EcgqrK4xqi4PbEAgHYWiU oqUjN8xTfhSfa7xCmYoYRt2+vpzM/YYUQTLkOmS/O4LV/9w= X-Google-Smtp-Source: ADFU+vufThdsJSUvb66q2WGJwRD3k5o7ZxEhvywIm63WAqWf9ZrEkIIR1jZvlpvORv6T4eupRYLmcIAIrDPqU7gkGtI= X-Received: by 2002:a1f:728a:: with SMTP id n132mr3436882vkc.1.1584448930005; Tue, 17 Mar 2020 05:42:10 -0700 (PDT) MIME-Version: 1.0 From: Daniel Nicolescu Date: Tue, 17 Mar 2020 14:41:59 +0200 Message-ID: Subject: USB device passthrough implementation To: freebsd-virtualization@freebsd.org, freebsd-hackers@freebsd.org Cc: patrascu.naina14@gmail.com, Mihai Carabas X-Rspamd-Queue-Id: 48hXql4wk0z3Qy0 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Ofzpr7YI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of danielnicolescu10@gmail.com designates 2607:f8b0:4864:20::a33 as permitted sender) smtp.mailfrom=danielnicolescu10@gmail.com X-Spamd-Result: default: False [-1.50 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.09), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[3.3.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Mar 2020 12:42:12 -0000 Hi, We want to implement the USB pass through feature for bhyve so that users will be able to pass through individual USB devices like in QEMU. We are trying to emulate the USB controller and pass to the virtual machine only one port. * We need to disconnect the USB device from host: we are currently not very familiar with the kernel API to perform this * We need to emulate a USB device controller: we are currently looking into pci_xhci.c and usb_mouse.c to reuse some of the code from there and we would like to know if there is some sort of in depth documentation regarding these. * We need to pass the USB device to the guest through an emulated port of the device controller: we know that project ACRN implements USB device passthrough and we are trying to figure out how they do it. Could you give us a few suggestions on how to tackle these? Thanks! From owner-freebsd-virtualization@freebsd.org Tue Mar 17 12:58:21 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 44182265520 for ; Tue, 17 Mar 2020 12:58:21 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from vito.onthenet.com.au (vito.onthenet.com.au [203.22.124.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48hYBN5rKdz48Jp for ; Tue, 17 Mar 2020 12:58:20 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by vito.onthenet.com.au (Postfix) with ESMTPS id 266D820B5B60 for ; Tue, 17 Mar 2020 22:58:16 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 0AB6720B49B0 for ; Tue, 17 Mar 2020 22:58:16 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 00FC820E467C for ; Tue, 17 Mar 2020 22:58:16 +1000 (AEST) Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id vlDQu4SkvZQY for ; Tue, 17 Mar 2020 22:58:15 +1000 (AEST) Received: from MacBook-Air-4.local (otn-120-29-62-248.broadband.onthenet.net [120.29.62.248]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 84B7220E467A; Tue, 17 Mar 2020 22:58:14 +1000 (AEST) Subject: Re: USB device passthrough implementation To: Daniel Nicolescu Cc: freebsd-virtualization@freebsd.org, patrascu.naina14@gmail.com, Mihai Carabas References: From: Peter Grehan Message-ID: <7043f9f6-448d-a76e-920d-9fa17cb0a2be@freebsd.org> Date: Tue, 17 Mar 2020 22:58:12 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=UIBp4xXy c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=DPSYfp6L38iglyTf480Cpw==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=IkcTkHD0fZMA:10 a=SS2py6AdgQ4A:10 a=8pif782wAAAA:8 a=jivqAgzXTCdQkgaRuWEA:9 a=QEXdDO2ut3YA:10 a=nhrbFlTIud0A:10 X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=d81BNirE c=1 sm=1 tr=0 a=eF5wA05EhuPcU0j7PNT2Vg==:117 a=DPSYfp6L38iglyTf480Cpw==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=IkcTkHD0fZMA:10 a=SS2py6AdgQ4A:10 a=8pif782wAAAA:8 a=jivqAgzXTCdQkgaRuWEA:9 a=QEXdDO2ut3YA:10 a=nhrbFlTIud0A:10 X-Rspamd-Queue-Id: 48hYBN5rKdz48Jp X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-0.01 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_LONG(-0.97)[-0.965,0]; TAGGED_RCPT(0.00)[]; NEURAL_SPAM_MEDIUM(0.96)[0.958,0]; ASN(0.00)[asn:9313, ipnet:203.22.124.0/23, country:AU] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Mar 2020 12:58:21 -0000 Hi Daniel, > * We need to disconnect the USB device from host: we are currently not very > familiar with the kernel API to perform this See usbconfig(8) and the detach_kernel_driver command. It's probably fine to request a user to issue that command, similar to how external commands are required to detach a PCI driver before using it for passthru in bhyve. > * We need to emulate a USB device controller: we are currently looking into > pci_xhci.c and usb_mouse.c to reuse some of the code from there and we > would like to know if there is some sort of in depth documentation > regarding these. pci_xhci.c is an implementation of a USB XHCI host controller so should be the place to start: it either has the functionality that is needed, or requires slight modifications. (Lots of info and pointer to the spec at https://en.wikipedia.org/wiki/Extensible_Host_Controller_Interface) usb_mouse.c is just an example of a USB device interface, that in turn passes through VNC mouse-movement events to the guest o/s. > * We need to pass the USB device to the guest through an emulated port of > the device controller: we know that project ACRN implements USB device > passthrough and we are trying to figure out how they do it. ACRN uses libusb as the interface to USB devices on the host (see devicemodel/hw/platform/usb_pmapper.c). later, Peter. From owner-freebsd-virtualization@freebsd.org Tue Mar 17 13:13:27 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B3E75265E9E; Tue, 17 Mar 2020 13:13:27 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48hYWn5NH1z4K5r; Tue, 17 Mar 2020 13:13:25 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2020.home.selasky.org (unknown [62.141.129.235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id CB5E2260133; Tue, 17 Mar 2020 14:13:20 +0100 (CET) Subject: Re: USB device passthrough implementation To: Daniel Nicolescu , freebsd-virtualization@freebsd.org, freebsd-hackers@freebsd.org Cc: patrascu.naina14@gmail.com, Mihai Carabas References: From: Hans Petter Selasky Message-ID: Date: Tue, 17 Mar 2020 14:12:19 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 48hYWn5NH1z4K5r X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hps@selasky.org designates 2a01:4f8:c17:6c4b::2 as permitted sender) smtp.mailfrom=hps@selasky.org X-Spamd-Result: default: False [-3.19 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.turbocat.net:c]; NEURAL_HAM_LONG(-1.00)[-0.996,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[selasky.org]; RCPT_COUNT_FIVE(0.00)[5]; SUSPICIOUS_RECIPS(1.50)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.72)[-0.722,0]; IP_SCORE(-2.67)[ip: (-9.20), ipnet: 2a01:4f8::/29(-2.57), asn: 24940(-1.56), country: DE(-0.02)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Mar 2020 13:13:27 -0000 Hi, On 2020-03-17 13:41, Daniel Nicolescu wrote: > Hi, > > We want to implement the USB pass through feature for bhyve so that users > will be able to pass through individual USB devices like in QEMU. We are > trying to emulate the USB controller and pass to the virtual machine only > one port. > > * We need to disconnect the USB device from host: we are currently not very > familiar with the kernel API to perform this See: libusb_detach_kernel_driver(). > * We need to emulate a USB device controller: we are currently looking into > pci_xhci.c and usb_mouse.c to reuse some of the code from there and we > would like to know if there is some sort of in depth documentation > regarding these. > * We need to pass the USB device to the guest through an emulated port of > the device controller: we know that project ACRN implements USB device > passthrough and we are trying to figure out how they do it. > > Could you give us a few suggestions on how to tackle these? USB has four transfer types: 1) Control transfers 2) Interrupt transfers 3) Bulk transfers 4) Isochronous transfers If your implementation is supposed to be fully USB compliant you need to support and test all transfer types. Please also pay attention to error conditions. USB has different speeds, LOW, HIGH, FULL, SUPER, which impose different characteristics on the USB wMaxPacketSize for example. You should know about the principle of short-terminated USB transfers. Is the wrapper supposed to be implemented in the kernel or in user-space? --HPS From owner-freebsd-virtualization@freebsd.org Tue Mar 17 14:09:13 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 77D9E267609; Tue, 17 Mar 2020 14:09:13 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48hZm72hPNz4TKF; Tue, 17 Mar 2020 14:09:11 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 02HE95qe075562 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 17 Mar 2020 15:09:05 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1584454146; bh=OHZRyhMFKdfNaSH6Csjd9xMwhMtnGlyUz/nAwZ+3mTU=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=Vtq8AnXXHRfG9gLhar4AACeL52WttAnVEB3qjFx5a/Z2zxoVfg4ZPMm/cPuAXxZab SB1j+l4uib6poNxGQij2mx5j5z3pr3U7Q8f5/dG49JdVMkGzyz1lzi7c1dSYwTQYWD z/G8HV/PtDulh7nAQBRbDMTprHKVuAV18pW7HK00= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 02HE95OY075559; Tue, 17 Mar 2020 15:09:05 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Tue, 17 Mar 2020 15:09:04 +0100 (CET) From: Wojciech Puchar To: Daniel Nicolescu cc: freebsd-virtualization@freebsd.org, freebsd-hackers@freebsd.org, patrascu.naina14@gmail.com, Mihai Carabas Subject: Re: USB device passthrough implementation In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 48hZm72hPNz4TKF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=fail (rsa verify failed) header.d=puchar.net header.s=default header.b=Vtq8AnXX; dmarc=none; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-2.29 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.92)[-0.924,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_DKIM_REJECT(1.00)[puchar.net:s=default]; R_SPF_ALLOW(-0.20)[+mx]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[puchar.net]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[puchar.net:-]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.57)[ip: (-6.78), ipnet: 194.1.144.0/24(-3.39), asn: 43476(-2.71), country: PL(0.06)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Mar 2020 14:09:13 -0000 > We want to implement the USB pass through feature for bhyve so that users > will be able to pass through individual USB devices like in QEMU. We are i want to be your beta tester. > trying to emulate the USB controller and pass to the virtual machine only > one port. > > * We need to disconnect the USB device from host: we are currently not very > familiar with the kernel API to perform this in most cases you actually don't have to. just use /dev/ugen/* devices simply make it as a parameter for bhyve usbpassthrough=a/device where a is virtual USB device number (i don't think you want to emulate more than one bus - one xhci is already done in bhyve), and device would be like /dev/ugenX.Y From owner-freebsd-virtualization@freebsd.org Tue Mar 17 15:34:18 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6593626999E; Tue, 17 Mar 2020 15:34:18 +0000 (UTC) (envelope-from daniel.nicolescu10@gmail.com) Received: from mail-vs1-xe30.google.com (mail-vs1-xe30.google.com [IPv6:2607:f8b0:4864:20::e30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48hcfJ4mmYz3LFg; Tue, 17 Mar 2020 15:34:16 +0000 (UTC) (envelope-from daniel.nicolescu10@gmail.com) Received: by mail-vs1-xe30.google.com with SMTP id a63so4182816vsa.8; Tue, 17 Mar 2020 08:34:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LDWk+3LJtnWLYySzyxUFjoOIs/+0ssowJSncWfOqnk4=; b=udHZAmCyS1+5ETFb2Xl6uu9gsx6VdPFZjMbj//7mWbjHFRwBDcPQWOOjdHDrjMmESx hH+JR3CLHKyFGx6Ldm1LomkGyOc6XnXxVpORDI633h7DfTsA6UToFf7ekmV5t1qFvcIu 3jE+jtJysxM1DBd8jtzmHUgqnJ6S2eP/thBGxXV8dCFgH/PD1sSxIlaNFBejTmPzIVAo xC9mDjpTgmIyjoGlkB4W0GrCaZ+UOYpZ6ziuSyO4wANCjTHYjWyz2Ux7Zhql+JD8k7J5 ALk+gy86Fe5r+e1Zcjn2HXO+FmyNPPNH7kKuiZ+k5lraeRjMwi28V5r5644nm2jfPRMF 5cBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LDWk+3LJtnWLYySzyxUFjoOIs/+0ssowJSncWfOqnk4=; b=DUxuSfSum3TLYl/VBcdxF1gS642+JrQX2/dPDU87EFkEhZkHzdmwK4I8V3SbQAHYSC jE+A5p2ggTnwgD+XFIMTrZ2uCfK9pzkEqPf8L50q2TzAwfaCh3UFBY+iRNQhRKHztfR3 reWHfrZGVFq2b/P1cXA5ltxIXGnO5e+Vps6bHlrF5xOOrp0sVfh8J7bqOXM2wMt3DDq8 TXFSEHoNDaF2KWFCLdEHIuEeHOqesNh6cAJQT/Hj1FmCpqy1xFpr9uTkYd+q1yBkMyao jM+9R5DJKeDvpXWKEav2shuUfbtghIlcm72UIrzyb9ATdAQNkYQqlXz+B1ujMKLFO7xg CBXg== X-Gm-Message-State: ANhLgQ3qN+LFKjUlmiMBH/cjjvzIOhxZRH2pO6xF1r09lusPLbY1pftR 1nJiQIWDNtrat4rWYRDKbw8NmPgM1zu5glRiZMI= X-Google-Smtp-Source: ADFU+vsMsbfxuQ4TaFM+PVbo2Z2X+65Cf34v6C9BzcAb1mgn9vShjX58YZ1PaqNyi685GMUXlMnDqlr+s1m5BRvXefo= X-Received: by 2002:a67:f6c9:: with SMTP id v9mr4123070vso.48.1584459254930; Tue, 17 Mar 2020 08:34:14 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Daniel Nicolescu Date: Tue, 17 Mar 2020 17:34:03 +0200 Message-ID: Subject: Re: USB device passthrough implementation To: Hans Petter Selasky Cc: freebsd-virtualization@freebsd.org, freebsd-hackers@freebsd.org, patrascu.naina14@gmail.com, Mihai Carabas X-Rspamd-Queue-Id: 48hcfJ4mmYz3LFg X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=udHZAmCy; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of danielnicolescu10@gmail.com designates 2607:f8b0:4864:20::e30 as permitted sender) smtp.mailfrom=danielnicolescu10@gmail.com X-Spamd-Result: default: False [-1.50 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.61), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[0.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Mar 2020 15:34:18 -0000 Hi, We would like to implement it in user-space. On Tue, Mar 17, 2020 at 3:13 PM Hans Petter Selasky wrote: > Hi, > > On 2020-03-17 13:41, Daniel Nicolescu wrote: > > Hi, > > > > We want to implement the USB pass through feature for bhyve so that users > > will be able to pass through individual USB devices like in QEMU. We are > > trying to emulate the USB controller and pass to the virtual machine only > > one port. > > > > * We need to disconnect the USB device from host: we are currently not > very > > familiar with the kernel API to perform this > > See: > libusb_detach_kernel_driver(). > > > * We need to emulate a USB device controller: we are currently looking > into > > pci_xhci.c and usb_mouse.c to reuse some of the code from there and we > > would like to know if there is some sort of in depth documentation > > regarding these. > > * We need to pass the USB device to the guest through an emulated port of > > the device controller: we know that project ACRN implements USB device > > passthrough and we are trying to figure out how they do it. > > > > Could you give us a few suggestions on how to tackle these? > > USB has four transfer types: > > 1) Control transfers > 2) Interrupt transfers > 3) Bulk transfers > 4) Isochronous transfers > > If your implementation is supposed to be fully USB compliant you need to > support and test all transfer types. > > Please also pay attention to error conditions. > > USB has different speeds, LOW, HIGH, FULL, SUPER, which impose different > characteristics on the USB wMaxPacketSize for example. > > You should know about the principle of short-terminated USB transfers. > > Is the wrapper supposed to be implemented in the kernel or in user-space? > > --HPS > From owner-freebsd-virtualization@freebsd.org Tue Mar 17 15:41:09 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0460D269D66; Tue, 17 Mar 2020 15:41:09 +0000 (UTC) (envelope-from daniel.nicolescu10@gmail.com) Received: from mail-vs1-xe36.google.com (mail-vs1-xe36.google.com [IPv6:2607:f8b0:4864:20::e36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48hcpD0C93z45Hf; Tue, 17 Mar 2020 15:41:07 +0000 (UTC) (envelope-from daniel.nicolescu10@gmail.com) Received: by mail-vs1-xe36.google.com with SMTP id t10so14194513vsp.1; Tue, 17 Mar 2020 08:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3vDtOR5lpMHvIxTa8hRILn5gtY9Q8tCMlNUXVSHVxKg=; b=p3LgGn8kAzEn+O79pvynf7uqfrG/n3IiikkVw6JfpgijYuvLiaYBgpPg7TwWWwRsha I4Oi4+292ew6p2d59jvnn7hZhKcsfZ4J/x+WnY96ucB+8a15knHa9uz6zp9DUGT2GYgB 2Cjd89UYYLvHVKvyKG2QUHZ9+YR0vCk5jkBn+49AE10WM18rAuy5twYpPFu1/T0tWaTg /z4aoJRtKyGaAT9GUrYKzxNYpDqfzHh20tsP6V73LaD2h5rpXvEwKlVkIDjQ8eh5xJ6B Ql5Lc5fYcfAr3R8Dk/XyMFYy0tTbqupYZHwv7eeBs2QmgYXtCEv/pAGZRjt5jADHAOxL kCag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3vDtOR5lpMHvIxTa8hRILn5gtY9Q8tCMlNUXVSHVxKg=; b=THbe12t3p6dh4O0NZ5VUYyzhinsVbRWjLgNXeTS96di3B2Y9oX0QDcrao8KI7L11xv A7bsZ+Ixbj0xEpJe78GvW+Fu4dWtwpE/kceVKXTn1LTZg8wmQELaSd7i6FMtaHc8LaDJ NNroW75DWrfCyk3/4v5AgT/gtjU+JBnnl0cQgRDGFISXRxRS69zO++obOlMsIYYtFUzJ 2qMcsVI84N1xLauCCF5a1wz1EVGFeOpMga3Wg9OuyWQMepClxko87eUtLikbMOJ+k++W 5xV5EkbssmHgeVTGQlNi7uB4ykQIeEbA4k6v6QEUM2dVUMyfXN9zHr7f+rD+8WYapBgo lkUg== X-Gm-Message-State: ANhLgQ2BcOS4sy8dwueRz3/xKhUqTyfZGGM6pUrLnvJjLkqWTbN9CVTv nIOZ7AP2tjgPMSAeTUXATtrSEClM+AxgT9X8o0k= X-Google-Smtp-Source: ADFU+vuP+T7cPgl/cyX11eZooU97msw3IgK7/CySiKwXQUiSD9A87Bq69dGrVPWzgH+tLQk3kUzy8TernyLEITHLV9Y= X-Received: by 2002:a67:2f83:: with SMTP id v125mr3751240vsv.100.1584459666913; Tue, 17 Mar 2020 08:41:06 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Daniel Nicolescu Date: Tue, 17 Mar 2020 17:40:55 +0200 Message-ID: Subject: Re: USB device passthrough implementation To: Wojciech Puchar Cc: freebsd-virtualization@freebsd.org, freebsd-hackers@freebsd.org, patrascu.naina14@gmail.com, Mihai Carabas X-Rspamd-Queue-Id: 48hcpD0C93z45Hf X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=p3LgGn8k; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of danielnicolescu10@gmail.com designates 2607:f8b0:4864:20::e36 as permitted sender) smtp.mailfrom=danielnicolescu10@gmail.com X-Spamd-Result: default: False [-1.50 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.59), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[6.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Mar 2020 15:41:09 -0000 Sure, when we will get there. Thank you On Tue, Mar 17, 2020 at 4:09 PM Wojciech Puchar wrote: > > We want to implement the USB pass through feature for bhyve so that users > > will be able to pass through individual USB devices like in QEMU. We are > > i want to be your beta tester. > > > trying to emulate the USB controller and pass to the virtual machine only > > one port. > > > > * We need to disconnect the USB device from host: we are currently not > very > > familiar with the kernel API to perform this > > in most cases you actually don't have to. > just use /dev/ugen/* devices > simply make it as a parameter for bhyve > > usbpassthrough=a/device > > where a is virtual USB device number (i don't think you want to emulate > more than one bus - one xhci is already done in bhyve), and device would > be like /dev/ugenX.Y > > From owner-freebsd-virtualization@freebsd.org Wed Mar 18 04:06:00 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 12E0F27B872 for ; Wed, 18 Mar 2020 04:06:00 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48hxKg0kRKz4f7Z for ; Wed, 18 Mar 2020 04:05:58 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-qv1-xf2a.google.com with SMTP id cy12so6294292qvb.7 for ; Tue, 17 Mar 2020 21:05:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Td9QaUtba5Rb8W/efKc3ylBCzI041eGd3Y+zlRlDM/g=; b=OBLhsOMPiMKlVqfYmTxKEn94RRPmjIs6Bl27BJo+YNtQXqyEzywjsGxu5X4QdNI1Eb BdUTfLEV1qKIBU4um+kGL8kDja8gumYUeo7mrk3qlLiM+ZvXLmFwroAuXdARvr5pz3oJ pOz/1O1FbCmxKGrUSw5XtaD68wCVFstYEVM5VkNpHnzpMNVqsbmzM6VsJtWCsRCvxilp 95Cc8EJB5wqO68pxMdxBHIakFa0FGulEFRavPAz8N4XCsBRvG5QEHCSki2FD5eW1+rq5 AruqUb7fehIV+fEL2zJCY5wI+38glAP8DKWlNdtYy4DkSLKHMJhzdIQLIRNqQ4CcVXLf UxAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Td9QaUtba5Rb8W/efKc3ylBCzI041eGd3Y+zlRlDM/g=; b=eRir6q5Lg5PRlnnOl0ahnBIokkiIZ01cm1NNWJ+02M64kLnC5bYTWDTO9zOy5IEGMf kDr6izf3LgIUr3uPhLtJ00NPWSmtnVQ47kQXC7zK5VYLWbcSHLTYAsjFWe2JmrEGuPAC AECmT7POGH7I8q8Mtvfm8wcAoOxXgDbNSb1NTrwXMltO8BM3PSMDkDY/5ZEIKBmx1rfv XbhNV2bovP3MD5eV7RP8ddXYQsRbVUwOh9gc2PMTju3uo4czYy5Oxnbmv++pcUmft1dn ZQkN9MGzUDp8IO+ijmULMZG+XeyMFalrn8PPgG0s4mApg/S10YRXwOIJA9yw9zLRh6V6 1q/Q== X-Gm-Message-State: ANhLgQ1KMyG4kC+NAdgmHhtKgJrvLqjpvDR7vq6EFu+yjtl9iH0xTAlr 0Qvsooh4ihM/9xuwlSFW0mB7JTcPO/y+RSraafb4MEguDYw= X-Google-Smtp-Source: ADFU+vsGMB7ej0RBeQKYOMfa5ngOGQMYU4IJUXZICwkb5CXgHX/xu1pzsb7iLX8aBM67hwkT0OMd97/EVfb7x6uPSfY= X-Received: by 2002:a0c:edc5:: with SMTP id i5mr2418942qvr.16.1584504357504; Tue, 17 Mar 2020 21:05:57 -0700 (PDT) MIME-Version: 1.0 From: Wanpeng Qian Date: Wed, 18 Mar 2020 13:05:40 +0900 Message-ID: Subject: bhyve: passthrough SMART info from host nvme controller To: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48hxKg0kRKz4f7Z X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=OBLhsOMP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of wanpengqian@gmail.com designates 2607:f8b0:4864:20::f2a as permitted sender) smtp.mailfrom=wanpengqian@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ip: (-5.56), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[a.2.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2020 04:06:00 -0000 Hello all. I have an idea about provide SMART of nvme controller. Currently bhyve's nvme controller didn't provide any SMART information. if we provide fake SMART, that is meaningless. I am thinking we can read the host nvme controller SMART and provide that to guest. such as nvme,disk1.img,maxq=16,qsz=8,ioslots=1,sectsz=512,ser=QNVME778,smart=nvme0 I have done some draft code. and it work. here is the output of nvmecontrol identify and logpage. root@smart:~ # nvmecontrol identify nvme0 Controller Capabilities/Features ================================ Vendor ID: fb5d Subsystem Vendor ID: 0000 Serial Number: QNVME778 Model Number: bhyve-NVMe Firmware Version: 1.0 Recommended Arb Burst: 4 IEEE OUI Identifier: 58 9c fc Multi-Path I/O Capabilities: Not Supported Max Data Transfer Size: 2097152 bytes Controller ID: 0x0000 Version: 1.3.0 Admin Command Set Attributes ============================ Security Send/Receive: Not Supported Format NVM: Supported Firmware Activate/Download: Not Supported Namespace Managment: Not Supported Device Self-test: Not Supported Directives: Not Supported NVMe-MI Send/Receive: Not Supported Virtualization Management: Not Supported Doorbell Buffer Config: Not Supported Get LBA Status: Not Supported Sanitize: Not Supported Abort Command Limit: 3 Async Event Request Limit: 5 Number of Firmware Slots: N/A Firmware Slot 1 Read-Only: N/A Per-Namespace SMART Log: No Error Log Page Entries: 1 Number of Power States: 2 Total NVM Capacity: 0 bytes Unallocated NVM Capacity: 0 bytes NVM Command Set Attributes ========================== Submission Queue Entry Size Max: 64 Min: 64 Completion Queue Entry Size Max: 16 Min: 16 Number of Namespaces: 1 Compare Command: Not Supported Write Uncorrectable Command: Not Supported Dataset Management Command: Not Supported Write Zeroes Command: Not Supported Save Features: Not Supported Reservations: Not Supported Timestamp feature: Not Supported Verify feature: Not Supported Fused Operation Support: Not Supported Format NVM Attributes: All-NVM Erase, All-NVM Format Volatile Write Cache: Not Present NVM Subsystem Name: root@smart:~ # nvmecontrol logpage -p 2 nvme0 SMART/Health Information Log ============================ Critical Warning State: 0x00 Available spare: 0 Temperature: 0 Device reliability: 0 Read only: 0 Volatile memory backup: 0 Temperature: 309 K, 35.85 C, 96.53 F Available spare: 100 Available spare threshold: 10 Percentage used: 0 Data units (512,000 byte) read: 173065 Data units written: 9381970 Host read commands: 4123220 Host write commands: 846568092 Controller busy time (minutes): 948 Power cycles: 918 Power on hours: 2631 Unsafe shutdowns: 812 Media errors: 0 No. error info log entries: 2546 Warning Temp Composite Time: 75 Error Temp Composite Time: 58 Temperature 1 Transition Count: 0 Temperature 2 Transition Count: 0 Total Time For Temperature 1: 0 Total Time For Temperature 2: 0 But currently bhyve has Capsicum capability, I cannot open /dev/nvme0 within pci_nvme.c without extra code. (currently I just disable the Capsicum capability) any feedback? From owner-freebsd-virtualization@freebsd.org Wed Mar 18 12:57:48 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8A65025F608 for ; Wed, 18 Mar 2020 12:57:48 +0000 (UTC) (envelope-from crowston@protonmail.com) Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48j97G0l0cz3yDv for ; Wed, 18 Mar 2020 12:57:45 +0000 (UTC) (envelope-from crowston@protonmail.com) Date: Wed, 18 Mar 2020 12:57:33 +0000 To: Wanpeng Qian From: Robert Crowston Cc: FreeBSD virtualization Reply-To: Robert Crowston Subject: Re: bhyve: passthrough SMART info from host nvme controller Message-ID: In-Reply-To: References: Feedback-ID: 2OVbcR1yHYpdkD8cgQllkFwcuMVZg_LiVMMPvptooFDfHD_03MuQO4ZaF626jWHZYFEhNR2cmIbZ53j4QGWMBQ==:Ext:ProtonMail MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=7.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mail.protonmail.ch X-Rspamd-Queue-Id: 48j97G0l0cz3yDv X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.09 / 15.00]; HAS_REPLYTO(0.00)[crowston@protonmail.com]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[protonmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.00)[ip: (-9.80), ipnet: 185.70.40.0/24(-4.89), asn: 62371(-3.91), country: CH(0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[131.40.70.185.list.dnswl.org : 127.0.5.1]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=default]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2020 12:57:48 -0000 I don't see why the guest needs to know about an implementation detail of t= he host hardware. If the host hardware is failing, that's something to be fixed on the host. = The guest should never realize that something was broken. Just my opinion. =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original Me= ssage =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 On Wednesday, 18 March 2020 04:05, Wanpeng Qian wro= te: > Hello all. > > I have an idea about provide SMART of nvme controller. > > Currently bhyve's nvme controller didn't provide any SMART information. > if we provide fake SMART, that is meaningless. > > I am thinking we can read the host nvme controller SMART and provide > that to guest. > > such as > > nvme,disk1.img,maxq=3D16,qsz=3D8,ioslots=3D1,sectsz=3D512,ser=3DQNVME778,= smart=3Dnvme0 > > I have done some draft code. and it work. > here is the output of nvmecontrol identify and logpage. > > root@smart:~ # nvmecontrol identify nvme0 > Controller Capabilities/Features > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Vendor ID: fb5d > Subsystem Vendor ID: 0000 > Serial Number: QNVME778 > Model Number: bhyve-NVMe > Firmware Version: 1.0 > Recommended Arb Burst: 4 > IEEE OUI Identifier: 58 9c fc > Multi-Path I/O Capabilities: Not Supported > Max Data Transfer Size: 2097152 bytes > Controller ID: 0x0000 > Version: 1.3.0 > > Admin Command Set Attributes > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Security Send/Receive: Not Supported > Format NVM: Supported > Firmware Activate/Download: Not Supported > Namespace Managment: Not Supported > Device Self-test: Not Supported > Directives: Not Supported > NVMe-MI Send/Receive: Not Supported > Virtualization Management: Not Supported > Doorbell Buffer Config: Not Supported > Get LBA Status: Not Supported > Sanitize: Not Supported > Abort Command Limit: 3 > Async Event Request Limit: 5 > Number of Firmware Slots: N/A > Firmware Slot 1 Read-Only: N/A > Per-Namespace SMART Log: No > Error Log Page Entries: 1 > Number of Power States: 2 > Total NVM Capacity: 0 bytes > Unallocated NVM Capacity: 0 bytes > > NVM Command Set Attributes > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Submission Queue Entry Size > Max: 64 > Min: 64 > Completion Queue Entry Size > Max: 16 > Min: 16 > Number of Namespaces: 1 > Compare Command: Not Supported > Write Uncorrectable Command: Not Supported > Dataset Management Command: Not Supported > Write Zeroes Command: Not Supported > Save Features: Not Supported > Reservations: Not Supported > Timestamp feature: Not Supported > Verify feature: Not Supported > Fused Operation Support: Not Supported > Format NVM Attributes: All-NVM Erase, All-NVM Format > Volatile Write Cache: Not Present > > NVM Subsystem Name: > root@smart:~ # nvmecontrol logpage -p 2 nvme0 > SMART/Health Information Log > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > > Critical Warning State: 0x00 > Available spare: 0 > Temperature: 0 > Device reliability: 0 > Read only: 0 > Volatile memory backup: 0 > Temperature: 309 K, 35.85 C, 96.53 F > Available spare: 100 > Available spare threshold: 10 > Percentage used: 0 > Data units (512,000 byte) read: 173065 > Data units written: 9381970 > Host read commands: 4123220 > Host write commands: 846568092 > Controller busy time (minutes): 948 > Power cycles: 918 > Power on hours: 2631 > Unsafe shutdowns: 812 > Media errors: 0 > No. error info log entries: 2546 > Warning Temp Composite Time: 75 > Error Temp Composite Time: 58 > Temperature 1 Transition Count: 0 > Temperature 2 Transition Count: 0 > Total Time For Temperature 1: 0 > Total Time For Temperature 2: 0 > > But currently bhyve has Capsicum capability, I cannot > open /dev/nvme0 within pci_nvme.c without extra code. > (currently I just disable the Capsicum capability) > > any feedback? > > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@freebsd.org Wed Mar 18 13:31:03 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A5B252600D0 for ; Wed, 18 Mar 2020 13:31:03 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48j9sf2TLbz4Fc0 for ; Wed, 18 Mar 2020 13:31:02 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-qv1-xf2a.google.com with SMTP id c28so12829678qvb.10 for ; Wed, 18 Mar 2020 06:31:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=xs+nrHcJFve25wL/Orkbl7t5/bKt/Yg6mRLOw2i1IFw=; b=oV8CIL/s1kDZ99mfJx0/sHYbGjaPKJHrvHNjzZRkuBKVftdEREVRJS6uQy0ioNIdTi sNu6VOheFetoMdw9B22eFoGG5W9DPj5oSpBEONmlwL48F8rMOOpRxcQwju9jner4yrv9 gWJX03Lbf1Rcgv8oBPD6XmNuUPigIEs78A2T+TvWAD9zJYMAJPzhxH3V32m2PFQ5/7xf 0zV5W1ZXipW/AwhsZFW6llypzSGhTOpR0IT1ZFYfM7+ReD33ul2CLhSZOCVooIuNV/82 KJL9kOxsHA9+q8HmgKsR4lLbM49wqy0wN5G/PPN8H4N9I699YSYNoHozB/aclmbHFpae mdDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=xs+nrHcJFve25wL/Orkbl7t5/bKt/Yg6mRLOw2i1IFw=; b=PVPFdZ8TNcPhyERRRt/YABXhmdvuSwJEyAHg735YtktjteMXi7EPg8xQI1UjM7PSx2 1BkvzjDn6VvgnC7v4KoCbUsoLyA8QqvtB8k9asjA/hwpe0vvTaeH0BrKKiv3KnYZB4BW RpciLywgmFhDbYImYHhVCK15+rA+R2Cu5d9/J+35ZmRuc7M1Zc8lr+Jo0M34Av4KIRQF 4FctCspO7X973jMfuwXTyHT85/GeunoEUP8k9K+4q88bbfg/CM8ee4ymkzuxzjK8Nsww Hgk+M8pAMJMZkh+iWsOHDPk55D+OFslTERw+PH51Z5jWNIIhJXVy78QWjbQX9PryWNXU 1YAA== X-Gm-Message-State: ANhLgQ2rj66B40+TMqaQF+QQrP3g0I2WYVq7QIWtgzbWf5ZUGzfA4yGh 8vi0XwbnSGD5eVB3n5CdXrGhqAnj/kz7eAhmic8= X-Google-Smtp-Source: ADFU+vvqZilX0fJYNcVulTgZAu57NSrHQHgCLaaWITExCQB7FVnj4+Lxhky6D8QFz24m0kKjVTTa9vRfubdY3kq2rkA= X-Received: by 2002:ad4:4a89:: with SMTP id h9mr4279334qvx.168.1584538260007; Wed, 18 Mar 2020 06:31:00 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Wanpeng Qian Date: Wed, 18 Mar 2020 22:30:44 +0900 Message-ID: Subject: Re: bhyve: passthrough SMART info from host nvme controller To: Robert Crowston Cc: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 X-Rspamd-Queue-Id: 48j9sf2TLbz4Fc0 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=oV8CIL/s; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of wanpengqian@gmail.com designates 2607:f8b0:4864:20::f2a as permitted sender) smtp.mailfrom=wanpengqian@gmail.com X-Spamd-Result: default: False [-2.90 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[protonmail.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-6.08), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[a.2.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2020 13:31:03 -0000 SGkgUm9iZXJ0DQoNCj4gSSBkb24ndCBzZWUgd2h5IHRoZSBndWVzdCBuZWVkcyB0byBrbm93IGFi b3V0IGFuIGltcGxlbWVudGF0aW9uIGRldGFpbCBvZiB0aGUgaG9zdCBoYXJkd2FyZS4NCj4gSWYg dGhlIGhvc3QgaGFyZHdhcmUgaXMgZmFpbGluZywgdGhhdCdzIHNvbWV0aGluZyB0byBiZSBmaXhl ZCBvbiB0aGUgaG9zdC4gVGhlIGd1ZXN0IHNob3VsZCBuZXZlciByZWFsaXplIHRoYXQgc29tZXRo aW5nIHdhcyBicm9rZW4uDQoNCkkgdG90YWxseSBhZ3JlZSB3aXRoIHlvdSBpbiB0aGlzIG9waW5p b24uIGV4Y2VwdCBvbmUgdGltZSB3aGVuIEkgYW0NCnJ1bm5pbmcgYSBOQVMgdmlydHVhbCBtYWNo aW5lLg0KcHJvdmlkaW5nIHR3byBudm1lIGRpc2tzIGZvciBjYWNoZSBwdXJwb3NlLiB0aGUgTkFT IHNlcnZlciBjYW5ub3QgdXNlDQp0aGVtIGF0IGZpcnN0Lg0KDQpjdXJyZW50bHkgdGhlIFNNQVJU IGRhdGEgc3RydWN0dXJlIGlzIGFsbCB6ZXJvLiBzbyBmb2xsb3dpbmcgdHdvIGZpZWxkDQpvZiBh cmUgemVybyB3aGVuIHByZXNlbnQgdG8gVk0uLg0KDQpBdmFpbGFibGUgc3BhcmU6IDANCkF2YWls YWJsZSBzcGFyZSB0aHJlc2hvbGQ6IDANCg0KVGhlIE5BUyBzZXJ2ZXIgaXMgdGhpbmtpbmcgdGhl IGRpc2tzIGFyZSBmYWxsaW5nIGFuZCBzdWdnZXN0IHJlcGxhY2luZyB0aGVtLg0KSSBoYXZlIHRv IHBhdGNoIGJoeXZlLCBwcm92aWRpbmcgc29tZSBmYWtlIGRhdGEuDQoNCkFuZCBJIGFtIHRoaW5r aW5nIGlmIHdlIGNhbiBwcm92aWRlIHRoZSByZWFsIGhhcmR3YXJlIFNNQVJUIHRvIHZpcnR1YWwg bWFjaGluZSwNCm1heSBiZSB0aGUgdmlydHVhbCBtYWNoaW5lIGNhbiBtb25pdG9yIHRoYXQgaW5m b3JtYXRpb24gYW5kIHNlbmQgYWxlcnQgd2hlbg0KaGFyZHdhcmUgaXMgZmFsbGluZy4gaW5zdGVh ZCBvZiB0aGUgdXNpbmcgc21hcnRtb250b29scyBpbiBGcmVlQlNEIGhvc3QuDQoNCkFuZCB0aGF0 IGlzIHlvdXIgZGVjaXNpb24gd2hldGhlciB5b3UgcGFzc3Rocm91Z2ggdGhpcyBpbmZvcm1hdGlv biBvciBub3QuDQp0aGUgc3lzdGVtIGp1c3QgcHJvdmlkZSB0aGlzIGFiaWxpdHkuDQoNCkJSLg0K DQpRaWFuDQoNCk9uIFdlZCwgTWFyIDE4LCAyMDIwIGF0IDk6NTcgUE0gUm9iZXJ0IENyb3dzdG9u IDxjcm93c3RvbkBwcm90b25tYWlsLmNvbT4gd3JvdGU6DQo+DQo+IEkgZG9uJ3Qgc2VlIHdoeSB0 aGUgZ3Vlc3QgbmVlZHMgdG8ga25vdyBhYm91dCBhbiBpbXBsZW1lbnRhdGlvbiBkZXRhaWwgb2Yg dGhlIGhvc3QgaGFyZHdhcmUuDQo+DQo+IElmIHRoZSBob3N0IGhhcmR3YXJlIGlzIGZhaWxpbmcs IHRoYXQncyBzb21ldGhpbmcgdG8gYmUgZml4ZWQgb24gdGhlIGhvc3QuIFRoZSBndWVzdCBzaG91 bGQgbmV2ZXIgcmVhbGl6ZSB0aGF0IHNvbWV0aGluZyB3YXMgYnJva2VuLg0KPg0KPiBKdXN0IG15 IG9waW5pb24uDQo+DQo+DQo+IOKAkOKAkOKAkOKAkOKAkOKAkOKAkCBPcmlnaW5hbCBNZXNzYWdl IOKAkOKAkOKAkOKAkOKAkOKAkOKAkA0KPiBPbiBXZWRuZXNkYXksIDE4IE1hcmNoIDIwMjAgMDQ6 MDUsIFdhbnBlbmcgUWlhbiA8d2FucGVuZ3FpYW5AZ21haWwuY29tPiB3cm90ZToNCj4NCj4gPiBI ZWxsbyBhbGwuDQo+ID4NCj4gPiBJIGhhdmUgYW4gaWRlYSBhYm91dCBwcm92aWRlIFNNQVJUIG9m IG52bWUgY29udHJvbGxlci4NCj4gPg0KPiA+IEN1cnJlbnRseSBiaHl2ZSdzIG52bWUgY29udHJv bGxlciBkaWRuJ3QgcHJvdmlkZSBhbnkgU01BUlQgaW5mb3JtYXRpb24uDQo+ID4gaWYgd2UgcHJv dmlkZSBmYWtlIFNNQVJULCB0aGF0IGlzIG1lYW5pbmdsZXNzLg0KPiA+DQo+ID4gSSBhbSB0aGlu a2luZyB3ZSBjYW4gcmVhZCB0aGUgaG9zdCBudm1lIGNvbnRyb2xsZXIgU01BUlQgYW5kIHByb3Zp ZGUNCj4gPiB0aGF0IHRvIGd1ZXN0Lg0KPiA+DQo+ID4gc3VjaCBhcw0KPiA+DQo+ID4gbnZtZSxk aXNrMS5pbWcsbWF4cT0xNixxc3o9OCxpb3Nsb3RzPTEsc2VjdHN6PTUxMixzZXI9UU5WTUU3Nzgs c21hcnQ9bnZtZTANCj4gPg0KPiA+IEkgaGF2ZSBkb25lIHNvbWUgZHJhZnQgY29kZS4gYW5kIGl0 IHdvcmsuDQo+ID4gaGVyZSBpcyB0aGUgb3V0cHV0IG9mIG52bWVjb250cm9sIGlkZW50aWZ5IGFu ZCBsb2dwYWdlLg0KPiA+DQo+ID4gcm9vdEBzbWFydDp+ICMgbnZtZWNvbnRyb2wgaWRlbnRpZnkg bnZtZTANCj4gPiBDb250cm9sbGVyIENhcGFiaWxpdGllcy9GZWF0dXJlcw0KPiA+DQo+ID4gPT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT0NCj4gPg0KPiA+IFZlbmRvciBJRDogZmI1ZA0KPiA+IFN1YnN5 c3RlbSBWZW5kb3IgSUQ6IDAwMDANCj4gPiBTZXJpYWwgTnVtYmVyOiBRTlZNRTc3OA0KPiA+IE1v ZGVsIE51bWJlcjogYmh5dmUtTlZNZQ0KPiA+IEZpcm13YXJlIFZlcnNpb246IDEuMA0KPiA+IFJl Y29tbWVuZGVkIEFyYiBCdXJzdDogNA0KPiA+IElFRUUgT1VJIElkZW50aWZpZXI6IDU4IDljIGZj DQo+ID4gTXVsdGktUGF0aCBJL08gQ2FwYWJpbGl0aWVzOiBOb3QgU3VwcG9ydGVkDQo+ID4gTWF4 IERhdGEgVHJhbnNmZXIgU2l6ZTogMjA5NzE1MiBieXRlcw0KPiA+IENvbnRyb2xsZXIgSUQ6IDB4 MDAwMA0KPiA+IFZlcnNpb246IDEuMy4wDQo+ID4NCj4gPiBBZG1pbiBDb21tYW5kIFNldCBBdHRy aWJ1dGVzDQo+ID4NCj4gPiA9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09DQo+ID4NCj4g PiBTZWN1cml0eSBTZW5kL1JlY2VpdmU6IE5vdCBTdXBwb3J0ZWQNCj4gPiBGb3JtYXQgTlZNOiBT dXBwb3J0ZWQNCj4gPiBGaXJtd2FyZSBBY3RpdmF0ZS9Eb3dubG9hZDogTm90IFN1cHBvcnRlZA0K PiA+IE5hbWVzcGFjZSBNYW5hZ21lbnQ6IE5vdCBTdXBwb3J0ZWQNCj4gPiBEZXZpY2UgU2VsZi10 ZXN0OiBOb3QgU3VwcG9ydGVkDQo+ID4gRGlyZWN0aXZlczogTm90IFN1cHBvcnRlZA0KPiA+IE5W TWUtTUkgU2VuZC9SZWNlaXZlOiBOb3QgU3VwcG9ydGVkDQo+ID4gVmlydHVhbGl6YXRpb24gTWFu YWdlbWVudDogTm90IFN1cHBvcnRlZA0KPiA+IERvb3JiZWxsIEJ1ZmZlciBDb25maWc6IE5vdCBT dXBwb3J0ZWQNCj4gPiBHZXQgTEJBIFN0YXR1czogTm90IFN1cHBvcnRlZA0KPiA+IFNhbml0aXpl OiBOb3QgU3VwcG9ydGVkDQo+ID4gQWJvcnQgQ29tbWFuZCBMaW1pdDogMw0KPiA+IEFzeW5jIEV2 ZW50IFJlcXVlc3QgTGltaXQ6IDUNCj4gPiBOdW1iZXIgb2YgRmlybXdhcmUgU2xvdHM6IE4vQQ0K PiA+IEZpcm13YXJlIFNsb3QgMSBSZWFkLU9ubHk6IE4vQQ0KPiA+IFBlci1OYW1lc3BhY2UgU01B UlQgTG9nOiBObw0KPiA+IEVycm9yIExvZyBQYWdlIEVudHJpZXM6IDENCj4gPiBOdW1iZXIgb2Yg UG93ZXIgU3RhdGVzOiAyDQo+ID4gVG90YWwgTlZNIENhcGFjaXR5OiAwIGJ5dGVzDQo+ID4gVW5h bGxvY2F0ZWQgTlZNIENhcGFjaXR5OiAwIGJ5dGVzDQo+ID4NCj4gPiBOVk0gQ29tbWFuZCBTZXQg QXR0cmlidXRlcw0KPiA+DQo+ID4gPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0KPiA+DQo+ID4gU3VibWlz c2lvbiBRdWV1ZSBFbnRyeSBTaXplDQo+ID4gTWF4OiA2NA0KPiA+IE1pbjogNjQNCj4gPiBDb21w bGV0aW9uIFF1ZXVlIEVudHJ5IFNpemUNCj4gPiBNYXg6IDE2DQo+ID4gTWluOiAxNg0KPiA+IE51 bWJlciBvZiBOYW1lc3BhY2VzOiAxDQo+ID4gQ29tcGFyZSBDb21tYW5kOiBOb3QgU3VwcG9ydGVk DQo+ID4gV3JpdGUgVW5jb3JyZWN0YWJsZSBDb21tYW5kOiBOb3QgU3VwcG9ydGVkDQo+ID4gRGF0 YXNldCBNYW5hZ2VtZW50IENvbW1hbmQ6IE5vdCBTdXBwb3J0ZWQNCj4gPiBXcml0ZSBaZXJvZXMg Q29tbWFuZDogTm90IFN1cHBvcnRlZA0KPiA+IFNhdmUgRmVhdHVyZXM6IE5vdCBTdXBwb3J0ZWQN Cj4gPiBSZXNlcnZhdGlvbnM6IE5vdCBTdXBwb3J0ZWQNCj4gPiBUaW1lc3RhbXAgZmVhdHVyZTog Tm90IFN1cHBvcnRlZA0KPiA+IFZlcmlmeSBmZWF0dXJlOiBOb3QgU3VwcG9ydGVkDQo+ID4gRnVz ZWQgT3BlcmF0aW9uIFN1cHBvcnQ6IE5vdCBTdXBwb3J0ZWQNCj4gPiBGb3JtYXQgTlZNIEF0dHJp YnV0ZXM6IEFsbC1OVk0gRXJhc2UsIEFsbC1OVk0gRm9ybWF0DQo+ID4gVm9sYXRpbGUgV3JpdGUg Q2FjaGU6IE5vdCBQcmVzZW50DQo+ID4NCj4gPiBOVk0gU3Vic3lzdGVtIE5hbWU6DQo+ID4gcm9v dEBzbWFydDp+ICMgbnZtZWNvbnRyb2wgbG9ncGFnZSAtcCAyIG52bWUwDQo+ID4gU01BUlQvSGVh bHRoIEluZm9ybWF0aW9uIExvZw0KPiA+DQo+ID4gPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09DQo+ID4NCj4gPiBDcml0aWNhbCBXYXJuaW5nIFN0YXRlOiAweDAwDQo+ID4gQXZh aWxhYmxlIHNwYXJlOiAwDQo+ID4gVGVtcGVyYXR1cmU6IDANCj4gPiBEZXZpY2UgcmVsaWFiaWxp dHk6IDANCj4gPiBSZWFkIG9ubHk6IDANCj4gPiBWb2xhdGlsZSBtZW1vcnkgYmFja3VwOiAwDQo+ ID4gVGVtcGVyYXR1cmU6IDMwOSBLLCAzNS44NSBDLCA5Ni41MyBGDQo+ID4gQXZhaWxhYmxlIHNw YXJlOiAxMDANCj4gPiBBdmFpbGFibGUgc3BhcmUgdGhyZXNob2xkOiAxMA0KPiA+IFBlcmNlbnRh Z2UgdXNlZDogMA0KPiA+IERhdGEgdW5pdHMgKDUxMiwwMDAgYnl0ZSkgcmVhZDogMTczMDY1DQo+ ID4gRGF0YSB1bml0cyB3cml0dGVuOiA5MzgxOTcwDQo+ID4gSG9zdCByZWFkIGNvbW1hbmRzOiA0 MTIzMjIwDQo+ID4gSG9zdCB3cml0ZSBjb21tYW5kczogODQ2NTY4MDkyDQo+ID4gQ29udHJvbGxl ciBidXN5IHRpbWUgKG1pbnV0ZXMpOiA5NDgNCj4gPiBQb3dlciBjeWNsZXM6IDkxOA0KPiA+IFBv d2VyIG9uIGhvdXJzOiAyNjMxDQo+ID4gVW5zYWZlIHNodXRkb3duczogODEyDQo+ID4gTWVkaWEg ZXJyb3JzOiAwDQo+ID4gTm8uIGVycm9yIGluZm8gbG9nIGVudHJpZXM6IDI1NDYNCj4gPiBXYXJu aW5nIFRlbXAgQ29tcG9zaXRlIFRpbWU6IDc1DQo+ID4gRXJyb3IgVGVtcCBDb21wb3NpdGUgVGlt ZTogNTgNCj4gPiBUZW1wZXJhdHVyZSAxIFRyYW5zaXRpb24gQ291bnQ6IDANCj4gPiBUZW1wZXJh dHVyZSAyIFRyYW5zaXRpb24gQ291bnQ6IDANCj4gPiBUb3RhbCBUaW1lIEZvciBUZW1wZXJhdHVy ZSAxOiAwDQo+ID4gVG90YWwgVGltZSBGb3IgVGVtcGVyYXR1cmUgMjogMA0KPiA+DQo+ID4gQnV0 IGN1cnJlbnRseSBiaHl2ZSBoYXMgQ2Fwc2ljdW0gY2FwYWJpbGl0eSwgSSBjYW5ub3QNCj4gPiBv cGVuIC9kZXYvbnZtZTAgd2l0aGluIHBjaV9udm1lLmMgd2l0aG91dCBleHRyYSBjb2RlLg0KPiA+ IChjdXJyZW50bHkgSSBqdXN0IGRpc2FibGUgdGhlIENhcHNpY3VtIGNhcGFiaWxpdHkpDQo+ID4N Cj4gPiBhbnkgZmVlZGJhY2s/DQo+ID4NCj4gPiBmcmVlYnNkLXZpcnR1YWxpemF0aW9uQGZyZWVi c2Qub3JnIG1haWxpbmcgbGlzdA0KPiA+IGh0dHBzOi8vbGlzdHMuZnJlZWJzZC5vcmcvbWFpbG1h bi9saXN0aW5mby9mcmVlYnNkLXZpcnR1YWxpemF0aW9uDQo+ID4gVG8gdW5zdWJzY3JpYmUsIHNl bmQgYW55IG1haWwgdG8gImZyZWVic2QtdmlydHVhbGl6YXRpb24tdW5zdWJzY3JpYmVAZnJlZWJz ZC5vcmciDQo+DQo+DQo= From owner-freebsd-virtualization@freebsd.org Wed Mar 18 17:59:06 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5AA17267D41 for ; Wed, 18 Mar 2020 17:59:06 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48jHpw51XKz4P72 for ; Wed, 18 Mar 2020 17:59:03 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 02IHx1XD019494 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 18 Mar 2020 10:59:01 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 02IHx17K019493; Wed, 18 Mar 2020 10:59:01 -0700 (PDT) (envelope-from jmg) Date: Wed, 18 Mar 2020 10:59:01 -0700 From: John-Mark Gurney To: Wanpeng Qian Cc: FreeBSD virtualization Subject: Re: bhyve: passthrough SMART info from host nvme controller Message-ID: <20200318175901.GI4213@funkthat.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Wed, 18 Mar 2020 10:59:01 -0700 (PDT) X-Rspamd-Queue-Id: 48jHpw51XKz4P72 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [2.60 / 15.00]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; IP_SCORE(-0.44)[ip: (-1.13), ipnet: 208.87.216.0/21(-0.57), asn: 32354(-0.45), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.93)[0.934,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.90)[0.902,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2020 17:59:06 -0000 Wanpeng Qian wrote this message on Wed, Mar 18, 2020 at 13:05 +0900: > But currently bhyve has Capsicum capability, I cannot > open /dev/nvme0 within pci_nvme.c without extra code. > (currently I just disable the Capsicum capability) > > any feedback? Can't you do what something like pci_passthru.c does in passthru_init, and open /dev/nvme0 in pci_nvme_init? Or am I missing something? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@freebsd.org Thu Mar 19 03:09:57 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BDB20278807 for ; Thu, 19 Mar 2020 03:09:57 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-qv1-xf36.google.com (mail-qv1-xf36.google.com [IPv6:2607:f8b0:4864:20::f36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48jX2X53KHz4Vmx for ; Thu, 19 Mar 2020 03:09:56 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-qv1-xf36.google.com with SMTP id cz10so291461qvb.0 for ; Wed, 18 Mar 2020 20:09:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=G0Kcm4lGHc/3RzmvjrCL4DUSGGU0m0vOnQT9PIfYFs0=; b=O/BQiIxe1Yb8T9bsiLWDyZElu8iqpp2H7q93ih7+/a7N78h7FvPfd63/7L1qS4Y2Dl 7SG6YwYvxQiG/TG+1AYVsTjgh4O1sG2aqnUAmJJVHVzQvULTz0XaeUEV+xpEsGcsuPBq ppmY0tvRMI9lEWdipS0kfU8VuffqeWkZ1ZcoV3lAiRtYk7IEX8dlG5I3psT+vFAgml62 yw+HNXOSgO7xh58CDjh6JuNm8tRHH/geEIZTCVCkcriJ8t3SB20Fy57GYirOUUA9i8q4 guKDkao+uWClpqtRRqfgiDLIXN+j8Db0ZN/wGSOUqQ+5wHGknrF7PhSQElzpY692qbm3 2xUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=G0Kcm4lGHc/3RzmvjrCL4DUSGGU0m0vOnQT9PIfYFs0=; b=eZUz98hptfelpaisyZhvswcVZJSByqcjG4Qft4Gesb2ZspP+2fi4FPPzN6aX/Gio3p 87y/yNFhRX1VdealU2h7kvn94LqRqDjV3ScACT8+C5WvzblIfndGe+FYgLGRWtOhJ321 bm+K4OZl8eOnMyS+c1epsag3HiRkJYi8o6PMSQmQLZgaayTF5lKlip7G5BLA9/nHI0Nr 9vaoCIgHNimtlw3ekujoicO9BWcZZtLONBkUw4EKNi3eOOelOTmgkACjE6id5UbRY2CS J/PDSpGuH0g+ws4wyh/9oF0Mv/FyiEM61FUjtQ44IUUvKmRMB9hh1uiF402VWfgkcPtP jCUA== X-Gm-Message-State: ANhLgQ3BbfU2XCH/e5FKEYdnEAP0ZK2re3uwthk6cDTH0iSrpAWpB6KW 77k6xnvrntGYsDs+QW8Y4+wezo206fDZCsZvFhYYJul6aHw= X-Google-Smtp-Source: ADFU+vvm04ADjYqvNlhWE236FK2Z/jI50aSjf6EZAyYU5E7Jll+x7CzqMHi/FJlwEKUGuAFg2mtcD2eNJT2WqVk4STo= X-Received: by 2002:a0c:d603:: with SMTP id c3mr1051795qvj.45.1584587395600; Wed, 18 Mar 2020 20:09:55 -0700 (PDT) MIME-Version: 1.0 References: <20200318175901.GI4213@funkthat.com> In-Reply-To: <20200318175901.GI4213@funkthat.com> From: Wanpeng Qian Date: Thu, 19 Mar 2020 12:09:37 +0900 Message-ID: Subject: Re: bhyve: passthrough SMART info from host nvme controller To: John-Mark Gurney Cc: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48jX2X53KHz4Vmx X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=O/BQiIxe; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of wanpengqian@gmail.com designates 2607:f8b0:4864:20::f36 as permitted sender) smtp.mailfrom=wanpengqian@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[6.3.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; IP_SCORE(0.00)[ip: (-6.16), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2020 03:09:57 -0000 > Can't you do what something like pci_passthru.c does in passthru_init, > and open /dev/nvme0 in pci_nvme_init? Yes, you are correct. but that will make /dev/nvme0 keep open all the time. I just thinking when guest fire a logpage command, open the /dev/nvme0 and get the SMART info. then close /dev/nvme0. On Thu, Mar 19, 2020 at 2:59 AM John-Mark Gurney wrote: > > Wanpeng Qian wrote this message on Wed, Mar 18, 2020 at 13:05 +0900: > > But currently bhyve has Capsicum capability, I cannot > > open /dev/nvme0 within pci_nvme.c without extra code. > > (currently I just disable the Capsicum capability) > > > > any feedback? > > Can't you do what something like pci_passthru.c does in passthru_init, > and open /dev/nvme0 in pci_nvme_init? > > Or am I missing something? > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@freebsd.org Thu Mar 19 03:52:45 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 476AC279B3C for ; Thu, 19 Mar 2020 03:52:45 +0000 (UTC) (envelope-from erleya@gmail.com) Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48jXzv3HV8z3Pbg; Thu, 19 Mar 2020 03:52:43 +0000 (UTC) (envelope-from erleya@gmail.com) Received: by mail-wr1-x42c.google.com with SMTP id z15so1004054wrl.1; Wed, 18 Mar 2020 20:52:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=q0Ta4zkR1LWfdJXVkir67uKukYfop+Tq/fSuZcAg6CA=; b=IbsdT/1z6EidJIxRzgNVoyE5k9NlPQqywFVPerXxvzzbWD7ZiDwN11XdOu+Zzghrmc HAumnuFFAhAPeEvoaBQrWoo+Cf9RwdLOEZmmO4FwAQGcJVoyPtIsRrsI8CjrTlbO197Y YIpG8NyWUvX8VKum1KHYxoPry6PgJxCXu3LNz8CchTOa5rVz2uBgfOfL7P6pEHhMPMmq iCuI3ixzUwZ49V9IefZ6lX146NAeO8LEz2IEAW46Ug+j4XGfgh3abctak3VAgvuAEiO8 Oj4cPOuJLXfF5mY7b9J4eGcs5ZmgZ3Ouc0M3KYQxGsRWTrN6Fj9V9cAiNtl0OGzaZfxv 3pXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=q0Ta4zkR1LWfdJXVkir67uKukYfop+Tq/fSuZcAg6CA=; b=Fd4xbOMl16wOy5973NuKbOW14Gdpr6ZnUYigBhNynj9QgKKK2qwvQYwpBqe0ujUFEx xPXGnrmJ4P/grMo104R1fVgcmM/981TkAFgIqX4LihU9Eirb/cyTfLlKZtRr8aFvY2cN m+9xVv5OVnqRT/SQQji+6+Kw/sHrjUhNr5Y2KMWRTp3anCPrmLao8GlvvW52TAJDsC6A UsPHjNeKx7adMNqsLdYRmh8RYYfCQWZfySY46drEzd0EVBaLqEWXtQhybf1yHPV6Ot7y sYpNBuRLof3TEjcwJc0QCvvsiqY6LRAaATT++fzJpVAZiThJyG53I+fSiPV/X5YhcoTb L8Lg== X-Gm-Message-State: ANhLgQ3x1y3kXuxj4XaXrhtB8i6w9ICmmANEabdRH51SkTm7BnAOhqVB Rl86W2NPa/eqQ3VCJrsP/CtBqxni X-Google-Smtp-Source: ADFU+vt8QPY6Y05+iDMBGWhlMFbF74OXDCe3Q+h4tkdX8J1pn+tgLa8K9ep3cmbyMmCuTwUHpWwdxw== X-Received: by 2002:adf:df82:: with SMTP id z2mr1324309wrl.46.1584589960155; Wed, 18 Mar 2020 20:52:40 -0700 (PDT) Received: from erley.ru (erley.ru. [83.153.157.67]) by smtp.gmail.com with ESMTPSA id t126sm1335864wmb.27.2020.03.18.20.52.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 18 Mar 2020 20:52:39 -0700 (PDT) Received: by erley.ru (OpenSMTPD) with ESMTPSA id 67f5e1ff (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO); Thu, 19 Mar 2020 04:52:32 +0100 (CET) Subject: Re: [GPU pass-through] no compatible bridge window for claimed BAR To: Robert Crowston Cc: Peter Grehan , "freebsd-virtualization@freebsd.org" References: <07921dcf-11d5-f440-a42f-d7ec950cab10@freebsd.org> From: Alex Erley Message-ID: <4674c0fc-2696-3476-55e4-608d11ebece2@gmail.com> Date: Thu, 19 Mar 2020 04:53:04 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 48jXzv3HV8z3Pbg X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=IbsdT/1z; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of erleya@gmail.com designates 2a00:1450:4864:20::42c as permitted sender) smtp.mailfrom=erleya@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[protonmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[67.157.153.83.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; IP_SCORE(0.00)[ip: (-9.10), ipnet: 2a00:1450::/32(-2.39), asn: 15169(-1.65), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[c.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2020 03:52:45 -0000 Hello, You are right about mapping 64-bit BARs. === (1) === The initial value of PCI_EMUL_MEMBASE64 is 0xD000000000 and it corresponds to the 64-bit mmio window set by QWordMemory(...) in DSDT from pci_bus_write_dsdt(), so guest VM gets from UEFI BIOS this 64-bit window: 0x000000D000000000-0x000000D0000FFFFF and it is ignored by guest kernel during boot: ACPI: PCI Root Bridge [PC00] (domain 0000 [bus 00-ff]) acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM acpi PNP0A03:00: host bridge window [mem 0xd000000000-0xd0000fffff window] (ignored, not CPU addressable) acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7]; [io 0x0000-0x0cf7 window] ignored 64-bit BAR code must be fixed to set proper memory region depending on host and guest resources. In general it makes sense only for huge BAR sizes, but in most cases all the guest devices should fit in available 32-bit mmio window. I decided to put it aside and concentrate my efforts on 32-bit BARs. So, I increased the max size in pci_emul_alloc_pbar() from 32 to 256 Mb to have all BARs allocated in 32-bit mmio window (for me it still looks like a hack). As I already tried it before, BHyve failed on VM start and I turned to debug what happened. Allocation goes in 32-bit mmio window 0xc0000000-0xdfffffff (512Mb). Keeping in mind these two requirements from PCI standard: - size of each region must be a power of two (it is already OK), - base address must be aligned on a boundary equal to the region size, BARs are allocated in the order of their indices: BAR[0]: type=2, size=0x1000000 => addr=0xc0000000, nextaddr=0xc1000000 BAR[1]: type=3, size=0x10000000=> addr=0xd0000000, nextaddr=0xe0000000 BAR[3]: type=3, size=0x2000000 => KO, 0xe0000000+size > 0xdfffffff BAR[5]: type=1, size=0x80 I fixed cfginitbar() in pci_passthru.c to allocate BARs in different order to make allocations more compact. They have to be allocated from bigger claimed size to smaller (btw, as it is done on host system). For this, in cfginitbar() I reordered BAR indices before calling pci_emul_alloc_pbar(). This solved all the problems with BAR allocation. (will share my patch when review it a bit more) === (2) === As you pointed before, now I faced the ROM mapping problem. All the ROM BARs are disabled in guest VM and quick look shows that as PCI_BARMAX==5 (=PCIR_MAX_BAR_0 from ), so only 6 BARs initialized in cfginitbar() at pci_passthru.c. It corresponds to registers 10h, 14h, 18h, 1ch, 20h, 24h. Expansion ROM Base Address register 30h is not initialized at all. I'd like to add missing code for it in pci_passthru.c. Physical guest memory map described in pci_emul.c says all ROM BARs must be allocated in 8Mb window 0xf0000000-0xf07fffff and it already matches generated DSDT. From PCI documentation dealing with ROM doesn't seem to be very complicated, but anyway, I'm interested in - What pitfalls will be on that way? - Do we have already some code for ROM BAR somewhere? Any help is welcome. Have a nice day, Alex On 3/15/20 3:20 PM, Robert Crowston wrote: > I suggest you map the BAR into the 32 bit address space, unless you have so many PCI devices that this is not feasible. Just raise the limit of the special 64 bit handling to 1 GB or something big. > - Many/most(?) consumer BIOS/UEFIs map 64 bit bars into the 32 bit address space by default, so this configuration is much more tested for device drivers and guest operating systems. > - Passthrough doesn't work for me at all above the 4GB memory window on my recent AMD system. It exits back to the hypervisor, which then crashes with a failed assert because it doesn't expect to handle this. > > With this tweak it is possible to use the open source nVidia driver on Linux. > > However, your next problem---if you want to use proprietary drivers---is that access to the ROM BAR is not supported or emulated. If that could be fixed, it would be a big step forwards. > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Saturday, 14 March 2020 12:50, Alex Erley wrote: > >> Hello, >> >> Some new findings to share. >> >> 1. Changing PCI_EMUL_MEMBASE64 from 0xD000000000 to any value >> below 0x0440000000 makes bhyve fail when starting VM with message: >> bhyve: failed to initialize BARs for PCI 1/0/0 >> device emulation initialization error: Cannot allocate memory >> >> 2. Having PCI_EMUL_MEMBASE64 set to 0x0440000000 (or above) guest VM >> can not configure BARs of pass-through device properly. >> >> == (a) == >> On BHyve host ppt device is: >> >> >>> devinfo -rv >> >> ... >> pci0 >> hostb0 at slot=0 function=0 dbsf=pci0:0:0:0 >> pcib1 at slot=1 function=0 dbsf=pci0:0:1:0 handle=\SB.PCI0.P0P2 >> I/O ports: 0xe000-0xefff >> I/O memory addresses: >> 0x00c0000000-0x00d30fffff <-- covers all child mem windows >> pci1 >> ppt0 at slot=0 function=0 dbsf=pci0:1:0:0 >> pcib1 I/O port window: 0xe000-0xe07f >> pcib1 memory window: >> 0x00c0000000-0x00cfffffff <-- 256M >> 0x00d0000000-0x00d1ffffff <-- 32M >> 0x00d2000000-0x00d2ffffff <-- 16M >> ppt1 at slot=0 function=1 dbsf=pci0:1:0:1 >> pcib1 memory window: >> 0xd3080000-0xd3083fff <-- 16K >> ... >> and there is no other device attached to pci1. >> >> == (b) == >> On guest VM dmesg shows (timestamps are removed): >> ... >> BIOS-provided physical RAM map: >> BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable >> BIOS-e820: [mem 0x0000000000100000-0x00000000bea95fff] usable >> BIOS-e820: [mem 0x00000000bea96000-0x00000000bea97fff] reserved >> BIOS-e820: [mem 0x00000000bea98000-0x00000000bea99fff] ACPI data >> BIOS-e820: [mem 0x00000000bea9a000-0x00000000beaa8fff] reserved >> BIOS-e820: [mem 0x00000000beaa9000-0x00000000bfb28fff] usable >> BIOS-e820: [mem 0x00000000bfb29000-0x00000000bfb58fff] type 20 >> BIOS-e820: [mem 0x00000000bfb59000-0x00000000bfb7cfff] reserved >> BIOS-e820: [mem 0x00000000bfb7d000-0x00000000bfb81fff] usable >> BIOS-e820: [mem 0x00000000bfb82000-0x00000000bfb88fff] ACPI data >> BIOS-e820: [mem 0x00000000bfb89000-0x00000000bfb8cfff] ACPI NVS >> BIOS-e820: [mem 0x00000000bfb8d000-0x00000000bffcffff] usable >> BIOS-e820: [mem 0x00000000bffd0000-0x00000000bffeffff] reserved >> BIOS-e820: [mem 0x00000000bfff0000-0x00000000bfffffff] usable >> BIOS-e820: [mem 0x0000000100000000-0x000000043fffffff] usable >> ^^^-upper limit for adressable memory >> ... >> PM: Registered nosave memory: [mem 0xc0000000-0xffffffff] >> [mem 0xc0000000-0xffffffff] available for PCI devices >> ... >> pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7] >> pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] >> pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] >> ^-- 128K >> pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff window] >> ^-- 512M >> pci_bus 0000:00: root bus resource [mem 0xf0000000-0xf07fffff window] >> ^-- 8M >> pci_bus 0000:00: root bus resource [bus 00-ff] >> >> == (c) == >> Until now all runs OK. >> >> Guest Linux then allocates memory regions for devices. >> Allocation is done from lower reg (0x10) to higher (0x30) >> for each device (i.e. from 00.0 to 1f.0) on PCI bus. >> >> Here I reordered dmesg output to groups to show continuous RAM regions: >> (pass-through device is marked with ) >> pci 0000:00:01.0: reg 0x24: [io 0x2000-0x207f] >> pci 0000:00:02.0: reg 0x10: [io 0x2080-0x209f] >> pci 0000:00:03.0: reg 0x10: [io 0x20c0-0x20ff] >> ... >> pci 0000:00:00.0: reg 0x30: [mem 0x00000000-0x000007ff pref] >> pci 0000:00:02.0: reg 0x30: [mem 0x00000000-0x000007ff pref] >> pci 0000:00:03.0: reg 0x30: [mem 0x00000000-0x000007ff pref] >> pci 0000:00:1d.0: reg 0x30: [mem 0x00000000-0x000007ff pref] >> pci 0000:00:1e.0: reg 0x30: [mem 0x00000000-0x000007ff pref] >> pci 0000:00:1f.0: reg 0x30: [mem 0x00000000-0x000007ff pref] >> ... >> pci 0000:00:01.0: reg 0x10:[mem 0xc0000000-0xc0ffffff] 16M... 0xc1000000-0xc1ffffff 16M gap >> pci 0000:00:01.0: reg 0x1c:[mem 0xc2000000-0xc3ffffff 64bit pref] 32M >> pci 0000:00:01.1: reg 0x10:[mem 0xc4000000-0xc4003fff]pci 0000:00:02.0: reg 0x14: [mem 0xc4004000-0xc4005fff] >> pci 0000:00:03.0: reg 0x14: [mem 0xc4006000-0xc4007fff] >> pci 0000:00:1d.0: reg 0x10: [mem 0xc4008000-0xc400807f] >> ... 0xc4008080-0xc4ffffff <16M gap >> pci 0000:00:1d.0: reg 0x14: [mem 0xc5000000-0xc5ffffff] 16M >> pci 0000:00:1e.0: reg 0x10: [mem 0xc6000000-0xc6000fff] >> ... 0xc6001000-0xd2ffffff <208M gap >> pci 0000:00:01.0: reg 0x30:[mem 0xd3000000-0xd307ffff pref] 512K >> 0xd3080000-0xdfffffff <208M gap >> pci 0000:00:01.0: reg0x14:[mem 0x440000000-0x44fffffff 64bit pref] 256M^^^- this value is outside allowed range >> >> == (d) == >> So, there is no window for 256M BAR, although there are 2 big gapes >> of 208M in 512M space provided for BAR allocation by PCI bus. >> >> So, BAR reg 0x14 of size 256M for device 01.0 must be inside provisioned >> 512M region 0xc0000000-0xdfffffff. >> But refering to (1) above, setting base address to any value below >> 0x440000000 breaks bhyve on start. >> According to (b), this value corresponds to upper addressable memory >> limit in guest VM. >> >> So I'm blocked here at the moment: >> >> - Guest VM requires a value which BHyve doesn't like. >> - Guest VM allocates BARs with huge gapes. >> >> I have little knowledge about PCI bus internals, although I already read >> some articles on internet. >> Could it be some ACPI trick to do? >> I'd be happy to hear any ideas... >> >> PS >> I suspect that if I take other OS as a guest VM or other pass-through >> GPU model, it would probably allocate BARs properly. >> But this is not what I want for this config. >> There should be a way to allocate 256M BAR in guest Linux. >> >> Have a nice day, >> Alex >> >> >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > > From owner-freebsd-virtualization@freebsd.org Thu Mar 19 03:58:12 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 706D6279C09 for ; Thu, 19 Mar 2020 03:58:12 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48jY6B47mBz48FV for ; Thu, 19 Mar 2020 03:58:10 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-ot1-f51.google.com with SMTP id e9so961739otr.12 for ; Wed, 18 Mar 2020 20:58:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vR7qbuM3QOtVmpSIDWXVZM/3KjDvONW96XkJHm+vz/U=; b=K4gLtMVgv8nPB0a9FgrHwPEC2Kb/7aVQ15Tnmoup7fsvYzk36V5vkD4pJ4+IojoVT8 g95PquMy2gw8z2zs3UMDg770SdZ71CKov1dp7bA1kdSr2tvHAHgru7YU1cUFV6tDYs3g z/X7eFm0y+yNMPlutVfObZF2eKSTsiwtv81EMsngV06YWAavAshdnLcB51TZzHy7x8SI UZEmLRvaiN7T7ooueJLZHifborFVTm8DXV8i6QKVzGNiTuoUbGx866L8nRe1b95f2nH8 5gfK669PUav1wB3Y1Z1CD0F7V5rdfd6U0Y8NrOAHFPJc5DBLau53P0UrrKwNLjUCzsVb iUTA== X-Gm-Message-State: ANhLgQ0Qs9VkHXLOWcXOQmxCnFaDAcd/HFovJ1JzLktO2y3B8fAiXcLO au9ftqcVPSsQPlFSKJaYhYY23hneNG3b4mW4LVo= X-Google-Smtp-Source: ADFU+vvIYwu3dScVCtCparu7H/vY0p3GcqsUfLo12UtXGmET7FBkgV6lIrPVK8RPmAJaXK3/zIOKqy15hNARQqPGjoc= X-Received: by 2002:a9d:228:: with SMTP id 37mr813797otb.52.1584590288291; Wed, 18 Mar 2020 20:58:08 -0700 (PDT) MIME-Version: 1.0 References: <20200318175901.GI4213@funkthat.com> In-Reply-To: From: Jason Tubnor Date: Thu, 19 Mar 2020 14:57:57 +1100 Message-ID: Subject: Re: bhyve: passthrough SMART info from host nvme controller To: Wanpeng Qian Cc: John-Mark Gurney , FreeBSD virtualization X-Rspamd-Queue-Id: 48jY6B47mBz48FV X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jtubnor@gmail.com designates 209.85.210.51 as permitted sender) smtp.mailfrom=jtubnor@gmail.com X-Spamd-Result: default: False [-3.13 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[tubnor.net]; RWL_MAILSPIKE_GOOD(0.00)[51.210.85.209.rep.mailspike.net : 127.0.0.18]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[51.210.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.13)[ip: (-0.99), ipnet: 209.85.128.0/17(-2.97), asn: 15169(-1.65), country: US(-0.05)]; FORGED_SENDER(0.30)[jason@tubnor.net,jtubnor@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[jason@tubnor.net,jtubnor@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2020 03:58:12 -0000 On Thu, 19 Mar 2020 at 14:09, Wanpeng Qian wrote: > > Can't you do what something like pci_passthru.c does in passthru_init, > > and open /dev/nvme0 in pci_nvme_init? > > Yes, you are correct. but that will make /dev/nvme0 keep open all the time. > I just thinking when guest fire a logpage command, open the /dev/nvme0 > and get the SMART info. then close /dev/nvme0. So are you implying that it is safe for a guest to send such a call at anytime? For those that use bhyve for isolation, this sort of facility would be problematic not to mention insecure. From owner-freebsd-virtualization@freebsd.org Thu Mar 19 06:16:38 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 56EFB27C3F6 for ; Thu, 19 Mar 2020 06:16:38 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-qv1-xf36.google.com (mail-qv1-xf36.google.com [IPv6:2607:f8b0:4864:20::f36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48jc9w63vmz4B9T for ; Thu, 19 Mar 2020 06:16:36 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-qv1-xf36.google.com with SMTP id v38so426520qvf.6 for ; Wed, 18 Mar 2020 23:16:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=xGklhA7JXM50xV8fpieUUckeH+CXLS7Y4rGrK0Z1q18=; b=AEkCAzJwgDEdBmvemyUstGaLOOJx05jt1ZvEYTqjhHUno0FlAJsO2PAuces9kW34G0 8cukkfTj8hyQkXAkRrij6PGZ+YI9A0s+P1jD9BcmzYA+keT6+DdRQiM1/6d2T6FUXavb olF6nW0qz+wwVdOe5nTQ6c02P0/3DDKPCgOYxn2k8EFQkMBgo9WH6OKoju53bamb8EZN IioEA03qdhz5X5xNUxCTP11v9EypRRCrnWGNPtuIH8QHsgAZJ1YJK63fbcFfTSpfrIQP eD/oXC+eufUqShHM2DkmESuFjJA1R/t9EfDKhoIwTzKeaV1vNPfmzDxhpvv1Cw3JA5Xe MoOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xGklhA7JXM50xV8fpieUUckeH+CXLS7Y4rGrK0Z1q18=; b=RGDo38ZwnfqQnDy1LVfM1kkd5j3aTZtel9c1xBm22zkB1ZHJlYn7VJCwHwYpWf5hF3 P9YksJnDbJXzIP9AkYZpCbL0R2aZqBrIMGsysme1vkeC+/Ja3f7LwEsjIsJ1+bnIcXup rcvZZFOySJrQkK05i3Ja0mCMFmLrPlwuwq5zVNRIFMNBD4Um1XnyC9CaSkR6GGG03Mu1 nAQLX4GtVtvVBXUMg71YpYU1QmVEphZh1qviO19jBVsjVXBU7iojWnNYXmYKtRr56Sxi e+vT2GCMr1nGzDeQPDjE/IvACvaDJFaD6C5ga05eKjQ0LbIH0BccncSyvzMfEGKFTt7U yrNQ== X-Gm-Message-State: ANhLgQ02ZdSoTEgY6YCoBYmkRJtNB0pX98fB5wn1S+t+sLhMQKZ4o1hr DptszAZLEQtimK8cXpPv1xrU5Ng6+GWj1e6i9NWr9nDe X-Google-Smtp-Source: ADFU+vvcSbZbQf7i9RxeupaOprFFvo9JpN5CJFeEocA1txVzadEWKz1NcVepzqkyx0sZhLnFDFW3waU19Nsxp90aVFY= X-Received: by 2002:a0c:c28c:: with SMTP id b12mr1513552qvi.26.1584598595125; Wed, 18 Mar 2020 23:16:35 -0700 (PDT) MIME-Version: 1.0 References: <20200318175901.GI4213@funkthat.com> In-Reply-To: From: Wanpeng Qian Date: Thu, 19 Mar 2020 15:16:15 +0900 Message-ID: Subject: Re: bhyve: passthrough SMART info from host nvme controller To: Jason Tubnor Cc: John-Mark Gurney , FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48jc9w63vmz4B9T X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=AEkCAzJw; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of wanpengqian@gmail.com designates 2607:f8b0:4864:20::f36 as permitted sender) smtp.mailfrom=wanpengqian@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[6.3.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(0.00)[ip: (-6.87), ipnet: 2607:f8b0::/32(-1.85), asn: 15169(-1.65), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2020 06:16:38 -0000 I dont think that is insecure. as bhyve can passthrough real device to VM. as your point, that make more insecure, right? Such configuration will not enable by default. if user intend to do it, system has this ability instead of not implement. Simple is best, less is secure. I know that. but real world has all kinds of requirements. if you provide more, more people using it. On Thu, Mar 19, 2020 at 12:58 PM Jason Tubnor wrote: > > > > On Thu, 19 Mar 2020 at 14:09, Wanpeng Qian wrote: >> >> > Can't you do what something like pci_passthru.c does in passthru_init, >> > and open /dev/nvme0 in pci_nvme_init? >> >> Yes, you are correct. but that will make /dev/nvme0 keep open all the time. >> I just thinking when guest fire a logpage command, open the /dev/nvme0 >> and get the SMART info. then close /dev/nvme0. > > > So are you implying that it is safe for a guest to send such a call at anytime? For those that use bhyve for isolation, this sort of facility would be problematic not to mention insecure. From owner-freebsd-virtualization@freebsd.org Thu Mar 19 10:20:03 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 392D825AF2E for ; Thu, 19 Mar 2020 10:20:03 +0000 (UTC) (envelope-from crowston@protonmail.com) Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48jjZm2CnWz3Pr6 for ; Thu, 19 Mar 2020 10:19:59 +0000 (UTC) (envelope-from crowston@protonmail.com) Date: Thu, 19 Mar 2020 10:19:54 +0000 To: Alex Erley From: Robert Crowston Cc: Peter Grehan , "freebsd-virtualization@freebsd.org" , Henrik Gulbrandsen Reply-To: Robert Crowston Subject: Re: [GPU pass-through] no compatible bridge window for claimed BAR Message-ID: In-Reply-To: <4674c0fc-2696-3476-55e4-608d11ebece2@gmail.com> References: <07921dcf-11d5-f440-a42f-d7ec950cab10@freebsd.org> <4674c0fc-2696-3476-55e4-608d11ebece2@gmail.com> Feedback-ID: 2OVbcR1yHYpdkD8cgQllkFwcuMVZg_LiVMMPvptooFDfHD_03MuQO4ZaF626jWHZYFEhNR2cmIbZ53j4QGWMBQ==:Ext:ProtonMail MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=7.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mail.protonmail.ch X-Rspamd-Queue-Id: 48jjZm2CnWz3Pr6 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.09 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; HAS_REPLYTO(0.00)[crowston@protonmail.com]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; DKIM_TRACE(0.00)[protonmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.00)[ip: (-9.80), ipnet: 185.70.40.0/24(-4.89), asn: 62371(-3.91), country: CH(0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[131.40.70.185.list.dnswl.org : 127.0.5.1]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=default]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-0.99)[-0.991,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2020 10:20:03 -0000 > This solved all the problems with BAR allocation. Nice! > Do we have already some code for ROM BAR somewhere? Henrik Gulbrandsen was working on expansion ROM support last July (CC'd). H= e posted his work up on https://www.gulbra.net/freebsd-bhyve/. =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original Me= ssage =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 On Thursday, 19 March 2020 03:52, Alex Erley wrote: > Hello, > > You are right about mapping 64-bit BARs. > > =3D=3D=3D (1) =3D=3D=3D > The initial value of PCI_EMUL_MEMBASE64 is 0xD000000000 and it > corresponds to the 64-bit mmio window set by QWordMemory(...) > in DSDT from pci_bus_write_dsdt(), so guest VM gets from > UEFI BIOS this 64-bit window: > 0x000000D000000000-0x000000D0000FFFFF > and it is ignored by guest kernel during boot: > ACPI: PCI Root Bridge [PC00] (domain 0000 [bus 00-ff]) > acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM > Segments MSI] > acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM > acpi PNP0A03:00: host bridge window [mem 0xd000000000-0xd0000fffff > window] (ignored, not CPU addressable) > acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7]; > [io 0x0000-0x0cf7 window] ignored > > 64-bit BAR code must be fixed to set proper memory region depending on > host and guest resources. > In general it makes sense only for huge BAR sizes, but in most cases all > the guest devices should fit in available 32-bit mmio window. > I decided to put it aside and concentrate my efforts on 32-bit BARs. > > So, I increased the max size in pci_emul_alloc_pbar() from 32 to 256 Mb > to have all BARs allocated in 32-bit mmio window (for me it still looks > like a hack). > As I already tried it before, BHyve failed on VM start and I turned to > debug what happened. > > Allocation goes in 32-bit mmio window 0xc0000000-0xdfffffff (512Mb). > Keeping in mind these two requirements from PCI standard: > > - size of each region must be a power of two (it is already OK), > > - base address must be aligned on a boundary equal to the region size, > BARs are allocated in the order of their indices: > BAR[0]: type=3D2, size=3D0x1000000 =3D> addr=3D0xc0000000, nextaddr= =3D0xc1000000 > BAR[1]: type=3D3, size=3D0x10000000=3D> addr=3D0xd0000000, nextaddr= =3D0xe0000000 > BAR[3]: type=3D3, size=3D0x2000000 =3D> KO, 0xe0000000+size > 0xdffff= fff > BAR[5]: type=3D1, size=3D0x80 > > I fixed cfginitbar() in pci_passthru.c to allocate BARs in different > order to make allocations more compact. They have to be allocated fro= m > bigger claimed size to smaller (btw, as it is done on host system). > For this, in cfginitbar() I reordered BAR indices before calling > pci_emul_alloc_pbar(). > This solved all the problems with BAR allocation. > (will share my patch when review it a bit more) > > =3D=3D=3D (2) =3D=3D=3D > As you pointed before, now I faced the ROM mapping problem. > > All the ROM BARs are disabled in guest VM and quick look shows that > as PCI_BARMAX=3D=3D5 (=3DPCIR_MAX_BAR_0 from ), > > > so only 6 BARs initialized in cfginitbar() at pci_passthru.c. > It corresponds to registers 10h, 14h, 18h, 1ch, 20h, 24h. > Expansion ROM Base Address register 30h is not initialized at all. > > I'd like to add missing code for it in pci_passthru.c. > Physical guest memory map described in pci_emul.c says all ROM BARs > must be allocated in 8Mb window 0xf0000000-0xf07fffff and it already > matches generated DSDT. > > From PCI documentation dealing with ROM doesn't seem to be very > complicated, but anyway, I'm interested in > > - What pitfalls will be on that way? > - Do we have already some code for ROM BAR somewhere? > > Any help is welcome. > > Have a nice day, > Alex > > On 3/15/20 3:20 PM, Robert Crowston wrote: > > > > I suggest you map the BAR into the 32 bit address space, unless you hav= e so many PCI devices that this is not feasible. Just raise the limit of th= e special 64 bit handling to 1 GB or something big. > > > > - Many/most(?) consumer BIOS/UEFIs map 64 bit bars into the 32 bit ad= dress space by default, so this configuration is much more tested for devic= e drivers and guest operating systems. > > - Passthrough doesn't work for me at all above the 4GB memory window = on my recent AMD system. It exits back to the hypervisor, which then crashe= s with a failed assert because it doesn't expect to handle this. > > > > With this tweak it is possible to use the open source nVidia driver on = Linux. > > However, your next problem---if you want to use proprietary drivers---i= s that access to the ROM BAR is not supported or emulated. If that could be= fixed, it would be a big step forwards. > > =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Origina= l Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80= =90 > > On Saturday, 14 March 2020 12:50, Alex Erley erleya@gmail.com wrote: > > > > > Hello, > > > Some new findings to share. > > > > > > 1. Changing PCI_EMUL_MEMBASE64 from 0xD000000000 to any value > > > below 0x0440000000 makes bhyve fail when starting VM with message= : > > > bhyve: failed to initialize BARs for PCI 1/0/0 > > > device emulation initialization error: Cannot allocate memory > > > > > > 2. Having PCI_EMUL_MEMBASE64 set to 0x0440000000 (or above) guest VM > > > can not configure BARs of pass-through device properly. > > > =3D=3D (a) =3D=3D > > > On BHyve host ppt device is: > > > > > > > > > > devinfo -rv > > > > > > ... > > > pci0 > > > hostb0 at slot=3D0 function=3D0 dbsf=3Dpci0:0:0:0 > > > pcib1 at slot=3D1 function=3D0 dbsf=3Dpci0:0:1:0 handle=3D\SB.PCI0.P0= P2 > > > I/O ports: 0xe000-0xefff > > > I/O memory addresses: > > > 0x00c0000000-0x00d30fffff <-- covers all child mem windows > > > pci1 > > > ppt0 at slot=3D0 function=3D0 dbsf=3Dpci0:1:0:0 > > > pcib1 I/O port window: 0xe000-0xe07f > > > pcib1 memory window: > > > 0x00c0000000-0x00cfffffff <-- 256M > > > 0x00d0000000-0x00d1ffffff <-- 32M > > > 0x00d2000000-0x00d2ffffff <-- 16M > > > ppt1 at slot=3D0 function=3D1 dbsf=3Dpci0:1:0:1 > > > pcib1 memory window: > > > 0xd3080000-0xd3083fff <-- 16K > > > ... > > > and there is no other device attached to pci1. > > > =3D=3D (b) =3D=3D > > > On guest VM dmesg shows (timestamps are removed): > > > ... > > > BIOS-provided physical RAM map: > > > BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable > > > BIOS-e820: [mem 0x0000000000100000-0x00000000bea95fff] usable > > > BIOS-e820: [mem 0x00000000bea96000-0x00000000bea97fff] reserved > > > BIOS-e820: [mem 0x00000000bea98000-0x00000000bea99fff] ACPI data > > > BIOS-e820: [mem 0x00000000bea9a000-0x00000000beaa8fff] reserved > > > BIOS-e820: [mem 0x00000000beaa9000-0x00000000bfb28fff] usable > > > BIOS-e820: [mem 0x00000000bfb29000-0x00000000bfb58fff] type 20 > > > BIOS-e820: [mem 0x00000000bfb59000-0x00000000bfb7cfff] reserved > > > BIOS-e820: [mem 0x00000000bfb7d000-0x00000000bfb81fff] usable > > > BIOS-e820: [mem 0x00000000bfb82000-0x00000000bfb88fff] ACPI data > > > BIOS-e820: [mem 0x00000000bfb89000-0x00000000bfb8cfff] ACPI NVS > > > BIOS-e820: [mem 0x00000000bfb8d000-0x00000000bffcffff] usable > > > BIOS-e820: [mem 0x00000000bffd0000-0x00000000bffeffff] reserved > > > BIOS-e820: [mem 0x00000000bfff0000-0x00000000bfffffff] usable > > > BIOS-e820: [mem 0x0000000100000000-0x000000043fffffff] usable > > > ^^^-upper limit for adressable memory > > > ... > > > PM: Registered nosave memory: [mem 0xc0000000-0xffffffff] > > > [mem 0xc0000000-0xffffffff] available for PCI devices > > > ... > > > pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7] > > > pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] > > > pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] > > > ^-- 128K > > > pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff window] > > > ^-- 512M > > > pci_bus 0000:00: root bus resource [mem 0xf0000000-0xf07fffff window] > > > ^-- 8M > > > pci_bus 0000:00: root bus resource [bus 00-ff] > > > =3D=3D (c) =3D=3D > > > Until now all runs OK. > > > Guest Linux then allocates memory regions for devices. > > > Allocation is done from lower reg (0x10) to higher (0x30) > > > for each device (i.e. from 00.0 to 1f.0) on PCI bus. > > > Here I reordered dmesg output to groups to show continuous RAM region= s: > > > (pass-through device is marked with ) > > > pci 0000:00:01.0: reg 0x24: [io 0x2000-0x207f] > > > pci 0000:00:02.0: reg 0x10: [io 0x2080-0x209f] > > > pci 0000:00:03.0: reg 0x10: [io 0x20c0-0x20ff] > > > ... > > > pci 0000:00:00.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > > > pci 0000:00:02.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > > > pci 0000:00:03.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > > > pci 0000:00:1d.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > > > pci 0000:00:1e.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > > > pci 0000:00:1f.0: reg 0x30: [mem 0x00000000-0x000007ff pref] > > > ... > > > pci 0000:00:01.0: reg 0x10:[mem 0xc0000000-0xc0ffffff] 16M... 0xc1000= 000-0xc1ffffff 16M gap > > > pci 0000:00:01.0: reg 0x1c:[mem 0xc2000000-0xc3ffffff 64bit pref] 32M > > > pci 0000:00:01.1: reg 0x10:[mem 0xc4000000-0xc4003fff]pci 0000:00:02.= 0: reg 0x14: [mem 0xc4004000-0xc4005fff] > > > pci 0000:00:03.0: reg 0x14: [mem 0xc4006000-0xc4007fff] > > > pci 0000:00:1d.0: reg 0x10: [mem 0xc4008000-0xc400807f] > > > ... 0xc4008080-0xc4ffffff <16M gap > > > pci 0000:00:1d.0: reg 0x14: [mem 0xc5000000-0xc5ffffff] 16M > > > pci 0000:00:1e.0: reg 0x10: [mem 0xc6000000-0xc6000fff] > > > ... 0xc6001000-0xd2ffffff <208M gap > > > pci 0000:00:01.0: reg 0x30:[mem 0xd3000000-0xd307ffff pref] 512K > > > 0xd3080000-0xdfffffff <208M gap > > > pci 0000:00:01.0: reg0x14:[mem 0x440000000-0x44fffffff 64bit pref] 25= 6M^^^- this value is outside allowed range > > > =3D=3D (d) =3D=3D > > > So, there is no window for 256M BAR, although there are 2 big gapes > > > of 208M in 512M space provided for BAR allocation by PCI bus. > > > So, BAR reg 0x14 of size 256M for device 01.0 must be inside provisio= ned > > > 512M region 0xc0000000-0xdfffffff. > > > But refering to (1) above, setting base address to any value below > > > 0x440000000 breaks bhyve on start. > > > According to (b), this value corresponds to upper addressable memory > > > limit in guest VM. > > > So I'm blocked here at the moment: > > > > > > - Guest VM requires a value which BHyve doesn't like. > > > > > > - Guest VM allocates BARs with huge gapes. > > > I have little knowledge about PCI bus internals, although I alrea= dy read > > > some articles on internet. > > > Could it be some ACPI trick to do? > > > I'd be happy to hear any ideas... > > > PS > > > I suspect that if I take other OS as a guest VM or other pass-thr= ough > > > GPU model, it would probably allocate BARs properly. > > > But this is not what I want for this config. > > > There should be a way to allocate 256M BAR in guest Linux. > > > Have a nice day, > > > Alex > > > > > > > > > freebsd-virtualization@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@= freebsd.org" From owner-freebsd-virtualization@freebsd.org Thu Mar 19 11:06:05 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 63A5B25BF0E for ; Thu, 19 Mar 2020 11:06:05 +0000 (UTC) (envelope-from henrik@gulbra.net) Received: from srv.gulbra.net (ec2-174-129-193-206.compute-1.amazonaws.com [174.129.193.206]) by mx1.freebsd.org (Postfix) with ESMTP id 48jkbw2pPqz3GkQ for ; Thu, 19 Mar 2020 11:06:04 +0000 (UTC) (envelope-from henrik@gulbra.net) Received: from srv.gulbra.net (localhost [127.0.0.1]) by srv.gulbra.net (Postfix) with ESMTP id 6888F2E4B for ; Thu, 19 Mar 2020 11:05:58 +0000 (UTC) Received: by srv.gulbra.net (Postfix, from userid 33) id 43F232E4D; Thu, 19 Mar 2020 11:05:58 +0000 (UTC) To: Robert Crowston Subject: Re: [GPU pass-through] no compatible bridge window for claimed BAR X-PHP-Originating-Script: 0:rcube.php MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 19 Mar 2020 12:05:58 +0100 From: Henrik Gulbrandsen Cc: Alex Erley , Peter Grehan , freebsd-virtualization@freebsd.org In-Reply-To: References: <07921dcf-11d5-f440-a42f-d7ec950cab10@freebsd.org> <4674c0fc-2696-3476-55e4-608d11ebece2@gmail.com> Message-ID: <531dd5f05f23ea3b82f4ef1c61ffe038@www.gulbra.net> X-Sender: henrik@gulbra.net User-Agent: Roundcube Webmail/1.2-beta X-Rspamd-Queue-Id: 48jkbw2pPqz3GkQ X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of henrik@gulbra.net has no SPF policy when checking 174.129.193.206) smtp.mailfrom=henrik@gulbra.net X-Spamd-Result: default: False [0.80 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.35)[-0.353,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; IP_SCORE(-0.43)[ipnet: 174.129.0.0/16(0.93), asn: 14618(-3.00), country: US(-0.05)]; HAS_X_POS(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[gulbra.net]; MIME_GOOD(-0.10)[text/plain]; AUTH_NA(1.00)[]; PHP_SCRIPT_ROOT(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_LONG(-0.92)[-0.919,0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[protonmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:174.129.0.0/16, country:US]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_WWW(0.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2020 11:06:05 -0000 On 2020-03-19 11:19, Robert Crowston wrote: > >> Do we have already some code for ROM BAR somewhere? > Henrik Gulbrandsen was working on expansion ROM support last July > (CC'd). He posted his work up on > https://www.gulbra.net/freebsd-bhyve/. > Yeah, I'm still on the list. I'm just keeping a low profile until I find some time to focus on the Video BIOS again. It will probably take some time to review thousands of lines of assembly code, but in the meantime, it would be a good thing to get more of the C code integrated in bhyve. The PCI stuff should be relatively easy to use. As I remember it, only the RFB changes got a bit tricky at the end. /Henrik From owner-freebsd-virtualization@freebsd.org Thu Mar 19 12:55:51 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3FBC425FC58 for ; Thu, 19 Mar 2020 12:55:51 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48jn2Y1t2cz4fNF for ; Thu, 19 Mar 2020 12:55:49 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-lf1-x136.google.com with SMTP id a28so1500915lfr.13 for ; Thu, 19 Mar 2020 05:55:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=KhvNtFg4baNc8z/wEY2frUAdZbPr0V8758lb3Few4Pk=; b=aucJSD/06QXpK9Kql3tB8ehpatpOgsue2Lrz3VPJgMs5ORrYqGM9TE1NaCkfu0L3pN 1rN3Wzv1Acb+oas2jxJJfogdKQ37gSlcK16Dyuvf/40gKL8mv1j91QzVwvXnh7eMC65U 0FuCcGJAIwk4htlmMA9Aer1BBICO3e8dp4B0WQirdAfE19eqisoQSA+L/eoOyFqSDXeF TJrlLgOHyO25C47wb3oVvi3ta9EuSkEY3V5WB7My12nsjkHq6uB4uAWp4WXGlFg7zWfg FRGVDYzTckkWNgBiz0j+uNfmazpgTqntsqcf1zLM8qfy0wlpvJUk99gyQuE0LQuOyZ0d th4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=KhvNtFg4baNc8z/wEY2frUAdZbPr0V8758lb3Few4Pk=; b=R9aqIwdayuG14Pd/o1iDKwm0S3lxy2aYO3X0QchONo6f5N08QDjyVyJk6aM6pDGoHY 3LpoUgbnWKgWBcMQfNd7zvvsAduS0SgRz5ozIKvzgj1FUWWr7y3vhdmOsqaqZwYHDi1c PmR4PlCI0Gh9kK8s5xRHH9xEP8WKW4HgKQaCRDFwuidp3Z/itfHdCD+osFyXdRDCKpei NB6mprJMJNQmPllu65KRgwRrAR67G6S2AyXON0Qmb/Q46IdYQLm5ZEeuS7x+YHbPFAhA gBemFG6FTB9CfUWP+ffgTIRnXeC9PTETg7uqwg0og16L+TPZD1gCzKXkjY1rJmDfVwB3 J6Mw== X-Gm-Message-State: ANhLgQ3Kssz54Nli32Gs79Az73TNEz0S7ZtblelzB36eRQncu/igK6iA dPKa8/1pPGpqhMpnIxr/5oOFD4+7u/K9FRnG8lvGispq/XI= X-Google-Smtp-Source: ADFU+vt1ezwgDbKk7Gb9HLvbPhMoHMt5VBY2NO8ChuGprS/5zle8iIwYxObtPATOiyF7hK0TILn7/E/gIUYv7QzQ1NQ= X-Received: by 2002:a19:8213:: with SMTP id e19mr2094453lfd.201.1584622546571; Thu, 19 Mar 2020 05:55:46 -0700 (PDT) MIME-Version: 1.0 From: Wanpeng Qian Date: Thu, 19 Mar 2020 21:54:59 +0900 Message-ID: Subject: bhyve: TRIM support in AHCI controller To: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48jn2Y1t2cz4fNF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=aucJSD/0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of wanpengqian@gmail.com designates 2a00:1450:4864:20::136 as permitted sender) smtp.mailfrom=wanpengqian@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ip: (-9.61), ipnet: 2a00:1450::/32(-2.39), asn: 15169(-1.65), country: US(-0.05)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[6.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2020 12:55:51 -0000 Hi all I found the output of diskinfo inside VM of AHCI disk as follow, no TRIM support. root@smart:~ # diskinfo -v ada0 ada0 512 # sectorsize 21474836480 # mediasize in bytes (20G) 41943040 # mediasize in sectors 131072 # stripesize 0 # stripeoffset 41610 # Cylinders according to firmware. 16 # Heads according to firmware. 63 # Sectors according to firmware. BHYVE SATA DISK # Disk descr. BHYVE-0B98-104F-E9A7 # Disk ident. No # TRIM/UNMAP support 0 # Rotation rate in RPM Not_Zoned # Zone Mode Also camcontrol identify ada0 -v show not support of TRIM Data Set Management (DSM/TRIM) no I check to pci_ahci.c, TRIM support is decided by block_if.c's candelete flag. while candelete is decided here strlcpy(arg.name, "GEOM::candelete", sizeof(arg.name)); arg.len = sizeof(arg.value.i); if (ioctl(fd, DIOCGATTR, &arg) == 0) candelete = arg.value.i; Obvious file backend will not have the candelete flag. If the backend storage support TRIM, I think we should present TRIM ability to VM. Any better idea to check whether backend storage support TRIM or not, so we can enable it. Thanks. From owner-freebsd-virtualization@freebsd.org Thu Mar 19 13:19:56 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 936ED260473 for ; Thu, 19 Mar 2020 13:19:56 +0000 (UTC) (envelope-from erleya@gmail.com) Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48jnZL6xgXz4Xc5; Thu, 19 Mar 2020 13:19:54 +0000 (UTC) (envelope-from erleya@gmail.com) Received: by mail-wm1-x336.google.com with SMTP id r7so4775280wmg.0; Thu, 19 Mar 2020 06:19:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=CV+XSiV0eCflokUmTtbYxk3Gg+rX+E4etLLw98hws1Q=; b=evJzPB00lVtr/jXEv/u5J1tabqy8sUu/bePh1TFofIvfvxz1MGiFqdRDYBo4D/bxZw IGb+weYfzowbTL6paeNf/Uwbme1/XDpRzTFwKC/cZrQ4JgKY2W5alJkRyE5oxuiCNBEH K6pb91qJrpd6aU6ccnJXJuz9UQ0+vWP9AjDdBeSSEhoyDF2MOaADyIN8dNQ8YFC7sAZI fNkgLsUND7RA8ICDCGX0/oIQkW82G3yeUeoc/3Xscjnonlq9wpO3J5T2fyAgcDRu1qLN UK4vb5eRMG/XwpniStOksbD/UORxwIbNDUDdnQ8P7LMI8fZav1x2RCmDp/oGcIXbrPWJ 5P3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=CV+XSiV0eCflokUmTtbYxk3Gg+rX+E4etLLw98hws1Q=; b=ufRvLzFj2naaN2QMFJv0VbvIJOWNLjhiAzqzTb13LXPJDhqR+Rfxem5Yk1OtRXGcMP fduq3HIxn+SU/oayaJlFhkc/F0rR0kT4d3sWm49bOL2v9AfeZTTJtB9ptJ0pAQltYz3p ISuwemwBYPn7oK8AXfQlpOV207Clh3M1gbOD8NIK77/zmAZquFQXv8wKNS0NurOQo2OX nS9vlt3Qzpi+w2I8W3F8iSvWwQK22t2dK6lKVl3dKp7L6mj0PDOWPSWnYNIemgWgTsVK TfXSOHS8WHpJUnGpiIta1U9yAfIz+ArtmlDoni57FRxecOaDyufDPHEc6/IyIshWRshx 6ICA== X-Gm-Message-State: ANhLgQ21KHlVLis8y27hRTz7nP2A4KYlQvaeMhs+eeiePKhOnfSPMVEv 5jccm7Aq7Hk3n7OCV+KOlswBR3aaURI= X-Google-Smtp-Source: ADFU+vstPybzlVuTMsqYjpPPgwgFVDmuKOUR9yCT89o04ocYzBF7G83TPGY+/CWtkHob7nE9bYKHwQ== X-Received: by 2002:a1c:dc55:: with SMTP id t82mr3716166wmg.6.1584623992978; Thu, 19 Mar 2020 06:19:52 -0700 (PDT) Received: from erley.ru (erley.ru. [83.153.157.67]) by smtp.gmail.com with ESMTPSA id n4sm3517409wrs.64.2020.03.19.06.19.51 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Mar 2020 06:19:52 -0700 (PDT) Received: by erley.ru (OpenSMTPD) with ESMTPSA id 4e20dd93 (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO); Thu, 19 Mar 2020 14:19:50 +0100 (CET) Subject: Re: [GPU pass-through] no compatible bridge window for claimed BAR To: Robert Crowston Cc: Peter Grehan , "freebsd-virtualization@freebsd.org" , Henrik Gulbrandsen References: <07921dcf-11d5-f440-a42f-d7ec950cab10@freebsd.org> <4674c0fc-2696-3476-55e4-608d11ebece2@gmail.com> From: Alex Erley Message-ID: <095fd4a5-f1c8-7b73-75e1-aee52f447a30@gmail.com> Date: Thu, 19 Mar 2020 14:20:22 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 48jnZL6xgXz4Xc5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=evJzPB00; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of erleya@gmail.com designates 2a00:1450:4864:20::336 as permitted sender) smtp.mailfrom=erleya@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[protonmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[67.157.153.83.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; IP_SCORE(0.00)[ip: (-9.40), ipnet: 2a00:1450::/32(-2.39), asn: 15169(-1.65), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[6.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2020 13:19:56 -0000 Hi, > > Do we have already some code for ROM BAR somewhere? > Henrik Gulbrandsen was working on expansion ROM support last July > (CC'd). > He posted his work up on https://www.gulbra.net/freebsd-bhyve/. Wow! Great news, this is definitely good source to accelerate things! Have to look at these patches + PCI docs more in depth... Thank you guys! Have a nice day, Alex From owner-freebsd-virtualization@freebsd.org Fri Mar 20 09:24:38 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 87F5B26107B for ; Fri, 20 Mar 2020 09:24:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48kJJP6ntTz3xjR for ; Fri, 20 Mar 2020 09:24:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 7F904261079; Fri, 20 Mar 2020 09:24:37 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7E978261078 for ; Fri, 20 Mar 2020 09:24:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48kJJM3d8Bz3xf6 for ; Fri, 20 Mar 2020 09:24:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A66DD216EE for ; Fri, 20 Mar 2020 09:24:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02K9OYZe084296 for ; Fri, 20 Mar 2020 09:24:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02K9OYMA084289 for virtualization@FreeBSD.org; Fri, 20 Mar 2020 09:24:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242894] bhyve: TCP / network disconnections between 12.0-RELEASE (guest) and 12.1-RELEASE (hypervisor) Date: Fri, 20 Mar 2020 09:24:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ole@free.de X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2020 09:24:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242894 Ole changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ole@free.de --- Comment #10 from Ole --- I ran into the same Problem with FreeBSD 12.0 and 12.1. It is documented he= re: https://lists.freebsd.org/pipermail/freebsd-questions/2019-February/284348.= html Solution was to configure the interface with -vlanhwtag on the host before starting the rest. (via rc.conf) It occurred with both igb and em drivers for me. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Mar 20 20:36:49 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 312132725E0 for ; Fri, 20 Mar 2020 20:36:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48kbD10bmCz3wdD for ; Fri, 20 Mar 2020 20:36:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 12B442725DD; Fri, 20 Mar 2020 20:36:49 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1163C2725DC for ; Fri, 20 Mar 2020 20:36:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48kbD06fNkz3wdB for ; Fri, 20 Mar 2020 20:36:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DF6D6147D for ; Fri, 20 Mar 2020 20:36:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02KKamfJ004911 for ; Fri, 20 Mar 2020 20:36:48 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02KKam37004910 for virtualization@FreeBSD.org; Fri, 20 Mar 2020 20:36:48 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236922] Virtio fails as QEMU-KVM guest with Q35 chipset on Ubuntu 18.04.2 LTS Date: Fri, 20 Mar 2020 20:36:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: tommyhp2@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2020 20:36:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236922 Tommy P changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #210737|0 |1 is obsolete| | --- Comment #54 from Tommy P --- Created attachment 212556 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D212556&action= =3Dedit Proper FreeBSD 12.x patch for VirtIO --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Mar 20 20:37:32 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CEDB827265F for ; Fri, 20 Mar 2020 20:37:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48kbDr443Bz3wxs for ; Fri, 20 Mar 2020 20:37:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 8B2DC27265D; Fri, 20 Mar 2020 20:37:32 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8A43B27265C for ; Fri, 20 Mar 2020 20:37:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48kbDr0GlGz3wxh for ; Fri, 20 Mar 2020 20:37:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 04A3F1481 for ; Fri, 20 Mar 2020 20:37:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02KKbVQH005756 for ; Fri, 20 Mar 2020 20:37:31 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02KKbVFL005755 for virtualization@FreeBSD.org; Fri, 20 Mar 2020 20:37:31 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236922] Virtio fails as QEMU-KVM guest with Q35 chipset on Ubuntu 18.04.2 LTS Date: Fri, 20 Mar 2020 20:37:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: tommyhp2@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2020 20:37:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236922 --- Comment #55 from Tommy P --- Created attachment 212558 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D212558&action= =3Dedit Proper FreeBSD 11.x patch for VirtIO --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Mar 20 20:38:34 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0B1D02726EC for ; Fri, 20 Mar 2020 20:38:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48kbG16cXyz3xFW for ; Fri, 20 Mar 2020 20:38:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id E30452726EA; Fri, 20 Mar 2020 20:38:33 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E2BDD2726E9 for ; Fri, 20 Mar 2020 20:38:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48kbG15jnkz3xFP for ; Fri, 20 Mar 2020 20:38:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BDBCD148F for ; Fri, 20 Mar 2020 20:38:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02KKcX9s007047 for ; Fri, 20 Mar 2020 20:38:33 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02KKcXuj007046 for virtualization@FreeBSD.org; Fri, 20 Mar 2020 20:38:33 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236922] Virtio fails as QEMU-KVM guest with Q35 chipset on Ubuntu 18.04.2 LTS Date: Fri, 20 Mar 2020 20:38:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: tommyhp2@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2020 20:38:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236922 Tommy P changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #210734|0 |1 is obsolete| | --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Mar 20 20:45:05 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 475542729E8 for ; Fri, 20 Mar 2020 20:45:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48kbPY1DwJz48JX for ; Fri, 20 Mar 2020 20:45:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 29F8F2729E7; Fri, 20 Mar 2020 20:45:05 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 29B8B2729E6 for ; Fri, 20 Mar 2020 20:45:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48kbPY0FLWz48JL for ; Fri, 20 Mar 2020 20:45:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 01F9B167F for ; Fri, 20 Mar 2020 20:45:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02KKj4Va026192 for ; Fri, 20 Mar 2020 20:45:04 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02KKj4dX026191 for virtualization@FreeBSD.org; Fri, 20 Mar 2020 20:45:04 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236922] Virtio fails as QEMU-KVM guest with Q35 chipset on Ubuntu 18.04.2 LTS Date: Fri, 20 Mar 2020 20:45:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: tommyhp2@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2020 20:45:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236922 Tommy P changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #212556|0 |1 is obsolete| | --- Comment #56 from Tommy P --- Created attachment 212559 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D212559&action= =3Dedit Proper FreeBSD 12.x patch for VirtIO --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Mar 20 20:45:30 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E2C0A272A43 for ; Fri, 20 Mar 2020 20:45:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48kbQ25n8bz48gR for ; Fri, 20 Mar 2020 20:45:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id C6418272A42; Fri, 20 Mar 2020 20:45:30 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C6047272A41 for ; Fri, 20 Mar 2020 20:45:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48kbQ24wMSz48gL for ; Fri, 20 Mar 2020 20:45:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A436D1685 for ; Fri, 20 Mar 2020 20:45:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02KKjU7R026995 for ; Fri, 20 Mar 2020 20:45:30 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02KKjUru026994 for virtualization@FreeBSD.org; Fri, 20 Mar 2020 20:45:30 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236922] Virtio fails as QEMU-KVM guest with Q35 chipset on Ubuntu 18.04.2 LTS Date: Fri, 20 Mar 2020 20:45:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: tommyhp2@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2020 20:45:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236922 Tommy P changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #212558|0 |1 is obsolete| | --- Comment #57 from Tommy P --- Created attachment 212560 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D212560&action= =3Dedit Proper FreeBSD 11.x patch for VirtIO --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Mar 20 23:45:50 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C9AB2276667 for ; Fri, 20 Mar 2020 23:45:50 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48kgQ55lrkz4R7B for ; Fri, 20 Mar 2020 23:45:49 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: by mailman.nyi.freebsd.org (Postfix) id 33CC3276666; Fri, 20 Mar 2020 23:45:49 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 338D6276665 for ; Fri, 20 Mar 2020 23:45:49 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [176.74.240.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48kgQ3206lz4R2n; Fri, 20 Mar 2020 23:45:46 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 61F78444CE; Sat, 21 Mar 2020 00:45:43 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V4zPCSKCW3gd; Sat, 21 Mar 2020 00:45:41 +0100 (CET) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 77927444CC; Sat, 21 Mar 2020 00:45:41 +0100 (CET) Subject: Re: [RFC] Adding a Rados block driver to bhyve To: cem@freebsd.org Cc: "freebsd-virtualization@freebsd.org" References: <9c7a8dea-ac8a-4d17-ed33-b6c4e882add8@digiware.nl> <936ed7c2-99d2-5df8-de3f-f64f28d2ba6f@digiware.nl> From: Willem Jan Withagen Message-ID: <2d86d513-efe0-d25f-ae20-0fae74afa1bc@digiware.nl> Date: Sat, 21 Mar 2020 00:45:41 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: nl X-Rspamd-Queue-Id: 48kgQ3206lz4R2n X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wjw@digiware.nl designates 176.74.240.9 as permitted sender) smtp.mailfrom=wjw@digiware.nl X-Spamd-Result: default: False [-5.70 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[digiware.nl]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-3.20)[ip: (-9.78), ipnet: 176.74.224.0/19(-4.89), asn: 28878(-1.38), country: NL(0.03)]; RCVD_IN_DNSWL_MED(-0.20)[9.240.74.176.list.dnswl.org : 127.0.9.2]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:28878, ipnet:176.74.224.0/19, country:NL]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2020 23:45:50 -0000 On 10-3-2020 19:08, Conrad Meyer wrote: > On Tue, Mar 10, 2020 at 9:28 AM Willem Jan Withagen > wrote: >>>> problem that libblock_rbd.so is stripped in such a way that the >>>> symbol I need is removed. >>> So either I'm doing it the wrong way, like special options on the >>> symbols oid. >>> However, I think there's something wrong with your library, too. The >>> library should be usable even if it's stripped. > Yes, strip only removes local symbols (.symtab), not exported ones > (.dynsym). >> I checked with objdump, and the symbol that I need is definitly not >> present in the stripped version. > How are you defining the symbol intended for export? Is the symbol a > function or data? Does the compiler flag -fvisibility=hidden get used? > Which symbol is missing and what are the symptoms? >> And it does not really matter if I declare it static or not. > Not declaring it "static" is necessary, if not sufficient. Looking at > your code on github, here are some issues: * In block_if.h, you define > an object blocklocal_backend. This is a header, and every compilation > unit that pulls in the header will get its own copy of > blocklocal_backend. You probably want 'extern block_backend_t > blocklocal_backend;' instead. I have fixed this in a different way, so I do not need to play tricks with     blocklocal_backend to handle the legacy case.So that is removed. > * You SET_DECLARE block_backend_set in block_if.c, but I think it > needs to be in block_if.h. Looked at the Macro definition, and it is `extern` so it can go into block_if.h > * There is some weirdness around linker sets being removed by the > linker if they are empty, so you may want to add blockbackend_local to > the linker set in the main program. (It's unclear to me why > blockbackend_local is treated specially regardless.) However, I'm not > quite sure why DATA_SET() in block_rbd.c is not creating > __start_set_block_backend_set / __stop_set_block_backend_set exported > symbols. As Alan asked, please provide 'nm -D foo.so'. Best, Conrad It is dynamic loaded with dlopen()/dlsym at the moment. But are you suggesting that a `DATA_SET()` actually adds to the set during dynamic loading? Note that the __start_set_block_backend_set / __stop_set_block_backend_setis avialable in the library. --WjW From owner-freebsd-virtualization@freebsd.org Sat Mar 21 00:49:42 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C827227824A for ; Sat, 21 Mar 2020 00:49:42 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48khqn6fPxz3J1m for ; Sat, 21 Mar 2020 00:49:41 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 02L0nchA085780 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 20 Mar 2020 17:49:38 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 02L0nchI085779; Fri, 20 Mar 2020 17:49:38 -0700 (PDT) (envelope-from jmg) Date: Fri, 20 Mar 2020 17:49:38 -0700 From: John-Mark Gurney To: Wanpeng Qian Cc: FreeBSD virtualization Subject: Re: bhyve: passthrough SMART info from host nvme controller Message-ID: <20200321004938.GO4213@funkthat.com> References: <20200318175901.GI4213@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Fri, 20 Mar 2020 17:49:38 -0700 (PDT) X-Rspamd-Queue-Id: 48khqn6fPxz3J1m X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [2.01 / 15.00]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; IP_SCORE(-0.41)[ip: (-1.05), ipnet: 208.87.216.0/21(-0.52), asn: 32354(-0.42), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.30)[0.297,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.92)[0.919,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Mar 2020 00:49:42 -0000 Wanpeng Qian wrote this message on Thu, Mar 19, 2020 at 12:09 +0900: > > Can't you do what something like pci_passthru.c does in passthru_init, > > and open /dev/nvme0 in pci_nvme_init? > > Yes, you are correct. but that will make /dev/nvme0 keep open all the time. > I just thinking when guest fire a logpage command, open the /dev/nvme0 > and get the SMART info. then close /dev/nvme0. But as you point out, the only way to have that happen is to remove capsicum, but that would make byhve overall LESS secure. Even if you were able to open /dev/nvme0 at any time after the process was locked down, it doesn't provide additional security, as any attacker could just open it up, and do the operation... So, I'm really confused as to what the benefit of not opening it at the start is.. > On Thu, Mar 19, 2020 at 2:59 AM John-Mark Gurney wrote: > > > > Wanpeng Qian wrote this message on Wed, Mar 18, 2020 at 13:05 +0900: > > > But currently bhyve has Capsicum capability, I cannot > > > open /dev/nvme0 within pci_nvme.c without extra code. > > > (currently I just disable the Capsicum capability) > > > > > > any feedback? > > > > Can't you do what something like pci_passthru.c does in passthru_init, > > and open /dev/nvme0 in pci_nvme_init? > > > > Or am I missing something? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@freebsd.org Sat Mar 21 00:55:20 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AA63D278468 for ; Sat, 21 Mar 2020 00:55:20 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com [IPv6:2607:f8b0:4864:20::f34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48khyH4NYZz3L6C for ; Sat, 21 Mar 2020 00:55:19 +0000 (UTC) (envelope-from wanpengqian@gmail.com) Received: by mail-qv1-xf34.google.com with SMTP id c28so4079784qvb.10 for ; Fri, 20 Mar 2020 17:55:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1kkAJOh9B9iAcK49q/OKxBSq4t0Z1YW24sCtQ8rqGRg=; b=dYsUpW6OMNqKBT4uSkjv1nZbtQPV3POnUg0Tr7yNFDCHRqCdB8Ct2O/a1xOxkDztVe qSzTJnOYC1ijobva3WyRj2aiU+c5YZukF7nECKZv/Jhnqujcy4v6sAQ/Ho2osXDI5sAm 7QmzopVs66nVfNrqk76PWEc9JxKzpFjg/5mOmIDKG1QrUd6I0irZvrFXcD8xZUXZxund 9dCCaY3ZlbX7/fSvNZYw2sLT4/TSfPQhssV1YW5gWmThzBlsJrStTU6mOZKPH3hQZ53d o1d+Hgk9vuQ+Btl7zTuCkQo35XRikArJ5HnmHWqL4Ts5O0TtRe/sueN8ej5/sZeI7y9y PL+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1kkAJOh9B9iAcK49q/OKxBSq4t0Z1YW24sCtQ8rqGRg=; b=LxPCgN+nl9vFOc0vbxi6qZotvcWI9zNLQKux4RGuIeVriUqZ0RS9KG1Q/lS0c6TTfd oORnP7ktVJiuGPoces6r16y6ZWuUyixC1lIoWZgbxICVuCABEaNzwrYjnnwbM4HUpkGg 5gGKekXNfFAEwTRyo1cm7yjaBdmBjydkj176ljyeMLYwshMuuHULOOdcgYRdpQjrwSLX /H2ZU6RNr+opEPAJy1849IhKzIn6s2qdYUe3R5e4vzSH7jo22UrUF+1u4IuANBoSuAbO ujUcyjfjdBDozp+t4ABS/kbQEXnEQWy7+OtMpKnc9W7W0rqT1dgXSz/BTHfT7tu2pkda KodQ== X-Gm-Message-State: ANhLgQ3Erv8qsUqRINALeiNYy2DCszL7BPsfQhcj5/Hgz44bU9xFZau+ QJ9mgBq1Gcc9pnpReDtb3aPzndxmiWUas+uDfWepcUBBCqM= X-Google-Smtp-Source: ADFU+vskVXLVBKu+ES4cGydKi70Ks2w927kNqXKtsQLdNjjHF6d/ixECx9asghdyniBBMEHhE2oRK1IVaPj0Rj+gQDc= X-Received: by 2002:a0c:fbcf:: with SMTP id n15mr10746931qvp.114.1584752118116; Fri, 20 Mar 2020 17:55:18 -0700 (PDT) MIME-Version: 1.0 References: <20200318175901.GI4213@funkthat.com> <20200321004938.GO4213@funkthat.com> In-Reply-To: <20200321004938.GO4213@funkthat.com> From: Wanpeng Qian Date: Sat, 21 Mar 2020 09:54:39 +0900 Message-ID: Subject: Re: bhyve: passthrough SMART info from host nvme controller To: John-Mark Gurney Cc: FreeBSD virtualization Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48khyH4NYZz3L6C X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=dYsUpW6O; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of wanpengqian@gmail.com designates 2607:f8b0:4864:20::f34 as permitted sender) smtp.mailfrom=wanpengqian@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[4.3.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; IP_SCORE(0.00)[ip: (-7.29), ipnet: 2607:f8b0::/32(-0.98), asn: 15169(-1.06), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Mar 2020 00:55:20 -0000 > But as you point out, the only way to have that happen is to remove > capsicum, but that would make byhve overall LESS secure. Yes, Capsicum is a necessary. for now I just don't know how to get around it. I just think on pci_nvme_init, we get the SMART info from real device. and return that info when guest query SMART. so it will not update all the time. This way, guest application that depended on SMART Info will benefit from this fix. On Sat, Mar 21, 2020 at 9:49 AM John-Mark Gurney wrote: > > Wanpeng Qian wrote this message on Thu, Mar 19, 2020 at 12:09 +0900: > > > Can't you do what something like pci_passthru.c does in passthru_init, > > > and open /dev/nvme0 in pci_nvme_init? > > > > Yes, you are correct. but that will make /dev/nvme0 keep open all the time. > > I just thinking when guest fire a logpage command, open the /dev/nvme0 > > and get the SMART info. then close /dev/nvme0. > > But as you point out, the only way to have that happen is to remove > capsicum, but that would make byhve overall LESS secure. > > Even if you were able to open /dev/nvme0 at any time after the process > was locked down, it doesn't provide additional security, as any > attacker could just open it up, and do the operation... > > So, I'm really confused as to what the benefit of not opening it at > the start is.. > > > On Thu, Mar 19, 2020 at 2:59 AM John-Mark Gurney wrote: > > > > > > Wanpeng Qian wrote this message on Wed, Mar 18, 2020 at 13:05 +0900: > > > > But currently bhyve has Capsicum capability, I cannot > > > > open /dev/nvme0 within pci_nvme.c without extra code. > > > > (currently I just disable the Capsicum capability) > > > > > > > > any feedback? > > > > > > Can't you do what something like pci_passthru.c does in passthru_init, > > > and open /dev/nvme0 in pci_nvme_init? > > > > > > Or am I missing something? > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@freebsd.org Sat Mar 21 14:28:44 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 854E42641AF for ; Sat, 21 Mar 2020 14:28:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48l30r29VHz3FDh for ; Sat, 21 Mar 2020 14:28:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 81C652641AE; Sat, 21 Mar 2020 14:28:43 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 818892641AD for ; Sat, 21 Mar 2020 14:28:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48l30q0Prdz3FBS for ; Sat, 21 Mar 2020 14:28:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7091CE12F for ; Sat, 21 Mar 2020 14:28:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 02LESf64023901 for ; Sat, 21 Mar 2020 14:28:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 02LESfWr023900 for virtualization@FreeBSD.org; Sat, 21 Mar 2020 14:28:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 242106] [panic] zfree(0xe4e9690,8224): wild pointer during install from 12.1R amd64 ISO in Parallels VM Date: Sat, 21 Mar 2020 14:28:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: sipovich@electronic.spb.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Mar 2020 14:28:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242106 Dima changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |sipovich@electronic.spb.ru --- Comment #2 from Dima --- I got this problem.=20 On the screen I have this. .. FreeBSD/x86 bootstrap loader revision 1.1 cd0: read 1 sector(s) from 8094719 to 0xffffe000 (8x8000): 0x1 panic: zfree(0xe4e9690,8224): wild pointer --> Press a key on the console to reboot <-- I use mainboard intel DX38BT with 8GB Ram hdd ssd MTRON 64GB.=20=20 What should I do ? --=20 You are receiving this mail because: You are the assignee for the bug.=