From owner-freebsd-arm@freebsd.org Wed Oct 16 16:51:50 2019 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A77B314AF63 for ; Wed, 16 Oct 2019 16:51:50 +0000 (UTC) (envelope-from 0100016dd57b89f3-034edaf3-4ae3-46f6-96ad-a3f0cf202c05-000000@amazonses.com) Received: from a8-52.smtp-out.amazonses.com (a8-52.smtp-out.amazonses.com [54.240.8.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46tdcP4ZvJz41YH for ; Wed, 16 Oct 2019 16:51:49 +0000 (UTC) (envelope-from 0100016dd57b89f3-034edaf3-4ae3-46f6-96ad-a3f0cf202c05-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57; d=tarsnap.com; t=1571244706; h=From:Subject:To:Message-ID:Date:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=BEOtDUV+yhxhVx61pWLp6J26veYj/1CNowcbo5W2Yw4=; b=lqBOICdJu76FxHSzVNz/lo2GjP+fpdSRJca+ufeqtAIyhULzP5tIqdh6txRIZEY3 SkdrlqzGgF7iesKmLzYYTvPzzVXlVSeRXrC+Vs40nOp+tpASaYSWmiOpkIbvxL5Y74l OXlBxaRpeUc14xCfIbEzmNPBtW0ToIkrnmARq5KI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1571244706; h=From:Subject:To:Message-ID:Date:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=BEOtDUV+yhxhVx61pWLp6J26veYj/1CNowcbo5W2Yw4=; b=TpHbUP1CdjJb2342lriIEozjdnIZ51e6kgHLqV1CFuR9FHLyJ+n21vGW+cONHiba iwSbX2cSneg3HWNwlsojM4hYr2tBaLB6kd78Ps0Zrg2rEJMCofhmqCpYKVVCX9IVLua 4LJMCe/0094SDqIwcpOI1w9Q7fowlyzxNPwQ+eLk= From: Colin Percival Subject: EC2 ARM64 "bare metal" instances To: freebsd-arm@freebsd.org Openpgp: preference=signencrypt Autocrypt: addr=cperciva@tarsnap.com; prefer-encrypt=mutual; keydata= mQGhBElrAAcRBACDfDys4ZtK+ErCJ1HAzYeteKpm3OEsvT/49AjUTLihkF79HhIKrCQU+1KC zv7BwHCMLb6hq30As9L7iFKG7n5QFLFC4Te/VcITUnWHMG/c3ViLOfJGvi+9/nOEHaM1dVJY D6tEp5yM1nHmVQpo9932j4KGuGFR0LhOK5IHXOSfGwCgxSFDPdgxe2OEjWxjGgY+oV3EafcD +JROXCTjlcQiG/OguQH4Vks3mhHfFnEppLxTkDuYgHZQiUtpcT9ssH5khgqoTyMar05OUdAj ZIhNbWDh4LgTj+7ZmvLhXT5Zxw8LX9d7T36aTB8XDQSenDqEtinMWOb0TCBBLbsB8EFG1WTT ESbZci9jJS5yhtktuZoY/eM8uXMD/3k4FWFO80VRRkELSp+XSy/VlSQjyi/rhl2nQq/oOA9F oJbDaB0yq9VNhxP+uFBzBWSqeIX0t1ZWLtNfVFr4TRP5hihI5ICrg/0OpqgisKsU2NFe9xyO hyJLYmfD8ebpDJ/9k30C7Iju9pVrwLm1QgS4S2fqJRcR+U4WbjvP7CgStCVDb2xpbiBQZXJj aXZhbCA8Y3BlcmNpdmFAdGFyc25hcC5jb20+iGEEExECACEFAklrALYCGwMHCwkIBwMCAQQV AggDBBYCAwECHgECF4AACgkQOM7KaQxqam6/igCgn+z2k3V5ggNppmWrZstt1U2lugsAoL7L wS9V9yLtil3oWmHtwpUqYruEuQINBElrAAcQCAD3ZLMIsP4CIDoJORg+YY0lqLVBgcnF7pFb 4Uy2+KvdWofN+DKH61rZLjgXXkNE9M4EQC1B4lGttBP8IY2gs41y3AUogGdyFbidq99rCBz7 LTsgARHwFxZoaHmXyiZLEU1QZuMqwPZV1mCviRhN5E3rRqYNXVcrnXAAuhBpvNyj/ntHvcDN 2/m+ochiuBYueU4kX3lHya7sOj+mTsndcWmQ9soOUyr8O0r/BG088bMn4qqtUw4dl5/pglXk jbl7uOOPinKf0WVd2r6M0wLPJCD4NPHrCWRLLLAjwfjrtoSRvXxDbXhCdgGBa72+K8eYLzVs hgq7tJOoBWzjVK6XRxR7AAMGB/9Mo3iJ2DxqDecd02KCB5BsFDICbJGhPltU7FwrtbC7djSb XUrwsEVLHi4st4cbdGNCWCrp0BRezXZKohKnNAPFOTK++ZfgeKxrV2sJod+Q9RILF86tQ4XF 7A7Yme5hy92t/WgiU4vc/fWbgP8gV/19f8nunaT2E9NSa70mZFjZNu4iuwThoUUO5CV3Wo0Y UISsnRK8XD1+LR3A2qVyLiFRwh/miC1hgLFCTGCQ3GLxZeZzIpYSlGdQJ0L5lixW5ZQD9r1I 8i/8zhE6qRFAM0upUMI3Gt1Oq2w03DiXrZU0Fu/R8Rm8rlnkQKA+95mRTUq1xL5P5NZIi4gJ Z569OPMFiEkEGBECAAkFAklrAAcCGwwACgkQOM7KaQxqam41igCfbaldnFTu5uAdrnrghESv EI3CAo8AoLkNMks1pThl2BJNRm4CtTK9xZeH Message-ID: <0100016dd57b89f3-034edaf3-4ae3-46f6-96ad-a3f0cf202c05-000000@email.amazonses.com> Date: Wed, 16 Oct 2019 16:51:46 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2019.10.16-54.240.8.52 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-Rspamd-Queue-Id: 46tdcP4ZvJz41YH X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tarsnap.com header.s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57 header.b=lqBOICdJ; dkim=pass header.d=amazonses.com header.s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw header.b=TpHbUP1C; dmarc=none; spf=pass (mx1.freebsd.org: domain of 0100016dd57b89f3-034edaf3-4ae3-46f6-96ad-a3f0cf202c05-000000@amazonses.com designates 54.240.8.52 as permitted sender) smtp.mailfrom=0100016dd57b89f3-034edaf3-4ae3-46f6-96ad-a3f0cf202c05-000000@amazonses.com X-Spamd-Result: default: False [-1.41 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[tarsnap.com:s=vnqrkfnvu6csdl6mwgk5t6ix3nnepx57,amazonses.com:s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[tarsnap.com]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DKIM_TRACE(0.00)[tarsnap.com:+,amazonses.com:+]; RCVD_IN_DNSWL_NONE(0.00)[52.8.240.54.list.dnswl.org : 127.0.15.0]; FORGED_SENDER(0.30)[cperciva@tarsnap.com,0100016dd57b89f3-034edaf3-4ae3-46f6-96ad-a3f0cf202c05-000000@amazonses.com]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-1.71)[ip: (-0.56), ipnet: 54.240.8.0/21(-4.62), asn: 14618(-3.30), country: US(-0.05)]; ASN(0.00)[asn:14618, ipnet:54.240.8.0/21, country:US]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; FROM_NEQ_ENVFROM(0.00)[cperciva@tarsnap.com, 0100016dd57b89f3-034edaf3-4ae3-46f6-96ad-a3f0cf202c05-000000@amazonses.com] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Oct 2019 16:51:50 -0000 Hi ARM experts, FreeBSD does not currently boot on Amazon's recently released "bare metal" ARM64 instances. It might be useful to get these working, since they're reasonably powered hardware (16 cores, 32 GB RAM) and should be able to run bhyve (that's the "bare metal" part). Right now the boot is failing because FreeBSD is not finding the disk (or the network adapter, for that matter): http://www.daemonology.net/tmp/a1.metal.boot_verbose Ed suggested that I get a listing of PCI devices from Linux (which does boot on these systems), and it looks like the missing devices are attached to PCI domains 2 and 3 (aka pci0 and pci1): http://www.daemonology.net/tmp/a1.metal.lspci-v I'm aware of a few patches these systems needed in order to work on Linux: 1. Respecting ACPI _DSM "PCI Boot Configuration": https://github.com/torvalds/linux/commit/a78cf9657ba5426f54aa93a067c10d097944c082 2. Adding the Annapurna Labs PCIe controller driver: https://github.com/torvalds/linux/commit/4166bfe53093b687a0b1b22e5d943e143b8089b2 https://github.com/torvalds/linux/commit/a8daea94754989f6c48dafda840482cbc9f882f9 3. Something I don't understand at all about MSI: https://github.com/torvalds/linux/commit/90b4c55586155cf13bbafbd4e55327f89681859d Can someone who knows more me about ARM64 / ACPI / PCIe / MSI take a look at these and figure out how to get this working in FreeBSD? I'll be happy to test patches or to walk someone through the process of using EC2 so you can hack on this directly. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid