Date: Mon, 11 Mar 2019 01:06:20 +0300 From: Greg V <greg@unrelenting.technology> To: Martin Karrer <martin@bmalum.com> Cc: freebsd-arm@freebsd.org, freebsd-hackers@freebsd.org, freebsd-cloud@freebsd.org Subject: Re: ARM Graviton AWS Processor (AMI Image) Message-ID: <1552255580.21373.0@unrelenting.technology> In-Reply-To: <1548182399.2864.0@smtp.migadu.com> References: <79CC79B9-81AF-4563-BABE-429E6A57F476@bmalum.com> <010201686fe5047f-ed14af85-2b25-4480-a62a-a893f062eedd-000000@eu-west-1.amazonses.com> <010201686fe5047f-ed14af85-2b25-4480-a62a-a893f062eedd-000000@eu-west-1.amazo>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jan 22, 2019 at 9:39 PM, Greg V <greg@unrelenting.technology>=20 wrote: > On Mon, Jan 21, 2019 at 1:11 PM, Martin Karrer <martin@bmalum.com>=20 > wrote: >> My question is if there are any plans yet to support the Graviton=20 >> ARM =7Finstances of AWS? >>=20 >> We have a heavy load on FreeBSD and would also use the ARM=20 >> instances. =7FAre there any other interested parties? >=20 > I have tried this. It should work very well in theory, e.g. the=20 > network card driver (if_ena) compiles with no changes for aarch64,=20 > and in fact NetBSD has ported this driver and is up and running on=20 > these instances: https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D4623 >=20 > But my result with FreeBSD was: nothing on the console after=20 > loader.efi hands control to the kernel. > [=E2=80=A6] Hello everyone, big update: FreeBSD/aarch64 on Amazon EC2 a1 (AWS Graviton) instances WORKS! https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D4813 And you can try it (well, my -CURRENT build, NO WARRANTY etc) right now: ami-0c2829a0b82a62ca6 in eu-west-1 (Ireland) ----- So, what I had to do / what should be done / how others can help get=20 this into a finished state: 1. Serial console: - I fixed it: https://reviews.freebsd.org/D19507 - (I learned some things about UARTs and their support in FreeBSD,=20 should write a blog post about that) 2. aarch64 build configuration: - if_ena network driver module should be enabled:=20 https://reviews.freebsd.org/D18372 - NVMe driver should be enabled in the GENERIC kernel config (device=20 nvme, device nvd) - BTW, why not also go with hw.nvme.use_nvd=3D"0" by default on=20 aarch64, IIRC that was done on powerpc64 3. VM image build system: - GPT+EFI should be used (amd64 was GPT with no EFI, and aarch64 was=20 MBR with EFI (???)): https://reviews.freebsd.org/D18371 - bsdec2-image-upload --arm64 flag should be supported: included=20 above ^^ - ec2.conf: amazon-ssm-agent shouldn't be installed when building=20 for aarch64 TARGET, since that's written in Go, and Go isn't ported to=20 FreeBSD/aarch64 yet:=20 https://github.com/myfreeweb/freebsd/commit/5b530ebf7385d8320b9076cf84f50aa= d01689bc=20 (untested patch, I actually used an interactive shell in between the=20 image build commands) - qemu-aarch64-static should be used for preinstalling pkgs when=20 chrooting into the image: rough version included above ^^ 4. ENA (Elastic Network Adapter) driver: - it works - except there's something funky with interrupt activation, and it=20 hits panic("Attempt to double activation of resource id: %u\n", res_id)=20 (for the management IRQ) on boot, so I applied the obvious silly=20 workaround of "don't panic":=20 https://github.com/myfreeweb/freebsd/commit/a7e7c6e48cdbdb0fdc6c4e0ba633922= 62938e62c - but still, it doesn't properly reactivate interrupts (and the box=20 becomes unreachable over the net) after going down and up again =E2=80=94=20 guess what does that on boot? dhclient applying the big jumbo MTU =E2=80=94= =20 so I set dhclient.conf to reject MTU changes:=20 https://github.com/myfreeweb/freebsd/commit/03ec4d417b0b4252285baaf4e294cc6= d8c870f7f Would be great if someone familiar with interrupts and stuff could help=20 debug the ena driver and make it work without these hacks :) =
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1552255580.21373.0>