From owner-freebsd-arm@freebsd.org Mon Oct 19 06:19:32 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 90B0A44633F for ; Mon, 19 Oct 2020 06:19:32 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-07-4.privateemail.com (mta-07-4.privateemail.com [68.65.122.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CF66W6hpQz48Zq for ; Mon, 19 Oct 2020 06:19:31 +0000 (UTC) (envelope-from freebsd-arm@darkain.com) Received: from MTA-07.privateemail.com (localhost [127.0.0.1]) by MTA-07.privateemail.com (Postfix) with ESMTP id 102AA60045 for ; Mon, 19 Oct 2020 02:19:29 -0400 (EDT) Received: from mail-io1-f42.google.com (unknown [10.20.151.249]) by MTA-07.privateemail.com (Postfix) with ESMTPA id C7FBF60047 for ; Mon, 19 Oct 2020 06:19:28 +0000 (UTC) Received: by mail-io1-f42.google.com with SMTP id q9so11661067iow.6 for ; Sun, 18 Oct 2020 23:19:28 -0700 (PDT) X-Gm-Message-State: AOAM533gohMt2B9Zf9Ch/iPhOmIYZwJR47gD1wc8vIjQPgQFR0CEgg1m 3/Dfj5rhtLU6v4eGrgtny3+4+HRBtVUhgZB9tV0= X-Google-Smtp-Source: ABdhPJyLN854e4W/DBQgPcw4bBhvXCpHd0UgdZaXNl59/rVCdjYyHG8fwoVuKDmFDixtitwF8/cHIUqNef6kVvrYXfY= X-Received: by 2002:a5d:815a:: with SMTP id f26mr10045347ioo.111.1603088366807; Sun, 18 Oct 2020 23:19:26 -0700 (PDT) MIME-Version: 1.0 From: Vincent Milum Jr Date: Sun, 18 Oct 2020 23:19:15 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: FreeBSD under VMware ESXi ARM Fling To: freebsd-arm@freebsd.org X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 4CF66W6hpQz48Zq X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-arm@darkain.com designates 68.65.122.27 as permitted sender) smtp.mailfrom=freebsd-arm@darkain.com X-Spamd-Result: default: False [-2.95 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.06)[-1.063]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[68.65.122.27:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:68.65.122.0/27]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.99)[-0.989]; RCVD_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[darkain.com]; NEURAL_HAM_SHORT(-0.60)[-0.600]; RCVD_IN_DNSWL_NONE(0.00)[68.65.122.27:from]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:22612, ipnet:68.65.122.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arm] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2020 06:19:32 -0000 Recently, VMware released what they're calling "ESXi ARM Fling" - an ARM64/Aarch64 version of VMware ESXi server. Details of the release along with the download can be found here: https://flings.vmware.com/esxi-arm-edition They support a number of ARM platforms already, most notably is the Raspberry Pi 4 (only 4GB and 8GB models). They have instructions on their site on how to install and configure the UEFI firmware on the Pi for getting the ESXi installer and then ESXi itself up and running. UEFI builds of FreeBSD ARM64/Aarch64 mostly work "out of the box" under ESXi ARM, whereas ARM 32-bit kernels are not supported by any guest OS under ESXi ARM. There are, however, some areas that could use some improvement, or need some massive work to get working. The largest issue currently is that SMP will cause the FreeBSD kernel to panic during boot. A VM with only one single vCPU however will boot and run just fine. Prior to the 13.0-CURRENT snapshot 2020-10-15, the CD-Rom driver was not included with the ARM64 builds, so the ISO would fail to mount itself on boot. This means the easiest way to try earlier versions, such as 12.1-RELEASE, you'll need to either convert the pre-built vmdk file from Workstation to ESXi format, or convert the raw disk image to ESXi vmdk format. This is how I initially got FreeBSD up and running prior to the 2020-10-15 snapshot. Every version of FreeBSD I've tested on ARM is lacking several drivers by default. uhci for USB 2.0 is not enabled by default. Ensure your VM is initially set up for USB 3.1 so the virtual keyboard and mouse will function. This is easy enough to fix post install with: uhci_load="YES" ums is the same as above. ESXi ARM uses USB instead of PS2 for the virtual mouse, so add this post install if you want mouse support: ums_load="YES" vmxnet and pvscsi are not pre-compiled. However, they compile and operate perfectly as expected without any modification on ARM. vmci has some x86 assembly that I've not dug into yet, so that is currently not working. I've documented in detail, and continue to update as I test more drivers over on my blog: https://vincerants.com/freebsd-on-esxi-arm-fling-fixing-virtual-hardware/ The ESXi UI has a known bug in that it'll list some virtual hardware that doesn't actually exist inside of the VM. These are all legacy devices, and should be easy enough to figure out. There are no virtual IDE/ATA or PS2 ports. Keyboard and mouse now use the virtual USB controller instead. open-vm-tools has required a bit of work to make functional on FreeBSD under ARM. There is currently a patch open with VMware with my changes to the codebase to make it functional. To make things easy, I've uploaded some pre-compiled binaries to my blog, as well as included steps to compile this customized package yourself. The blog will also be updated as development progresses upstream with VMware. https://vincerants.com/open-vm-tools-on-freebsd-under-vmware-esxi-arm-fling/ Xorg desktop runs, and is mostly functional. I've yet to get the xf86-input-vmmouse or xf86-video-vmware working. Xorg can use sysmouse and the console framebuffer for rendering, and this is working okay for basic testing right now. I also have an early dmesg, but this was from before getting most drivers working. https://dmesgd.nycbug.org/index.cgi?do=view&id=5693 I welcome any additional feedback on getting FreeBSD to 100% under ESXi ARM! - Vince (darkain)