From nobody Sat Nov 25 10:37:12 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ScpG80fr1z51ghl for ; Sat, 25 Nov 2023 10:37:52 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ScpG7120kz3dY7 for ; Sat, 25 Nov 2023 10:37:51 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=hE7l8wUl; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2a00:1450:4864:20::629 as permitted sender) smtp.mailfrom=marietto2008@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a00d5b0ec44so378038466b.0 for ; Sat, 25 Nov 2023 02:37:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700908669; x=1701513469; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=nWXjJUAM1lKpeIoqtGeYf/W5+lxDgMw4djI0JFAclZ8=; b=hE7l8wUly/1vHELi08Xf0PY+t2pzJezq7gT0OIXrLSfffr5n7AEdQc6kzuER7+sE/E P69LujkCQ41R0vmjx1tWYmUxfK14YAUdDNKAPWEXVkECggvglidSrlHEcqozR//0LVDM V1RxwtWZ4A08eIyOd+xZtWMg3si1UqJY+1gDCslymg9zC0TeeH6NBvHPwjjHPsYLsSmD +7tC498/AoQ/ywhgbgISEcspUtgFbMEtbX/8Zbg4KGnUUd540fg/NfXHtxGnlY3z4Cs3 x+NoQTwpAq7SDd9SFkaxBBMj58K/UOwjFZT2x3JYtCJKYDy1wf+nOqSuV65K5N2nmAyR q/BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700908669; x=1701513469; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=nWXjJUAM1lKpeIoqtGeYf/W5+lxDgMw4djI0JFAclZ8=; b=dMxVmK6vGNM3PggepdRcWWxaB41pY3nX4Hbs8jCqA0IfjCe4bpZUOiIOhx5O6yzHlq 6ycXlXt/dwp5xpdzlykSkWRrrhVQXARpSSu7ySslLirR4QVGvDVk4KEa7tBIQwDlwJZt xgPIWzaDy30nC7L4AZ8Y2esCy+aUvyxfnDQjd43Z56J5vBkuDCg5L196S3XENMk9qnsA qwFyvpYO0HQH1BERqduogK4KL3EcAbCaWFBSKpXA8dCJUKF8xpyHHRETE9szqtH3M2Ry VCqaFuklzrl8EkkVGMlzTSi4X4ytGS2ZATJv4n6K8Jbm2hk/FGZWej3yKfQLdw6SEVnk xYnQ== X-Gm-Message-State: AOJu0YzYEVxpB6TXIWZPoEtyChb2nZ0TWgfofb92w1jp1+G2/V6f1w46 YFa8trKfhDQFeTPoTjAk0rLy+QtLGUKPVt9oEKU0Dmwn2kyurw== X-Google-Smtp-Source: AGHT+IF8PXL6FKMpScGNYwD2W1LsTPXvNornsb87FKTlUJAF5NCKfx1Wfa9D0M0Ri8GacmamhSGrTNCIQIKncbruwrY= X-Received: by 2002:a17:906:290:b0:9e6:18fe:7447 with SMTP id 16-20020a170906029000b009e618fe7447mr4064969ejf.9.1700908669166; Sat, 25 Nov 2023 02:37:49 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 From: Mario Marietto Date: Sat, 25 Nov 2023 11:37:12 +0100 Message-ID: Subject: Should we boot the FreeBSD kernel in ELF format or in zImage format ? How? To: FreeBSD virtualization Content-Type: multipart/alternative; boundary="0000000000000568a1060af7a944" X-Spamd-Result: default: False [0.66 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_SPAM_LONG(0.94)[0.936]; NEURAL_SPAM_MEDIUM(0.72)[0.720]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::629:from]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org] X-Rspamd-Queue-Id: 4ScpG7120kz3dY7 X-Spamd-Bar: / --0000000000000568a1060af7a944 Content-Type: text/plain; charset="UTF-8" Hello. we have just virtualized Debian 12 on our arm (32 bit) Chromebook. As host / dom0 we have chosen Devuan 5,and for guest / domU,Debian 12. It works great. But our goal is different. We want to virtualize FreeBSD as domU. Can we have a working Xen PV network driver for a FreeBSD arm guest ?. I found that Julien Grall has ported the Xen drivers to FreeBSD on arm. I would like to know if Julien's work was accepted upstream by FreeBSD, in which case FreeBSD as a Xen guest on arm should work if we enable the Xen PV drivers in the FreeBSD on arm kernel. If Julien's work was not accepted upstream by FreeBSD, we will have to find his patches and apply them ourselves to the FreeBSD on arm kernel. We found these slides : https://events.static.linuxfound.org/sites/events/files/slides/Porting%20FreeBSD%20on%20Xen%20on%20ARM%20.pdf Slide 13 refers to a XENHVM FreeBSD on arm kernel config - that is what we want to find. It looks like when that slide presentation was written, there were some limitations on FreeBSD Xen guests. For example, for our debian bookworm guest, I am using vcpus = '2' to match the number of real cpus on our Chromebook, but slide 13 mentions support for only 1 VCPU with a FreeBSD guest, so I will need to change that vcpus = '1' in the FreeBSD guest config unless support for 2 or more vcpus was added later, which is possible because that slide presentation is 9 years old. Here is where I would expect to find the XENHVM FreeBSD on arm kernel config file: https://cgit.freebsd.org/src/tree/sys/arm/conf But it is not there unless I am not understanding something correctly. For now, unfortunately conclude that the support for Xen on arm that Julien Grall mentioned in that slide presentation 9 years ago was never added to the official FreeBSD source code. I am searching the web now to see if the patches that Julien Grall wrote are still posted somewhere online. If we cannot find them, we can ask here and on the xen-users mailing list. Julien regularly reads that list and responds to question about Xen on arm, so I think he will tell us how to find the patches if we cannot find them online. According to this page from the FreeBSD wiki: https://wiki.freebsd.org/Xen I think FreeBSD only supports Xen on x86, not arm. So this is going to be a bit of a challenge to get a Xen FreeBSD guest on arm working. We know Julien Grall has some patches that made it work in the past ! I found a slightly newer slide presentation by Julien here: https://www.slideshare.net/xen_com_mgr/bsdcan-2015-how-to-port-your-bsd It is about the same, but it mentions the GENERIC FreeBSD kernel supports Xen on arm64, but still says we need the XENHVM FreeBSD config for Xen on arm 32 bit, which I haven't found online yet. Please,take a look at this output of the linux kernel that can boot on Xen, and the FreeBSD kernel that cannot : % file zImage-6.1.59-stb-xen-cbe+ zImage-6.1.59-stb-xen-cbe+: Linux kernel ARM boot executable zImage (little-endian) % file FREEBSD-XENVIRT FREEBSD-XENVIRT: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /red/herring, for FreeBSD 11.0 (1100048), not stripped The FreeBSD kernel that won't boot is in ELF format but the Linux kernel that does boot is in zImage format. I spent time reading the docs on xenbits.xenproject.org, and according to those docs Xen on arm only knows how to boot a kernel in the zImage format, so the FreeBSD kernel is in a format that modern Xen incorrectly detects as an x86 kernel. I also watched Julien Grall's 30 minute video presentation of his work to boot FreeBSD/arm on Xen at FOSDEM 2014 here : https://archive.fosdem.org/2014/schedule/event/freebsd_xen_arm/ In that video, and in other places, Julien mentions that the boot ABI for FreeBSD/arm on Xen was not yet developed and he was getting occasional crashes and needed to investigate the problem. He mentioned the zImage ABI that Linux uses, but pointed out FreeBSD does not use that format, and back then it was an open question which format to use to boot FreeBSD/arm on Xen. Unfortunately, nine years later, the only supported format is still the zImage format that Linux uses. It looks like Julien's work back then was using an ELF binary to boot FreeBSD/arm on Xen instead of the supported zImage format that Linux uses and the modern Xen toolstack exits with an error when trying to boot the FreeBSD ELF formatted binary that Julien's patch creates. So the best solution would be to try to port the rules to build a FreeBSD kernel in the zImage format instead of the ELF format. I have been studying the Makefiles in Linux to see how Linux builds the Linux arm kernel in the zImage format, but it is not trivial to understand. -- Mario. --0000000000000568a1060af7a944 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
=20
=20
=20
=20

Hello.

we have just virtualized Debian 12 on our arm (32 bit) = Chromebook. As host / dom0 we have chosen Devuan 5,and for guest / domU,Debian 12. It=20 works great. But our goal is different. We want to virtualize FreeBSD as domU. Can we have a working Xen PV network driver for a FreeBSD arm=20 guest ?. I found that Julien Grall has ported the Xen drivers to FreeBSD on arm. I would like to know if Julien's work was accepted upstream by= =20 FreeBSD, in which case FreeBSD as a Xen guest on arm should work if we=20 enable the Xen PV drivers in the FreeBSD on arm kernel. If Julien's wor= k was not accepted upstream by FreeBSD, we will have to find his patches=20 and apply them ourselves to the FreeBSD on arm kernel.

We found these slides :

https://events.static.linuxfound.org/sites/events/files/slides/Porting%2= 0FreeBSD%20on%20Xen%20on%20ARM%20.pdf

Slide 13 refers to a XENHVM FreeBSD on arm kernel config - that is what = we want to find.

It looks like when that slide presentation was written, there were=20 some limitations on FreeBSD Xen guests. For example, for our debian=20 bookworm guest, I am using vcpus =3D '2' to match the number of rea= l cpus=20 on our Chromebook, but slide 13 mentions support for only 1 VCPU with a=20 FreeBSD guest, so I will need to change that vcpus =3D '1' in the F= reeBSD=20 guest config unless support for 2 or more vcpus was added later, which=20 is possible because that slide presentation is 9 years old.

Here is where I would expect to find the XENHVM FreeBSD on arm kernel co= nfig file:

https://cgit.freebsd.org/src/tree/sys/arm/conf

But it is not there unless I am not understanding something=20 correctly. For now, unfortunately conclude that the support for Xen on=20 arm that Julien Grall mentioned in that slide presentation 9 years ago=20 was never added to the official FreeBSD source code. I am searching the=20 web now to see if the patches that Julien Grall wrote are still posted=20 somewhere online. If we cannot find them, we can ask here and on the=20 xen-users mailing list. Julien regularly reads that list and responds to question about Xen on arm, so I think he will tell us how to find the=20 patches if we cannot find them online.

According to this page from the FreeBSD wiki:

htt= ps://wiki.freebsd.org/Xen

I think FreeBSD only supports Xen on x86, not arm. So this is going=20 to be a bit of a challenge to get a Xen FreeBSD guest on arm working. We know Julien Grall has some patches that made it work in the past !

I found a slightly newer slide presentation by Julien here:

https://www.slideshare.net/xen_com_= mgr/bsdcan-2015-how-to-port-your-bsd

It is about the same, but it mentions the GENERIC FreeBSD kernel=20 supports Xen on arm64, but still says we need the XENHVM FreeBSD config=20 for Xen on arm 32 bit, which I haven't found online yet.

Please,take a look at this output of the linux kernel that can boot on X= en, and the FreeBSD kernel that cannot :


% file zImage-6.1.59-stb-xen-cbe+
zImage-6.1.59-stb-xen-cbe+: Linux kernel ARM boot executable zImage (little=
-endian)

% file FREEBSD-XENVIRT         =20
FREEBSD-XENVIRT: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dy=
namically linked, interpreter /red/herring, for FreeBSD 11.0 (1100048), not=
 stripped


The FreeBSD kernel that won't boot is in ELF format but t= he Linux kernel that does boot is in zImage format.

I spent time reading the docs on xenbits.xenproject.org, and=20 according to those docs Xen on arm only knows how to boot a kernel in=20 the zImage format, so the FreeBSD kernel is in a format that modern Xen=20 incorrectly detects as an x86 kernel.

I also watched Julien Grall's 30 minute video presentation of his wo= rk to boot FreeBSD/arm on Xen at FOSDEM 2014 here :

https://archive.fosdem.org/2014/schedule/ev= ent/freebsd_xen_arm/

In that video, and in other places, Julien mentions that the boot ABI for FreeBSD/arm on Xen was not yet developed and he was getting=20 occasional crashes and needed to investigate the problem. He mentioned=20 the zImage ABI that Linux uses, but pointed out FreeBSD does not use=20 that format, and back then it was an open question which format to use=20 to boot FreeBSD/arm on Xen. Unfortunately, nine years later, the only=20 supported format is still the zImage format that Linux uses.

It looks like Julien's work back then was using an ELF binary to boo= t FreeBSD/arm on Xen instead of the supported zImage format that Linux=20 uses and the modern Xen toolstack exits with an error when trying to=20 boot the FreeBSD ELF formatted binary that Julien's patch creates. So= =20 the best solution would be to try to port the rules to build a FreeBSD=20 kernel in the zImage format instead of the ELF format. I have been=20 studying the Makefiles in Linux to see how Linux builds the Linux arm=20 kernel in the zImage format, but it is not trivial to understand.

=
--
Mario.
--0000000000000568a1060af7a944-- From nobody Sun Nov 26 21:00:52 2023 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sdh2Y4523z522q0 for ; Sun, 26 Nov 2023 21:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sdh2Y0PDHz4G3P for ; Sun, 26 Nov 2023 21:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1701032453; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TgXrmQtfKCOFOeSfvPqlkA9d4w32loyAXccwdLhvtKg=; b=HQofu7RQ9W54jbt19o4zMLBOGYaNcyPGLwL4glJK6o8SphjF6udeTpsjYg+u+FqqZLdjc2 LdfQt2QaBBMKzJzNwbcl1THXqkcuDMVi72cJw15aUCZaT69Nq0NFnoh8HtKSqPYMStsOZt Hdw03TrkL9HLVXzjGolyTLh+FTY9wISdwTb2t6sxtjQVpml6nqt6D6VxApzI8yhbIESiIy kIzzYbmeWlqNbnXpLfQnV8LeuUoB50NqfaIAZwOQc9rjc/pm35CKSuabXxURQZ/ZhbSe9Y rqkoxYdnb0UPTXIfiDXRBlx5wemTrRcedTyGLBKweKYfx9/XIKrCxrQW/fE/RQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1701032453; a=rsa-sha256; cv=none; b=X6udftu8mbC3AgeYoGzXr2EgVkMWhnSUtdzgH7kLRAMvPCdG72jUjfp63JnFH8RVDwAbEc SFE4IcywhSIapD96Wzwtc3rm3D/bPpJOxETJbNeMVdANYJ+Ywps5QqcxOFXXYJnL+R5PGo gLfn6qkitWjhggc1Plkz6y9lzguKYvbH61F/NNa07w2nq9T+Tu8+fotEJXN+EcJP56t3z7 8QkXldQdrfQbbK7bkk+y+kEsciIltHsYuajhZYl9e3+krEvW4FSyHptvgaTR8653dw3snA oWAsXIFhTdFDdHv7rcuUU1/5tEqkkRLdXtSM7y3VObb1+kFAMDNO1etcR7Q20g== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Sdh2X6bC7z19cD for ; Sun, 26 Nov 2023 21:00:52 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3AQL0qYo014163 for ; Sun, 26 Nov 2023 21:00:52 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3AQL0q8b014162 for virtualization@FreeBSD.org; Sun, 26 Nov 2023 21:00:52 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202311262100.3AQL0q8b014162@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 26 Nov 2023 21:00:52 +0000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="17010324525.745799DB.10853" Content-Transfer-Encoding: 7bit --17010324525.745799DB.10853 Date: Sun, 26 Nov 2023 21:00:52 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 247208 | mpt(4): VMWare virtualized LSI controller panics New | 240945 | [hyper-v] [netvsc] hn network driver incorrectly Open | 244838 | "bectl activate -t" does not honor the -t flag in 3 problems total for which you should take action. --17010324525.745799DB.10853 Date: Sun, 26 Nov 2023 21:00:52 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
In Progress |    247208 | mpt(4): VMWare virtualized LSI controller panics 
New         |    240945 | [hyper-v] [netvsc] hn network driver incorrectly 
Open        |    244838 | "bectl activate -t" does not honor the -t flag in

3 problems total for which you should take action.
--17010324525.745799DB.10853-- From nobody Sun Nov 26 21:10:56 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SdhGB61xTz5288t for ; Sun, 26 Nov 2023 21:10:58 +0000 (UTC) (envelope-from 0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@amazonses.com) Received: from a48-103.smtp-out.amazonses.com (a48-103.smtp-out.amazonses.com [54.240.48.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SdhG972vpz4LrV for ; Sun, 26 Nov 2023 21:10:57 +0000 (UTC) (envelope-from 0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@amazonses.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=amazonses.com header.s=224i4yxa5dv7c2xz3womw6peuasteono header.b=JL72VmhC; spf=pass (mx1.freebsd.org: domain of 0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@amazonses.com designates 54.240.48.103 as permitted sender) smtp.mailfrom=0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@amazonses.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=acm.org (policy=none) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1701033056; h=Message-ID:Date:MIME-Version:To:From:Subject:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=KTtH5miv5E5yyzFuW7HSof3HQwpCzSChnSawBPtDSSw=; b=JL72VmhCZZ4pG6HF39dn2dk6Mb4zu5GWmPUUOq/MB5uPrPW9JhHVt3WXxM+f5YfD S/Omepfay6T4vI/T1XV5pxan7xvW1l1MjkAzyoXPLHHVyac03UhdjF5bMek6kWoqANI gemr0w+3CCKdowGiXpmh0ltsyx2N4c5y4XpdCyfA= Message-ID: <0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@email.amazonses.com> Date: Sun, 26 Nov 2023 21:10:56 +0000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: freebsd-virtualization@freebsd.org Content-Language: en-US From: Thomas Laus Subject: Bhyve won't boot OpenBSD 7.4 Autocrypt: addr=lausts@acm.org; keydata= xsDiBDx1NWwRBADARalI5I8kGeBYYYWnZB73T1fU4333yCuRokRvzlAZ5Zhb3hqsNdTEMheN FDjZSL8J5jeJtvSRinY2p09CxpAMoJR9zHLmHl+zEOY8fInbB+KiFtSfGf0blSEY9/+isQP9 xmUIQWUj0kwVtrns7m1HrYLiI07NVFzbHNKqQcbPuwCg0n/KKi+VJiUs5MqLKwGuPotGeZME AIluMetTQwfLyovundMwFYlSZ/Z8JjkMybqgKuiRrZnaBVVZ80NjAYZI73yAZPfQh9mvFxW9 ipc2tSALwDy/tYDpQRK0k+0EsDmwG/wM6OarkqSuFcYx+tP86+2+6Xitn6E/hriIWa/ZQVef /fx7dZzdwhXH6fd34v8o/BuqhawLBACs4MTMGbdSmyI56vCMXWY1yxRPmuygd4vUnXqYwlrM Ee/LjQdreg1zTAJnnW1K+PgOUW/jvS+uAbgxLa3i59/Z4Uu7nB6G1y+Y1cThojUsLnvoJlt1 4XE1U/vnOcvO3evo6knB1qjbAMsZGaVleiVKDq+7XE7swe4WtBJKbYJthc0cVGhvbWFzIExh dXMgPGxhdXN0c0BhY20ub3JnPsJlBBMRAgAlAheAAh4BBgsJCAcDAgYVCAIJCgsEFgIDAQUC Y++t4wUJKy+wdwAKCRDr/4LUYgg2z3viAJ4iIhfj6EQW8rbJjdAmLX6K6I0WCACfVo9g+WUL nfSUcrH4QERMagahtmrOwU0ERbKYHBAIAI17IvVqqMACme3dosere34mILtkARlqt+jbzfnD wSSLtAp2oC9b+FtVK0Vv9HCJy84KpVq0lO0hYMtfqxgZbrDB3/P32hnmQ9F8DHWIEyGHY3Su VWdBmkVlLNC2xwDiUJDhURyzABBvMK3N39ca57lSeCTzgPNZE5k172E8kDVyQcknNKDUoL0r rfGgWzRgWYcPVNld8l155zHlkfPg2K64mHTnUXiu0tCzTfEz2OtDycdyizfb1GPQm+S8tDWr xd8ERD0KmJTX0zKBvxgP5/FwekgqDI+96CgssBS23JAVIxbJe8E3iir1OvFfJFhu+tn7HF4T oUXbhExq42tPTG8AAwUH/1osnJnXzbQeNfr70x0Ks/bZt2RwFtshOdwsX0Kb5ckvysJ1oC45 f4ldEh0Q3nXtDSeckxKj1Yaqu1gnZalbBz0qVxX78joGzZoz5jmP2tZ1uFJ1kqgO24EL5LYW oYaC3h85FPGCrqVtJtOUSAaakBEeh8ATpLE6uLFlf5VMtT9FU/BqqjyzZJSntgRLvioboS1e l8BBt1axzaGOv0eQjBrwYWW7YlFYgmZLZsSQIxatFQnLo+/jqOdnquAS9rzK8y1/tiG9SUlo nerdDfoK18W69pr+x9ldv+FaNFyNyCsT19ZnRsvUQZG0VBslfQPEZWktEE5177MWU4zp5o8T 1+vCTwQYEQIADwIbDAUCY++t4wUJKy+wdwAKCRDr/4LUYgg2zxt8AJ46/dSLMaIS2PVrhQ7k aW2h2upY6ACfckKS5ugbpEjWsl3Bxqt5WtA9W4o= Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Feedback-ID: 1.us-east-1.9pbSdi8VQuDGy3n7CRAr3/hYnLCug78GrsPo0xSgBOs=:AmazonSES X-SES-Outgoing: 2023.11.26-54.240.48.103 X-Spamd-Result: default: False [-0.67 / 15.00]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.997]; NEURAL_HAM_SHORT(-0.98)[-0.982]; FORGED_SENDER(0.30)[lausts@acm.org,0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@amazonses.com]; R_DKIM_ALLOW(-0.20)[amazonses.com:s=224i4yxa5dv7c2xz3womw6peuasteono]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18:c]; DMARC_POLICY_SOFTFAIL(0.10)[acm.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[54.240.48.103:from]; XM_UA_NO_VERSION(0.01)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:14618, ipnet:54.240.48.0/23, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ZERO(0.00)[0]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[amazonses.com:+]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[54.240.48.103:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[lausts@acm.org,0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@amazonses.com]; TO_DN_NONE(0.00)[]; DWL_DNSWL_NONE(0.00)[amazonses.com:dkim] X-Rspamd-Queue-Id: 4SdhG972vpz4LrV X-Spamd-Bar: / I have 2 production servers that I want to upgrade to FreeBSD 14.0 that use Bhyve to run OpenBSD 7.3 packet packet filters and a few other instances. Since this is an operating system upgrade to FreeBSD 14.0, I wanted to also upgrade the OpenBSD 7.3 instance to OpenBSD 7.4 that was released last month. I tried the in-place OpenBSD 'sysupgrade' tool unsuccessfully. This has worked for me successfully for the past several years. I tried booting from an OpenBSD 7.4 CDROM to attempt the upgrade using this method. It was also without success. Thinking that I had a bad .iso file, I burned it to a CDROM and used it to install OpenBSD 7.4 on a spare drive in an old laptop. It worked like a champ. I have not tried to load OpenBSD 7.4 on any other hypervisors like Vultr or Amazon EC2 yet to see if that version has issues with all hypervisors. Here is my vm-bhyve log for the installation of OpenBSD 7.3, the in-place upgrade to 7.4 and finally the boot attempt from the .iso file: Nov 26 10:51:09: initialising Nov 26 10:51:09: [loader: grub] Nov 26 10:51:09: [cpu: 1] Nov 26 10:51:09: [memory: 768M] Nov 26 10:51:09: [hostbridge: standard] Nov 26 10:51:09: [com ports: com1] Nov 26 10:51:09: [uuid: af22d4e4-2a87-11eb-b8fd-a0369f7299dd] Nov 26 10:51:09: [debug mode: no] Nov 26 10:51:09: [primary disk: disk0.img] Nov 26 10:51:09: [primary disk dev: file] Nov 26 10:51:09: initialising network device tap0 Nov 26 10:51:10: booting Nov 26 10:51:10: create file /virtual/openbsd/device.map Nov 26 10:51:10: -> (cd0) /virtual/.iso/install73.iso Nov 26 10:51:10: -> (hd0) /virtual/openbsd/disk0.img Nov 26 10:51:10: create file /virtual/openbsd/grub.cfg Nov 26 10:51:10: -> timeout=3 Nov 26 10:51:10: -> menuentry 'openbsd (bhyve install)' { Nov 26 10:51:10: -> root=cd0 Nov 26 10:51:10: -> kopenbsd -h com0 /7.3/amd64/bsd.rd Nov 26 10:51:10: -> } Nov 26 10:51:10: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S -m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd Nov 26 10:51:13: [bhyve options: -c 1 -m 768M -AHP -w -U af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S] Nov 26 10:51:13: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/virtual/openbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0] Nov 26 10:51:13: [bhyve console: -l com1,/dev/nmdm-openbsd.1A] Nov 26 10:51:13: [bhyve iso device: -s 3:0,ahci-cd,/virtual/.iso/install73.iso,ro] Nov 26 10:51:13: starting bhyve (run 1) Nov 26 10:57:04: bhyve exited with status 0 Nov 26 10:57:04: restarting Nov 26 10:57:05: create file /virtual/openbsd/device.map Nov 26 10:57:05: -> (hd0) /virtual/openbsd/disk0.img Nov 26 10:57:05: create file /virtual/openbsd/grub.cfg Nov 26 10:57:05: -> timeout=3 Nov 26 10:57:05: -> menuentry 'openbsd (bhyve run)' { Nov 26 10:57:05: -> root=hd0,1 Nov 26 10:57:05: -> kopenbsd -h com0 -r sd0a /bsd Nov 26 10:57:05: -> } Nov 26 10:57:05: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S -m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd Nov 26 10:57:08: [bhyve options: -c 1 -m 768M -AHP -w -U af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S] Nov 26 10:57:08: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/virtual/openbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0] Nov 26 10:57:08: [bhyve console: -l com1,/dev/nmdm-openbsd.1A] Nov 26 10:57:08: [bhyve iso device: -s 3:0,ahci-cd,/virtual/.iso/install73.iso,ro] Nov 26 10:57:08: starting bhyve (run 2) Nov 26 10:59:01: bhyve exited with status 0 Nov 26 10:59:01: restarting Nov 26 10:59:02: create file /virtual/openbsd/device.map Nov 26 10:59:02: -> (hd0) /virtual/openbsd/disk0.img Nov 26 10:59:02: create file /virtual/openbsd/grub.cfg Nov 26 10:59:02: -> timeout=3 Nov 26 10:59:02: -> menuentry 'openbsd (bhyve run)' { Nov 26 10:59:02: -> root=hd0,1 Nov 26 10:59:02: -> kopenbsd -h com0 -r sd0a /bsd Nov 26 10:59:02: -> } Nov 26 10:59:02: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S -m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd Nov 26 10:59:06: [bhyve options: -c 1 -m 768M -AHP -w -U af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S] Nov 26 10:59:06: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/virtual/openbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0] Nov 26 10:59:06: [bhyve console: -l com1,/dev/nmdm-openbsd.1A] Nov 26 10:59:06: [bhyve iso device: -s 3:0,ahci-cd,/virtual/.iso/install73.iso,ro] Nov 26 10:59:06: starting bhyve (run 3) Nov 26 11:01:46: bhyve exited with status 0 Nov 26 11:01:46: restarting Nov 26 11:01:47: create file /virtual/openbsd/device.map Nov 26 11:01:47: -> (hd0) /virtual/openbsd/disk0.img Nov 26 11:01:47: create file /virtual/openbsd/grub.cfg Nov 26 11:01:47: -> timeout=3 Nov 26 11:01:47: -> menuentry 'openbsd (bhyve run)' { Nov 26 11:01:47: -> root=hd0,1 Nov 26 11:01:47: -> kopenbsd -h com0 -r sd0a /bsd Nov 26 11:01:47: -> } Nov 26 11:01:47: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S -m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd Nov 26 11:03:35: [bhyve options: -c 1 -m 768M -AHP -w -U af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S] Nov 26 11:03:35: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/virtual/openbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0] Nov 26 11:03:35: [bhyve console: -l com1,/dev/nmdm-openbsd.1A] Nov 26 11:03:35: [bhyve iso device: -s 3:0,ahci-cd,/virtual/.iso/install73.iso,ro] Nov 26 11:03:35: starting bhyve (run 4) Nov 26 11:04:16: bhyve exited with status 1 Nov 26 11:04:16: destroying network device tap0 Nov 26 11:04:16: stopped Nov 26 11:07:32: initialising Nov 26 11:07:32: [loader: grub] Nov 26 11:07:32: [cpu: 1] Nov 26 11:07:32: [memory: 768M] Nov 26 11:07:32: [hostbridge: standard] Nov 26 11:07:32: [com ports: com1] Nov 26 11:07:32: [uuid: af22d4e4-2a87-11eb-b8fd-a0369f7299dd] Nov 26 11:07:32: [debug mode: no] Nov 26 11:07:32: [primary disk: disk0.img] Nov 26 11:07:32: [primary disk dev: file] Nov 26 11:07:32: initialising network device tap0 Nov 26 11:07:32: booting Nov 26 11:07:32: create file /virtual/openbsd/device.map Nov 26 11:07:32: -> (cd0) /virtual/.iso/openbsd74.iso Nov 26 11:07:32: -> (hd0) /virtual/openbsd/disk0.img Nov 26 11:07:32: create file /virtual/openbsd/grub.cfg Nov 26 11:07:32: -> timeout=3 Nov 26 11:07:32: -> menuentry 'openbsd (bhyve install)' { Nov 26 11:07:32: -> root=cd0 Nov 26 11:07:32: -> kopenbsd -h com0 /7.4/amd64/bsd.rd Nov 26 11:07:32: -> } Nov 26 11:07:32: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S -m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd Nov 26 11:07:35: [bhyve options: -c 1 -m 768M -AHP -w -U af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S] Nov 26 11:07:35: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/virtual/openbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0] Nov 26 11:07:35: [bhyve console: -l com1,/dev/nmdm-openbsd.1A] Nov 26 11:07:35: [bhyve iso device: -s 3:0,ahci-cd,/virtual/.iso/openbsd74.iso,ro] Nov 26 11:07:35: starting bhyve (run 1) Nov 26 11:07:40: bhyve exited with status 0 Nov 26 11:07:40: restarting Nov 26 11:07:41: create file /virtual/openbsd/device.map Nov 26 11:07:41: -> (hd0) /virtual/openbsd/disk0.img Nov 26 11:07:41: create file /virtual/openbsd/grub.cfg Nov 26 11:07:41: -> timeout=3 Nov 26 11:07:41: -> menuentry 'openbsd (bhyve run)' { Nov 26 11:07:41: -> root=hd0,1 Nov 26 11:07:41: -> kopenbsd -h com0 -r sd0a /bsd Nov 26 11:07:41: -> } Nov 26 11:07:41: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S -m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd Nov 26 11:07:44: [bhyve options: -c 1 -m 768M -AHP -w -U af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S] Nov 26 11:07:44: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/virtual/openbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0] Nov 26 11:07:44: [bhyve console: -l com1,/dev/nmdm-openbsd.1A] Nov 26 11:07:44: [bhyve iso device: -s 3:0,ahci-cd,/virtual/.iso/openbsd74.iso,ro] Nov 26 11:07:44: starting bhyve (run 2) Nov 26 11:09:13: bhyve exited with status 1 Nov 26 11:09:13: destroying network device tap0 Nov 26 11:09:13: stopped Nov 26 11:10:02: initialising Nov 26 11:10:02: [loader: grub] Nov 26 11:10:02: [cpu: 1] Nov 26 11:10:02: [memory: 768M] Nov 26 11:10:02: [hostbridge: standard] Nov 26 11:10:02: [com ports: com1] Nov 26 11:10:02: [uuid: af22d4e4-2a87-11eb-b8fd-a0369f7299dd] Nov 26 11:10:02: [debug mode: no] Nov 26 11:10:02: [primary disk: disk0.img] Nov 26 11:10:02: [primary disk dev: file] Nov 26 11:10:02: initialising network device tap0 Nov 26 11:10:02: booting Nov 26 11:10:02: create file /virtual/openbsd/device.map Nov 26 11:10:02: -> (cd0) /virtual/.iso/openbsd74.iso Nov 26 11:10:02: -> (hd0) /virtual/openbsd/disk0.img Nov 26 11:10:02: create file /virtual/openbsd/grub.cfg Nov 26 11:10:02: -> timeout=3 Nov 26 11:10:02: -> menuentry 'openbsd (bhyve install)' { Nov 26 11:10:02: -> root=cd0 Nov 26 11:10:02: -> kopenbsd -h com0 /7.4/amd64/bsd.rd Nov 26 11:10:02: -> } Nov 26 11:10:02: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S -m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd Nov 26 11:10:06: [bhyve options: -c 1 -m 768M -AHP -U af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S] Nov 26 11:10:06: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/virtual/openbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0] Nov 26 11:10:06: [bhyve console: -l com1,/dev/nmdm-openbsd.1A] Nov 26 11:10:06: [bhyve iso device: -s 3:0,ahci-cd,/virtual/.iso/openbsd74.iso,ro] Nov 26 11:10:06: starting bhyve (run 1) Nov 26 11:10:10: bhyve exited with status 0 Nov 26 11:10:10: restarting Nov 26 11:10:11: create file /virtual/openbsd/device.map Nov 26 11:10:11: -> (hd0) /virtual/openbsd/disk0.img Nov 26 11:10:11: create file /virtual/openbsd/grub.cfg Nov 26 11:10:11: -> timeout=3 Nov 26 11:10:11: -> menuentry 'openbsd (bhyve run)' { Nov 26 11:10:11: -> root=hd0,1 Nov 26 11:10:11: -> kopenbsd -h com0 -r sd0a /bsd Nov 26 11:10:11: -> } Nov 26 11:10:11: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S -m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd Nov 26 11:10:15: [bhyve options: -c 1 -m 768M -AHP -U af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S] Nov 26 11:10:15: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/virtual/openbsd/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0] Nov 26 11:10:15: [bhyve console: -l com1,/dev/nmdm-openbsd.1A] Nov 26 11:10:15: [bhyve iso device: -s 3:0,ahci-cd,/virtual/.iso/openbsd74.iso,ro] Nov 26 11:10:15: starting bhyve (run 2) Nov 26 11:10:52: bhyve exited with status 1 Nov 26 11:10:52: destroying network device tap0 Nov 26 11:10:52: stopped -- Public Keys: PGP KeyID = 0x5F22FDC1 GnuPG KeyID = 0x620836CF From nobody Mon Nov 27 03:41:16 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sdrwh2GWqz51q5R for ; Mon, 27 Nov 2023 03:41:24 +0000 (UTC) (envelope-from SRS0=aJx3=HI=freebsd.org=grehan@iredmail.onthenet.com.au) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sdrwg5gjDz3WnL for ; Mon, 27 Nov 2023 03:41:22 +0000 (UTC) (envelope-from SRS0=aJx3=HI=freebsd.org=grehan@iredmail.onthenet.com.au) Authentication-Results: mx1.freebsd.org; none Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id B965320B4981 for ; Mon, 27 Nov 2023 13:41:17 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id B114921419F5 for ; Mon, 27 Nov 2023 13:41:17 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nt.com.au; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:references:to:content-language:subject:subject :user-agent:mime-version:date:date:message-id; s=dkim; t= 1701056477; x=1703648478; bh=cxBBOGTh+ZuACC8HHNd/bMeGeqcW58zo9Nw YZRZxpeM=; b=VmEgvJyyMFIKLK0s+BAUU0SBQLRxQXwyj6HdaHbxGp37tDLIQ/G amfg6RAlIpej4VA2g4BxQagSdkNAxq6W8tmVMcVAKdaBGm+lz/pgaqnJegxxxGkl oQi+hUU4a3dz1w+FsAThLbDElVT45+wdaiI01L41pHQHIqiwav3nyOJo= Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id u5DhxES08KTr for ; Mon, 27 Nov 2023 13:41:17 +1000 (AEST) Received: from [192.168.1.108] (otn-120-29-24-174.broadband.onthenet.net [120.29.24.174]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 8463521419F2; Mon, 27 Nov 2023 13:41:16 +1000 (AEST) Message-ID: <5fd7aded-2422-4bec-b4ae-f944b1052508@freebsd.org> Date: Mon, 27 Nov 2023 13:41:16 +1000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Bhyve won't boot OpenBSD 7.4 Content-Language: en-US To: Thomas Laus References: <0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@email.amazonses.com> Cc: freebsd-virtualization@freebsd.org From: Peter Grehan In-Reply-To: <0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@email.amazonses.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=SvcLFMG0 c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=Zeo2tHO+Msv1PodVRVHjvw==:17 a=IkcTkHD0fZMA:10 a=BNY50KLci1gA:10 a=NEAV23lmAAAA:8 a=vTEbAenKoJUt3ks-aTAA:9 a=QEXdDO2ut3YA:10 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:9313, ipnet:203.13.68.0/24, country:AU] X-Rspamd-Queue-Id: 4Sdrwg5gjDz3WnL Hi Thomas, > I have 2 production servers that I want to upgrade to FreeBSD 14.0 that= =20 > use Bhyve to run OpenBSD 7.3 packet packet filters and a few other=20 > instances.=C2=A0 Since this is an operating system upgrade to FreeBSD 1= 4.0, I=20 > wanted to also upgrade the OpenBSD 7.3 instance to OpenBSD 7.4 that was= =20 > released last month. ... > Nov 26 10:51:10: /usr/local/sbin/grub-bhyve ^^^^^^^^^^ The issue is described here by Yuichiro Naito https://github.com/grehan-freebsd/grub2-bhyve/pull/16 Just need to test that out and update the port, and hopefully all=20 should be fixed. later, Peter. From nobody Mon Nov 27 12:40:24 2023 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sf4tf2VXtz52D4j for ; Mon, 27 Nov 2023 12:40:26 +0000 (UTC) (envelope-from 0100018c10cb8d45-3a75ba9f-30df-4601-841f-630a3b3f489c-000000@amazonses.com) Received: from a8-86.smtp-out.amazonses.com (a8-86.smtp-out.amazonses.com [54.240.8.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sf4td3DQTz4VG6 for ; Mon, 27 Nov 2023 12:40:25 +0000 (UTC) (envelope-from 0100018c10cb8d45-3a75ba9f-30df-4601-841f-630a3b3f489c-000000@amazonses.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=amazonses.com header.s=224i4yxa5dv7c2xz3womw6peuasteono header.b=eqbpyMZ8; spf=pass (mx1.freebsd.org: domain of 0100018c10cb8d45-3a75ba9f-30df-4601-841f-630a3b3f489c-000000@amazonses.com designates 54.240.8.86 as permitted sender) smtp.mailfrom=0100018c10cb8d45-3a75ba9f-30df-4601-841f-630a3b3f489c-000000@amazonses.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=acm.org (policy=none) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1701088824; h=Message-ID:Date:MIME-Version:Subject:To:References:From:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=Lrjc1PALuJvE3AbCGryoOClxa+DS1Qwym4jOd/15nQ4=; b=eqbpyMZ8SjTvtavEYa7QM1nZiZFIUKJ23VhgoOjS03mklvvqrV5e/cx6JrBSif6e FmuezzpNHve/maomlwhiWerl0g1Iyd6GjnLr5zJRMbfAmPDWne3c8U4lWnO3PeWtGie quBV7V4TmxNk15ib//mQkTDQmcKB4wZlonhKyhnM= Message-ID: <0100018c10cb8d45-3a75ba9f-30df-4601-841f-630a3b3f489c-000000@email.amazonses.com> Date: Mon, 27 Nov 2023 12:40:24 +0000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Bhyve won't boot OpenBSD 7.4 Content-Language: en-US To: virtualization@freebsd.org References: <0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@email.amazonses.com> <5fd7aded-2422-4bec-b4ae-f944b1052508@freebsd.org> From: Thomas Laus Autocrypt: addr=lausts@acm.org; keydata= xsDiBDx1NWwRBADARalI5I8kGeBYYYWnZB73T1fU4333yCuRokRvzlAZ5Zhb3hqsNdTEMheN FDjZSL8J5jeJtvSRinY2p09CxpAMoJR9zHLmHl+zEOY8fInbB+KiFtSfGf0blSEY9/+isQP9 xmUIQWUj0kwVtrns7m1HrYLiI07NVFzbHNKqQcbPuwCg0n/KKi+VJiUs5MqLKwGuPotGeZME AIluMetTQwfLyovundMwFYlSZ/Z8JjkMybqgKuiRrZnaBVVZ80NjAYZI73yAZPfQh9mvFxW9 ipc2tSALwDy/tYDpQRK0k+0EsDmwG/wM6OarkqSuFcYx+tP86+2+6Xitn6E/hriIWa/ZQVef /fx7dZzdwhXH6fd34v8o/BuqhawLBACs4MTMGbdSmyI56vCMXWY1yxRPmuygd4vUnXqYwlrM Ee/LjQdreg1zTAJnnW1K+PgOUW/jvS+uAbgxLa3i59/Z4Uu7nB6G1y+Y1cThojUsLnvoJlt1 4XE1U/vnOcvO3evo6knB1qjbAMsZGaVleiVKDq+7XE7swe4WtBJKbYJthc0cVGhvbWFzIExh dXMgPGxhdXN0c0BhY20ub3JnPsJlBBMRAgAlAheAAh4BBgsJCAcDAgYVCAIJCgsEFgIDAQUC Y++t4wUJKy+wdwAKCRDr/4LUYgg2z3viAJ4iIhfj6EQW8rbJjdAmLX6K6I0WCACfVo9g+WUL nfSUcrH4QERMagahtmrOwU0ERbKYHBAIAI17IvVqqMACme3dosere34mILtkARlqt+jbzfnD wSSLtAp2oC9b+FtVK0Vv9HCJy84KpVq0lO0hYMtfqxgZbrDB3/P32hnmQ9F8DHWIEyGHY3Su VWdBmkVlLNC2xwDiUJDhURyzABBvMK3N39ca57lSeCTzgPNZE5k172E8kDVyQcknNKDUoL0r rfGgWzRgWYcPVNld8l155zHlkfPg2K64mHTnUXiu0tCzTfEz2OtDycdyizfb1GPQm+S8tDWr xd8ERD0KmJTX0zKBvxgP5/FwekgqDI+96CgssBS23JAVIxbJe8E3iir1OvFfJFhu+tn7HF4T oUXbhExq42tPTG8AAwUH/1osnJnXzbQeNfr70x0Ks/bZt2RwFtshOdwsX0Kb5ckvysJ1oC45 f4ldEh0Q3nXtDSeckxKj1Yaqu1gnZalbBz0qVxX78joGzZoz5jmP2tZ1uFJ1kqgO24EL5LYW oYaC3h85FPGCrqVtJtOUSAaakBEeh8ATpLE6uLFlf5VMtT9FU/BqqjyzZJSntgRLvioboS1e l8BBt1axzaGOv0eQjBrwYWW7YlFYgmZLZsSQIxatFQnLo+/jqOdnquAS9rzK8y1/tiG9SUlo nerdDfoK18W69pr+x9ldv+FaNFyNyCsT19ZnRsvUQZG0VBslfQPEZWktEE5177MWU4zp5o8T 1+vCTwQYEQIADwIbDAUCY++t4wUJKy+wdwAKCRDr/4LUYgg2zxt8AJ46/dSLMaIS2PVrhQ7k aW2h2upY6ACfckKS5ugbpEjWsl3Bxqt5WtA9W4o= In-Reply-To: <5fd7aded-2422-4bec-b4ae-f944b1052508@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Feedback-ID: 1.us-east-1.9pbSdi8VQuDGy3n7CRAr3/hYnLCug78GrsPo0xSgBOs=:AmazonSES X-SES-Outgoing: 2023.11.27-54.240.8.86 X-Spamd-Result: default: False [-0.68 / 15.00]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.995]; FORGED_SENDER(0.30)[lausts@acm.org,0100018c10cb8d45-3a75ba9f-30df-4601-841f-630a3b3f489c-000000@amazonses.com]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18]; R_DKIM_ALLOW(-0.20)[amazonses.com:s=224i4yxa5dv7c2xz3womw6peuasteono]; DMARC_POLICY_SOFTFAIL(0.10)[acm.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; RWL_MAILSPIKE_GOOD(-0.10)[54.240.8.86:from]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; RCVD_COUNT_ZERO(0.00)[0]; ARC_NA(0.00)[]; ASN(0.00)[asn:14618, ipnet:54.240.8.0/21, country:US]; MIME_TRACE(0.00)[0:+]; FROM_NEQ_ENVFROM(0.00)[lausts@acm.org,0100018c10cb8d45-3a75ba9f-30df-4601-841f-630a3b3f489c-000000@amazonses.com]; MLMMJ_DEST(0.00)[virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[amazonses.com:+]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[54.240.8.86:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; DWL_DNSWL_NONE(0.00)[amazonses.com:dkim] X-Rspamd-Queue-Id: 4Sf4td3DQTz4VG6 X-Spamd-Bar: / On 11/26/23 22:41, Peter Grehan wrote: >> Nov 26 10:51:10: /usr/local/sbin/grub-bhyve >                                    ^^^^^^^^^^ > >  The issue is described here by Yuichiro Naito >     https://github.com/grehan-freebsd/grub2-bhyve/pull/16 > > Just need to test that out and update the port, and hopefully all > should be fixed. > This update to grub2-bhyve has been tested for OpenBSD releases up to version 7.3. I don't have any issues with 7.3. Has this change been tested for OpenBSD 7.4? Is my problem related to console definition changes in the OpenBSD OS or something else? Tom -- Public Keys: PGP KeyID = 0x5F22FDC1 GnuPG KeyID = 0x620836CF From nobody Mon Nov 27 12:43:52 2023 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sf4yf6ktfz52FmC for ; Mon, 27 Nov 2023 12:43:54 +0000 (UTC) (envelope-from 0100018c10ceb8a3-173a19c8-7b25-4a64-9e7e-613292bc0cc4-000000@amazonses.com) Received: from a48-89.smtp-out.amazonses.com (a48-89.smtp-out.amazonses.com [54.240.48.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sf4yf1Qpxz4WpQ for ; Mon, 27 Nov 2023 12:43:54 +0000 (UTC) (envelope-from 0100018c10ceb8a3-173a19c8-7b25-4a64-9e7e-613292bc0cc4-000000@amazonses.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=amazonses.com header.s=224i4yxa5dv7c2xz3womw6peuasteono header.b=EupNut5g; spf=pass (mx1.freebsd.org: domain of 0100018c10ceb8a3-173a19c8-7b25-4a64-9e7e-613292bc0cc4-000000@amazonses.com designates 54.240.48.89 as permitted sender) smtp.mailfrom=0100018c10ceb8a3-173a19c8-7b25-4a64-9e7e-613292bc0cc4-000000@amazonses.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=acm.org (policy=none) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1701089032; h=Message-ID:Date:MIME-Version:Subject:To:References:From:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=wVTMOZy3rbfypw8o/Q5kzgaZhXGzxrTxQlYASZiwQJk=; b=EupNut5gmjnkunAgTKt7rMJGWEI4Txs4hD+0HGbMsaQWObUuDRKCGIwROcSgG6OD iRpE9r+BmVG60F3zzNAnnqXN32Iwvkr4ipxeTkP5KctA05qQAKJRQhxANEpyySCi+It hghtLqGDHC3IqDD4mnKE93ex6RZv1n9iCvfYET0A= Message-ID: <0100018c10ceb8a3-173a19c8-7b25-4a64-9e7e-613292bc0cc4-000000@email.amazonses.com> Date: Mon, 27 Nov 2023 12:43:52 +0000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Bhyve won't boot OpenBSD 7.4 Content-Language: en-US To: virtualization@freebsd.org References: <0100018c0d78981c-e969143a-6ba2-4e15-8889-25259e392848-000000@email.amazonses.com> <5fd7aded-2422-4bec-b4ae-f944b1052508@freebsd.org> From: Thomas Laus Autocrypt: addr=lausts@acm.org; keydata= xsDiBDx1NWwRBADARalI5I8kGeBYYYWnZB73T1fU4333yCuRokRvzlAZ5Zhb3hqsNdTEMheN FDjZSL8J5jeJtvSRinY2p09CxpAMoJR9zHLmHl+zEOY8fInbB+KiFtSfGf0blSEY9/+isQP9 xmUIQWUj0kwVtrns7m1HrYLiI07NVFzbHNKqQcbPuwCg0n/KKi+VJiUs5MqLKwGuPotGeZME AIluMetTQwfLyovundMwFYlSZ/Z8JjkMybqgKuiRrZnaBVVZ80NjAYZI73yAZPfQh9mvFxW9 ipc2tSALwDy/tYDpQRK0k+0EsDmwG/wM6OarkqSuFcYx+tP86+2+6Xitn6E/hriIWa/ZQVef /fx7dZzdwhXH6fd34v8o/BuqhawLBACs4MTMGbdSmyI56vCMXWY1yxRPmuygd4vUnXqYwlrM Ee/LjQdreg1zTAJnnW1K+PgOUW/jvS+uAbgxLa3i59/Z4Uu7nB6G1y+Y1cThojUsLnvoJlt1 4XE1U/vnOcvO3evo6knB1qjbAMsZGaVleiVKDq+7XE7swe4WtBJKbYJthc0cVGhvbWFzIExh dXMgPGxhdXN0c0BhY20ub3JnPsJlBBMRAgAlAheAAh4BBgsJCAcDAgYVCAIJCgsEFgIDAQUC Y++t4wUJKy+wdwAKCRDr/4LUYgg2z3viAJ4iIhfj6EQW8rbJjdAmLX6K6I0WCACfVo9g+WUL nfSUcrH4QERMagahtmrOwU0ERbKYHBAIAI17IvVqqMACme3dosere34mILtkARlqt+jbzfnD wSSLtAp2oC9b+FtVK0Vv9HCJy84KpVq0lO0hYMtfqxgZbrDB3/P32hnmQ9F8DHWIEyGHY3Su VWdBmkVlLNC2xwDiUJDhURyzABBvMK3N39ca57lSeCTzgPNZE5k172E8kDVyQcknNKDUoL0r rfGgWzRgWYcPVNld8l155zHlkfPg2K64mHTnUXiu0tCzTfEz2OtDycdyizfb1GPQm+S8tDWr xd8ERD0KmJTX0zKBvxgP5/FwekgqDI+96CgssBS23JAVIxbJe8E3iir1OvFfJFhu+tn7HF4T oUXbhExq42tPTG8AAwUH/1osnJnXzbQeNfr70x0Ks/bZt2RwFtshOdwsX0Kb5ckvysJ1oC45 f4ldEh0Q3nXtDSeckxKj1Yaqu1gnZalbBz0qVxX78joGzZoz5jmP2tZ1uFJ1kqgO24EL5LYW oYaC3h85FPGCrqVtJtOUSAaakBEeh8ATpLE6uLFlf5VMtT9FU/BqqjyzZJSntgRLvioboS1e l8BBt1axzaGOv0eQjBrwYWW7YlFYgmZLZsSQIxatFQnLo+/jqOdnquAS9rzK8y1/tiG9SUlo nerdDfoK18W69pr+x9ldv+FaNFyNyCsT19ZnRsvUQZG0VBslfQPEZWktEE5177MWU4zp5o8T 1+vCTwQYEQIADwIbDAUCY++t4wUJKy+wdwAKCRDr/4LUYgg2zxt8AJ46/dSLMaIS2PVrhQ7k aW2h2upY6ACfckKS5ugbpEjWsl3Bxqt5WtA9W4o= In-Reply-To: <5fd7aded-2422-4bec-b4ae-f944b1052508@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Feedback-ID: 1.us-east-1.9pbSdi8VQuDGy3n7CRAr3/hYnLCug78GrsPo0xSgBOs=:AmazonSES X-SES-Outgoing: 2023.11.27-54.240.48.89 X-Spamd-Result: default: False [-0.69 / 15.00]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.995]; FORGED_SENDER(0.30)[lausts@acm.org,0100018c10ceb8a3-173a19c8-7b25-4a64-9e7e-613292bc0cc4-000000@amazonses.com]; R_DKIM_ALLOW(-0.20)[amazonses.com:s=224i4yxa5dv7c2xz3womw6peuasteono]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18:c]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[acm.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; RWL_MAILSPIKE_GOOD(-0.10)[54.240.48.89:from]; XM_UA_NO_VERSION(0.01)[]; MLMMJ_DEST(0.00)[virtualization@freebsd.org]; ASN(0.00)[asn:14618, ipnet:54.240.48.0/23, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ZERO(0.00)[0]; RCVD_IN_DNSWL_NONE(0.00)[54.240.48.89:from]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[amazonses.com:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[lausts@acm.org,0100018c10ceb8a3-173a19c8-7b25-4a64-9e7e-613292bc0cc4-000000@amazonses.com]; TO_DN_NONE(0.00)[]; DWL_DNSWL_NONE(0.00)[amazonses.com:dkim] X-Rspamd-Queue-Id: 4Sf4yf1Qpxz4WpQ X-Spamd-Bar: / On 11/26/23 22:41, Peter Grehan wrote: > Hi Thomas, >  The issue is described here by Yuichiro Naito >     https://github.com/grehan-freebsd/grub2-bhyve/pull/16 > > Just need to test that out and update the port, and hopefully all > should be fixed. > I now see that it works on OpenBSD CURRENT as of September of this year, so I should be good to go as soon as the port hits the tree. Thanks for the work. Tom -- Public Keys: PGP KeyID = 0x5F22FDC1 GnuPG KeyID = 0x620836CF