From owner-freebsd-virtualization@freebsd.org Sun Apr 7 17:32:00 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE0BD1565A5C for ; Sun, 7 Apr 2019 17:31:59 +0000 (UTC) (envelope-from jonathans.email@icloud.com) Received: from pv50p00im-ztdg10021201.me.com (pv50p00im-ztdg10021201.me.com [17.58.6.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D060A6A8F7 for ; Sun, 7 Apr 2019 17:31:57 +0000 (UTC) (envelope-from jonathans.email@icloud.com) Received: from jonathans-mbp.home (unknown [94.1.145.187]) by pv50p00im-ztdg10021201.me.com (Postfix) with ESMTPSA id A43FCA4000B for ; Sun, 7 Apr 2019 17:31:49 +0000 (UTC) From: Jonathan Rowley Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: unsubscribe Date: Sun, 7 Apr 2019 18:31:46 +0100 References: To: freebsd-virtualization@freebsd.org In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3445.102.3) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-04-07_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=11 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1812120000 definitions=main-1904070165 X-Rspamd-Queue-Id: D060A6A8F7 X-Spamd-Bar: -------- X-Spamd-Result: default: False [-8.64 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_COMPOSITE_RCVD_IN_DNSWL_MED_DWL_DNSWL_LOW(0.00)[]; FREEMAIL_FROM(0.00)[icloud.com]; R_SPF_ALLOW(-0.20)[+ip4:17.58.0.0/16]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DKIM_TRACE(0.00)[icloud.com:+]; RCVD_IN_DNSWL_MED(-0.20)[45.6.58.17.list.dnswl.org : 127.0.5.2]; DMARC_POLICY_ALLOW(-0.50)[icloud.com,quarantine]; MX_GOOD(-0.01)[mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, m x6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.co m, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud .com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icloud.com, mx6.mail.icloud.com, mx4.mail.icloud.com, mx5.mail.icloud.com, mx2.mail.icloud.com, mx3.mail.icloud.com, mx1.mail.icl oud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com,mx1.mail.icloud.com,mx6.mail.icloud.com,mx4.mail.icloud.com,mx5.mail.icloud.com,mx2.mail.icloud.com,mx3.mail.icloud.com]; NEURAL_HAM_SHORT(-0.93)[-0.932,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[icloud.com]; ASN(0.00)[asn:714, ipnet:17.58.0.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[icloud.com:s=04042017]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(0.00)[icloud.com.dwl.dnswl.org : 127.0.5.1]; IP_SCORE(-1.50)[ip: (-3.24), ipnet: 17.58.0.0/20(-2.10), asn: 714(-2.11), country: US(-0.06)]; WHITELIST_SPF_DKIM(-3.00)[icloud.com:d:+,icloud.com:s:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2019 17:32:00 -0000 > On 7 Apr 2019, at 13:00, freebsd-virtualization-request@freebsd.org = wrote: >=20 > Send freebsd-virtualization mailing list submissions to > freebsd-virtualization@freebsd.org >=20 > To subscribe or unsubscribe via the World Wide Web, visit > = https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > or, via email, send a message with subject or body 'help' to > freebsd-virtualization-request@freebsd.org >=20 > You can reach the person managing the list at > freebsd-virtualization-owner@freebsd.org >=20 > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-virtualization digest..." >=20 >=20 > Today's Topics: >=20 > 1. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 2. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 3. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes) > 4. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes) > 5. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 6. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) >=20 >=20 > ---------------------------------------------------------------------- >=20 > Message: 1 > Date: Sun, 7 Apr 2019 09:37:43 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407023743.GB99339@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>=20 >>>> You can usually use the host by doing mdconfig -f = >>>=20 >>> Unfortunately mdconfig does not work with zvols: >>>=20 >>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>=20 >> If its a zvol cant you just do >> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>=20 >> and=20 >> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >=20 > No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >=20 > This is the default for a reason: it's no good exposing scores of = always > coming and going guest geoms to the host system. I think you can even > get a conflict of labels or something like that one day. >=20 >>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>> prompt and with a few commands: >>>>=20 >>>> Yes, the timeout is very long, and I do not know that we >>>> document anyplace that if you wait long enough at a failed >>>> boot you do get a EFI shell prompt eventually. >>>=20 >>> Can I press some key to escape to the EFI shell? >> Not that I am aware of. >=20 > It's a major problem! There must be a well-known way to break the boot > sequence any time and enter the EFI shell. >=20 >>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), = for >>>>> example? >>>>=20 >>>> I suspect that what ever guest you installed installed something >>>> else someplace, either within the eft partition, or possibly in >>>> the MBR? >>>=20 >>> Do you mean to say, the guest installing something else someplace = can >>> influence the boot sequence of bhyve efi? >>=20 >> The guest created all of the bits on that zvol, >> it can influence many things. There is probably a tiny initial >> stub that efi loads that has this bath to grubx64.efi codded in >> it and that is what is causing this issue. >=20 > It is very important to find and debug it because Oracle VirtualBox in > UEFI mode installs and runs this guest just fine. So it must be some > issue in bhyve itself. >=20 > Here is the complete archive of everything the guest created in the = EFI > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > can you find those confusing bits? >=20 > The standard procedure should be as follows: >=20 > Automated detection relies on standardized file paths to the OS > loader, with the path varying depending on the computer architecture. > The format of the file path is defined as > /EFI/BOOT/BOOT.EFI; for > example, the file path to the OS loader on an x86-64 system is > /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture.=20= >=20 > Nothing about grub*.efi. But only bhyve is confused, VirtualBox is = not. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = >=20 > ------------------------------ >=20 > Message: 2 > Date: Sun, 7 Apr 2019 10:12:37 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407031237.GA7489@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Victor Sudakov wrote: >>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>> example? >>>>>=20 >>>>> I suspect that what ever guest you installed installed something >>>>> else someplace, either within the eft partition, or possibly in >>>>> the MBR? >>>>=20 >>>> Do you mean to say, the guest installing something else someplace = can >>>> influence the boot sequence of bhyve efi? >>>=20 >>> The guest created all of the bits on that zvol, >>> it can influence many things. There is probably a tiny initial >>> stub that efi loads that has this bath to grubx64.efi codded in >>> it and that is what is causing this issue. >>=20 >> It is very important to find and debug it because Oracle VirtualBox = in >> UEFI mode installs and runs this guest just fine. So it must be some >> issue in bhyve itself. >>=20 >> Here is the complete archive of everything the guest created in the = EFI >> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >> can you find those confusing bits? >=20 > I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but > BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some > kind of chain loader. >=20 > Watch the interactive session below. It does not however mean that = there is > nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and = runs this > guest just fine. >=20 > FS0:\> cd EFI > FS0:\EFI\> ls > Directory of: FS0:\EFI\ > 04/04/2019 15:53 2,048 . > 04/04/2019 15:53 0 .. > 04/04/2019 16:26 2,048 centos > 04/06/2019 04:19 2,048 BOOT > 0 File(s) 0 bytes > 4 Dir(s) > FS0:\EFI\> cd BOOT > FS0:\EFI\BOOT\> ls > Directory of: FS0:\EFI\BOOT\ > 04/04/2019 16:18 2,048 . > 04/04/2019 16:18 2,048 .. > 08/31/2017 21:30 1,296,176 BOOTX64.EFI > 08/31/2017 21:30 79,048 fbx64.efi > 2 File(s) 1,375,224 bytes > 2 Dir(s) > FS0:\EFI\BOOT\> BOOTX64.EFI > Failed to set MokListRT: Invalid Parameter > Failed to open \EFI\BOOT\grubx64.efi - Not Found > Failed to load image \EFI\BOOT\grubx64.efi: Not Found > start_image() returned Not Found > FS0:\EFI\BOOT\>=20 >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = >=20 > ------------------------------ >=20 > Message: 3 > Date: Sat, 6 Apr 2019 21:19:37 -0700 (PDT) > From: "Rodney W. Grimes" > To: Victor Sudakov > Cc: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <201904070419.x374JbIp048373@gndrsh.dnsmgr.net> > Content-Type: text/plain; charset=3DUS-ASCII >=20 >> Rodney W. Grimes wrote: >>>>>=20 >>>>> You can usually use the host by doing mdconfig -f = >>>>=20 >>>> Unfortunately mdconfig does not work with zvols: >>>>=20 >>>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>>=20 >>> If its a zvol cant you just do >>> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>>=20 >>> and=20 >>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >>=20 >> No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >>=20 >> This is the default for a reason: it's no good exposing scores of = always >> coming and going guest geoms to the host system. I think you can even >> get a conflict of labels or something like that one day. >=20 > So it may take a few more commands but it should be > possible to do this from the host side using host > side tools without having to boot a guest to make > these corrections. >=20 >>>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>>> prompt and with a few commands: >>>>>=20 >>>>> Yes, the timeout is very long, and I do not know that we >>>>> document anyplace that if you wait long enough at a failed >>>>> boot you do get a EFI shell prompt eventually. >>>>=20 >>>> Can I press some key to escape to the EFI shell? >>> Not that I am aware of. >>=20 >> It's a major problem! There must be a well-known way to break the = boot >> sequence any time and enter the EFI shell. >=20 > Agreed, hopefully those working on edk2 take note and either > chime in with what that way is, or create a bug and track > so that someone may fix this issue. >=20 >>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>> example? >>>>>=20 >>>>> I suspect that what ever guest you installed installed something >>>>> else someplace, either within the eft partition, or possibly in >>>>> the MBR? >>>>=20 >>>> Do you mean to say, the guest installing something else someplace = can >>>> influence the boot sequence of bhyve efi? >>>=20 >>> The guest created all of the bits on that zvol, >>> it can influence many things. There is probably a tiny initial >>> stub that efi loads that has this bath to grubx64.efi codded in >>> it and that is what is causing this issue. >>=20 >> It is very important to find and debug it because Oracle VirtualBox = in >> UEFI mode installs and runs this guest just fine. So it must be some >> issue in bhyve itself. >=20 > As I stated earlier bhyve is missing percistant efi variables, > and that is most likely the reason that VirtualBox just works > and bhyve does not. >=20 >> Here is the complete archive of everything the guest created in the = EFI >> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >> can you find those confusing bits? >=20 > Probably you well find in your VirtualBox directory a > file that is used to store efivars, that is where the > difference occurs. >=20 >> The standard procedure should be as follows: >>=20 >> Automated detection relies on standardized file paths to the OS >> loader, with the path varying depending on the computer architecture. >> The format of the file path is defined as >> /EFI/BOOT/BOOT.EFI; = for >> example, the file path to the OS loader on an x86-64 system is >> /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 = architecture.=20 >>=20 >> Nothing about grub*.efi. But only bhyve is confused, VirtualBox is = not. >=20 > bhyve is not really confused, it is simply missing a feature > that this guest depends on for its boot procedures. This is > a well known miss-feature, but I do not know of anyone actively > working on fixing it. >=20 >> Victor Sudakov, VAS4-RIPE, VAS47-RIPN >> 2:5005/49@fidonet http://vas.tomsk.ru/ > --=20 > Rod Grimes = rgrimes@freebsd.org >=20 >=20 > ------------------------------ >=20 > Message: 4 > Date: Sat, 6 Apr 2019 21:26:06 -0700 (PDT) > From: "Rodney W. Grimes" > To: Victor Sudakov > Cc: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <201904070426.x374Q641048406@gndrsh.dnsmgr.net> > Content-Type: text/plain; charset=3DUS-ASCII >=20 >> Victor Sudakov wrote: >>>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>>> example? >>>>>>=20 >>>>>> I suspect that what ever guest you installed installed something >>>>>> else someplace, either within the eft partition, or possibly in >>>>>> the MBR? >>>>>=20 >>>>> Do you mean to say, the guest installing something else someplace = can >>>>> influence the boot sequence of bhyve efi? >>>>=20 >>>> The guest created all of the bits on that zvol, >>>> it can influence many things. There is probably a tiny initial >>>> stub that efi loads that has this bath to grubx64.efi codded in >>>> it and that is what is causing this issue. >>>=20 >>> It is very important to find and debug it because Oracle VirtualBox = in >>> UEFI mode installs and runs this guest just fine. So it must be some >>> issue in bhyve itself. >>>=20 >>> Here is the complete archive of everything the guest created in the = EFI >>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >>> can you find those confusing bits? >>=20 >> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but >> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some >> kind of chain loader. >=20 > And it brobably tries to read a efivariable, and if that variable > is not set it defaults to grubx64.efi. This bootx64.efi is something > the guest installed into the EFI partition, hence my assertion that > the issue is with something the guest installed is some what valid. >=20 > There are some 3rd party EFI boot managers that might help > resolve this problem, or simply use the work around that > I provided earlier until we can get efivars working in > bhyve. >=20 >=20 >> Watch the interactive session below. It does not however mean that = there is >> nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and = runs this >> guest just fine. >>=20 >> FS0:\> cd EFI >> FS0:\EFI\> ls >> Directory of: FS0:\EFI\ >> 04/04/2019 15:53 2,048 . >> 04/04/2019 15:53 0 .. >> 04/04/2019 16:26 2,048 centos >> 04/06/2019 04:19 2,048 BOOT >> 0 File(s) 0 bytes >> 4 Dir(s) >> FS0:\EFI\> cd BOOT >> FS0:\EFI\BOOT\> ls >> Directory of: FS0:\EFI\BOOT\ >> 04/04/2019 16:18 2,048 . >> 04/04/2019 16:18 2,048 .. >> 08/31/2017 21:30 1,296,176 BOOTX64.EFI >> 08/31/2017 21:30 79,048 fbx64.efi >> 2 File(s) 1,375,224 bytes >> 2 Dir(s) >> FS0:\EFI\BOOT\> BOOTX64.EFI >> Failed to set MokListRT: Invalid Parameter >> Failed to open \EFI\BOOT\grubx64.efi - Not Found >> Failed to load image \EFI\BOOT\grubx64.efi: Not Found >> start_image() returned Not Found >> FS0:\EFI\BOOT\>=20 >>=20 >> --=20 >> Victor Sudakov, VAS4-RIPE, VAS47-RIPN >> 2:5005/49@fidonet http://vas.tomsk.ru/ >=20 > --=20 > Rod Grimes = rgrimes@freebsd.org >=20 >=20 > ------------------------------ >=20 > Message: 5 > Date: Sun, 7 Apr 2019 15:02:35 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407080235.GA40361@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>>>=20 >>>>>> You can usually use the host by doing mdconfig -f = >>>>>=20 >>>>> Unfortunately mdconfig does not work with zvols: >>>>>=20 >>>>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0=20 >>>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >>>>=20 >>>> If its a zvol cant you just do >>>> gpart show /dev/zvol/d02/vm/freepbx/disk0 >>>>=20 >>>> and=20 >>>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 >>>=20 >>> No I can't if the zvol is in the "volmode=3Ddev" mode which is the = default.=20 >>>=20 >>> This is the default for a reason: it's no good exposing scores of = always >>> coming and going guest geoms to the host system. I think you can = even >>> get a conflict of labels or something like that one day. >>=20 >> So it may take a few more commands but it should be >> possible to do this from the host side using host >> side tools without having to boot a guest to make >> these corrections. >=20 > I'm not aware of such commands. If anyone knows them please share with = us. >=20 > Moreover, I already asked a similar question in February under the > topic "mounting/exporting/importing a zfs volume" and nobody gave a > recipe. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = >=20 > ------------------------------ >=20 > Message: 6 > Date: Sun, 7 Apr 2019 15:14:45 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407081445.GB40361@admin.sibptus.ru> > Content-Type: text/plain; charset=3D"us-ascii" >=20 > Rodney W. Grimes wrote: >>>>>>>> I can guess that it looks for a FAT16 partition in the GPT with = the type >>>>>>>> "efi" but the rest is a mystery for me. Why is it trying to = find >>>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is = present), for >>>>>>>> example? >>>>>>>=20 >>>>>>> I suspect that what ever guest you installed installed something >>>>>>> else someplace, either within the eft partition, or possibly in >>>>>>> the MBR? >>>>>>=20 >>>>>> Do you mean to say, the guest installing something else someplace = can >>>>>> influence the boot sequence of bhyve efi? >>>>>=20 >>>>> The guest created all of the bits on that zvol, >>>>> it can influence many things. There is probably a tiny initial >>>>> stub that efi loads that has this bath to grubx64.efi codded in >>>>> it and that is what is causing this issue. >>>>=20 >>>> It is very important to find and debug it because Oracle VirtualBox = in >>>> UEFI mode installs and runs this guest just fine. So it must be = some >>>> issue in bhyve itself. >>>>=20 >>>> Here is the complete archive of everything the guest created in the = EFI >>>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz >>>> can you find those confusing bits? >>>=20 >>> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, = but >>> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be = some >>> kind of chain loader. >>=20 >> And it brobably tries to read a efivariable, and if that variable >> is not set it defaults to grubx64.efi. This bootx64.efi is something >> the guest installed into the EFI partition, hence my assertion that >> the issue is with something the guest installed is some what valid. >=20 > Do you think the guest OS installer set some efi variable during the > installation process, which bhyve did not save? That would explain a > lot. >=20 >>>>>>> Moreover, I waited (for a long time!) for the EFI interactive = shell >>>>>>> prompt and with a few commands: >>>>>>=20 >>>>>> Yes, the timeout is very long, and I do not know that we >>>>>> document anyplace that if you wait long enough at a failed >>>>>> boot you do get a EFI shell prompt eventually. >>>>>=20 >>>>> Can I press some key to escape to the EFI shell? >>>> Not that I am aware of. >>>=20 >>> It's a major problem! There must be a well-known way to break the = boot >>> sequence any time and enter the EFI shell. >>=20 >> Agreed, hopefully those working on edk2 take note and either >> chime in with what that way is, or create a bug and track >> so that someone may fix this issue. >=20 > Would it be useful to create a PR in the FreeBSD bugtracker with a > feature request? >=20 >>>=20 >>> It is very important to find and debug it because Oracle VirtualBox = in >>> UEFI mode installs and runs this guest just fine. So it must be some >>> issue in bhyve itself. >>=20 >> As I stated earlier bhyve is missing percistant efi variables, >> and that is most likely the reason that VirtualBox just works >> and bhyve does not. >>=20 >> Probably you well find in your VirtualBox directory a >> file that is used to store efivars, that is where the >=20 > I'll look into the VirtualBox directory tomorrow and report here. I = was > under the impression that efivars are stored in a configuration file = in > the EFI partition but I was probably wrong, they are kept in NVRAM > somewhere, like BIOS settings, and not on a disk. >=20 > --=20 > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/49@fidonet http://vas.tomsk.ru/ > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 455 bytes > Desc: not available > URL: = >=20 > ------------------------------ >=20 > Subject: Digest Footer >=20 > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to = "freebsd-virtualization-unsubscribe@freebsd.org" >=20 >=20 > ------------------------------ >=20 > End of freebsd-virtualization Digest, Vol 436, Issue 7 > ******************************************************