Date: Sun, 19 Apr 2020 16:41:53 -0700 From: Conrad Meyer <cem@freebsd.org> To: Shawn Webb <shawn.webb@hardenedbsd.org> Cc: src-committers <src-committers@freebsd.org>, svn-src-all <svn-src-all@freebsd.org>, svn-src-head <svn-src-head@freebsd.org> Subject: Re: svn commit: r359950 - head/usr.sbin/bhyve Message-ID: <CAG6CVpUsmDv6tAMLMmqOVy5Trw9SOE6Hr-_6Tf0v7eyyj6sNvA@mail.gmail.com> In-Reply-To: <20200419231353.fkx5uwtqq7h7yolh@mutt-hbsd> References: <202004150200.03F20I3H001953@repo.freebsd.org> <20200419224713.xaxvnuo334v6bhtj@mutt-hbsd> <CAG6CVpUkVPFJLE_4KsQ1-AUG2OATQY45WNu5Fq%2BJDhem0Gd2jQ@mail.gmail.com> <20200419231353.fkx5uwtqq7h7yolh@mutt-hbsd>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks! I believe we're running into VM_MAX_MEMMAPS (4) due to: 1. Low memory (0 through PCI hole) 2. High memory (4GB+) 3. Framebuffer 4. VMgenid's segment 5. EFI firmware's segment As a temporary workaround, if you do not need any of: >3GB RAM, framebuffer, or EFI boot, turning off any one of these should free a segment and allow boot. Unless there is an ABI problem, I think we should probably go ahead and bump VM_MAX_MEMMAPS, or make it dynamically sized. And as a userspace workaround for the issue that doesn't require a reboot (or reload of vmm.ko), we could add a knob to bhyve(8) to disable vmgenid. Best, Conrad On Sun, Apr 19, 2020 at 4:14 PM Shawn Webb <shawn.webb@hardenedbsd.org> wrote: > > This is the full output from bhyve: > > fbuf frame buffer base: 0x69191a00000 [sz 16777216] > bhyve: bootrom_alloc: vm_mmap_mapseg: No space left on device > bhyve: vmgenc_init: bootrom_alloc > > Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > GPG Key ID: 0xFF2E67A277F8E1FA > GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9 3633 C85B 0AF8 AB23 0FB2 > https://git-01.md.hardenedbsd.org/HardenedBSD/pubkeys/src/branch/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > > On Sun, Apr 19, 2020 at 04:04:16PM -0700, Conrad Meyer wrote: > > Hey Shawn, > > > > I will take a look. Thanks for the report and especially the repro example. > > What sort of bad symptoms are you observing (or will it be super obvious > > when I try this)? > > > > Thanks, > > Conrad > > > > On Sun, Apr 19, 2020 at 15:53 Shawn Webb <shawn.webb@hardenedbsd.org> wrote: > > > > > On Wed, Apr 15, 2020 at 02:00:18AM +0000, Conrad Meyer wrote: > > > > Author: cem > > > > Date: Wed Apr 15 02:00:17 2020 > > > > New Revision: 359950 > > > > URL: https://svnweb.freebsd.org/changeset/base/359950 > > > > > > > > Log: > > > > bhyve(8): Add VM Generation Counter ACPI device > > > > > > > > Add an implementatation of the 'Virtual Machine Generation ID' spec to > > > > Bhyve. The spec provides a randomly generated GUID (at bhyve start) in > > > > device memory, along with an ACPI device with _CID VM_Gen_Counter and > > > ADDR > > > > evaluating to a Package pointing at that GUID. > > > > > > > > A GPE is defined which Notifies the ACPI Device when the generation > > > changes > > > > (such as when a snapshot is rolled back). At this time, Bhyve does not > > > > support snapshotting, so the GPE is never actually raised. > > > > > > > > Suggested by: rpokala > > > > Discussed with: grehan > > > > Differential Revision: https://reviews.freebsd.org/D23165 > > > > > > > > Added: > > > > head/usr.sbin/bhyve/vmgenc.c (contents, props changed) > > > > head/usr.sbin/bhyve/vmgenc.h (contents, props changed) > > > > Modified: > > > > head/usr.sbin/bhyve/Makefile > > > > head/usr.sbin/bhyve/acpi.c > > > > head/usr.sbin/bhyve/acpi.h > > > > head/usr.sbin/bhyve/bhyverun.c > > > > head/usr.sbin/bhyve/pm.c > > > > > > Hey Conrad, > > > > > > Something about this commit broke bhyve in UEFI mode. Reverting this > > > specific change caused bhyve to work again. Here's a sample command: > > > > > > /usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve \ > > > -c 4 \ > > > -m 16g \ > > > -H \ > > > -A \ > > > -P \ > > > -S \ > > > -g 0 \ > > > -s 0:0,hostbridge \ > > > -s 1:0,lpc \ > > > -s 29,fbuf,tcp=127.0.0.1:5910,w=1024,h=768,wait \ > > > -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ > > > -s 2:0,virtio-net,tap1 \ > > > -s 3:0,virtio-blk,/dev/zvol/rpool/bhyve/hbsd-cross-dso-cfi-01/disk-01 \ > > > -l com1,/dev/nmdm-hbsd-cross-dso-cfi-01-A \ > > > -s 31:0,ahci-cd,/ISO/HardenedBSD/12-stable_amd64/2020-04-19_disc1.iso \ > > > hbsd-cdcfi-01 > > > > > > Thanks, > > > > > > -- > > > Shawn Webb > > > Cofounder / Security Engineer > > > HardenedBSD > > > > > > GPG Key ID: 0xFF2E67A277F8E1FA > > > GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9 3633 C85B 0AF8 AB23 0FB2 > > > > > > https://git-01.md.hardenedbsd.org/HardenedBSD/pubkeys/src/branch/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAG6CVpUsmDv6tAMLMmqOVy5Trw9SOE6Hr-_6Tf0v7eyyj6sNvA>