Date: Tue, 13 Jan 2026 19:50:58 +0100 From: Roman Bogorodskiy <novel@freebsd.org> To: Bojan =?utf-8?B?Tm92a292acSH?= <bnovkov@freebsd.org> Cc: freebsd-virtualization@freebsd.org Subject: Re: RFC: ACPI-based PCI device hotplugging for bhyve Message-ID: <aWaUEkIV4GHAJpPG@tulp> In-Reply-To: <66a1c389-6b87-4062-aa2d-a1123f9edc97@freebsd.org>
index | next in thread | previous in thread | raw e-mail
Bojan Novković wrote:
> Hi everyone,
>
> Over the last couple of months I've been working on implementing PCI device
> hotplugging in bhyve and would like to get some feedback on the changes.
> The patch series from D54641 to D54656 implements an ACPI-based hotplugging
> mechanism
> that is accessible through an nvlist(9)-based IPC protocol.
> Users can add to or remove an emulated PCI device from a running virtual
> machine
> by sending an appropriately crafted nvlist(9) object to a UNIX socket
> located at '/var/run/bhyve/<VM_name>'.
> The patch series also adds new command-line options to 'bhyvectl'
> ('--pci-{add,remove}') to facilitate these operations.
> Since the implementation ended up touching a lot of files and adding a lot
> of
> changes to the PCI emulation layer, I'll try to give a brief overview of the
> problems each change is meant to solve and outline the major changes in this
> patch series.
>
> Aside from implementing the ACPI-based hotplugging mechanism, adding PCI
> hotplugging to bhyve required solving the following problems:
> 1) bhyve's PCI BAR allocator can't allocate and free BARs during runtime,
> 2) bhyve is sandboxed, which makes it impossible to initialize certain
> emulated
> PCI devices during runtime since they might need to open files
> during initialization,
> 3) Allowing users to request hotplug or hotremove operations from a running
> bhyve instance.
>
> The first problem was solved using 'libuvmem(3)', a userspace port of the
> 'vmem(9)' general-purpose allocator. This allows us to manage resources
> owned by
> each virtual PCI bus during runtime. The second problem was solved by
> delegating
> parts of the PCI device initialization to 'bhyvectl'. Under this scheme,
> 'bhyvectl'
> opens all required files and sends the appropriate file descriptors to bhyve
> over
> a UNIX socket. To avoid unnecessary code duplication, parts of the bhyve's
> device-specific initialization routines were moved to 'libbhyve', a new
> private library.
> The third problem was solved by generalizing bhyve's VM snapshotting IPC
> thread.
> This allows end users to invoke bhyve's PCI hotplugging routines through an
> nvlist(9)-based IPC protocol.
>
> In summary, here's a brief overview of the changes made by this patch
> series.
> - Replacing bhyve's static PCI BAR allocation algorithm with 'libuvmem(3)'
> (D54641),
> - Adding ACPI methods that allow us to inject hotplugging events into the
> guest (D54642 - D54646),
> - Generalizing bhyve's IPC thread and introducing an nvlist(9)-based IPC
> protocol (D54650),
> - Refactoring and moving parts of the device initialization and
> configuration code to 'libbhyve' (D54648),
> - Adding hotplugging IPC commands to 'bhyve' and the corresponding
> '--pci-{add,remove}' options to 'bhyvectl' (D54651 - D54653),
> - Making the 'e1000' and 'virtio-net' network devices hotpluggable - (D54654
> - D54656).
>
> The network devices were chosen as a proof of concept. Other devices will
> be added after this patch series gets reviewed.
> Please feel free to leave comments on any of the reviews from D54641 to
> D54656.
>
> Thanks,
> Bojan
>
Hi Bojan,
This is an exciting feature for bhyve!
Do you have a public branch with all these changes? It's a bit
cumbersome to pull them from arcanist.
Roman
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?aWaUEkIV4GHAJpPG>