From nobody Sun Feb 5 15:33:40 2023 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4P8thh4PwLz3kS3H for ; Sun, 5 Feb 2023 15:33:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4P8thh2z0Dz44k6 for ; Sun, 5 Feb 2023 15:33:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675611220; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EiQ+Zib6yv7UxslPi9tx8tWSwYwe5Aoy4HFU0D1hRXQ=; b=jLW3qgID2HseN8YNnFWp/SAqv8xz2QxArNIoO77J3whzTak9Erad1XLFGE1Wq10q6XoRrt XPhn0HCrw+aR2u+v62mkqE5KNOFfM1PAdNJPnl36cBItjOLJ4rYNFW1wHkuw+GJOweTFQO qx01fpEPC1ijte8xTEPFpLOnDD4hTqU6AcA8Qmdz8rNV3+uUgtqmoGaaVgPwDfwncSJv3j XDCOSkarIZhifEAv2it9xZej1XZL/MOzlygJByfjNDc2rUW8yfYCLJgw9PCWwEFzQQiGnX iFXgq+TcrK6hTxd9fIc6+3Nqa4hfDKbzTOsM/aii0BznBpu5UJghEVRHmvWVkw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1675611220; a=rsa-sha256; cv=none; b=wcOQwUxYkIL+rLzwOUkCj0yguD6/B6nXDNTX55aUDUNUNb0ijnhTx/AFlZpE81GfFtjFqA 2/vj1FENXhKNnUV2icomu42m3BS2BjWq4W0wB2VcdxwwvYQoDXZXQiFN/gRBZagqFyU83t +UQ9NHK2eFxLvJrfkQ1PSsF/VYs4hVj6gHS6PUMaRNhf9Yj+O97sh8RPZTxLUGDnvpFctO GO77jKRKHIEZ4+xVsEq5S360jffjfQU6YOnUVqNy+lBwDr7e6hXyMTq35kAT3hccc60fr4 FHIQWFBnfydtcXCBEZYy9GNDprtps2Vk6vmq4IF647SWCsobrZhbABTBHi18XQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4P8thh241Nz100V for ; Sun, 5 Feb 2023 15:33:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 315FXeHs047066 for ; Sun, 5 Feb 2023 15:33:40 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 315FXe0L047065 for virtualization@FreeBSD.org; Sun, 5 Feb 2023 15:33:40 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 221074] Hyper V Gen 2 install has no mouse Date: Sun, 05 Feb 2023 15:33:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221074 --- Comment #32 from commit-hook@FreeBSD.org --- A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3De4d3f1e40ab3327e4ff6f689f54a64bd2= ebc0568 commit e4d3f1e40ab3327e4ff6f689f54a64bd2ebc0568 Author: Yuri AuthorDate: 2023-02-05 15:32:08 +0000 Commit: Vladimir Kondratyev CommitDate: 2023-02-05 15:32:08 +0000 hv_hid: Hyper-V HID driver Hyper-V HID driver using hidbus/hms. Reviewed by: wulf MFC after: 1 week PR: 221074 Differential revision: https://reviews.freebsd.org/D38140 sys/amd64/conf/NOTES | 1 + sys/conf/files.x86 | 1 + sys/dev/hid/hidbus.c | 1 + sys/dev/hyperv/input/hv_hid.c (new) | 564 ++++++++++++++++++++++++++++++= ++++ sys/i386/conf/NOTES | 1 + sys/modules/hyperv/Makefile | 2 +- sys/modules/hyperv/hid/Makefile (new) | 12 + 7 files changed, 581 insertions(+), 1 deletion(-) --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sun Feb 5 21:00:20 2023 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4P91xd1XBJz3kqm3 for ; Sun, 5 Feb 2023 21:00:21 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4P91xc6PBPz3LC2 for ; Sun, 5 Feb 2023 21:00:20 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675630820; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=pTSF7C3r4ISUdFNZEnsOmqKykTiBnku7NNyxIv6B7xI=; b=Gd+XATXKFfM/B6Lnl8ZGmbHaBmkPIGZ4d/PFMwe/ovnEbYT4KIg7Zqlb408n7xZNzx/pzF aIv4leUujJb0wyvIWzXb8QxPHvnwNp5sylT3+OKaQyIaSCN/soGK9CkdrIwgtQ87/ZZP/B R7zUWxQyB5UXfPrfDr1oxgprn1dAf0AlnoSL5RsEQKDuIxGy/UC2131wGEFsw02fdoVVth k1SNVWpA3uIupqtkbTzMJjOn8JNEzCQKQJ21bxNCI5EOECbXCPL/Suzurm4pUr8XjWruuq yY0HWcxFdX+Spie8WJKUqNSn+CTK1FrD+jXTwG/QULUombI8TBqcQxKPNMTbpw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1675630820; a=rsa-sha256; cv=none; b=p9H/MOueAV6K2pzm1asGUPHjdVXCNGc1fq2hZQJYs59iVQJVgoQ00GtiUqxjrpp/s1xV4j Woyej5+uqBCZ7eRQs7YzrUsNZoWyKRPGIvNdnGB7RvMruktzq8nxah/fY5r5dW/BvTzH+8 00L4sZ27Uq6UDieGbO2m4qC48Sezs73AfDhleGPyF5Pg1dQL+AheHlKjwejfGbdeqdRX0U sCsI444fvaW5O2G7F2XtYEqf3KepQ9N33KZQv6uq5p45ha9BB59SQS8ekveEHB0kOEM3Fh ai9fxl496ORyFsV5UaSgmG/MyN6iqhcE7IjZIGkBHFrTTpIcfSChdgY+FRKp3Q== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4P91xc5Vn3z17yX for ; Sun, 5 Feb 2023 21:00:20 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 315L0K4L022255 for ; Sun, 5 Feb 2023 21:00:20 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 315L0K9V022254 for virtualization@FreeBSD.org; Sun, 5 Feb 2023 21:00:20 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202302052100.315L0K9V022254@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 5 Feb 2023 21:00:20 +0000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="16756308203.d33d6b.20182" Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N --16756308203.d33d6b.20182 Date: Sun, 5 Feb 2023 21:00:20 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 247208 | mpt(4): VMWare virtualized LSI controller panics New | 240945 | [hyper-v] [netvsc] hn network driver incorrectly Open | 244838 | "bectl activate -t" does not honor the -t flag in 3 problems total for which you should take action. --16756308203.d33d6b.20182 Date: Sun, 5 Feb 2023 21:00:20 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
In Progress |    247208 | mpt(4): VMWare virtualized LSI controller panics 
New         |    240945 | [hyper-v] [netvsc] hn network driver incorrectly 
Open        |    244838 | "bectl activate -t" does not honor the -t flag in

3 problems total for which you should take action.
--16756308203.d33d6b.20182-- From nobody Wed Feb 8 10:36:48 2023 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PBbym3Fzfz3n1Kc for ; Wed, 8 Feb 2023 10:36:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PBbym003bz42GY for ; Wed, 8 Feb 2023 10:36:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675852608; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gaqlpZT/3qOoJ2LJZ04V4S/TOEiR7w+/h3zwPDJ/HS4=; b=OmoFFPBruSAAknTx4kiAcBT2hldVOnh476YhoVrjmc/tfk4McuMyY5Gp70Uut3rO/Yt4Lu 7kSOnDlE4eoYRh5vLE0jzpzEGjwYMjvBmYiqKS2bJA6AoDST76hExIsRektHy4SMerE7FV OLfZ2Y5i6oCX0jhmlWbE+r6sMMi5a4hy0RETm90/FUeFMX70p7lu7IHq9BVaHTx5tPVLSY dUc3e2bUWPoxBXBWxWZel1wGr/kiTAjT/aqpreZ6rlkhJ/3h+ytsmkgqEB0IFYlrhSUNg5 nKCd65Exl7gJrpftUJML5LanFQvpnPrzWPEduKUdrx9P7ngO3Icnwi6lsP3Ivw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1675852608; a=rsa-sha256; cv=none; b=rNsCdw2l3UNiqIXaVk15tPBzcMSGWfEE8QZ3r0xxo9IBLAD7cUv/hxAf+CCK0jO4VteO3H MV/pvTIZPp/mjBgTTCCN1/cgXM96/LGSz2urQgPs543k3swznm/k3i1tkBz6rVMH97dXED IzIvYMhpi0wEiYlM6zY/W+8cjn1DmTJd3nCQto+nWTwiIfH1c/BRnh+Q5+YyzG6HYSShP1 qqHij9742mxaoGHEJKlYY1Og/VVo9RS1PhfGGS1mSAIlyJR5sg3FdeTIo8yYjvEt6PYkny 8kKmUVCYzqrqCvn87FEdwvHXrUSaMJqSbJq+UBh1ZyQub0LI6DlgOlYtys4zHw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PBbyl69RdztqP for ; Wed, 8 Feb 2023 10:36:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 318AalEA038207 for ; Wed, 8 Feb 2023 10:36:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 318AaldS038206 for virtualization@FreeBSD.org; Wed, 8 Feb 2023 10:36:47 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 268794] Simultaneous vcpu_lock_all() and vm_handle_rendezvous() can deadlock vmm Date: Wed, 08 Feb 2023 10:36:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 13.1-STABLE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: corvink@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D268794 --- Comment #14 from Corvin K=C3=B6hne --- The easiest test to check if the rom works correctly would be to boot a lin= ux guest and checking the dmesg output. Additionally, if it fails, it'll give = us more information about the failure. The VBIOS of older gpus don't contain an EFI driver which is required for graphical output on boot. You can test it with the EfiRom tool included in edk2's BaseTools. To get the tool, clone https://github.com/tianocore/edk2 = and run make in the BaseTools folder. Running `edksetup.sh` in the base folder = of the repo adds the command to your path. After that run `EfiRom -d your-vbios.rom`. It should contain EFI ROM header like: ``` Image 1 -- Offset 0x0 ROM header contents Signature 0xAA55 PCIR offset 0x001C Signature PCIR Vendor ID 0x8086 Device ID 0x5912 Length 0x001C Revision 0x0003 DeviceListOffset 0x00 Class Code 0x000000 Image size 0x11C00 Code revision: 0x0000 MaxRuntimeImageLength 0x00 ConfigUtilityCodeHeaderOffset 0x00 DMTFCLPEntryPointOffset 0x00 Indicator 0x80 (last image) Code type 0x03 (EFI image) EFI ROM header contents EFI Signature 0x0EF1 Compression Type 0x0000 (not compressed) Machine type 0x8664 (X64) Subsystem 0x000B (EFI boot service driver) EFI image offset 0x01E0 (@0x1E0) ``` --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed Feb 8 18:05:20 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PBnwP4htHz3nk40 for ; Wed, 8 Feb 2023 18:05:25 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-il1-x12e.google.com (mail-il1-x12e.google.com [IPv6:2607:f8b0:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PBnwN5X5Tz3nrb; Wed, 8 Feb 2023 18:05:24 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=UbukKn7b; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::12e as permitted sender) smtp.mailfrom=markjdb@gmail.com; dmarc=none Received: by mail-il1-x12e.google.com with SMTP id v15so967147ilc.10; Wed, 08 Feb 2023 10:05:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=zQOHdUDbGP1ViKIYaGVZKyXs9Q4XZmpU3BUfnv2lpRI=; b=UbukKn7bGosd1s9/WRvbCu/m1ltWNigAtk3k2QDnjJDz4+z4SSRBr+v0gjJB2xoLFI 25FJfuNLnQKNOUFjuDt8wQIGpMReujswb4Uo/eJYMdjRoOxRMSumv+oGTuBEwzNb6cks XgS9Fkz4rM1PmY9QUJwY82m733BtJ31/y6DMwxuQ+4xZq4bITMpuQJtJ1pqeogqks2Kg r31kWd8G/yEt/u9ihQa5E8VmGDZN8V+SjncUFNY8lp2iEfNYNNbGzkppiYrsHNn1GnxT 4av6RhlJhqd73EGDo8mRPF44yhPWF4RrwX8Pl/NkXM3dNADFd22iwmuEvzQwgVYyr0RJ Kt2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zQOHdUDbGP1ViKIYaGVZKyXs9Q4XZmpU3BUfnv2lpRI=; b=faSmSwcQtudGjqLcrFKVeJRlk1YbUNSpM+GYV+Ly9HXMrhaQOuWyhDAJ+6AL9dA4rO MqDYVbv0MAY7rJZkO9B49oPl7qIuPTOLDaoH6OgenkST8rwuBrrzuXlbnhsMqCIjnLLV ukXaSthh0I8e7mAXKYrsz86RIc1iUA221YbVDu7FYnduAJedeK+N8pZ6D3n1QSWoakIw xquslK/Odv01jm2PZpOxX7VvAjWrwAcNaRKnfcw+7zINTpTmjZ57XeaznbAKmw1PyST6 UE58Y9iyH+z9Xo3KL2HVZLGbk2RY276oTYCu1rxjq0wHxZE/onP2tn0560BZzUXcYjH9 74vA== X-Gm-Message-State: AO0yUKUk8Ebn/FUl01DEU+jo0t4HbWFGQvCfZtYi+oALHzpQkmpL/KWV omoXWhXjSzz5AiLJDHTH3n7UT/e1urY= X-Google-Smtp-Source: AK7set9X9uaq6iNs/Ae7biFZ5LDhC7FCNWSrN2PAzcgRh0v9j86J6Lz2ZFJIBWuRelPRvdn/rfzUxw== X-Received: by 2002:a05:6e02:8ab:b0:313:e431:10fb with SMTP id a11-20020a056e0208ab00b00313e43110fbmr5752457ilt.2.1675879523276; Wed, 08 Feb 2023 10:05:23 -0800 (PST) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id y24-20020a02ce98000000b003c488204c6fsm28356jaq.76.2023.02.08.10.05.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Feb 2023 10:05:22 -0800 (PST) Date: Wed, 8 Feb 2023 13:05:20 -0500 From: Mark Johnston To: Corvin =?iso-8859-1?Q?K=F6hne?= , jhb@freebsd.org Cc: freebsd-virtualization@freebsd.org Subject: bhyve 13.1 compatibility breakage Message-ID: References: <202211230800.2AN80G58068419@gitrepo.freebsd.org> List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spamd-Result: default: False [-2.60 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.90)[-0.902]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::12e:from]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[freebsd.org]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4PBnwN5X5Tz3nrb X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N On Sun, Jan 15, 2023 at 10:46:21AM -0500, Mark Johnston wrote: > On Wed, Nov 23, 2022 at 08:00:16AM +0000, Corvin Köhne wrote: > > The branch main has been updated by corvink: > > > > URL: https://cgit.FreeBSD.org/src/commit/?id=7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b > > > > commit 7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b > > Author: Corvin Köhne > > AuthorDate: 2022-11-21 14:00:04 +0000 > > Commit: Corvin Köhne > > CommitDate: 2022-11-23 08:00:04 +0000 > > > > vmm: don't lock a mtx in the icr_low write handler > > > > x2apic accesses are handled by a wrmsr exit. This handler is called in a > > critical section. So, we can't lock a mtx in the icr_low handler. > > > > Reported by: kp, pho > > Tested by: kp, pho > > Approved by: manu (mentor) > > Fixes: c0f35dbf19c3c8825bd2b321d8efd582807d1940 vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs. > > MFC after: 1 week > > MFC with: c0f35dbf19c3c8825bd2b321d8efd582807d1940 > > Sponsored by: Beckhoff Automation GmbH & Co. KG > > Differential Revision: https://reviews.freebsd.org/D37452 > > --- > > Hi Corvin, > > This seems to break AP startup when using bhyve/libvmmapi from FreeBSD > 13.1 on a kernel built from main. It looks like the commit somehow > regresses commit 769b884e2e2, but I'm not sure yet exactly how. I debugged this further and am not quite sure how to fix the problem, which isn't specific to this commit after all. I'll try to describe it here. Suppose we're booting a VM with 2 vCPUs. When the BSP raises the INIT IPI to start AP 1, vlapic_icrlo_write_handler() looks up the destination vCPU with vlapic_calcdest(), which only returns active vCPUs. However, old bhyve executables activate APs (i.e., call vm_activate_cpu()) lazily, only upon receiving a VM_EXITCODE_SPINUP_AP message. Thus, vm_handle_ipi() simply doesn't doesn't do anything since "dmask" is empty, so APs don't boot up. To further complicate things, new vmm.ko allocates vCPUs lazily. New bhyve executables call vm_activate_cpu() for all vCPUs before running the BSP, but as I said above, old bhyve executables do not. So merely fixing "dmask" in vlapic_icrlo_write_handler() to include not-yet-activated vCPUs doesn't work, and we can't allocate a new vCPU in that context. In general it seems that we want an INIT IPI to trigger allocation of a vcpu structure to preserve compatibility with old bhyve, but I don't see a good way to implement that. I would quite like to fix this since I make heavy use of 13.1-RELEASE bhyve+jails on a kernel running main. I believe bhyve from stable/13 is unaffected, but 13.2 is not yet released. Any suggestions would be appreciated. From nobody Wed Feb 8 19:08:31 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PBqKT6RKCz3npM6 for ; Wed, 8 Feb 2023 19:08:45 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PBqKT5dmBz46pW; Wed, 8 Feb 2023 19:08:45 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675883325; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BjsGsTWF0LRyiKZdMANKIK9ch5fSy6aQRSXfKngXPOg=; b=RcZ+w5PLnquG1wjiMpVGcl6UeWcZFLcdk60O9JkDKcagI2FUAgz5Gza6QOk7kEkBHJludZ iUuwKki8PrcIJ3tVXs4NGuCPJTQfX5PPc27ba59E2hkFpktInrX+90BvFxSNAqzxblYQX9 BxEqUdqmWJWx39sLywP9XpRBg8y6ltqyV9iCfUTHbGnYgVhz+5KIfdhtognoPOThwJ7Mg/ +bGpB1wzkeoBQZusl1wpBLl1HToSnX135Fiz1bPsVhptKbYvsWPrmtuk47suBU3DMc3mxp XMJZeXFRYnEC0kANJP3DpU3aiebcI7ehhfDv8i0yi3uYNf8/IfnXUvq1T1OCRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675883325; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BjsGsTWF0LRyiKZdMANKIK9ch5fSy6aQRSXfKngXPOg=; b=tWcgO3TFfbaELup/W9X+Dk/X3u+jRnWJ8n+8dKG3mUl9TWs9KKl0oC+DQ5qHocvuK/0NZf DFII2P4R8ouO47ptxiudGssllZlG2XP5LcRq6CWICs4Fle6/U87cRJ85xfc/S4enMHWRR9 VPw9x66zwNrUNg7dkSAsypBYKsCJtbFHWFVyPeKYsppZDKNrXfuq7C5PB0miSKpN/85Txw wuKQkfOx7UYrl50+YHnY+/MS3wqyzKK0cTlj9CIr+U+4xQJQWNVftcADnpcH4QtEoupuXp ldv2+iNhn6B7H1oNU4dVAJRoxlQ0MpmCq6nbsC80gUaYNGFRFUgr+KmOfu1JmA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1675883325; a=rsa-sha256; cv=none; b=m/YKMD9R5peRTYkG+PmoTwOwmVIHtxy3BMKyjl2U8Kp2Pz3mdPoFIDcmQ2M6pgFRPC/0nk zkmBbNGYpBtdLlCgpGbffmKIVGYyuxjvBHOlAJD1dX8exW5tR80SttFy/AddZGsRw4nMMn 77FU8E7Tk6q9OB8MpPQypQ4P+ZgGTfRsK5eQb8h4TOBsq2VOrJGAfYfwFWp0K+a5XFDsa5 2zQPQ7JGloumHj4y5RTmJFp4o6220e2jaPogwP/GUIrAGnnlUoz1z7IpyEqa/dJX8vFx3g hJAzvJ3CXwlYAc09Mi9/1DZoEkJe18d57bF93jQhtJ1KBH81K53OYUcEQuHS6Q== Received: from [10.0.1.4] (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PBqKT1tnzz132Q; Wed, 8 Feb 2023 19:08:45 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <8aba2bc4-93da-44d7-1d14-8914c4111190@FreeBSD.org> Date: Wed, 8 Feb 2023 11:08:31 -0800 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: bhyve 13.1 compatibility breakage Content-Language: en-US To: Mark Johnston , =?UTF-8?Q?Corvin_K=c3=b6hne?= Cc: freebsd-virtualization@freebsd.org References: <202211230800.2AN80G58068419@gitrepo.freebsd.org> From: John Baldwin In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ThisMailContainsUnwantedMimeParts: N On 2/8/23 10:05 AM, Mark Johnston wrote: > On Sun, Jan 15, 2023 at 10:46:21AM -0500, Mark Johnston wrote: >> On Wed, Nov 23, 2022 at 08:00:16AM +0000, Corvin Köhne wrote: >>> The branch main has been updated by corvink: >>> >>> URL: https://cgit.FreeBSD.org/src/commit/?id=7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b >>> >>> commit 7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b >>> Author: Corvin Köhne >>> AuthorDate: 2022-11-21 14:00:04 +0000 >>> Commit: Corvin Köhne >>> CommitDate: 2022-11-23 08:00:04 +0000 >>> >>> vmm: don't lock a mtx in the icr_low write handler >>> >>> x2apic accesses are handled by a wrmsr exit. This handler is called in a >>> critical section. So, we can't lock a mtx in the icr_low handler. >>> >>> Reported by: kp, pho >>> Tested by: kp, pho >>> Approved by: manu (mentor) >>> Fixes: c0f35dbf19c3c8825bd2b321d8efd582807d1940 vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs. >>> MFC after: 1 week >>> MFC with: c0f35dbf19c3c8825bd2b321d8efd582807d1940 >>> Sponsored by: Beckhoff Automation GmbH & Co. KG >>> Differential Revision: https://reviews.freebsd.org/D37452 >>> --- >> >> Hi Corvin, >> >> This seems to break AP startup when using bhyve/libvmmapi from FreeBSD >> 13.1 on a kernel built from main. It looks like the commit somehow >> regresses commit 769b884e2e2, but I'm not sure yet exactly how. > > I debugged this further and am not quite sure how to fix the problem, > which isn't specific to this commit after all. I'll try to describe it > here. > > Suppose we're booting a VM with 2 vCPUs. When the BSP raises the INIT > IPI to start AP 1, vlapic_icrlo_write_handler() looks up the destination > vCPU with vlapic_calcdest(), which only returns active vCPUs. However, > old bhyve executables activate APs (i.e., call vm_activate_cpu()) > lazily, only upon receiving a VM_EXITCODE_SPINUP_AP message. Thus, > vm_handle_ipi() simply doesn't doesn't do anything since "dmask" is > empty, so APs don't boot up. > > To further complicate things, new vmm.ko allocates vCPUs lazily. New > bhyve executables call vm_activate_cpu() for all vCPUs before running > the BSP, but as I said above, old bhyve executables do not. So merely > fixing "dmask" in vlapic_icrlo_write_handler() to include > not-yet-activated vCPUs doesn't work, and we can't allocate a new vCPU > in that context. In general it seems that we want an INIT IPI to > trigger allocation of a vcpu structure to preserve compatibility with > old bhyve, but I don't see a good way to implement that. > > I would quite like to fix this since I make heavy use of 13.1-RELEASE > bhyve+jails on a kernel running main. I believe bhyve from stable/13 is > unaffected, but 13.2 is not yet released. Any suggestions would be > appreciated. Hmm, I thought I had fixed this by using the bitmask of started CPUs rather than requiring the vCPU to be allocated. I was definitely testing an old bhyve binary from head against the vCPU branch while working on it and remember hitting this exact case, but I thought I had fixed it. Oh, hmm, my fix was the commit quoted above (769b884e2e2). -- John Baldwin From nobody Wed Feb 8 19:24:09 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PBqgK6h6dz3nqxj for ; Wed, 8 Feb 2023 19:24:13 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PBqgK4rV2z4cSy; Wed, 8 Feb 2023 19:24:13 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-io1-xd2e.google.com with SMTP id j17so5474682ioa.9; Wed, 08 Feb 2023 11:24:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=FjznaNAOs1m0R2+C6A+qSCozFue1eFJXbRJmQHma1Pk=; b=cl7XYtZQwoz96V0deQfuDkrvWgbab0mrIV3BKNomJCLHadzQPGe+Kv6GdxqjWePoky c2s3yWezHMGscSUmK4vCS+MdQB0BB+wbKJkSTm3EUhkzV0Vvd5sj/wfWHDGeYaxAbXrR Ov7yS3X9BymkMbBiRiJX8tc7DjdyASdMi4bGk+0kka0UJ6chk5L2ZwcT+PjiRuUWr8Dz eGEOWXGpux6CV9LRyBp5CcE/MN9Fmu8OQek/uuKgA8v6fmTbsMgKPXcJIzeev0glWgaB XTujM8+i2QDRsB6rYnKirDZomPGO0xcDrZk4l1SomawbOSVua2oUG5r2Cfs0VxF5doTt F8TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FjznaNAOs1m0R2+C6A+qSCozFue1eFJXbRJmQHma1Pk=; b=OJeBVu3HgVWG6COYIUsdXya1sZdg6A4GptN10CbMSGtaCBU58Jx1fAcX4mMngcue+o aOCMfuT4XqxKbfjQraZ8MPoqTlMWWj1WMTjuhIiMoANERuqBnHXkx3qAaNiNS1atvYoC /lQKtn5H/8XTE1sCm2k4tJpNgCjc8zEa+FmwCDO6vgMYaaGSPouaaPP3V9ThXVuRcap2 NKNO9YfQKqsGZPUA+CRWU5064uQsBp5mzNuqmkq+4yDAfpd/hgRQQ1sPBDRXfG5Zjz/T 016F8MFiCgFKWGtwrDkMHRDWWERZPOA0pMB2k5lOLCZoMEQwgb6101rAjTU/qkAMn8/t bDgg== X-Gm-Message-State: AO0yUKVjKOIR5xifTBrINegc3xSWUvsCli83sIxQxNY/VKRR0T8aeRHd df2pTOhJNtLOfyD27yRbbbmivT7vhQg= X-Google-Smtp-Source: AK7set9Jl26i0jiJgB4KTuxmFeFcPYD76xkWLIklhxVTFLdqDiel5z4A2SqNsCHferQ6l9rCRmOk6A== X-Received: by 2002:a5e:8914:0:b0:734:e4df:cd04 with SMTP id k20-20020a5e8914000000b00734e4dfcd04mr5360322ioj.12.1675884251973; Wed, 08 Feb 2023 11:24:11 -0800 (PST) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id e4-20020a6b5004000000b006884b050a0asm4793610iob.18.2023.02.08.11.24.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Feb 2023 11:24:11 -0800 (PST) Date: Wed, 8 Feb 2023 14:24:09 -0500 From: Mark Johnston To: John Baldwin Cc: Corvin =?iso-8859-1?Q?K=F6hne?= , freebsd-virtualization@freebsd.org Subject: Re: bhyve 13.1 compatibility breakage Message-ID: References: <202211230800.2AN80G58068419@gitrepo.freebsd.org> <8aba2bc4-93da-44d7-1d14-8914c4111190@FreeBSD.org> List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8aba2bc4-93da-44d7-1d14-8914c4111190@FreeBSD.org> X-Rspamd-Queue-Id: 4PBqgK4rV2z4cSy X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Wed, Feb 08, 2023 at 11:08:31AM -0800, John Baldwin wrote: > On 2/8/23 10:05 AM, Mark Johnston wrote: > > On Sun, Jan 15, 2023 at 10:46:21AM -0500, Mark Johnston wrote: > >> On Wed, Nov 23, 2022 at 08:00:16AM +0000, Corvin Köhne wrote: > >>> The branch main has been updated by corvink: > >>> > >>> URL: https://cgit.FreeBSD.org/src/commit/?id=7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b > >>> > >>> commit 7c326ab5bb9aced8dcbc2465ac1c9ff8df2ba46b > >>> Author: Corvin Köhne > >>> AuthorDate: 2022-11-21 14:00:04 +0000 > >>> Commit: Corvin Köhne > >>> CommitDate: 2022-11-23 08:00:04 +0000 > >>> > >>> vmm: don't lock a mtx in the icr_low write handler > >>> > >>> x2apic accesses are handled by a wrmsr exit. This handler is called in a > >>> critical section. So, we can't lock a mtx in the icr_low handler. > >>> > >>> Reported by: kp, pho > >>> Tested by: kp, pho > >>> Approved by: manu (mentor) > >>> Fixes: c0f35dbf19c3c8825bd2b321d8efd582807d1940 vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs. > >>> MFC after: 1 week > >>> MFC with: c0f35dbf19c3c8825bd2b321d8efd582807d1940 > >>> Sponsored by: Beckhoff Automation GmbH & Co. KG > >>> Differential Revision: https://reviews.freebsd.org/D37452 > >>> --- > >> > >> Hi Corvin, > >> > >> This seems to break AP startup when using bhyve/libvmmapi from FreeBSD > >> 13.1 on a kernel built from main. It looks like the commit somehow > >> regresses commit 769b884e2e2, but I'm not sure yet exactly how. > > > > I debugged this further and am not quite sure how to fix the problem, > > which isn't specific to this commit after all. I'll try to describe it > > here. > > > > Suppose we're booting a VM with 2 vCPUs. When the BSP raises the INIT > > IPI to start AP 1, vlapic_icrlo_write_handler() looks up the destination > > vCPU with vlapic_calcdest(), which only returns active vCPUs. However, > > old bhyve executables activate APs (i.e., call vm_activate_cpu()) > > lazily, only upon receiving a VM_EXITCODE_SPINUP_AP message. Thus, > > vm_handle_ipi() simply doesn't doesn't do anything since "dmask" is > > empty, so APs don't boot up. > > > > To further complicate things, new vmm.ko allocates vCPUs lazily. New > > bhyve executables call vm_activate_cpu() for all vCPUs before running > > the BSP, but as I said above, old bhyve executables do not. So merely > > fixing "dmask" in vlapic_icrlo_write_handler() to include > > not-yet-activated vCPUs doesn't work, and we can't allocate a new vCPU > > in that context. In general it seems that we want an INIT IPI to > > trigger allocation of a vcpu structure to preserve compatibility with > > old bhyve, but I don't see a good way to implement that. > > > > I would quite like to fix this since I make heavy use of 13.1-RELEASE > > bhyve+jails on a kernel running main. I believe bhyve from stable/13 is > > unaffected, but 13.2 is not yet released. Any suggestions would be > > appreciated. > > Hmm, I thought I had fixed this by using the bitmask of started CPUs > rather than requiring the vCPU to be allocated. I was definitely testing > an old bhyve binary from head against the vCPU branch while working on it > and remember hitting this exact case, but I thought I had fixed it. Oh, > hmm, my fix was the commit quoted above (769b884e2e2). After looking further, I think this is actually not so painful to fix. I have a patch which seems to work, but I need to test further. Lazy allocation of the vcpu structure is mostly fine. The only problem is that the INIT handler in vm_handle_ipi() wants to reinitialize the vlapic on all destination vCPUs and this doesn't work if any are not yet allocated. But if they're not yet allocated, then we can rely on a later vm_alloc_vcpu() to initialize vlapic state, so it doesn't really matter. So if I change vlapic_calcdest() to report inactive destination CPUs for INIT and STARTUP IPIs, and I change vm_handle_ipi() to only invoke vlapic_handle_init() on active vCPUs, and I fix a small bug in this commit (CPU_FFS() is 1-indexed not 0-indexed) then I can boot multicore VMs using a 13.1-RELEASE bhyve again.