Date: Mon, 21 Nov 2022 15:18:37 +0100 From: =?UTF-8?Q?Corvin_K=c3=b6hne?= <corvink@FreeBSD.org> To: Kristof Provost <kp@FreeBSD.org>, John Baldwin <jhb@FreeBSD.org> Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: c0f35dbf19c3 - main - vmm: Use a cpuset_t for vCPUs waiting for STARTUP IPIs. Message-ID: <39b67bfb-6533-74ef-2bdb-782fb162951f@FreeBSD.org> In-Reply-To: <B7E34E6F-924F-412E-88CC-ED4A65936001@FreeBSD.org> References: <202211181826.2AIIQssl030757@gitrepo.freebsd.org> <B7E34E6F-924F-412E-88CC-ED4A65936001@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------AMx8XKtN7ijnKPl15jRb3Rqa
Content-Type: multipart/mixed; boundary="------------HNQIGZa4vhvfzWnws28XTU10";
protected-headers="v1"
From: =?UTF-8?Q?Corvin_K=c3=b6hne?= <corvink@FreeBSD.org>
To: Kristof Provost <kp@FreeBSD.org>, John Baldwin <jhb@FreeBSD.org>
Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org,
dev-commits-src-main@FreeBSD.org
Message-ID: <39b67bfb-6533-74ef-2bdb-782fb162951f@FreeBSD.org>
Subject: Re: git: c0f35dbf19c3 - main - vmm: Use a cpuset_t for vCPUs waiting
for STARTUP IPIs.
References: <202211181826.2AIIQssl030757@gitrepo.freebsd.org>
<B7E34E6F-924F-412E-88CC-ED4A65936001@FreeBSD.org>
In-Reply-To: <B7E34E6F-924F-412E-88CC-ED4A65936001@FreeBSD.org>
--------------HNQIGZa4vhvfzWnws28XTU10
Content-Type: multipart/alternative;
boundary="------------2nNzeS0r8mpJ3AS0wfJoygtT"
--------------2nNzeS0r8mpJ3AS0wfJoygtT
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64
T24gMTEvMjEvMjAyMiAyOjE1IFBNLCBLcmlzdG9mIFByb3Zvc3Qgd3JvdGU6DQo+DQo+IE9u
IDE4IE5vdiAyMDIyLCBhdCAxOToyNiwgSm9obiBCYWxkd2luIHdyb3RlOg0KPg0KPiAgICAg
VGhlIGJyYW5jaCBtYWluIGhhcyBiZWVuIHVwZGF0ZWQgYnkgamhiOg0KPg0KPiAgICAgVVJM
Og0KPiAgICAgaHR0cHM6Ly9jZ2l0LkZyZWVCU0Qub3JnL3NyYy9jb21taXQvP2lkPWMwZjM1
ZGJmMTljM2M4ODI1YmQyYjMyMWQ4ZWZkNTgyODA3ZDE5NDANCj4gICAgIDxodHRwczovL2Nn
aXQuRnJlZUJTRC5vcmcvc3JjL2NvbW1pdC8/aWQ9YzBmMzVkYmYxOWMzYzg4MjViZDJiMzIx
ZDhlZmQ1ODI4MDdkMTk0MD4NCj4NCj4gICAgIGNvbW1pdCBjMGYzNWRiZjE5YzNjODgyNWJk
MmIzMjFkOGVmZDU4MjgwN2QxOTQwDQo+ICAgICBBdXRob3I6IEpvaG4gQmFsZHdpbiA8amhi
QEZyZWVCU0Qub3JnPg0KPiAgICAgQXV0aG9yRGF0ZTogMjAyMi0xMS0xOCAxODowNToxMCAr
MDAwMA0KPiAgICAgQ29tbWl0OiBKb2huIEJhbGR3aW4gPGpoYkBGcmVlQlNELm9yZz4NCj4g
ICAgIENvbW1pdERhdGU6IDIwMjItMTEtMTggMTg6MjU6MzggKzAwMDANCj4NCj4gICAgIHZt
bTogVXNlIGEgY3B1c2V0X3QgZm9yIHZDUFVzIHdhaXRpbmcgZm9yIFNUQVJUVVAgSVBJcy4N
Cj4NCj4gICAgIFJldGlyZSB0aGUgYm9vdF9zdGF0ZSBtZW1iZXIgb2Ygc3RydWN0IHZsYXBp
YyBhbmQgaW5zdGVhZCB1c2UgYQ0KPiAgICAgY3B1c2V0DQo+ICAgICBpbiB0aGUgVk0gdG8g
dHJhY2sgdkNQVXMgd2FpdGluZyBmb3IgU1RBUlRVUCBJUElzLiBJTklUIElQSXMgYWRkDQo+
ICAgICB2Q1BVcyB0byB0aGlzIHNldCwgYW5kIFNUQVJUVVAgSVBJcyByZW1vdmUgdkNQVXMg
ZnJvbSB0aGUgc2V0Lg0KPiAgICAgU1RBUlRVUCBJUElzIGFyZSBvbmx5IHJlcG9ydGVkIHRv
IHVzZXJsYW5kIGZvciB2Q1BVcyB0aGF0IHdlcmUNCj4gICAgIHJlbW92ZWQNCj4gICAgIGZy
b20gdGhlIHNldC4NCj4NCj4gICAgIEluIHBhcnRpY3VsYXIsIHRoaXMgcGVybWl0cyBhIHN1
YnNlcXVlbnQgY2hhbmdlIHRvIGFsbG9jYXRlIHZDUFVzIG9uDQo+ICAgICBkZW1hbmQgd2hl
biB0aGUgdkNQVSBtYXkgbm90IGJlIGFsbG9jYXRlZCB1bnRpbCBhZnRlciBhIFNUQVJUVVAN
Cj4gICAgIElQSSBpcw0KPiAgICAgcmVwb3J0ZWQgdG8gdXNlcmxhbmQuDQo+DQo+ICAgICBS
ZXZpZXdlZCBieTogY29ydmluaywgbWFya2oNCj4gICAgIERpZmZlcmVudGlhbCBSZXZpc2lv
bjogaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0QzNzE3Mw0KPiAgICAgLS0tDQo+ICAg
ICBzeXMvYW1kNjQvaW5jbHVkZS92bW0uaCB8IDMgKysrDQo+ICAgICBzeXMvYW1kNjQvdm1t
L2lvL3ZsYXBpYy5jIHwgNDYNCj4gICAgICsrKysrKysrKystLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLQ0KPiAgICAgc3lzL2FtZDY0L3ZtbS9pby92bGFwaWNfcHJpdi5oIHwg
NyAtLS0tLS0tDQo+ICAgICBzeXMvYW1kNjQvdm1tL3ZtbS5jIHwgMjcgKysrKysrKysrKysr
KysrKysrKysrKysrKw0KPiAgICAgNCBmaWxlcyBjaGFuZ2VkLCA0MSBpbnNlcnRpb25zKCsp
LCA0MiBkZWxldGlvbnMoLSkNCj4NCj4NCj4gSeKAmW0gc2VlaW5nIGEgcGFuaWMgc3RhcnRp
bmcgYSBiaHl2ZSBndWVzdCwgYW5kIEkgdGhpbmsgdGhpcyBjb21taXQgDQo+IG1pZ2h0IGJl
IHRoZSByZXNwb25zaWJsZSBvbmU6DQo+DQo+IHxsb2dpbjogcGFuaWM6IGFjcXVpcmluZyBi
bG9ja2FibGUgc2xlZXAgbG9jayB3aXRoIHNwaW5sb2NrIG9yIA0KPiBjcml0aWNhbCBzZWN0
aW9uIGhlbGQgKHNsZWVwIG11dGV4KSB2bSByZW5kZXp2b3VzIGxvY2sgQCANCj4gL3Vzci9z
cmMvc3lzL2FtZDY0L3ZtbS92bW0uYzoyNTA4IGNwdWlkID0gMTQgdGltZSA9IDE2NjkwMzUy
MTIgS0RCOiANCj4gc3RhY2sgYmFja3RyYWNlOiBkYl90cmFjZV9zZWxmX3dyYXBwZXIoKSBh
dCANCj4gZGJfdHJhY2Vfc2VsZl93cmFwcGVyKzB4MmIvZnJhbWUgMHhmZmZmZmUwMTVmMmRk
NTMwIHZwYW5pYygpIGF0IA0KPiB2cGFuaWMrMHgxNTEvZnJhbWUgMHhmZmZmZmUwMTVmMmRk
NTgwIHBhbmljKCkgYXQgcGFuaWMrMHg0My9mcmFtZSANCj4gMHhmZmZmZmUwMTVmMmRkNWUw
IHdpdG5lc3NfY2hlY2tvcmRlcigpIGF0IA0KPiB3aXRuZXNzX2NoZWNrb3JkZXIrMHhkM2Uv
ZnJhbWUgMHhmZmZmZmUwMTVmMmRkN2EwIF9fbXR4X2xvY2tfZmxhZ3MoKSANCj4gYXQgX19t
dHhfbG9ja19mbGFncysweDk0L2ZyYW1lIDB4ZmZmZmZlMDE1ZjJkZDdmMCB2bV9zdGFydF9j
cHVzKCkgYXQgDQo+IHZtX3N0YXJ0X2NwdXMrMHgzMS9mcmFtZSAweGZmZmZmZTAxNWYyZGQ4
MjAgDQo+IHZsYXBpY19pY3Jsb193cml0ZV9oYW5kbGVyKCkgYXQgdmxhcGljX2ljcmxvX3dy
aXRlX2hhbmRsZXIrMHgzMGMvZnJhbWUgDQo+IDB4ZmZmZmZlMDE1ZjJkZDhhMCB2bXhfcnVu
KCkgYXQgdm14X3J1bisweDIwZWQvZnJhbWUgDQo+IDB4ZmZmZmZlMDE1ZjJkZDljMCB2bV9y
dW4oKSBhdCB2bV9ydW4rMHgxZDIvZnJhbWUgMHhmZmZmZmUwMTVmMmRkYWEwIA0KPiB2bW1k
ZXZfaW9jdGwoKSBhdCB2bW1kZXZfaW9jdGwrMHg2NDQvZnJhbWUgMHhmZmZmZmUwMTVmMmRk
YjQwIA0KPiBkZXZmc19pb2N0bCgpIGF0IGRldmZzX2lvY3RsKzB4Y2QvZnJhbWUgMHhmZmZm
ZmUwMTVmMmRkYjkwIHZuX2lvY3RsKCkgDQo+IGF0IHZuX2lvY3RsKzB4MTMxL2ZyYW1lIDB4
ZmZmZmZlMDE1ZjJkZGNhMCBkZXZmc19pb2N0bF9mKCkgYXQgDQo+IGRldmZzX2lvY3RsX2Yr
MHgxZS9mcmFtZSAweGZmZmZmZTAxNWYyZGRjYzAga2Vybl9pb2N0bCgpIGF0IA0KPiBrZXJu
X2lvY3RsKzB4MjAyL2ZyYW1lIDB4ZmZmZmZlMDE1ZjJkZGQzMCBzeXNfaW9jdGwoKSBhdCAN
Cj4gc3lzX2lvY3RsKzB4MTJhL2ZyYW1lIDB4ZmZmZmZlMDE1ZjJkZGUwMCBhbWQ2NF9zeXNj
YWxsKCkgYXQgDQo+IGFtZDY0X3N5c2NhbGwrMHgxMmUvZnJhbWUgMHhmZmZmZmUwMTVmMmRk
ZjMwIGZhc3Rfc3lzY2FsbF9jb21tb24oKSBhdCANCj4gZmFzdF9zeXNjYWxsX2NvbW1vbisw
eGY4L2ZyYW1lIDB4ZmZmZmZlMDE1ZjJkZGYzMCAtLS0gc3lzY2FsbCAoNTQsIA0KPiBGcmVl
QlNEIEVMRjY0LCBpb2N0bCksIHJpcCA9IDB4M2RkOWNiZDdmOTRhLCByc3AgPSAweDNkZGMx
ZDQ0YWU1OCwgcmJwIA0KPiA9IDB4M2RkYzFkNDRhZjEwIC0tLSB8DQo+DQo+IEFuZCBrZ2Ri
4oCZcyBiYWNrdHJhY2U6DQo+DQo+IHwoa2dkYikgYnQgIzAgX19jdXJ0aHJlYWQgKCkgYXQg
DQo+IC91c3Ivc3JjL3N5cy9hbWQ2NC9pbmNsdWRlL3BjcHVfYXV4Lmg6NTkgIzEgZHVtcF9z
YXZlY3R4ICgpIGF0IA0KPiAvdXNyL3NyYy9zeXMva2Vybi9rZXJuX3NodXRkb3duLmM6NDA1
ICMyIDB4ZmZmZmZmZmY4MGJlYzY3OCBpbiBkdW1wc3lzIA0KPiAoZGk9MHgwKSBhdCAvdXNy
L3NyYy9zeXMveDg2L2luY2x1ZGUvZHVtcC5oOjg3ICMzIGRvYWR1bXAgDQo+ICh0ZXh0ZHVt
cD10ZXh0ZHVtcEBlbnRyeT0wKSBhdCAvdXNyL3NyYy9zeXMva2Vybi9rZXJuX3NodXRkb3du
LmM6NDM0IA0KPiAjNCAweGZmZmZmZmZmODA0YjQwM2EgaW4gZGJfZHVtcCAoZHVtbXk9PG9w
dGltaXplZCBvdXQ+LCANCj4gZHVtbXkyPTx1bmF2YWlsYWJsZT4sIGR1bW15Mz08dW5hdmFp
bGFibGU+LCBkdW1teTQ9PHVuYXZhaWxhYmxlPikgYXQgDQo+IC91c3Ivc3JjL3N5cy9kZGIv
ZGJfY29tbWFuZC5jOjU5MyAjNSAweGZmZmZmZmZmODA0YjNlNDAgaW4gZGJfY29tbWFuZCAN
Cj4gKGxhc3RfY21kcD08b3B0aW1pemVkIG91dD4sIGNtZF90YWJsZT08b3B0aW1pemVkIG91
dD4sIGRvcGFnZXI9dHJ1ZSkgDQo+IGF0IC91c3Ivc3JjL3N5cy9kZGIvZGJfY29tbWFuZC5j
OjUwNiAjNiAweGZmZmZmZmZmODA0YjNiMGQgaW4gDQo+IGRiX2NvbW1hbmRfbG9vcCAoKSBh
dCAvdXNyL3NyYy9zeXMvZGRiL2RiX2NvbW1hbmQuYzo1NTMgIzcgDQo+IDB4ZmZmZmZmZmY4
MDRiNzFhNiBpbiBkYl90cmFwICh0eXBlPTxvcHRpbWl6ZWQgb3V0PiwgY29kZT08b3B0aW1p
emVkIA0KPiBvdXQ+KSBhdCAvdXNyL3NyYy9zeXMvZGRiL2RiX21haW4uYzoyNzAgIzggMHhm
ZmZmZmZmZjgwYzNiODllIGluIA0KPiBrZGJfdHJhcCAodHlwZT10eXBlQGVudHJ5PTMsIGNv
ZGU9PHVuYXZhaWxhYmxlPiwgY29kZUBlbnRyeT0wLCANCj4gdGY9dGZAZW50cnk9MHhmZmZm
ZmUwMTVmMmRkNDcwKSBhdCAvdXNyL3NyYy9zeXMva2Vybi9zdWJyX2tkYi5jOjc0NSAjOSAN
Cj4gMHhmZmZmZmZmZjgxMGNlNTc3IGluIHRyYXAgKGZyYW1lPTB4ZmZmZmZlMDE1ZjJkZDQ3
MCkgYXQgDQo+IC91c3Ivc3JjL3N5cy9hbWQ2NC9hbWQ2NC90cmFwLmM6NjExICMxMCA8c2ln
bmFsIGhhbmRsZXIgY2FsbGVkPiAjMTEgDQo+IGtkYl9lbnRlciAod2h5PTxvcHRpbWl6ZWQg
b3V0PiwgbXNnPTxvcHRpbWl6ZWQgb3V0PikgYXQgDQo+IC91c3Ivc3JjL3N5cy9rZXJuL3N1
YnJfa2RiLmM6NTA5ICMxMiAweGZmZmZmZmZmODBiZWM4MjIgaW4gdnBhbmljIA0KPiAoZm10
PTxvcHRpbWl6ZWQgb3V0PiwgYXA9YXBAZW50cnk9MHhmZmZmZmUwMTVmMmRkNWMwKSBhdCAN
Cj4gL3Vzci9zcmMvc3lzL2tlcm4va2Vybl9zaHV0ZG93bi5jOjk2NyAjMTMgMHhmZmZmZmZm
ZjgwYmVjNWMzIGluIHBhbmljIA0KPiAoZm10PTB4ZmZmZmZmZmY4MWU4Y2U3MCA8Y25wdXRz
X210eD4gIlwzMTQ6KVwyMDFcMzc3XDM3N1wzNzdcMzc3IikgYXQgDQo+IC91c3Ivc3JjL3N5
cy9rZXJuL2tlcm5fc2h1dGRvd24uYzo5MDMgIzE0IDB4ZmZmZmZmZmY4MGM1ZWE0ZSBpbiAN
Cj4gd2l0bmVzc19jaGVja29yZGVyIChsb2NrPTB4ZmZmZmY4MDRkNTg1ZDEzOCwgZmxhZ3M9
PG9wdGltaXplZCBvdXQ+LCANCj4gZmlsZT08b3B0aW1pemVkIG91dD4sIGxpbmU9MjUwOCwg
aW50ZXJsb2NrPTxvcHRpbWl6ZWQgb3V0PikgYXQgDQo+IC91c3Ivc3JjL3N5cy9rZXJuL3N1
YnJfd2l0bmVzcy5jOjEyMDIgIzE1IDB4ZmZmZmZmZmY4MGJjNmQwNCBpbiANCj4gX19tdHhf
bG9ja19mbGFncyAoYz0weGZmZmZmODA0ZDU4NWQxNTAsIG9wdHM9MCwgDQo+IGZpbGU9MHhm
ZmZmZmZmZjgzMjI1NDJkICIvdXNyL3NyYy9zeXMvYW1kNjQvdm1tL3ZtbS5jIiwgbGluZT0y
NTA4KSBhdCANCj4gL3Vzci9zcmMvc3lzL2tlcm4va2Vybl9tdXRleC5jOjI3OCAjMTYgMHhm
ZmZmZmZmZjgzMjA0MTAxIGluIA0KPiB2bV9zdGFydF9jcHVzICh2bT0weGZmZmZmODA0ZDU4
NWQwMDAsIA0KPiB0b3N0YXJ0PXRvc3RhcnRAZW50cnk9MHhmZmZmZmUwMTVmMmRkODU4KSBh
dCANCj4gL3Vzci9zcmMvc3lzL2FtZDY0L3ZtbS92bW0uYzoyNTA4ICMxNyAweGZmZmZmZmZm
ODMyMTFiNWMgaW4gDQo+IHZsYXBpY19pY3Jsb193cml0ZV9oYW5kbGVyICh2bGFwaWM9dmxh
cGljQGVudHJ5PTB4ZmZmZmY4MDQ4ZGMxNGE4MCwgDQo+IHJldHU9cmV0dUBlbnRyeT0weGZm
ZmZmZTAxNWYyZGQ5NTApIGF0IA0KPiAvdXNyL3NyYy9zeXMvYW1kNjQvdm1tL2lvL3ZsYXBp
Yy5jOjExNzIgIzE4IDB4ZmZmZmZmZmY4MzIxOTc3ZCBpbiANCj4gdm14X2hhbmRsZV9hcGlj
X3dyaXRlICh2Y3B1PTB4ZmZmZmY4MDQ4ZGIzYWMwMCwgDQo+IHZsYXBpYz0weGZmZmZmODA0
OGRjMTRhODAsIHF1YWw9NzY4KSBhdCANCj4gL3Vzci9zcmMvc3lzL2FtZDY0L3ZtbS9pbnRl
bC92bXguYzoyMTg0ICMxOSB2bXhfZXhpdF9wcm9jZXNzIA0KPiAodm14PTB4ZmZmZmY4MDRk
NTg1YjAwMCwgdmNwdT0weGZmZmZmODA0OGRiM2FjMDAsIA0KPiB2bWV4aXQ9MHhmZmZmZjgw
NGE3YzFhNjg4KSBhdCAvdXNyL3NyYy9zeXMvYW1kNjQvdm1tL2ludGVsL3ZteC5jOjI3Njcg
DQo+ICMyMCB2bXhfcnVuICh2Y3B1aT0weGZmZmZmODA0OGRiM2FjMDAsIHJpcD08b3B0aW1p
emVkIG91dD4sIA0KPiBwbWFwPTB4ZmZmZmZlMDAzZGNlMTUzMCwgZXZpbmZvPTB4ZmZmZmZl
MDE1ZjJkZDlkOCkgYXQgDQo+IC91c3Ivc3JjL3N5cy9hbWQ2NC92bW0vaW50ZWwvdm14LmM6
MzE3NCAjMjEgMHhmZmZmZmZmZjgzMjAyNTcyIGluIA0KPiB2bV9ydW4gKHZjcHU9dmNwdUBl
bnRyeT0weGZmZmZmODA0YTdjMWE2MDAsIA0KPiB2bWVfdXNlcj12bWVfdXNlckBlbnRyeT0w
eGZmZmZmODAwMzI2MDFiMDgpIGF0IA0KPiAvdXNyL3NyYy9zeXMvYW1kNjQvdm1tL3ZtbS5j
OjE4NzMgIzIyIDB4ZmZmZmZmZmY4MzIwNWFiNCBpbiANCj4gdm1tZGV2X2lvY3RsIChjZGV2
PTxvcHRpbWl6ZWQgb3V0PiwgY21kPTMyMzA2OTI4NjUsIGRhdGE9PG9wdGltaXplZCANCj4g
b3V0PiwgZmZsYWc9PG9wdGltaXplZCBvdXQ+LCB0ZD08b3B0aW1pemVkIG91dD4pIGF0IA0K
PiAvdXNyL3NyYy9zeXMvYW1kNjQvdm1tL3ZtbV9kZXYuYzo1NjUgIzIzIDB4ZmZmZmZmZmY4
MGE3YmU3ZCBpbiANCj4gZGV2ZnNfaW9jdGwgKGFwPTB4ZmZmZmZlMDE1ZjJkZGJhOCkgYXQg
DQo+IC91c3Ivc3JjL3N5cy9mcy9kZXZmcy9kZXZmc192bm9wcy5jOjkzMyAjMjQgMHhmZmZm
ZmZmZjgwY2Y2NDIxIGluIA0KPiB2bl9pb2N0bCAoZnA9MHhmZmZmZjgwMDBjZTJjYjQwLCBj
b209PG9wdGltaXplZCBvdXQ+LCANCj4gZGF0YT0weGZmZmZmODAwMzI2MDFiMDAsIGFjdGl2
ZV9jcmVkPTB4ZmZmZmY4MDAwYzRkMDgwMCwgdGQ9MHgxMCkgYXQgDQo+IC91c3Ivc3JjL3N5
cy9rZXJuL3Zmc192bm9wcy5jOjE2OTkgIzI1IDB4ZmZmZmZmZmY4MGE3YzUyZSBpbiANCj4g
ZGV2ZnNfaW9jdGxfZiAoZnA9MHhmZmZmZmZmZjgxZThjZTcwIDxjbnB1dHNfbXR4PiwgY29t
PTAsIA0KPiBkYXRhPTB4ZmZmZmZmZmY4MTJhMDAwMCwgY3JlZD0weDEsIHRkPTB4MTApIGF0
IA0KPiAvdXNyL3NyYy9zeXMvZnMvZGV2ZnMvZGV2ZnNfdm5vcHMuYzo4NjQgIzI2IDB4ZmZm
ZmZmZmY4MGM2NDY3MiBpbiANCj4gZm9faW9jdGwgKGZwPTB4ZmZmZmY4MDAwY2UyY2I0MCwg
Y29tPTMyMzA2OTI4NjUsIGRhdGE9MHgxZDAsIA0KPiBhY3RpdmVfY3JlZD0weDEsIHRkPTxv
cHRpbWl6ZWQgb3V0PikgYXQgL3Vzci9zcmMvc3lzL3N5cy9maWxlLmg6MzY1IA0KPiAjMjcg
a2Vybl9pb2N0bCAodGQ9dGRAZW50cnk9MHhmZmZmZmUwMTYwOTM2MDAwLCBmZD08b3B0aW1p
emVkIG91dD4sIA0KPiBjb209Y29tQGVudHJ5PTMyMzA2OTI4NjUsIGRhdGE9MHgxZDAgPGVy
cm9yOiBDYW5ub3QgYWNjZXNzIG1lbW9yeSBhdCANCj4gYWRkcmVzcyAweDFkMD4sIGRhdGFA
ZW50cnk9MHhmZmZmZjgwMDMyNjAxYjAwICIiKSBhdCANCj4gL3Vzci9zcmMvc3lzL2tlcm4v
c3lzX2dlbmVyaWMuYzo4MDMgIzI4IDB4ZmZmZmZmZmY4MGM2NDNiYSBpbiANCj4gc3lzX2lv
Y3RsICh0ZD0weGZmZmZmZTAxNjA5MzYwMDAsIHVhcD0weGZmZmZmZTAxNjA5MzYzZjgpIGF0
IA0KPiAvdXNyL3NyYy9zeXMva2Vybi9zeXNfZ2VuZXJpYy5jOjcxMSAjMjkgMHhmZmZmZmZm
ZjgxMGNmM2JlIGluIA0KPiBzeXNjYWxsZW50ZXIgKHRkPTxvcHRpbWl6ZWQgb3V0PikgYXQg
DQo+IC91c3Ivc3JjL3N5cy9hbWQ2NC9hbWQ2NC8uLi8uLi9rZXJuL3N1YnJfc3lzY2FsbC5j
OjE4OSAjMzAgDQo+IGFtZDY0X3N5c2NhbGwgKHRkPTB4ZmZmZmZlMDE2MDkzNjAwMCwgdHJh
Y2VkPTApIGF0IA0KPiAvdXNyL3NyYy9zeXMvYW1kNjQvYW1kNjQvdHJhcC5jOjEyMDAgIzMx
IDxzaWduYWwgaGFuZGxlciBjYWxsZWQ+ICMzMiANCj4gMHgwMDAwM2RkOWNiZDdmOTRhIGlu
ID8/ICgpIEJhY2t0cmFjZSBzdG9wcGVkOiBDYW5ub3QgYWNjZXNzIG1lbW9yeSBhdCANCj4g
YWRkcmVzcyAweDNkZGMxZDQ0YWU1OCAoa2dkYikgZnIgMTYgIzE2IDB4ZmZmZmZmZmY4MzIw
NDEwMSBpbiANCj4gdm1fc3RhcnRfY3B1cyAodm09MHhmZmZmZjgwNGQ1ODVkMDAwLCANCj4g
dG9zdGFydD10b3N0YXJ0QGVudHJ5PTB4ZmZmZmZlMDE1ZjJkZDg1OCkgYXQgDQo+IC91c3Iv
c3JjL3N5cy9hbWQ2NC92bW0vdm1tLmM6MjUwOCAyNTA4IG10eF9sb2NrKCZ2bS0+cmVuZGV6
dm91c19tdHgpOyB8DQo+DQo+IEkgYmVsaWV2ZSBXSVRORVNTIGlzIHVwc2V0IHRoYXQgd2Xi
gJlyZSBnb2luZyB0byBwb3RlbnRpYWxseSBzbGVlcCBkb2luZyANCj4gbXR4X2xvY2soJnZt
LT5yZW5kZXp2b3VzX210eCk7IGluIHZtX3N0YXJ0X2NwdXMoKSB3aGVuIHdl4oCZdmUgZG9u
ZSANCj4gY3JpdGljYWxfZW50ZXIoKSBpbiB2bV9ydW4oKS4NCj4NCj4gQmVzdCByZWdhcmRz
LA0KPiBLcmlzdG9mDQo+DQoNCkNhbiB5b3UgcGxlYXNlIHRlc3QgaHR0cHM6Ly9yZXZpZXdz
LmZyZWVic2Qub3JnL0QzNzQ1Mj8NCg0KDQotLSANCkJlc3QgcmVnYXJkcywNCkNvcnZpbg0K
DQo=
--------------2nNzeS0r8mpJ3AS0wfJoygtT
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DUTF=
-8">
</head>
<body>
<div class=3D"moz-cite-prefix">On 11/21/2022 2:15 PM, Kristof Provost=
wrote:<br>
</div>
<blockquote type=3D"cite"
cite=3D"mid:B7E34E6F-924F-412E-88CC-ED4A65936001@FreeBSD.org">
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DU=
TF-8">
<div style=3D"font-family: sans-serif;">
<div class=3D"markdown" style=3D"white-space: normal;">
<p dir=3D"auto">On 18 Nov 2022, at 19:26, John Baldwin wrote:</=
p>
</div>
<div class=3D"plaintext" style=3D"white-space: normal;">
<blockquote style=3D"margin: 0 0 5px; padding-left: 5px;
border-left: 2px solid #136BCE; color: #136BCE;">
<p dir=3D"auto">The branch main has been updated by jhb:</p>
<p dir=3D"auto">URL: <a
href=3D"https://cgit.FreeBSD.org/src/commit/?id=3Dc0f35dbf19c3c8825bd2b32=
1d8efd582807d1940"
moz-do-not-send=3D"true">https://cgit.FreeBSD.org/src/com=
mit/?id=3Dc0f35dbf19c3c8825bd2b321d8efd582807d1940</a></p>
<p dir=3D"auto">commit
c0f35dbf19c3c8825bd2b321d8efd582807d1940
<br>
Author: John Baldwin <a class=3D"moz-txt-link-rfc2396E" hre=
f=3D"mailto:jhb@FreeBSD.org"><jhb@FreeBSD.org></a>
<br>
AuthorDate: 2022-11-18 18:05:10 +0000
<br>
Commit: John Baldwin <a class=3D"moz-txt-link-rfc2396E" hre=
f=3D"mailto:jhb@FreeBSD.org"><jhb@FreeBSD.org></a>
<br>
CommitDate: 2022-11-18 18:25:38 +0000</p>
<p dir=3D"auto"> vmm: Use a cpuset_t for vCPUs waiting for
STARTUP IPIs.</p>
<p dir=3D"auto"> Retire the boot_state member of struct vlapi=
c
and instead use a cpuset
<br>
in the VM to track vCPUs waiting for STARTUP IPIs. INIT
IPIs add
<br>
vCPUs to this set, and STARTUP IPIs remove vCPUs from the
set.
<br>
STARTUP IPIs are only reported to userland for vCPUs that
were removed
<br>
from the set.</p>
<p dir=3D"auto"> In particular, this permits a subsequent
change to allocate vCPUs on
<br>
demand when the vCPU may not be allocated until after a
STARTUP IPI is
<br>
reported to userland.</p>
<p dir=3D"auto"> Reviewed by: corvink, markj
<br>
Differential Revision: <a
href=3D"https://reviews.freebsd.org/D37173"
moz-do-not-send=3D"true" class=3D"moz-txt-link-freetext">=
https://reviews.freebsd.org/D37173</a>;
<br>
---
<br>
sys/amd64/include/vmm.h | 3 +++
<br>
sys/amd64/vmm/io/vlapic.c | 46
++++++++++--------------------------------
<br>
sys/amd64/vmm/io/vlapic_priv.h | 7 -------
<br>
sys/amd64/vmm/vmm.c | 27 +++++++++++++++++++++++++
<br>
4 files changed, 41 insertions(+), 42 deletions(-)</p>
<br>
</blockquote>
</div>
<div class=3D"markdown" style=3D"white-space: normal;">
<p dir=3D"auto">I=E2=80=99m seeing a panic starting a bhyve gue=
st, and I
think this commit might be the responsible one:</p>
<pre style=3D"margin-left: 15px; margin-right: 15px; padding: 5=
px; border: thin solid gray; overflow-x: auto; max-width: 90vw; backgroun=
d-color: #E4E4E4;"><code style=3D"padding: 0 0.25em; background-color: #E=
4E4E4;">login: panic: acquiring blockable sleep lock with spinlock or cri=
tical section held (sleep mutex) vm rendezvous lock @ /usr/src/sys/amd64/=
vmm/vmm.c:2508
cpuid =3D 14
time =3D 1669035212
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe015f2=
dd530
vpanic() at vpanic+0x151/frame 0xfffffe015f2dd580
panic() at panic+0x43/frame 0xfffffe015f2dd5e0
witness_checkorder() at witness_checkorder+0xd3e/frame 0xfffffe015f2dd7a0=
__mtx_lock_flags() at __mtx_lock_flags+0x94/frame 0xfffffe015f2dd7f0
vm_start_cpus() at vm_start_cpus+0x31/frame 0xfffffe015f2dd820
vlapic_icrlo_write_handler() at vlapic_icrlo_write_handler+0x30c/frame 0x=
fffffe015f2dd8a0
vmx_run() at vmx_run+0x20ed/frame 0xfffffe015f2dd9c0
vm_run() at vm_run+0x1d2/frame 0xfffffe015f2ddaa0
vmmdev_ioctl() at vmmdev_ioctl+0x644/frame 0xfffffe015f2ddb40
devfs_ioctl() at devfs_ioctl+0xcd/frame 0xfffffe015f2ddb90
vn_ioctl() at vn_ioctl+0x131/frame 0xfffffe015f2ddca0
devfs_ioctl_f() at devfs_ioctl_f+0x1e/frame 0xfffffe015f2ddcc0
kern_ioctl() at kern_ioctl+0x202/frame 0xfffffe015f2ddd30
sys_ioctl() at sys_ioctl+0x12a/frame 0xfffffe015f2dde00
amd64_syscall() at amd64_syscall+0x12e/frame 0xfffffe015f2ddf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe015f2ddf3=
0
--- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x3dd9cbd7f94a, rsp =3D 0=
x3ddc1d44ae58, rbp =3D 0x3ddc1d44af10 ---
</code></pre>
<p dir=3D"auto">And kgdb=E2=80=99s backtrace:</p>
<pre style=3D"margin-left: 15px; margin-right: 15px; padding: 5=
px; border: thin solid gray; overflow-x: auto; max-width: 90vw; backgroun=
d-color: #E4E4E4;"><code style=3D"padding: 0 0.25em; background-color: #E=
4E4E4;">(kgdb) bt
#0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:59
#1 dump_savectx () at /usr/src/sys/kern/kern_shutdown.c:405
#2 0xffffffff80bec678 in dumpsys (di=3D0x0) at /usr/src/sys/x86/include/=
dump.h:87
#3 doadump (textdump=3Dtextdump@entry=3D0) at /usr/src/sys/kern/kern_shu=
tdown.c:434
#4 0xffffffff804b403a in db_dump (dummy=3D<optimized out>, dummy2=3D=
<unavailable>, dummy3=3D<unavailable>, dummy4=3D<unavailab=
le>) at /usr/src/sys/ddb/db_command.c:593
#5 0xffffffff804b3e40 in db_command (last_cmdp=3D<optimized out>, =
cmd_table=3D<optimized out>, dopager=3Dtrue) at /usr/src/sys/ddb/db=
_command.c:506
#6 0xffffffff804b3b0d in db_command_loop () at /usr/src/sys/ddb/db_comma=
nd.c:553
#7 0xffffffff804b71a6 in db_trap (type=3D<optimized out>, code=3D&=
lt;optimized out>) at /usr/src/sys/ddb/db_main.c:270
#8 0xffffffff80c3b89e in kdb_trap (type=3Dtype@entry=3D3, code=3D<una=
vailable>, code@entry=3D0, tf=3Dtf@entry=3D0xfffffe015f2dd470) at /usr=
/src/sys/kern/subr_kdb.c:745
#9 0xffffffff810ce577 in trap (frame=3D0xfffffe015f2dd470) at /usr/src/s=
ys/amd64/amd64/trap.c:611
#10 <signal handler called>
#11 kdb_enter (why=3D<optimized out>, msg=3D<optimized out>) =
at /usr/src/sys/kern/subr_kdb.c:509
#12 0xffffffff80bec822 in vpanic (fmt=3D<optimized out>, ap=3Dap@en=
try=3D0xfffffe015f2dd5c0) at /usr/src/sys/kern/kern_shutdown.c:967
#13 0xffffffff80bec5c3 in panic (fmt=3D0xffffffff81e8ce70 <cnputs_mtx&=
gt; "\314:)\201\377\377\377\377") at /usr/src/sys/kern/kern_shutdown.c:90=
3
#14 0xffffffff80c5ea4e in witness_checkorder (lock=3D0xfffff804d585d138, =
flags=3D<optimized out>, file=3D<optimized out>, line=3D2508,=
interlock=3D<optimized out>)
at /usr/src/sys/kern/subr_witness.c:1202
#15 0xffffffff80bc6d04 in __mtx_lock_flags (c=3D0xfffff804d585d150, opts=3D=
0, file=3D0xffffffff8322542d "/usr/src/sys/amd64/vmm/vmm.c", line=3D2508)=
at /usr/src/sys/kern/kern_mutex.c:278
#16 0xffffffff83204101 in vm_start_cpus (vm=3D0xfffff804d585d000, tostart=
=3Dtostart@entry=3D0xfffffe015f2dd858) at /usr/src/sys/amd64/vmm/vmm.c:25=
08
#17 0xffffffff83211b5c in vlapic_icrlo_write_handler (vlapic=3Dvlapic@ent=
ry=3D0xfffff8048dc14a80, retu=3Dretu@entry=3D0xfffffe015f2dd950) at /usr/=
src/sys/amd64/vmm/io/vlapic.c:1172
#18 0xffffffff8321977d in vmx_handle_apic_write (vcpu=3D0xfffff8048db3ac0=
0, vlapic=3D0xfffff8048dc14a80, qual=3D768) at /usr/src/sys/amd64/vmm/int=
el/vmx.c:2184
#19 vmx_exit_process (vmx=3D0xfffff804d585b000, vcpu=3D0xfffff8048db3ac00=
, vmexit=3D0xfffff804a7c1a688) at /usr/src/sys/amd64/vmm/intel/vmx.c:2767=
#20 vmx_run (vcpui=3D0xfffff8048db3ac00, rip=3D<optimized out>, pma=
p=3D0xfffffe003dce1530, evinfo=3D0xfffffe015f2dd9d8) at /usr/src/sys/amd6=
4/vmm/intel/vmx.c:3174
#21 0xffffffff83202572 in vm_run (vcpu=3Dvcpu@entry=3D0xfffff804a7c1a600,=
vme_user=3Dvme_user@entry=3D0xfffff80032601b08) at /usr/src/sys/amd64/vm=
m/vmm.c:1873
#22 0xffffffff83205ab4 in vmmdev_ioctl (cdev=3D<optimized out>, cmd=
=3D3230692865, data=3D<optimized out>, fflag=3D<optimized out>=
;, td=3D<optimized out>) at /usr/src/sys/amd64/vmm/vmm_dev.c:565
#23 0xffffffff80a7be7d in devfs_ioctl (ap=3D0xfffffe015f2ddba8) at /usr/s=
rc/sys/fs/devfs/devfs_vnops.c:933
#24 0xffffffff80cf6421 in vn_ioctl (fp=3D0xfffff8000ce2cb40, com=3D<op=
timized out>, data=3D0xfffff80032601b00, active_cred=3D0xfffff8000c4d0=
800, td=3D0x10) at /usr/src/sys/kern/vfs_vnops.c:1699
#25 0xffffffff80a7c52e in devfs_ioctl_f (fp=3D0xffffffff81e8ce70 <cnpu=
ts_mtx>, com=3D0, data=3D0xffffffff812a0000, cred=3D0x1, td=3D0x10) at=
/usr/src/sys/fs/devfs/devfs_vnops.c:864
#26 0xffffffff80c64672 in fo_ioctl (fp=3D0xfffff8000ce2cb40, com=3D323069=
2865, data=3D0x1d0, active_cred=3D0x1, td=3D<optimized out>) at /us=
r/src/sys/sys/file.h:365
#27 kern_ioctl (td=3Dtd@entry=3D0xfffffe0160936000, fd=3D<optimized ou=
t>, com=3Dcom@entry=3D3230692865, data=3D0x1d0 <error: Cannot acces=
s memory at address 0x1d0>, data@entry=3D0xfffff80032601b00 "")
at /usr/src/sys/kern/sys_generic.c:803
#28 0xffffffff80c643ba in sys_ioctl (td=3D0xfffffe0160936000, uap=3D0xfff=
ffe01609363f8) at /usr/src/sys/kern/sys_generic.c:711
#29 0xffffffff810cf3be in syscallenter (td=3D<optimized out>) at /u=
sr/src/sys/amd64/amd64/../../kern/subr_syscall.c:189
#30 amd64_syscall (td=3D0xfffffe0160936000, traced=3D0) at /usr/src/sys/a=
md64/amd64/trap.c:1200
#31 <signal handler called>
#32 0x00003dd9cbd7f94a in ?? ()
Backtrace stopped: Cannot access memory at address 0x3ddc1d44ae58
(kgdb) fr 16
#16 0xffffffff83204101 in vm_start_cpus (vm=3D0xfffff804d585d000, tostart=
=3Dtostart@entry=3D0xfffffe015f2dd858) at /usr/src/sys/amd64/vmm/vmm.c:25=
08
2508 mtx_lock(&vm->rendezvous_mtx);
</code></pre>
<p dir=3D"auto">I believe WITNESS is upset that we=E2=80=99re g=
oing to
potentially sleep doing
mtx_lock(&vm->rendezvous_mtx); in vm_start_cpus()
when we=E2=80=99ve done critical_enter() in vm_run().</p>
<p dir=3D"auto">Best regards,<br>
Kristof</p>
</div>
</div>
</blockquote>
<p><br>
</p>
<p>Can you please test <a class=3D"moz-txt-link-freetext" href=3D"htt=
ps://reviews.freebsd.org/D37452">https://reviews.freebsd.org/D37452</a>?<=
br>
</p>
<p><br>
</p>
<pre class=3D"moz-signature" cols=3D"72">--=20
Best regards,
Corvin</pre>
</body>
</html>
--------------2nNzeS0r8mpJ3AS0wfJoygtT--
--------------HNQIGZa4vhvfzWnws28XTU10--
--------------AMx8XKtN7ijnKPl15jRb3Rqa
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"
-----BEGIN PGP SIGNATURE-----
wsF5BAABCAAjFiEEgvRSla3m2t/H2U9G2FTaVjFeAmoFAmN7iL0FAwAAAAAACgkQ2FTaVjFeAmp9
qRAAwtA9RPCim2H0RC1EseE205i8Dnipl53OZwY73hifamTJsxVOKerubqOwm+7zU5CCwXxRFixR
E3MNOG3kME6QGDuNtncPwa2paFXyGIJ3sPgh7mhjAfQ2Td4lqf4VxdwCp5GRFSRvRUfb4eIM6+Iy
2Xl//MHeFxFiACtaak1m/5BpUlPxeeQD5Ek69U1Kmq1Sa5hT0U7GLhTconl/7spNi0yZIrsBr6aO
djPo9KeQHtVpmI26Y9j9K7hL3+RulufMddkRpbs0c27E+5M3CkfPYBlZD1mobPeeeoLgxJ4HjWuv
J4WSsNOstVN65QIxXCPGJ9zhGmPZQONuJmvE3bSMxbfCbWuuNbWzTb3sOnlQ2v9JpQGJ6Sw0blR2
0Jj6/MOgpXz5PZ3MdYvR4L8B3wCRTlNU7zC7Mce/L6qkjB/hCIz34CVfAO5zvjDguy02pDJN/Dms
XzkQe5gTdo2YAM1ivUSL+M7RsWiZV347J6Hu0Sim8kizGqoCIYwVn/trNiE2nfR5F3Q5eL3+LFsn
3jGI60heHYN9NhNtMY+FOm92tcK3jQYGj0jlO+fCyFsTg8zdcEHf79skGTOeSmSmy6WZQ+astsOW
wDlWSwNko6IztT+3Hzdt8hGphXfrNX3zpXWqCR6UDA5nqd8FmOQngWnT6LGJL20Jnom1taGu+cQU
zKc=
=EtqY
-----END PGP SIGNATURE-----
--------------AMx8XKtN7ijnKPl15jRb3Rqa--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?39b67bfb-6533-74ef-2bdb-782fb162951f>