From nobody Mon Mar 27 06:40:11 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PlNV32vmQz41qWg; Mon, 27 Mar 2023 06:40:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PlNV32N3Hz41F2; Mon, 27 Mar 2023 06:40:11 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1679899211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=mh7fLWG59nXCnmKWHJo5SwnN3i4BLJZm9BzGIpJCnvY=; b=VzvO1BtzyYddbdm9X10wnya0hIn6paEiXpMgNVPGTtkLz9ucZEBCWaEMvgDRcBlPRdejC5 98rYzdVk6jppj/Ya/41LTO5krIO9IuDxhrauq884AZfuYLdPlfW//TpvWuaqcW9htrcpEU O+I41Eb3zNJBIuywcyCrw0X2HG+dr+jijg9+gErjzFlAagItTi/wCesYEW8oIgwDZfr4hg pM4c55CvzQXdbIQZqDVuEa+CquSM5WcTRI/i1WnLSRBvZGvci9XbhyJJ2BSL4JkHF15oFS 3rq3XFRTR2ddava06LYCJIoiscPr6fOYUgxeb6PZ+ij/5VbPLHvgcgcvvQNVZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1679899211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=mh7fLWG59nXCnmKWHJo5SwnN3i4BLJZm9BzGIpJCnvY=; b=AYEu9mME0KuYgYrmvhaZ+cWL+NJpo0q9cc9UISAcJ3Atva6e2RIU+Hy4icSES9oXtEDa74 Nd4J5qY5N5Ey/dMSZ4AEYm3fdcD2TGWXKzoh+WYG0e0FSswWJounUlr5c3KPz15t6EkGpt 8UZG3RlPHnZynXoxv5xY+hKnnON1tA2po8F23gxpdt32gKhF7zXOKYRSUxYlCIfsbPd1Is wtvGpIYEjG6+Vu0qAJx9caH7tuDoOCoxUSP7FdkXr5iK9AW0WTzbPRQ5X/cE/ifl7z2p3n aMUYfSvZoF9WJvabJzRl7vMhoySN1HSd58sEOEpSTtyMNTaP+iLiOKUg9K9L6w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1679899211; a=rsa-sha256; cv=none; b=pdl0Xc3HCtY8UaM84o9azDFV+jU8VI/gezb5XrsBPu0yu8w0mHFQsNWys8nXz7BkxEusqy 5yywVCqD50iZnmaGYfo9BMPKCT+H6dzynHcfBo7CZRhVmmpFShG3Bgk0N+5SdCgRqeZuNo Vl2U7HpLAzjPW+bK+nxLpwZpNT5+2s0VvPFVScmDyAtSGwiH+o61CC4U6y2IUEzliBqJEA bK1jGV5wkWLVBN4QT67lVWrVpa02lZ80reE3aoYCnX03XJCbsJvXOat+RoNffIC8kmFZgk Kzrn1goQs6viHyoHh5ni1lt8rIHarivv7OnKSQDyjZ6mKncs0+hBCsTNsTvwFw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PlNV31QZtz1BGS; Mon, 27 Mar 2023 06:40:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 32R6eBrm086573; Mon, 27 Mar 2023 06:40:11 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 32R6eBYb086570; Mon, 27 Mar 2023 06:40:11 GMT (envelope-from git) Date: Mon, 27 Mar 2023 06:40:11 GMT Message-Id: <202303270640.32R6eBYb086570@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: =?utf-8?Q?Corvin=20K=C3=B6hne?= Subject: git: 6dace6a5cc84 - stable/13 - vmm: fix missing ipi statistic List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: corvink X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 6dace6a5cc84644668a66d58502b779243a666f0 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by corvink: URL: https://cgit.FreeBSD.org/src/commit/?id=6dace6a5cc84644668a66d58502b779243a666f0 commit 6dace6a5cc84644668a66d58502b779243a666f0 Author: Vitaliy Gusev AuthorDate: 2023-03-17 09:17:22 +0000 Commit: Corvin Köhne CommitDate: 2023-03-27 06:33:00 +0000 vmm: fix missing ipi statistic ipi counters are missing in bhyvectl's output because vm_maxcpu is 0 when initializing them. That's because vmm_stat_register is executed before vmm_init. Instead of directly fixing it, there's a better solution in illumos which is cherry picked: https://github.com/illumos/illumos-gate/commit/65a3bc83734e5fb0fc2c19df3e5112b87dcdc3f8 It replaces the matrix statistic by two counters per vcpu. One for counting the ipis to the vcpu and one counting the ipis received by the vcpu. This has several advantages: - A matrix statistic becomes huge when using many vcpus. - A matrix statistic easily reaches the MAX_VMM_STAT_ELEMS limit. - Two counters are enough in most cases. DTrace can be used for more advanced debugging purposes. - A matrix statistic wastes memory. The matrix size is determined by vm_maxcpu regardless of the number of vcpus assigned to the vm. Reviewed by: corvink, markj Fixes: ee98f99d7a68b284a669fefb969cbfc31df2d0ab ("vmm: Convert VM_MAXCPU into a loader tunable hw.vmm.maxcpu.") MFC after: 1 week Sponsored by: vStack Differential Revision: https://reviews.freebsd.org/D39038 (cherry picked from commit 94a3876d7e18ada9596464623829d37d186da856) --- sys/amd64/vmm/io/vlapic.c | 9 ++++++--- sys/amd64/vmm/vmm_stat.c | 3 --- sys/amd64/vmm/vmm_stat.h | 2 -- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/sys/amd64/vmm/io/vlapic.c b/sys/amd64/vmm/io/vlapic.c index 884e232b1422..2144cbabd979 100644 --- a/sys/amd64/vmm/io/vlapic.c +++ b/sys/amd64/vmm/io/vlapic.c @@ -905,7 +905,8 @@ vlapic_calcdest(struct vm *vm, cpuset_t *dmask, uint32_t dest, bool phys, } } -static VMM_STAT_ARRAY(IPIS_SENT, VMM_STAT_NELEMS_VCPU, "ipis sent to vcpu"); +static VMM_STAT(VLAPIC_IPI_SEND, "ipis sent from vcpu"); +static VMM_STAT(VLAPIC_IPI_RECV, "ipis received by vcpu"); static void vlapic_set_tpr(struct vlapic *vlapic, uint8_t val) @@ -1102,7 +1103,8 @@ vlapic_icrlo_write_handler(struct vlapic *vlapic, bool *retu) CPU_FOREACH_ISSET(i, &dmask) { vcpu = vm_vcpu(vlapic->vm, i); lapic_intr_edge(vcpu, vec); - vmm_stat_array_incr(vlapic->vcpu, IPIS_SENT, i, 1); + vmm_stat_incr(vlapic->vcpu, VLAPIC_IPI_SEND, 1); + vmm_stat_incr(vcpu, VLAPIC_IPI_RECV, 1); VLAPIC_CTR2(vlapic, "vlapic sending ipi %d to vcpuid %d", vec, i); } @@ -1223,7 +1225,8 @@ vlapic_self_ipi_handler(struct vlapic *vlapic, uint64_t val) vec = val & 0xff; lapic_intr_edge(vlapic->vcpu, vec); - vmm_stat_array_incr(vlapic->vcpu, IPIS_SENT, vlapic->vcpuid, 1); + vmm_stat_incr(vlapic->vcpu, VLAPIC_IPI_SEND, 1); + vmm_stat_incr(vlapic->vcpu, VLAPIC_IPI_RECV, 1); VLAPIC_CTR1(vlapic, "vlapic self-ipi %d", vec); } diff --git a/sys/amd64/vmm/vmm_stat.c b/sys/amd64/vmm/vmm_stat.c index 2750982185aa..2df4423bc60f 100644 --- a/sys/amd64/vmm/vmm_stat.c +++ b/sys/amd64/vmm/vmm_stat.c @@ -70,9 +70,6 @@ vmm_stat_register(void *arg) if (vst->scope == VMM_STAT_SCOPE_AMD && !vmm_is_svm()) return; - if (vst->nelems == VMM_STAT_NELEMS_VCPU) - vst->nelems = vm_maxcpu; - if (vst_num_elems + vst->nelems >= MAX_VMM_STAT_ELEMS) { printf("Cannot accommodate vmm stat type \"%s\"!\n", vst->desc); return; diff --git a/sys/amd64/vmm/vmm_stat.h b/sys/amd64/vmm/vmm_stat.h index 050d3c13dda1..e40a960ec82a 100644 --- a/sys/amd64/vmm/vmm_stat.h +++ b/sys/amd64/vmm/vmm_stat.h @@ -58,8 +58,6 @@ struct vmm_stat_type { void vmm_stat_register(void *arg); -#define VMM_STAT_NELEMS_VCPU (-1) - #define VMM_STAT_FDEFINE(type, nelems, desc, func, scope) \ struct vmm_stat_type type[1] = { \ { -1, nelems, desc, func, scope } \