From nobody Fri Jul 21 08:59:51 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R6k5h16TSz4dQlP; Fri, 21 Jul 2023 08:59:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R6k5h0cF8z4KJ4; Fri, 21 Jul 2023 08:59:52 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689929992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vKuYbjn8G4S++wn71jPuaQJh/WnobrGGEdVobPH2X/U=; b=fIt8GV3H8ScCpBHSG7pRdMIZwmjvjxZQGMx1gAMO4OW4Sk+cdZkVX6UamC9soClWdnf4rO VUx4/hzLkm6YjsuaVSl7dmIVs1noTrmlEOPczhswKmlcvMc4Ds4z8isEeJ3+tkzp1b5FlY qEdjrqxk7YoUcRIRBP3ND9wAz7+tS6Ql341pkthHTjjHTNqDsrPscLHqkXXqVrYpNkwrh+ Lu7GyU8ZXxBKZT1i2IXFBGG7tBC/SFF367HH28E+bAczQfytBPBAjX9Aodz6Vfmpkglou0 4YywLawyeih1plYeLgiLskEYM+kEL+R+T+dhycS6rs7PjSviNN0UTc06hkfWUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689929992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vKuYbjn8G4S++wn71jPuaQJh/WnobrGGEdVobPH2X/U=; b=aj81+N2xHKt2XUG7Ng+ie/IFNGxFx2utdrEU9nJrvVOM4hMmEkPXCalaadKbMm3oCK36hJ EwU0AtHNyEk1MbbD00pGYGn5DZ0+MJyavbaP5O+q++stySujeuuu9qOk0Me3uTQQnbReB8 sUkFVF6bR0250xe7b+JvddQmThvrNP30oHYeqYVyJkvYCYkJkHm1HMB7AP3a/Nn5gHKONy A46wdnvpvQwpmGv7D7XSffenvrON/On8VbeXxaDiYFkz8Yf87iC46fh8GIm1Dcy2m9iEia I2WJDXJ1VSMSKmMd3ku5a7DTuUKPLUwtg5cKlt5QVPFSoMm2yimQoiDgtth0iw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1689929992; a=rsa-sha256; cv=none; b=YJCJTYPmmHdghC6YrLo/bRDGpn6E7VSLetwChxnX0ok265pVQo0w4BxWnPcZzO29QbYEnZ Xd/Sei4w+z/ScNByhb2D2iNXel7a5xzIJtaBfjCkLUdfv8waKxQNX3moVoc3bUYU3JwTK7 Y6P55eMEV26bOc8E0QYKZJCHtEvcYG+rEB6w0yAl8+/wzlKyA75PYWK4rjnaITW3NkycXH OGBqQP00mpfYs7MRfDDo2fs0O4Gg6+2ddDy5U5Ci5vFQHorPB73HnQLVJjJ56+YkaWGnE+ OkgjAEY6hiap5aY+/sjlbYGMo2t+d0HSXapG/aUlYSUsbUXHr3ubFroqekwrTA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4R6k5g6mT0z161X; Fri, 21 Jul 2023 08:59:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 36L8xpM6046379; Fri, 21 Jul 2023 08:59:51 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 36L8xp7v046378; Fri, 21 Jul 2023 08:59:51 GMT (envelope-from git) Date: Fri, 21 Jul 2023 08:59:51 GMT Message-Id: <202307210859.36L8xp7v046378@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Roger=20Pau=20Monn=C3=A9?= Subject: git: 20fc5bf7df1d - main - xen: move vcpu_info to common, leave hook for setup List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: royger X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 20fc5bf7df1db698f2651eaa04a3bc71290e1636 Auto-Submitted: auto-generated The branch main has been updated by royger: URL: https://cgit.FreeBSD.org/src/commit/?id=20fc5bf7df1db698f2651eaa04a3bc71290e1636 commit 20fc5bf7df1db698f2651eaa04a3bc71290e1636 Author: Elliott Mitchell AuthorDate: 2023-06-08 21:13:17 +0000 Commit: Roger Pau Monné CommitDate: 2023-07-21 08:59:12 +0000 xen: move vcpu_info to common, leave hook for setup vcpu_info is crucial for the Xen event channel core. Since both the data and setup steps are identical between architectures, move them to the common file. Since there is no cross-architecture method to call a function on every processor during bring-up, simply leave the setup function. The number of vcpu_info structures available on the shared information page varies by architecture. Instead of hard-coding the count use nitems(). Add a warning message for this being used. Switch to XEN_VCPUID() and use Xen's typedefs. panic() on failure since >32 processors is no longer unusual. royger: Specify 64-byte alignment for vcpu_info to try to defend against vcpu_info crossing a page boundary. Add detection for this limit. Reviewed by: royger --- sys/x86/xen/hvm.c | 25 +++---------------------- sys/xen/xen-os.h | 5 +++++ sys/xen/xen_common.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 61 insertions(+), 22 deletions(-) diff --git a/sys/x86/xen/hvm.c b/sys/x86/xen/hvm.c index d59c3901d2d7..38c93bf25998 100644 --- a/sys/x86/xen/hvm.c +++ b/sys/x86/xen/hvm.c @@ -88,8 +88,7 @@ int xen_vector_callback_enabled; bool xen_evtchn_needs_ack; /*------------------------------- Per-CPU Data -------------------------------*/ -DPCPU_DEFINE(struct vcpu_info, vcpu_local_info); -DPCPU_DEFINE(struct vcpu_info *, vcpu_info); +DPCPU_DECLARE(struct vcpu_info *, vcpu_info); /*------------------------------ Sysctl tunables -----------------------------*/ int xen_disable_pv_disks = 0; @@ -416,10 +415,8 @@ SYSINIT(xen_hvm_init, SI_SUB_HYPERVISOR, SI_ORDER_FIRST, xen_hvm_sysinit, NULL); static void xen_hvm_cpu_init(void) { - struct vcpu_register_vcpu_info info; - struct vcpu_info *vcpu_info; uint32_t regs[4]; - int cpu, rc; + int rc; if (!xen_domain()) return; @@ -459,23 +456,7 @@ xen_hvm_cpu_init(void) rc); } - /* - * Set the vCPU info. - * - * NB: the vCPU info for vCPUs < 32 can be fetched from the shared info - * page, but in order to make sure the mapping code is correct always - * attempt to map the vCPU info at a custom place. - */ - vcpu_info = DPCPU_PTR(vcpu_local_info); - cpu = PCPU_GET(vcpu_id); - info.mfn = vtophys(vcpu_info) >> PAGE_SHIFT; - info.offset = vtophys(vcpu_info) - trunc_page(vtophys(vcpu_info)); - - rc = HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info); - if (rc != 0) - DPCPU_SET(vcpu_info, &HYPERVISOR_shared_info->vcpu_info[cpu]); - else - DPCPU_SET(vcpu_info, vcpu_info); + xen_setup_vcpu_info(); } SYSINIT(xen_hvm_cpu_init, SI_SUB_INTR, SI_ORDER_FIRST, xen_hvm_cpu_init, NULL); diff --git a/sys/xen/xen-os.h b/sys/xen/xen-os.h index e1d03b7322bc..4919a9fc0702 100644 --- a/sys/xen/xen-os.h +++ b/sys/xen/xen-os.h @@ -44,6 +44,11 @@ #include #include +/* + * Setup function which needs to be called on each processor by architecture + */ +extern void xen_setup_vcpu_info(void); + static inline vm_paddr_t xen_get_xenstore_mfn(void) { diff --git a/sys/xen/xen_common.c b/sys/xen/xen_common.c index d0f0ecdeed59..95fe00b2ba98 100644 --- a/sys/xen/xen_common.c +++ b/sys/xen/xen_common.c @@ -33,11 +33,16 @@ __FBSDID("$FreeBSD$"); #include /* required by xen/xen-os.h */ +#include +#include + #include /* required by xen/xen-os.h */ #include #include +#include + /*-------------------------------- Global Data -------------------------------*/ enum xen_domain_type xen_domain_type = XEN_NATIVE; @@ -49,3 +54,51 @@ uint32_t hvm_start_flags; /*------------------ Hypervisor Access Shared Memory Regions -----------------*/ shared_info_t *HYPERVISOR_shared_info; + +/*------------------------------- Per-CPU Data -------------------------------*/ +DPCPU_DEFINE(struct vcpu_info *, vcpu_info); + +void +xen_setup_vcpu_info(void) +{ + /* This isn't directly accessed outside this function */ + DPCPU_DEFINE_STATIC(vcpu_info_t, vcpu_local_info) + __attribute__((aligned(64))); + + vcpu_info_t *vcpu_info = DPCPU_PTR(vcpu_local_info); + vcpu_register_vcpu_info_t info = { + .mfn = vtophys(vcpu_info) >> PAGE_SHIFT, + .offset = vtophys(vcpu_info) & PAGE_MASK, + }; + unsigned int cpu = XEN_VCPUID(); + int rc; + + KASSERT(xen_domain(), ("%s(): invoked when not on Xen?", __func__)); + + _Static_assert(sizeof(struct vcpu_info) <= 64, + "struct vcpu_info is larger than supported limit of 64 bytes"); + + /* + * Set the vCPU info. + * + * NB: the vCPU info for some vCPUs can be fetched from the shared info + * page, but in order to make sure the mapping code is correct always + * attempt to map the vCPU info at a custom place. + */ + rc = HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info); + if (rc == 0) + DPCPU_SET(vcpu_info, vcpu_info); + else if (cpu < nitems(HYPERVISOR_shared_info->vcpu_info)) { + static bool warned = false; + + DPCPU_SET(vcpu_info, &HYPERVISOR_shared_info->vcpu_info[cpu]); + + if (bootverbose && !warned) { + warned = true; + printf( + "WARNING: Xen vCPU %u failed to setup vcpu_info rc = %d\n", + cpu, rc); + } + } else + panic("Unable to register vCPU %u, rc=%d\n", cpu, rc); +}