From nobody Wed Apr 19 05:29:51 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1TrW22X7z4500p; Wed, 19 Apr 2023 05:30:03 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1TrW18BFz3Pxb; Wed, 19 Apr 2023 05:30:03 +0000 (UTC) (envelope-from kevans@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681882203; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WM5uTmN4cwWtfzgLPX5jxplWgTi/P2R1GaPoCwkWUx8=; b=R2WbPCv0HORxcx5pfOaAeG6fyKD9JamScOiWlBVlZgkCOEVYdMpChIjX+iU7A1iAotpd8m eQUGwpppUeYd/K4ZMWcn3lxmKWs2r6SKtAGd4l6d25hlPAK48W7EuMgmpyHP5hWDqwkIiz RVHZW2pDY4IndqBWIFGXWvPHzWlPxk5X2KFXXP/7cWAtk4dQ6XpHpN1xOBeWi1r7+KJMGj z9NPw88+xh6rFY9WO/ckK40F9TxT6ArXtR4++1C8eZGf1Wnu37zIepoKeJ/d3cqqzb3QPr BXcFn8nY14kpUQgISQ8OOZvoKR35q95Sx0WmET9E+y5Z1tjABZKIdob6lmPykg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681882203; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WM5uTmN4cwWtfzgLPX5jxplWgTi/P2R1GaPoCwkWUx8=; b=kK5PGj3zfVIu5J3Rj/1pJSOx87/ES/kI1d+OniBSROiUuFaq7PeZHSITOtAMSJOl9Ipe91 X33hm3e9mci+WplXXOE0uz9n6fv2PAW4VXJIUbsdnXOzzadl8yeIuyawgmfeyo9BeT4gmC 5jDrGTFay/rNa/+1mXTEth2bzFFM0/6D+fj0KlgEqrHiO6shNb5fiiaRjsSqRsOLJkRvhm 5XyvBJXTDHeU8IUIb3TcPqlAvCSOirl9T8Gi5Wg5/Ae2vOPN6d1VpUmIGybO7amdFT8YcW /Uq2JNPDcRuoOMRj60AhwM7LzHizvMN36dcPoqAztj5/GRrsP2darqp570Z+aw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681882203; a=rsa-sha256; cv=none; b=B7wrigwzwloJeOKt1vFZd2RX9ZoXgfhTRgbAlYhO4bL74cRj62XYiF5y9FXGP4w1Ew7wou uyru4v0jQPphPf3u/BHo/SgNRx6SyIkr7z5HOn8ikuYhR8Ap4aqLOz3TeagwuAAg32jI30 ngome1uGoMV8uoBJ8ZDtYfac88S4Vv7myQLVQRWlCEVBZAJkwbjhaAlJOqkWo4wwu3BcV8 p1SFULYWDw4AFwI4GqXSLLsrx/daYT7jEtHmlS9kR4+Zr/tz9aK/y1h3ArIEAHBGvsaMXs mJbLmqax8oO9b3tEMShwJvYLHlHR0O1HGWNL8sk7IMn8+nAuRzSGXE4jeOQmYw== Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Q1TrW007yz1M0D; Wed, 19 Apr 2023 05:30:02 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qv1-f48.google.com with SMTP id m16so20071126qvx.9; Tue, 18 Apr 2023 22:30:02 -0700 (PDT) X-Gm-Message-State: AAQBX9eG35K/0Hj6vN09/TNrQFyQq6jiIOYHTCO6afq0PVYUerv3mDLA 4ZW6yo2xPWsh5EEVmZkLcLsrJyvcsMsuyCFjGXY= X-Google-Smtp-Source: AKy350bYTE1epwi6cYlveTfXJ67zgqP2n4sBHOCKPwMMnuPaRk9xxYd4hx2m3zsIT3mMvvpJLyh2HAK5WidRcfdS3W8= X-Received: by 2002:ad4:4bb4:0:b0:5a5:c0d:b620 with SMTP id i20-20020ad44bb4000000b005a50c0db620mr2921293qvw.1.1681882202156; Tue, 18 Apr 2023 22:30:02 -0700 (PDT) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 References: <202210271354.29RDsUoH077155@gitrepo.freebsd.org> In-Reply-To: <202210271354.29RDsUoH077155@gitrepo.freebsd.org> From: Kyle Evans Date: Wed, 19 Apr 2023 00:29:51 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: git: 9729f076e4d9 - main - arm64: Hyper-V: enablement for ARM64 in Hyper-V (Part 3, final) To: Wei Hu , Souradeep Chakrabarti Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-ThisMailContainsUnwantedMimeParts: N On Thu, Oct 27, 2022 at 8:54=E2=80=AFAM Wei Hu wrote: > > The branch main has been updated by whu: > > URL: https://cgit.FreeBSD.org/src/commit/?id=3D9729f076e4d93c5a37e78d427b= fe0f1ab99bbcc6 > > commit 9729f076e4d93c5a37e78d427bfe0f1ab99bbcc6 > Author: Souradeep Chakrabarti > AuthorDate: 2022-10-27 13:46:08 +0000 > Commit: Wei Hu > CommitDate: 2022-10-27 13:53:22 +0000 > > arm64: Hyper-V: enablement for ARM64 in Hyper-V (Part 3, final) > > This is the last part for ARM64 Hyper-V enablement. This includes > commone files and make file changes to enable the ARM64 FreeBSD > guest on Hyper-V. With this patch, it should be able to build > the ARM64 image and install it on Hyper-V. > Hi, First off- thanks for doing this work! I can't seem to boot a -CURRENT image under Hyper-V on a Volterra machine, seemingly due to vmbus. It stalls right after "vmbus: the irq 18" but before emitting a version number, I have some other comments here... > [... snip ...] > diff --git a/sys/dev/hyperv/vmbus/vmbus.c b/sys/dev/hyperv/vmbus/vmbus.c > index b0cd750b26c8..f370f2a75b99 100644 > --- a/sys/dev/hyperv/vmbus/vmbus.c > +++ b/sys/dev/hyperv/vmbus/vmbus.c > [... snip ...] > @@ -107,7 +113,7 @@ static uint32_t vmbus_get_vcpu_id= _method(device_t bus, > device_t dev, int cpu); > static struct taskqueue *vmbus_get_eventtq_method(device_= t, device_t, > int); > -#ifdef EARLY_AP_STARTUP > +#if defined(EARLY_AP_STARTUP) || defined(__aarch64__) > static void vmbus_intrhook(void *); > #endif > My gut reaction to this is that this is a red flag. EARLY_AP_STARTUP implies characteristics that aarch64 doesn't exhibit; it's hard to see why this conditional is OK, or whether it's testing EARLY_AP_STARTUP as a bad way to write __i386__ || __amd64__. > [... snip ...] > @@ -760,7 +736,7 @@ vmbus_synic_setup(void *xsc) > > if (hyperv_features & CPUID_HV_MSR_VP_INDEX) { > /* Save virtual processor id. */ > - VMBUS_PCPU_GET(sc, vcpuid, cpu) =3D rdmsr(MSR_HV_VP_INDEX= ); > + VMBUS_PCPU_GET(sc, vcpuid, cpu) =3D RDMSR(MSR_HV_VP_INDEX= ); > } else { > /* Set virtual processor id to 0 for compatibility. */ > VMBUS_PCPU_GET(sc, vcpuid, cpu) =3D 0; This one, vmbus_synic_setup(), is invoked via vmbus_intrhook -> vmbus_doattach -> smp_rendezvous. On !EARLY_AP_STARTUP (e.g., aarch64), SMP isn't functional in intrhooks and smp_rendezvous() will just call vmbus_synic_setup() on the boot AP. There's nothing that will initialize the pcpu data on every other AP, AFAICT. That said, the !EARLY_AP_STARTUP path is also wrong. Quoting lines that weren't in this e-mail from vmbus_attach(): 1527 #else /* !EARLY_AP_STARTUP */ 1528 /* 1529 * If the system has already booted and thread 1530 * scheduling is possible indicated by the global 1531 * cold set to zero, we just call the driver 1532 * initialization directly. 1533 */ 1534 if (!cold) 1535 vmbus_doattach(vmbus_sc); 1536 #endif /* EARLY_AP_STARTUP and aarch64 */ The two immediate issues I see is that in a device_attach, SMP won't be started. It's also going to be before SI_SUB_CONFIGURE, so `cold` will never be 0 here -- this is effectively dead code, and if it weren't dead code it would suffer the same problem as above where other APs aren't started yet so we won't set their pcpu data. This is OK, though, because then we have this sysinit later on that does the same thing for !EARLY_AP_STARTUP && !__aarch64__`, but that should really just be `!EARLY_AP_STARTUP` as aarch64 also needs to invoke vmbus_doattach() at SI_SUB_SMP (much later than SI_SUB_DRIVERS). Thanks, Kyle Evans