From nobody Thu Oct 20 10:43:23 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MtPMl5BN7z4gTRS for ; Thu, 20 Oct 2022 10:43:31 +0000 (UTC) (envelope-from andrew@fubar.geek.nz) Received: from fry.fubar.geek.nz (fry.fubar.geek.nz [139.59.165.16]) by mx1.freebsd.org (Postfix) with ESMTP id 4MtPMj6N1Jz3Xjl for ; Thu, 20 Oct 2022 10:43:29 +0000 (UTC) (envelope-from andrew@fubar.geek.nz) Received: from smtpclient.apple (cpc91214-cmbg18-2-0-cust234.5-4.cable.virginm.net [81.102.75.235]) by fry.fubar.geek.nz (Postfix) with ESMTPSA id 908DA4E630; Thu, 20 Oct 2022 10:43:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fubar.geek.nz; s=mail; t=1666262603; bh=yq3fGugMW3Ql/oEZWh0+dfaVPuAd/Nja2FdVsdnRWeU=; h=From:Subject:Date:In-Reply-To:Cc:To:References; b=Qxb8zLM8vxAdMv45A7unt5I+qbmJ0fK2fhgAr18QQD/EhdzBgjnftGUpTe3mC6E4o tiRhjxiHrBbXBwnTxLXwa2U5nw4gezxmq52444NNXdpadldn4v8JScV6Av0Btnco5p XI7sDfecdximGbNmhPrcxdjwRO2HjTd5gJy0zvKRNgyMh6bxt9+y6uJqxBu3QGZOgF AkKYzJzobuQH9xqcM1u5bVMMdDwxw7dxkid3RKqhFEWTSj97MU31hhwaha6YAx9lCp Lcr30JocVnbV5eqYpXfmKpiqD3ezaT2hgL30H0ggQ8w6HsWlYgh9WqtgLdTz6C3GvU 20OGd1iYoVo4HPkUSiEFpMZMOxtnKfo9j9IooEXIAD+rAZHlqx2pU5qBMDYaFaELH+ 6pIb9TxStKNfTLZ78g3A5itmBJyOG0xbnKgdG4piveR4JaSrvI7cmjgSBH4ppRV9CW SHGRJM3YRjKVCFgEbm2nz+2oKj9NAn5qgz4KEtVhQ+R6R5Z6omWuxsADQaGpmStRGM 6B/iiXhDEFyEQXbepZBMQXcOueF+FcEpV2E2MzI4QPvNz1/o0NSGIddvbpuDrq2Ckm +Kbe2toysS51E6glqR72MF8bjjGhc8/NoRrNWe1D8pc72N74w7LyAh+Bne1jZzwCKS 6WJroBIG44mQUGnQMzMpVEtA= From: Andrew Turner Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_8D832B57-8A2A-4CAC-96CA-1E8A51A498FD" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: [EXTERNAL] allocating IRQ mentioned in _CRS of ACPI Date: Thu, 20 Oct 2022 11:43:23 +0100 In-Reply-To: Cc: Warner Losh , "freebsd-hackers@freebsd.org" , Wei Hu To: Souradeep Chakrabarti References: X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4MtPMj6N1Jz3Xjl X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fubar.geek.nz header.s=mail header.b=Qxb8zLM8; dmarc=pass (policy=none) header.from=fubar.geek.nz; spf=softfail (mx1.freebsd.org: 139.59.165.16 is neither permitted nor denied by domain of andrew@fubar.geek.nz) smtp.mailfrom=andrew@fubar.geek.nz X-Spamd-Result: default: False [0.30 / 15.00]; VIOLATED_DIRECT_SPF(3.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50)[]; R_DKIM_ALLOW(-0.20)[fubar.geek.nz:s=mail]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_NO_TLS_LAST(0.10)[]; FREEFALL_USER(0.00)[andrew]; FROM_HAS_DN(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; ARC_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:139.59.160.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[fubar.geek.nz:+]; FROM_EQ_ENVFROM(0.00)[]; DMARC_POLICY_ALLOW(0.00)[fubar.geek.nz,none]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_8D832B57-8A2A-4CAC-96CA-1E8A51A498FD Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Based on my reading of the ACPI tables, and looking how Linux handles it = I don=E2=80=99t think the vmbus_res driver should be used. If the vmbus = driver was to attach to the =E2=80=9CVMBUS" _HID it would be able to = manage the interrupt directly. Ideally acpi_syscontainer would become a proper bus, but that would need = checking that existing drivers that depend on the current behaviour = still work. Andrew > On 19 Oct 2022, at 18:51, Souradeep Chakrabarti = wrote: >=20 > Hi Warner, > =20 > 1) PCI mmio resource in HID "ACPI0004", which is needed by the FreeBSD = guest for SRIOV devices. > =20 > Device (\_SB.VMOD) <-- This is currently owned by = acpi_syscontainer module on FreeBSD > { > Name (_HID, "ACPI0004" /* Module Device */) // _HID: Hardware = ID > Name (_UID, Zero) // _UID: Unique ID > Name (_CRS, ResourceTemplate () // _CRS: Current Resource = Settings > { > ... > } > CreateDWordField (_CRS, \_SB.VMOD._Y00._MIN, MIN6) // _MIN: = Minimum Base Address > CreateDWordField (_CRS, \_SB.VMOD._Y00._MAX, MAX6) // _MAX: = Maximum Base Address > CreateDWordField (_CRS, \_SB.VMOD._Y00._LEN, LEN6) // _LEN: = Length > CreateQWordField (_CRS, \_SB.VMOD._Y01._MIN, MIN7) // _MIN: = Minimum Base Address > CreateQWordField (_CRS, \_SB.VMOD._Y01._MAX, MAX7) // _MAX: = Maximum Base Address > CreateQWordField (_CRS, \_SB.VMOD._Y01._LEN, LEN7) // _LEN: = Length > Method (_INI, 0, NotSerialized) // _INI: Initialize > { > MIN6 =3D MG2B /* \MG2B */ > LEN6 =3D MG2L /* \MG2L */ > Local0 =3D MG2L /* \MG2L */ > MAX6 =3D (MIN6 + Local0--) > Local1 =3D (HMIB << 0x14) > Local2 =3D (HMIL << 0x14) > MIN7 =3D Local1 > LEN7 =3D Local2 > Local0 =3D Local2 > MAX7 =3D (MIN7 + Local0--) > } > } > =20 > 2) Vmbus IRQ resource in HID "VMBus", which is needed to get Hyper-V = vmbus interrupt to work on guests. > Device (\_SB.VMOD.VMBS) <--- currently owned by vmbus_res module on = FreeBSD > { > ... > Name (_HID, "VMBus") // _HID: Hardware ID > Name (_UID, Zero) // _UID: Unique ID > ... > Name (_CRS, ResourceTemplate () // _CRS: Current Resource = Settings > { > Interrupt (ResourceConsumer, Edge, ActiveHigh, Exclusive, = ,, ) > { > 0x00000012, > } > }) > } > =20 > > =20 > Today I was able to get both IRQ and MMIO allocation successful using = below snippet in vmbus: > =20 > + device_t dev =3D = devclass_get_device(devclass_find("vmbus_res"), 0); > + sc->ires =3D bus_alloc_resource_any(dev, > SYS_RES_IRQ, &sc->vector, RF_ACTIVE | RF_SHAREABLE); > =20 > =20 > Thanks & Regards, > Souradeep > =20 > From: Warner Losh =20 > Sent: Wednesday, October 19, 2022 11:09 PM > To: Souradeep Chakrabarti > Cc: freebsd-hackers@freebsd.org; Wei Hu > Subject: [EXTERNAL] Re: allocating IRQ mentioned in _CRS of ACPI > =20 > Sorry for the late reply... I've been busy with some things for = work... > =20 > I think you'll need to get the parent of vmbus to allow a pass through = allocation. What bus is that > currently? > =20 > Warner > =20 > On Tue, Oct 18, 2022 at 12:33 PM Souradeep Chakrabarti = > wrote: > Hi, > It will be a great help, if someone can help here with some idea. > As it is blocking the FreeBSD on Hyper-V ARM64. >=20 > Thanks & Regards, > Souradeep >=20 > > -----Original Message----- > > From: Souradeep Chakrabarti > > Sent: Friday, October 14, 2022 1:24 PM > > To: 'Warner Losh' > > > Cc: freebsd-hackers@freebsd.org = ; Wei Hu > > > Subject: RE: allocating IRQ mentioned in _CRS of ACPI > >=20 > > Last mail was having incorrect FreeBSD hacker alias. Replacing that = with correct > > one here. > >=20 > >=20 > > > -----Original Message----- > > > From: Souradeep Chakrabarti > > > Sent: Friday, October 14, 2022 1:19 PM > > > To: Warner Losh > > > > Cc: hacker@freebsd.org ; Wei Hu = > > > > Subject: allocating IRQ mentioned in _CRS of ACPI > > > > > > Hi, > > > I would like to allocate IRQ to a device, mentioned in the _CRS of > > > that device in ACPI table. > > > I have tried with bus_alloc_resource_any(), but it is failing as = the > > > parent of that device is not owning the IRQ. > > > > > > Current ACPI topo for the device : > > > ACPI0->SB.VMOD(HID ACPI0004, has SYS_RES_MEM for MMIO in _CRS)- > > > >VMBUS( it has SYS_RES_IRQ in it's _CRS). > > > > > > How can I get here both SYS_RES_IRQ and SYS_RES_MEM allocated to = VMBUS? > > > > > > Thanks & Regards, > > > Souradeep --Apple-Mail=_8D832B57-8A2A-4CAC-96CA-1E8A51A498FD Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Based= on my reading of the ACPI tables, and looking how Linux handles it I = don=E2=80=99t think the vmbus_res driver should be used. If the vmbus = driver was to attach to the =E2=80=9CVMBUS" _HID it would be able to = manage the interrupt directly.

Ideally acpi_syscontainer would become a proper bus, but that = would need checking that existing drivers that depend on the current = behaviour still work.

Andrew

On 19 = Oct 2022, at 18:51, Souradeep Chakrabarti <schakrabarti@microsoft.com> wrote:

Hi Warner,
 
1) PCI mmio resource in HID "ACPI0004", which = is needed by the FreeBSD guest for SRIOV devices.
 
   Device = (\_SB.VMOD)    <-- This is currently owned by = acpi_syscontainer module on FreeBSD
   = {
        = Name (_HID, "ACPI0004" /* Module Device */)  // _HID: Hardware = ID
        = Name (_UID, Zero)  // _UID: Unique ID
        Name (_CRS, = ResourceTemplate ()  // _CRS: Current Resource Settings
        {
          &nb= sp; ...
        = }
        = CreateDWordField (_CRS, \_SB.VMOD._Y00._MIN, MIN6)  // _MIN: = Minimum Base Address
        CreateDWordField = (_CRS, \_SB.VMOD._Y00._MAX, MAX6)  // _MAX: Maximum Base = Address
        = CreateDWordField (_CRS, \_SB.VMOD._Y00._LEN, LEN6)  // _LEN: = Length
        = CreateQWordField (_CRS, \_SB.VMOD._Y01._MIN, MIN7)  // _MIN: = Minimum Base Address
        CreateQWordField = (_CRS, \_SB.VMOD._Y01._MAX, MAX7)  // _MAX: Maximum Base = Address
        = CreateQWordField (_CRS, \_SB.VMOD._Y01._LEN, LEN7)  // _LEN: = Length
        = Method (_INI, 0, NotSerialized)  // _INI: Initialize
        {
          &nb= sp; MIN6 =3D MG2B /* \MG2B */
          &nb= sp; LEN6 =3D MG2L /* \MG2L */
          &nb= sp; Local0 =3D MG2L /* \MG2L */
          &nb= sp; MAX6 =3D (MIN6 + Local0--)
          &nb= sp; Local1 =3D (HMIB << 0x14)
          &nb= sp; Local2 =3D (HMIL << 0x14)
          &nb= sp; MIN7 =3D Local1
          &nb= sp; LEN7 =3D Local2
   =         Local0 =3D Local2
          &nb= sp; MAX7 =3D (MIN7 + Local0--)
        }
    }
 
2) Vmbus IRQ resource in HID "VMBus", which is = needed to get Hyper-V vmbus interrupt to work on guests.
Device (\_SB.VMOD.VMBS)   <--- currently owned = by vmbus_res module on FreeBSD
   = {
        = ...
        = Name (_HID, "VMBus")  // _HID: Hardware ID
        Name (_UID, = Zero)  // _UID: Unique ID
        ...
        Name (_CRS, = ResourceTemplate ()  // _CRS: Current Resource Settings
        {
          &nb= sp; Interrupt (ResourceConsumer, Edge, ActiveHigh, Exclusive, ,, )
          &nb= sp; {
          &nb= sp;     0x00000012,
          &nb= sp; }
        = })
    }
 
<image001.png>
 
Today I was able to get both IRQ and MMIO allocation = successful using below snippet in vmbus:
 
+       device_t dev =3D  = devclass_get_device(devclass_find("vmbus_res"), 0);
+       sc->ires =3D = bus_alloc_resource_any(dev,
          &nb= sp; SYS_RES_IRQ, &sc->vector, RF_ACTIVE | RF_SHAREABLE);
 
 
Thanks= & Regards,
 Souradeep
 
From: Warner Losh <imp@bsdimp.com> 
Sent: Wednesday, October 19, 2022 = 11:09 PM
To: Souradeep Chakrabarti = <schakrabarti@microsoft.com>
Cc: freebsd-hackers@freebsd.org; Wei Hu <weh@microsoft.com>
Subject: [EXTERNAL] Re: allocating = IRQ mentioned in _CRS of ACPI
 
Sorry for the late reply... I've been busy = with some things for work...
 
I = think you'll need to get the parent of vmbus to allow a pass through = allocation. What bus is that
currently?
 
Warner
 
On Tue, Oct 18, 2022 at 12:33 PM Souradeep = Chakrabarti <schakrabarti@microsoft.com> wrote:
Hi,
It will = be a great help, if someone can help here with some idea.
As= it is blocking the FreeBSD on Hyper-V ARM64.

Thanks & Regards,
Souradeep
> -----Original Message-----
> From: = Souradeep Chakrabarti
> Sent: Friday, October 14, 2022 = 1:24 PM
> To: 'Warner Losh' <imp@bsdimp.com>
> Cc: freebsd-hackers@freebsd.org; Wei Hu <weh@microsoft.com>> Subject: RE: allocating IRQ mentioned in _CRS of = ACPI
> 
> Last = mail was having incorrect FreeBSD hacker alias. Replacing that with = correct
> one here.
> 
> 
> > = -----Original Message-----
> > From: Souradeep = Chakrabarti
> > Sent: Friday, October 14, 2022 1:19 = PM
> > To: Warner Losh <imp@bsdimp.com>
> > Cc: hacker@freebsd.org; = Wei Hu <weh@microsoft.com>
> > Subject: = allocating IRQ mentioned in _CRS of ACPI
> >
> > Hi,
> > I would like to = allocate IRQ to a device, mentioned in the _CRS of
> = > that device in ACPI table.
> > I have tried = with bus_alloc_resource_any(), but it is failing as the
>= > parent of that device is not owning the IRQ.
> = >
> > Current ACPI topo for the device :
> > ACPI0->SB.VMOD(HID ACPI0004, has SYS_RES_MEM for = MMIO in _CRS)-
> > >VMBUS( it has SYS_RES_IRQ in = it's _CRS).
> >
> > How can I = get here both SYS_RES_IRQ and SYS_RES_MEM allocated to VMBUS?
> >
> > Thanks & Regards,
> > = Souradeep

= --Apple-Mail=_8D832B57-8A2A-4CAC-96CA-1E8A51A498FD--