From nobody Thu Nov 3 12:50:53 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N33XW2T7Pz4gws4 for ; Thu, 3 Nov 2022 12:51:07 +0000 (UTC) (envelope-from bT.5ifi7mpc30=t8uhswtnaitc=ay9tx06eyt@em790814.fubar.geek.nz) Received: from e2i251.smtp2go.com (e2i251.smtp2go.com [103.2.140.251]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4N33XT6ZhHz3V0d for ; Thu, 3 Nov 2022 12:51:04 +0000 (UTC) (envelope-from bT.5ifi7mpc30=t8uhswtnaitc=ay9tx06eyt@em790814.fubar.geek.nz) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=smtpservice.net; s=mgy720.a1-4.dyn; x=1667480764; h=Feedback-ID: X-Smtpcorp-Track:To:Message-Id:Date:From:Subject:Reply-To:Sender: List-Unsubscribe; bh=qvxsBb4OXAS/daAE19v34ervw9SeLWtPWFDvJEnpoB8=; b=vTmyU8jd EERO5NcaQPfSEos2cnhFc8nnPHPtOTqfCcnFN2wk1QsYOqGu9iqhfsa2QWYPnGaxyREKAInbD10RM yiVfjsKTAwC8txK8KUbI0dsKkyNB9798g2QcwULLhZbtqj+ek2rUUgVSfHVQvgUkb2tf1kPqv6SLP RyMCAdr3n10+pFKgTczLOYIPbU5fBq7ry2Np2tZdKF4gQpucjFy/XLQ2YgE9YEhQ6t/aB4758rQah 7C4Y2ZEOqwcZoxAkLu0pEDFgbjwzlfSKtnwMAAoOMpVOha2nt56uiNBLdufP1Sv4/vMbxDWyLdvFv 4shmz154MF5nX12vZZ+uRAUr2g==; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fubar.geek.nz; i=@fubar.geek.nz; q=dns/txt; s=s790814; t=1667479864; h=from : subject : to : message-id : date; bh=qvxsBb4OXAS/daAE19v34ervw9SeLWtPWFDvJEnpoB8=; b=ge7Z1rViKAArbfu9VmjoGj+01RmiJ0cy8eye7f2thFgVJwKnaWI44s3hBKbow7iQlCOr8 n1yrPrKMcS7rK1qRPsWPJk6bxguzt5IOqFQFp7VevYUt3d7vICCD6Tr689ACIJi5Upk9crT a6BCkQYd45k0tZh9xkryhtpgZt1jsa4RvOo0fqXH5biNir7JXdrOQPkEvUvrNU5e+xEWU08 EqyKTLNPV7fVM/VuqgS+Xrsw88FM/ONVwDxPHd5QzOj8OdMOmdO+crbdTK97bO7pghq/Ol3 2pbFVzmfD9ILK4M/xLlq2C25bWNAgkbDNDT9kQbcMx54PqYRQ6SBJhbnEBNw== Received: from [10.139.162.187] (helo=SmtpCorp) by smtpcorp.com with esmtpsa (TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.94.2-S2G) (envelope-from ) id 1oqZgP-Y8PHaa-KE; Thu, 03 Nov 2022 12:50:57 +0000 Received: from [10.162.55.164] (helo=morbo.fubar.geek.nz) by smtpcorp.com with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96-S2G) (envelope-from ) id 1oqZgP-4XfowD-1O; Thu, 03 Nov 2022 12:50:57 +0000 Received: from smtpclient.apple (cpc91214-cmbg18-2-0-cust234.5-4.cable.virginm.net [81.102.75.235]) by morbo.fubar.geek.nz (Postfix) with ESMTPSA id 264F39152; Thu, 3 Nov 2022 12:50:54 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: [EXTERNAL] pcib msix allocation in arm64 From: Andrew Turner In-Reply-To: Date: Thu, 3 Nov 2022 12:50:53 +0000 Cc: Warner Losh , Wei Hu , "freebsd-hackers@FreeBSD.org" Content-Transfer-Encoding: quoted-printable Message-Id: <947DBD67-6443-480F-82DA-2BDDF44C3D03@fubar.geek.nz> References: <146F5D18-6366-4953-A8D9-61FE7EC67F71@fubar.geek.nz> To: Souradeep Chakrabarti X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Smtpcorp-Track: 1oqZge4bfowD1O.Y2kzP6gvSXqIi Feedback-ID: 790814m:790814amQcrys:790814sNwJM4lss3 X-Report-Abuse: Please forward a copy of this message, including all headers, to X-Rspamd-Queue-Id: 4N33XT6ZhHz3V0d X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=smtpservice.net header.s=mgy720.a1-4.dyn header.b=vTmyU8jd; dkim=pass header.d=fubar.geek.nz header.s=s790814 header.b=ge7Z1rVi; dmarc=pass (policy=none) header.from=fubar.geek.nz; spf=pass (mx1.freebsd.org: domain of "bT.5ifi7mpc30=t8uhswtnaitc=ay9tx06eyt@em790814.fubar.geek.nz" designates 103.2.140.251 as permitted sender) smtp.mailfrom="bT.5ifi7mpc30=t8uhswtnaitc=ay9tx06eyt@em790814.fubar.geek.nz" X-Spamd-Result: default: False [-3.89 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; NEURAL_HAM_LONG(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[fubar.geek.nz,none]; MV_CASE(0.50)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; FORGED_SENDER(0.30)[andrew@fubar.geek.nz,bT.5ifi7mpc30=t8uhswtnaitc=ay9tx06eyt@em790814.fubar.geek.nz]; RCVD_IN_DNSWL_MED(-0.20)[103.2.140.251:from]; R_SPF_ALLOW(-0.20)[+ip4:103.2.140.0/22]; R_DKIM_ALLOW(-0.20)[smtpservice.net:s=mgy720.a1-4.dyn,fubar.geek.nz:s=s790814]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_NEQ_ENVFROM(0.00)[andrew@fubar.geek.nz,bT.5ifi7mpc30=t8uhswtnaitc=ay9tx06eyt@em790814.fubar.geek.nz]; RCVD_COUNT_THREE(0.00)[4]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[smtpservice.net:+,fubar.geek.nz:+]; ASN(0.00)[asn:23352, ipnet:103.2.140.0/22, country:US]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hi Souradeep, For the vmbus_pcib driver you=E2=80=99ll need to implement the = pcib_alloc_msi, pcib_release_msi, and the msix versions, and = pcib_map_msi. For alloc and release you can just call into the same intr_* function as = pci_host_generic_acpi.c. You=E2=80=99ll need to pass in the xref for the = interrupt controller. It needs to be the xref the MSI controller = registered. For the GICv2m it=E2=80=99s ACPI_MSI_XREF. For pcib_map_msi I am unsure if the current code is valid on arm64. If = not you=E2=80=99ll need to call intr_map_msi. Andrew > On 2 Nov 2022, at 22:27, Souradeep Chakrabarti = wrote: >=20 > Hi Andrew, > Thanks for the reply. Regarding generic_pcie_acpi_alloc_msix( ), it = can be called if the > PCI device is child of the generic pcib ( DRIVER_MODULE(pcib, acpi, = generic_pcie_acpi_driver, 0, 0) .=20 > But if the PCI device is communicating with a different pcib driver = (like vmbus_pcib), > in that case do we need to implement all these functions of = pci_host_generic_acpi.c ? >=20 > Or there are some ways to reuse them? >=20 >> -----Original Message----- >> From: Andrew Turner >> Sent: Wednesday, November 2, 2022 6:54 PM >> To: Souradeep Chakrabarti >> Cc: Warner Losh ; Wei Hu ; = freebsd- >> hackers@FreeBSD.org >> Subject: [EXTERNAL] Re: pcib msix allocation in arm64 >>=20 >>=20 >>> On 2 Nov 2022, at 12:56, Souradeep Chakrabarti = >> wrote: >>>=20 >>> Hi, >>> I can see in x86 nexus.c has implemented pcib_alloc_msix using >> nexus_alloc_msix(). >>> Which calls msix_alloc() for msix allocation. >>>=20 >>> But in case of arm64 I don't find similar pcib_alloc_msix = implementation in >> nexus.c . >>> So, on arm64 what is correct way to get allocate msix ? >>=20 >> For an arm64 system with ACPI it is most likely handled in >> generic_pcie_acpi_release_msix. For FDT it can depend on which PCI = driver is >> used. >>=20 >> In either case it will call into intr_release_msix that then calls = into the MSI >> controller to allocate the vectors. For a GICv3 driver it will either = be >> gicv3_its_alloc_msix if you have an ITS device, or gic_v3_alloc_msix = if using MBI >> ranges. >>=20 >> On ACPI we don=E2=80=99t currently support MBI ranges, although it = looks like this could >> be handled by the existing gicv2m driver. This driver should already = work as a >> child of the GICv3, however it appears to be FDT only, so will need = some work to >> add ACPI support. >>=20 >> Andrew >=20