Date: Wed, 19 Mar 2014 16:02:36 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: Ryan Stone <rysto32@gmail.com> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>, Neel Natu <neelnatu@gmail.com> Subject: Re: [PATCH] Support PCIe Alternative RID Interpretation (ARI) Message-ID: <20140319140236.GM21331@kib.kiev.ua> In-Reply-To: <CAFMmRNwCGVhyn5cU29YpsVq44Q5i51C38GVsz33xGeqNyemx0Q@mail.gmail.com> References: <CAFMmRNzL3uBZ-djWgpnKi3XDQdq4c1ODAL_8E-Vpy-dPLa-hog@mail.gmail.com> <20140316141216.GA21331@kib.kiev.ua> <CAFMmRNwormaaPXk6rJ-JJGePS6fDNFsdKAfmmW2jGLNRscf1Pw@mail.gmail.com> <CAFgRE9F632zLseG7MobxgV5CdvD0KyMn28CBSwYqVtZKuLBwRw@mail.gmail.com> <CAFMmRNwCGVhyn5cU29YpsVq44Q5i51C38GVsz33xGeqNyemx0Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--kunpHVz1op/+13PW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 18, 2014 at 01:51:47PM -0400, Ryan Stone wrote: > On Mon, Mar 17, 2014 at 5:20 PM, Neel Natu <neelnatu@gmail.com> wrote: > > Hi Ryan, > > In any case it seems that the VT-d implementation in bhyve will work > > fine with ARI enabled devices. >=20 > There was an assert that would trip for the function number being > greater than 8. >=20 >=20 > I've put together the following patch series: >=20 > http://people.freebsd.org/~rstone/patches/ari/0001-Add-a-method-to-get-th= e-PCI-RID-for-a-device.patch >=20 I do not like this, in fact. Not the general idea, but the implementation. I think that what is needed is method which would return combined slot/function number (or just function number for ARI-enabled device). Then, existing pci_get_bus() and this new method are enough for proper construction of the translating structures. > This adds a method to get the RID that will be consumed by the VT-d > drivers. This patch is non-ARI only. >=20 >=20 > http://people.freebsd.org/~rstone/patches/ari/0002-Re-implement-the-DMAR-= I-O-MMU-code-in-terms-of-PCI-R.patch >=20 > This reworks the busdma DMAR code to work in terms of RIDs where > necessary. This should be a no-op. I tested this with > hw.dmar.enable=3D1 on a Nehalem with the em driver and a Sandy Bridge > with the igb and ixgbe driver and was able to pass traffic. Again, I do not like this. IMO the patch is too conservative. Almost all occurences of the s/f in the IOMMU code must be removed, and replaced by the half-rid returned by the new method from above. In particular, context must be stripped of s/f at all. As before, if you want, omit this part altogether, and I will try to do the pass later. >=20 >=20 > http://people.freebsd.org/~rstone/patches/ari/0003-Re-write-bhyve-s-I-O-M= MU-handling-in-terms-of-PCI-RI.patch >=20 > Same thing, but for bhyve. I'm not sure that passing the rid down to > the CPU-dependent layer is the right thing to do, because I'm not sure > what the AMD VT-d equivalent requires. Should I just pass down the > entire device_t and let the CPU layer deal with it? I tested loading > vmm.ko with a device assigned to the ppt driver but I didn't go as far > as starting a VM and using PCI passthrough. >=20 > (Also, as you'd probably expected doing this with hw.dmar.enable=3D1 > causes all hell to break loose). >=20 >=20 > http://people.freebsd.org/~rstone/patches/ari/0004-Add-support-for-PCIe-A= RI.patch >=20 > This is a slightly reworked version of the previous patch. The main > difference are that there is a new implementation of pcib_get_rid that > understands ARI RIDs. I also fixed a bug where the default > implementation of pcib_numslots was not actually being used because I > misspelled DEFAULT as default in pcib_if.m. >=20 >=20 > http://people.freebsd.org/~rstone/patches/ari/0005-Print-status-of-ARI-ca= pability-in-pciconf-c.patch >=20 > This makes pciconf -c dump the status of ARI on PCIe downstream ports. --kunpHVz1op/+13PW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJTKaN7AAoJEJDCuSvBvK1BRVsP/AsDqJ1VK7EfC21CI21l+6ym Xrp55HNV7CUDA6joK1pctpKKz33Ve1RoX4wINpt3cHXYygCDBfSH4VdTKGGrWywD Rxi73aZQH+0KCszm/Cm0TH0ldFD8EvAOPfQWba2tyGn3eDXg5pq8gXMhRLkv4t/8 JtDLhkqbwWQXTSvKHI6m9cqSLaxZPqpSH0sqv0C2L8py2mYQW3/5RcygS45hU61T VmLxaUC/bPjf3GWm7xkf8q9fqhZzplRtX8hAadqpXExzUhbJVyaUcymkwlPGZvOj nGIfZh6Jn+BQ5zLYAgfSPxpDMcprpSRJ4eThOzyH2s3sgxHmxpI4SSmesx9/7JC8 9KQnvUpt0YZdZhJE4RMLlx7/9d5sZ7WtsDZ5NvDJa4gOQl8axrT1alkWA0kcHU0U 5fm88bAZyTiT9H8Nad0a9rdSDbZF/4u/lEZ5kqt9YM0iISZsI6fkgpWkV/uEUtkg h8/Nph8ALfH688o3CjQgk/QiD/C61c2+iuw8BWEMWg31rXKsU0kXtiHeU6LSzVNF 3b6gStixo2e5dgJeXbstO/sCSOS6VtQiF4q4hbVnxR3JT9uaVJEcigL5hseIEZyD GodshFKhKUVWPa+Pa06zJcGkPyqCQrKcZwGBRd6Ymi+1ooF9E2GvtvpM46oIjP6I i0nZoR1KUEG/wnaFl39A =1lCL -----END PGP SIGNATURE----- --kunpHVz1op/+13PW--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140319140236.GM21331>