Date: Mon, 26 May 2014 21:38:31 -0400 From: Ryan Stone <rysto32@gmail.com> To: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: PCI SR-IOV patches for review Message-ID: <CAFMmRNwHM3i6WPbwftnn_O4TyMJsLHu%2B=X_CvzRfus6MHUaYcg@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
My work on the PCI SR-IOV driver framework is complete and ready for review. This includes: - extending the PCI infrastructure to be able to create SR-IOV Virtual Functions, allocating PCI resources to them and probing/attaching the VF driver - bhyve integration to make it easy to pass VFs to bhyve VMs as PCI passthrough devices - a flexible infrastructure for defining driver-specific configuration parameters - a unified configuration tool, iovctl, for configuring SR-IOV on PF devices At this point I do not yet have a fully working PF driver. I'm still working on the i40e and ixgbe drivers at this time, and I don't have an estimated completion date yet. The main reason why I would like this reviewed and committed ahead of the drivers is that some driver maintainers would like access to the SR-IOV infrastructure so that they can do the work in their own drivers. Ideally, we can have multiple SR-IOV drivers in place in time for 10.1-RELEASE. This work was sponsored by Sandvine Inc. Thanks to John Baldwin, who provided advice on how to integrate this into the pci driver, and to Jack Vogel, who gave pointers on the Intel drivers and hardware. I described the driver interface on a post to -arch earlier in the month: http://lists.freebsd.org/pipermail/freebsd-arch/2014-May/015332.html The interface is largely unchanged (but yes, I did eliminate the header pollution caused by nvlist_t :) ). The main difference is that the get_iov_config_schema method has been eliminated. Now, the PF and VF schemas are passed directly to pci_iov_attach() during the device_attach method. I did this because the proposed API could be abused by a PF driver that dynamically changed the schema that it provided at runtime, which was not the intention of the API. As the schema is administrator-facing I felt that changing it dynamically would be potentially very confusing to an administrator, and allowing it really doesn't bring any benefits. A driver might want to modify the schema based on the hardware type, but the driver can just as easily detect the hardware at attach as they can at runtime, so I don't think that this will be a problem. The driver interface and iovctl have manpages, so please refer to those before asking about the details of the interface from the driver maintaner's or administrator's perspective. The internal representation of the configuration schema and the SR-IOV configuration itself is described in depth in comments in <sys/iov.h>. I've also put sample versions of both in the comments, which should prove useful when reviewing code that deals with those data structures. The patch series can be see here: http://people.freebsd.org/~rstone/patches/iov/ This is a fairly large change, so I've done my best to break it up into a lot of smaller, more digestible patches. Keep in mind that an individual patch may seem pointless in isolation (for example, it introduces a new API that is unused). Rest assured that another patch will be consuming the new APIs that I have added. The patches can be broken up into the following series of patches. I plan on start separate email threads on the relevant list for review of individual patches. If you have any comments or questions about an individual patch, please respond to the thread for that patch series. I will note the mailing list that I intended to send each patch series to below. This work depends on a port of libnv to the kernel. This work is nearly complete; I'm working with pjd@ to get some technical details ironed out and then I'll send those patches out for review. Any comments that anybody can offer would be greatly appreciated. Series 1/7: PCI Infrastructure Refactoring (freebsd-hackers) http://people.freebsd.org/~rstone/patches/iov/0001-Refactor-PCI-device-creation.patch http://people.freebsd.org/~rstone/patches/iov/0002-Refactor-PCI-resource-allocation.patch http://people.freebsd.org/~rstone/patches/iov/0003-Add-some-pcib-methods-to-get-ARI-related-information.patch Series 2/7: bhyve integration (freebsd-virtualization) http://people.freebsd.org/~rstone/patches/iov/0004-Allow-passthrough-devices-to-be-hinted.patch Series 3/7: manual pages (freebsd-doc) http://people.freebsd.org/~rstone/patches/iov/0005-Document-pci_iov_attach-detach-in-pci.9.patch http://people.freebsd.org/~rstone/patches/iov/0006-Add-manpages-for-SR-IOV-enable-disable-driver-interf.patch http://people.freebsd.org/~rstone/patches/iov/0014-Add-a-manpage-for-iovctl-8.patch http://people.freebsd.org/~rstone/patches/iov/0015-Add-manpage-documenting-iovctl-config-file-format.patch http://people.freebsd.org/~rstone/patches/iov/0020-Document-the-interface-for-defining-a-configuration-.patch Series 4/7: Core SR-IOV infrastructure in pci driver (freebsd-hackers) http://people.freebsd.org/~rstone/patches/iov/0007-Implement-interface-to-create-SR-IOV-Virtual-Functio.patch http://people.freebsd.org/~rstone/patches/iov/0008-Emulate-the-Device-ID-and-Vendor-ID-registers-for-VF.patch http://people.freebsd.org/~rstone/patches/iov/0009-Allocate-PCI-I-O-memory-spaces-for-VFs.patch http://people.freebsd.org/~rstone/patches/iov/0010-Add-interface-to-destroy-SR-IOV-VFs.patch Series 5/7: SR-IOV Configuration Interface (freebsd-hackers) http://people.freebsd.org/~rstone/patches/iov/0011-Add-infrastructure-for-exporting-config-schema-from-.patch http://people.freebsd.org/~rstone/patches/iov/0012-Add-function-to-validate-the-consistency-of-SR-IOV-c.patch http://people.freebsd.org/~rstone/patches/iov/0013-Pass-SR-IOV-configuration-to-kernel-using-an-nvlist.patch http://people.freebsd.org/~rstone/patches/iov/0021-Validate-the-schema-that-the-PF-driver-passed-to-us.patch Series 6/7: iovctl Configuration Tool (freebsd-hackers) http://people.freebsd.org/~rstone/patches/iov/0016-Add-iovctl-functions-for-validating-config.patch http://people.freebsd.org/~rstone/patches/iov/0017-Add-functions-for-parsing-the-iovctl-config-file.patch http://people.freebsd.org/~rstone/patches/iov/0018-Add-main-for-iovctl-and-hook-iovctl-into-build.patch Series 7/7: rc.d script (freebsd-rc) http://people.freebsd.org/~rstone/patches/iov/0019-Add-an-rc.d-script-to-invoke-iovctl-8-during-boot.patch
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFMmRNwHM3i6WPbwftnn_O4TyMJsLHu%2B=X_CvzRfus6MHUaYcg>