Date: Thu, 19 Feb 2015 12:33:38 -0500 From: John Baldwin <jhb@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: "freebsd-arch@freebsd.org" <arch@freebsd.org>, Adrian Chadd <adrian@freebsd.org> Subject: Re: RFC: bus_get_cpus(9) Message-ID: <6632720.8QN4idWR9d@ralph.baldwin.cx> In-Reply-To: <C5D32E2D-0618-4F5E-AEC7-064502438138@bsdimp.com> References: <1848011.eGOHhpCEMm@ralph.baldwin.cx> <2650364.MV3AvSBuVe@ralph.baldwin.cx> <C5D32E2D-0618-4F5E-AEC7-064502438138@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thursday, February 19, 2015 10:15:56 AM Warner Losh wrote:
> > On Feb 19, 2015, at 10:03 AM, John Baldwin <jhb@freebsd.org> wrote:
> >
> > On Thursday, February 19, 2015 08:28:34 AM Adrian Chadd wrote:
> >> On 19 February 2015 at 07:37, John Baldwin <jhb@freebsd.org> wrote:
> >>> There's nothing preventing the RSS code from calling bus_get_cpus()
> >>> internally to populate the info it returns in its APIs.
> >>>
> >>> That is, I imagine something like:
> >>>
> >>> #ifdef RSS
> >>>
> >>> queue_info = fetch_rss_info(dev);
> >>> for (queue in queue_info) {
> >>>
> >>> create queue for CPU queue->cpu
> >>>
> >>> }
> >>>
> >>> #else
> >>>
> >>> /* Use bus_get_cpus directly and do 1:1 */
> >>>
> >>> #endif
> >>>
> >>> That is, I think RSS should provide a layer on top of new-bus, not be a
> >>> bus_foo API. At some point all drivers might only have the #ifdef RSS
> >>> case
> >>> and not use bus_get_cpus() directly at all, but it doesn't seem like the
> >>> RSS API is quite there yet.
> >>
> >> I wasn't suggesting we have RSS as a newbus method, just that drivers
> >> don't necessarily need to call the bus method and iterate themselves.
> >>
> >> I was suggesting that we do what i've done for rss, but as a generic
> >> "how should devices create queues and map them to cpusets / interrupt
> >> locality" and that calls the bus method(s) to discover topology and
> >> query local-interrupt and local-memory sets to do things
> >> appropriately.
> >>
> >> Then RSS is just a flavour of that API call - network drivers could
> >> either be RSS aware and call it to get the mapping, or call some
> >> higher level bus API call to do the "generic" hints or whatever based
> >> mapping.
> >
> > Can you provide a sample API (function prototype, etc.)? Aside from RSS
> > (which will have its own API for other reasons), I don't know of another
> > use case that is well understood enough to let us build an abstraction on
> > yet (we all know the perils of abstracting from one use case), so I'm
> > hesitant to go much further than "these are the best place to do
> > interrupts”.
>
> Newer LSI cards could benefit from that, but the rest of the storage stack
> may need tweaks to allow for true multi-queue implementations. Interrupts
> would be part of that.
Right, storage in particular I think we don't really know what model we want
yet, so aren't really ready for an API that tries to be abstract across both
network and storage.
--
John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6632720.8QN4idWR9d>