Date: Thu, 19 Feb 2015 10:15:56 -0700 From: Warner Losh <imp@bsdimp.com> To: John Baldwin <jhb@freebsd.org> Cc: "freebsd-arch@freebsd.org" <arch@freebsd.org>, Adrian Chadd <adrian@freebsd.org> Subject: Re: RFC: bus_get_cpus(9) Message-ID: <C5D32E2D-0618-4F5E-AEC7-064502438138@bsdimp.com> In-Reply-To: <2650364.MV3AvSBuVe@ralph.baldwin.cx> References: <1848011.eGOHhpCEMm@ralph.baldwin.cx> <6147240.5Rne9DUXyM@ralph.baldwin.cx> <CAJ-Vmom0ZEAqgoeJLhu2ORk6_z=-e4-pae8ArDAZJRbD7MgPzQ@mail.gmail.com> <2650364.MV3AvSBuVe@ralph.baldwin.cx>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Feb 19, 2015, at 10:03 AM, John Baldwin <jhb@freebsd.org> wrote:
>
> On Thursday, February 19, 2015 08:28:34 AM Adrian Chadd wrote:
>> On 19 February 2015 at 07:37, John Baldwin <jhb@freebsd.org> wrote:
>>> There's nothing preventing the RSS code from calling bus_get_cpus()
>>> internally to populate the info it returns in its APIs.
>>>
>>> That is, I imagine something like:
>>>
>>> #ifdef RSS
>>>
>>> queue_info = fetch_rss_info(dev);
>>> for (queue in queue_info) {
>>>
>>> create queue for CPU queue->cpu
>>>
>>> }
>>>
>>> #else
>>>
>>> /* Use bus_get_cpus directly and do 1:1 */
>>>
>>> #endif
>>>
>>> That is, I think RSS should provide a layer on top of new-bus, not be a
>>> bus_foo API. At some point all drivers might only have the #ifdef RSS
>>> case
>>> and not use bus_get_cpus() directly at all, but it doesn't seem like the
>>> RSS API is quite there yet.
>>
>> I wasn't suggesting we have RSS as a newbus method, just that drivers
>> don't necessarily need to call the bus method and iterate themselves.
>>
>> I was suggesting that we do what i've done for rss, but as a generic
>> "how should devices create queues and map them to cpusets / interrupt
>> locality" and that calls the bus method(s) to discover topology and
>> query local-interrupt and local-memory sets to do things
>> appropriately.
>>
>> Then RSS is just a flavour of that API call - network drivers could
>> either be RSS aware and call it to get the mapping, or call some
>> higher level bus API call to do the "generic" hints or whatever based
>> mapping.
>
> Can you provide a sample API (function prototype, etc.)? Aside from RSS
> (which will have its own API for other reasons), I don't know of another use
> case that is well understood enough to let us build an abstraction on yet (we
> all know the perils of abstracting from one use case), so I'm hesitant to go
> much further than "these are the best place to do interrupts”.
Newer LSI cards could benefit from that, but the rest of the storage stack
may need tweaks to allow for true multi-queue implementations. Interrupts
would be part of that.
Warner
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C5D32E2D-0618-4F5E-AEC7-064502438138>