Date: Thu, 19 Feb 2015 10:15:56 -0700 From: Warner Losh <imp@bsdimp.com> To: John Baldwin <jhb@freebsd.org> Cc: "freebsd-arch@freebsd.org" <arch@freebsd.org>, Adrian Chadd <adrian@freebsd.org> Subject: Re: RFC: bus_get_cpus(9) Message-ID: <C5D32E2D-0618-4F5E-AEC7-064502438138@bsdimp.com> In-Reply-To: <2650364.MV3AvSBuVe@ralph.baldwin.cx> References: <1848011.eGOHhpCEMm@ralph.baldwin.cx> <6147240.5Rne9DUXyM@ralph.baldwin.cx> <CAJ-Vmom0ZEAqgoeJLhu2ORk6_z=-e4-pae8ArDAZJRbD7MgPzQ@mail.gmail.com> <2650364.MV3AvSBuVe@ralph.baldwin.cx>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Feb 19, 2015, at 10:03 AM, John Baldwin <jhb@freebsd.org> wrote:
>=20
> On Thursday, February 19, 2015 08:28:34 AM Adrian Chadd wrote:
>> On 19 February 2015 at 07:37, John Baldwin <jhb@freebsd.org> wrote:
>>> There's nothing preventing the RSS code from calling bus_get_cpus()
>>> internally to populate the info it returns in its APIs.
>>>=20
>>> That is, I imagine something like:
>>>=20
>>> #ifdef RSS
>>>=20
>>> queue_info =3D fetch_rss_info(dev);
>>> for (queue in queue_info) {
>>>=20
>>> create queue for CPU queue->cpu
>>>=20
>>> }
>>>=20
>>> #else
>>>=20
>>> /* Use bus_get_cpus directly and do 1:1 */
>>>=20
>>> #endif
>>>=20
>>> That is, I think RSS should provide a layer on top of new-bus, not =
be a
>>> bus_foo API. At some point all drivers might only have the #ifdef =
RSS
>>> case
>>> and not use bus_get_cpus() directly at all, but it doesn't seem like =
the
>>> RSS API is quite there yet.
>>=20
>> I wasn't suggesting we have RSS as a newbus method, just that drivers
>> don't necessarily need to call the bus method and iterate themselves.
>>=20
>> I was suggesting that we do what i've done for rss, but as a generic
>> "how should devices create queues and map them to cpusets / interrupt
>> locality" and that calls the bus method(s) to discover topology and
>> query local-interrupt and local-memory sets to do things
>> appropriately.
>>=20
>> Then RSS is just a flavour of that API call - network drivers could
>> either be RSS aware and call it to get the mapping, or call some
>> higher level bus API call to do the "generic" hints or whatever based
>> mapping.
>=20
> Can you provide a sample API (function prototype, etc.)? Aside from =
RSS=20
> (which will have its own API for other reasons), I don't know of =
another use=20
> case that is well understood enough to let us build an abstraction on =
yet (we=20
> all know the perils of abstracting from one use case), so I'm hesitant =
to go=20
> much further than "these are the best place to do interrupts=E2=80=9D.
Newer LSI cards could benefit from that, but the rest of the storage =
stack
may need tweaks to allow for true multi-queue implementations. =
Interrupts
would be part of that.
Warner=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C5D32E2D-0618-4F5E-AEC7-064502438138>