Date: Fri, 14 May 2010 12:22:13 -0400 From: Andrew Gallatin <gallatin@cs.duke.edu> To: Alexander Sack <pisymbol@gmail.com> Cc: Murat Balaban <murat@enderunix.org>, freebsd-net@freebsd.org, freebsd-performance@freebsd.org Subject: Re: Intel 10Gb Message-ID: <4BED78B5.8000906@cs.duke.edu> In-Reply-To: <AANLkTim9y9B7kW4OH74LLa2Iwffkvu1rYoSODGYvRMwy@mail.gmail.com> References: <AANLkTimMrsM08Rmdr-l6RFu83VkqFw0Pk2sHxpV5Yl5x@mail.gmail.com> <4BE52856.3000601@unsane.co.uk> <1273323582.3304.31.camel@efe> <20100511135103.GA29403@grapeape2.cs.duke.edu> <AANLkTikROvNKUmpax-CbhEyj5o7TW0hfV_x79Bm_nU2V@mail.gmail.com> <4BED5929.5020302@cs.duke.edu> <AANLkTikAow9ZdK4XokeWXkbmusva2rKxeLO2EBBe3tsZ@mail.gmail.com> <4BED6F1B.7070602@cs.duke.edu> <AANLkTim9y9B7kW4OH74LLa2Iwffkvu1rYoSODGYvRMwy@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Alexander Sack wrote:
>> To use DCA you need:
>>
>> - A DCA driver to talk to the IOATDMA/DCA pcie device, and obtain the tag
>> table
>> - An interface that a client device (eg, NIC driver) can use to obtain
>> either the tag table, or at least the correct tag for the CPU
>> that the interrupt handler is bound to. The basic support in
>> a NIC driver boils down to something like:
>>
>> nic_interrupt_handler()
>> {
>> if (sc->dca.enabled && (curcpu != sc->dca.last_cpu)) {
>> sc->dca.last_cpu = curcpu;
>> tag = dca_get_tag(curcpu);
>> WRITE_REG(sc, DCA_TAG, tag);
>> }
>> }
>
> Drew, at least in the Intel documentation, it seems the NIC uses the
> LAPIC id to tell the PCIe TLPs where to put inbound NIC I/O (in the
> TLP the DCA info is stored) to the appropriate core's cache. i.e. the
> heuristic you gave above is more granular than what I think Intel
The pseudo-code above was intended to be the MSI-X interrupt handler
for a single queue, not some dispatcher for multiple queues.
Sorry that wasn't clear. So yes, the DCA tag value may be different
per queue.
> does. I could be wrong, maybe Jack can chime in and correct me. But
> it seems with Intel chipsets it is a per queue parameter which allows
> you to bind a core cache's to a queue via DCA. The added piece to
> this for at least bpf(4) consumers is to have bpf(4) subscribe to
> these queues AND to allow an interface for libpcap applications to
> know where what queue is on what core and THEN bind to it.
Yes, everything associated with a queue must be bound to the same
core (or at least to cores which share a cache).
Drew
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4BED78B5.8000906>