Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 1 Apr 2015 09:23:00 +0300
From:      Slawa Olhovchenkov <slw@zxy.spb.ru>
To:        Adrian Chadd <adrian@freebsd.org>
Cc:        "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>
Subject:   Re: irq cpu binding
Message-ID:  <20150401062259.GD23643@zxy.spb.ru>
In-Reply-To: <CAJ-VmonDgch%2BRDyczkMMgJr5CXdgYyAJO6MqCE_uGzyNU%2BXN4g@mail.gmail.com>
References:  <20150328154031.GA23643@zxy.spb.ru> <CAJ-VmokpWNy-arZ_=4hNbyAvzBYgDWu4F=vYSDwqPgpjaGHk=w@mail.gmail.com> <20150328181026.GB23643@zxy.spb.ru> <CAJ-Vmok=0=C%2BHLvLXEM5pbYp_zP6C498wudAfEA40s4AVpGa_w@mail.gmail.com> <20150328183147.GC23643@zxy.spb.ru> <CAJ-VmokPdnOSASLcrzHtqs_r6GfUW6Q5ap7r9Tnp1OvKhDh20A@mail.gmail.com> <20150328192505.GD23643@zxy.spb.ru> <CAJ-Vmo=R5J=LT9ntgZJ4xHZwn8_WEAUC=3wSqPKn3=z8ki_N6A@mail.gmail.com> <20150331225947.GC23643@zxy.spb.ru> <CAJ-VmonDgch%2BRDyczkMMgJr5CXdgYyAJO6MqCE_uGzyNU%2BXN4g@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Mar 31, 2015 at 04:14:04PM -0700, Adrian Chadd wrote:

> You also have to do the taskqueue threads.

# procstat -ta | grep ix
    0 101305 kernel           ix0 que            6    8 sleep   -         
    0 101306 kernel           ix0 que            7    8 sleep   -         
    0 101307 kernel           ix0 que            8    8 sleep   -         
    0 101308 kernel           ix0 linkq          7    8 sleep   -         
    0 101309 kernel           ix1 que            9    8 sleep   -         
    0 101310 kernel           ix1 que           10    8 sleep   -         
    0 101311 kernel           ix1 que           11    8 sleep   -         
    0 101312 kernel           ix1 linkq          1    8 sleep   -         
   12 100064 intr             irq270: ix0:que    6    8 run     -         
   12 100066 intr             irq271: ix0:que    7    8 run     -         
   12 100068 intr             irq272: ix0:que    8    8 run     -         
   12 100070 intr             irq273: ix0:link   5    8 wait    -         
   12 100072 intr             irq274: ix1:que    9    8 wait    -         
   12 100074 intr             irq275: ix1:que   10    8 wait    -         
   12 100076 intr             irq276: ix1:que   11    8 run     -         
   12 100078 intr             irq277: ix1:link   9    8 wait    -         
# cpuset -g -t 101305
tid 101305 mask: 6
# cpuset -g -t 101306
tid 101306 mask: 7
# cpuset -g -t 101307
tid 101307 mask: 8
# cpuset -g -t 101309
tid 101309 mask: 9
# cpuset -g -t 101310
tid 101310 mask: 10
# cpuset -g -t 101311
tid 101311 mask: 11
# cpuset -g -t 100064
tid 100064 mask: 6
# cpuset -g -t 100066
tid 100066 mask: 7
# cpuset -g -t 100068
tid 100068 mask: 8
# cpuset -g -t 100072
tid 100072 mask: 9
# cpuset -g -t 100074
tid 100074 mask: 10
# cpuset -g -t 100076
tid 100076 mask: 11

Or you talk abot somewere different?

> (I mean, pmc could also be broken...)

Why?

> -a
> 
> 
> On 31 March 2015 at 15:59, Slawa Olhovchenkov <slw@zxy.spb.ru> wrote:
> > On Sat, Mar 28, 2015 at 12:33:52PM -0700, Adrian Chadd wrote:
> >
> >> That's done deferred by the bus interrupt wiring. That's something
> >> John's been looking into as part of the general NUMA work (and I'm
> >> trying to debug right now, on dual-socket boxes with ixgbe. :-)
> >>
> >> Look at bus_bind_intr() and the twisty path to intr_event_bind(), then
> >> x86/x86/intr_machdep.c:intr_assign_cpu(), then intr_shuffle_cpus() at
> >> boot, versus what happens via calls to pic_assign_cpu to setup the
> >> wiring.
> >
> > I am do simple, ugle hack ixgbe driver for let start cpu binding.
> > I am still see ixgbe in pmc output.
> >
> > What may be wrong?
> > What may be miss?
> >
> > =====
> > static int ixgbe_start_cpu = 0;
> > TUNABLE_INT("hw.ix.start_cpu", &ixgbe_start_cpu);
> > SYSCTL_INT(_hw_ix, OID_AUTO, start_cpu, CTLFLAG_RDTUN, &ixgbe_start_cpu, 0,
> >     "Start CPU for next IRQ binding");
> >
> > [...]
> >                 if (adapter->num_queues > 1)
> >                         bus_bind_intr(dev, que->res, i+ixgbe_start_cpu);
> >
> > #ifndef IXGBE_LEGACY_TX
> >                 TASK_INIT(&txr->txq_task, 0, ixgbe_deferred_mq_start, txr);
> > #endif
> >                 TASK_INIT(&que->que_task, 0, ixgbe_handle_que, que);
> >                 que->tq = taskqueue_create_fast("ixgbe_que", M_NOWAIT,
> >                     taskqueue_thread_enqueue, &que->tq);
> >                 taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que",
> >                     device_get_nameunit(adapter->dev));
> >         }
> >         ixgbe_start_cpu += adapter->num_queues;
> >



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150401062259.GD23643>