Date: Wed, 1 Apr 2015 09:23:00 +0300 From: Slawa Olhovchenkov <slw@zxy.spb.ru> To: Adrian Chadd <adrian@freebsd.org> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: irq cpu binding Message-ID: <20150401062259.GD23643@zxy.spb.ru> In-Reply-To: <CAJ-VmonDgch%2BRDyczkMMgJr5CXdgYyAJO6MqCE_uGzyNU%2BXN4g@mail.gmail.com> References: <20150328154031.GA23643@zxy.spb.ru> <CAJ-VmokpWNy-arZ_=4hNbyAvzBYgDWu4F=vYSDwqPgpjaGHk=w@mail.gmail.com> <20150328181026.GB23643@zxy.spb.ru> <CAJ-Vmok=0=C%2BHLvLXEM5pbYp_zP6C498wudAfEA40s4AVpGa_w@mail.gmail.com> <20150328183147.GC23643@zxy.spb.ru> <CAJ-VmokPdnOSASLcrzHtqs_r6GfUW6Q5ap7r9Tnp1OvKhDh20A@mail.gmail.com> <20150328192505.GD23643@zxy.spb.ru> <CAJ-Vmo=R5J=LT9ntgZJ4xHZwn8_WEAUC=3wSqPKn3=z8ki_N6A@mail.gmail.com> <20150331225947.GC23643@zxy.spb.ru> <CAJ-VmonDgch%2BRDyczkMMgJr5CXdgYyAJO6MqCE_uGzyNU%2BXN4g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Mar 31, 2015 at 04:14:04PM -0700, Adrian Chadd wrote:
> You also have to do the taskqueue threads.
# procstat -ta | grep ix
0 101305 kernel ix0 que 6 8 sleep -
0 101306 kernel ix0 que 7 8 sleep -
0 101307 kernel ix0 que 8 8 sleep -
0 101308 kernel ix0 linkq 7 8 sleep -
0 101309 kernel ix1 que 9 8 sleep -
0 101310 kernel ix1 que 10 8 sleep -
0 101311 kernel ix1 que 11 8 sleep -
0 101312 kernel ix1 linkq 1 8 sleep -
12 100064 intr irq270: ix0:que 6 8 run -
12 100066 intr irq271: ix0:que 7 8 run -
12 100068 intr irq272: ix0:que 8 8 run -
12 100070 intr irq273: ix0:link 5 8 wait -
12 100072 intr irq274: ix1:que 9 8 wait -
12 100074 intr irq275: ix1:que 10 8 wait -
12 100076 intr irq276: ix1:que 11 8 run -
12 100078 intr irq277: ix1:link 9 8 wait -
# cpuset -g -t 101305
tid 101305 mask: 6
# cpuset -g -t 101306
tid 101306 mask: 7
# cpuset -g -t 101307
tid 101307 mask: 8
# cpuset -g -t 101309
tid 101309 mask: 9
# cpuset -g -t 101310
tid 101310 mask: 10
# cpuset -g -t 101311
tid 101311 mask: 11
# cpuset -g -t 100064
tid 100064 mask: 6
# cpuset -g -t 100066
tid 100066 mask: 7
# cpuset -g -t 100068
tid 100068 mask: 8
# cpuset -g -t 100072
tid 100072 mask: 9
# cpuset -g -t 100074
tid 100074 mask: 10
# cpuset -g -t 100076
tid 100076 mask: 11
Or you talk abot somewere different?
> (I mean, pmc could also be broken...)
Why?
> -a
>
>
> On 31 March 2015 at 15:59, Slawa Olhovchenkov <slw@zxy.spb.ru> wrote:
> > On Sat, Mar 28, 2015 at 12:33:52PM -0700, Adrian Chadd wrote:
> >
> >> That's done deferred by the bus interrupt wiring. That's something
> >> John's been looking into as part of the general NUMA work (and I'm
> >> trying to debug right now, on dual-socket boxes with ixgbe. :-)
> >>
> >> Look at bus_bind_intr() and the twisty path to intr_event_bind(), then
> >> x86/x86/intr_machdep.c:intr_assign_cpu(), then intr_shuffle_cpus() at
> >> boot, versus what happens via calls to pic_assign_cpu to setup the
> >> wiring.
> >
> > I am do simple, ugle hack ixgbe driver for let start cpu binding.
> > I am still see ixgbe in pmc output.
> >
> > What may be wrong?
> > What may be miss?
> >
> > =====
> > static int ixgbe_start_cpu = 0;
> > TUNABLE_INT("hw.ix.start_cpu", &ixgbe_start_cpu);
> > SYSCTL_INT(_hw_ix, OID_AUTO, start_cpu, CTLFLAG_RDTUN, &ixgbe_start_cpu, 0,
> > "Start CPU for next IRQ binding");
> >
> > [...]
> > if (adapter->num_queues > 1)
> > bus_bind_intr(dev, que->res, i+ixgbe_start_cpu);
> >
> > #ifndef IXGBE_LEGACY_TX
> > TASK_INIT(&txr->txq_task, 0, ixgbe_deferred_mq_start, txr);
> > #endif
> > TASK_INIT(&que->que_task, 0, ixgbe_handle_que, que);
> > que->tq = taskqueue_create_fast("ixgbe_que", M_NOWAIT,
> > taskqueue_thread_enqueue, &que->tq);
> > taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que",
> > device_get_nameunit(adapter->dev));
> > }
> > ixgbe_start_cpu += adapter->num_queues;
> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150401062259.GD23643>