Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 19 Jan 2013 07:47:30 -0800 (PST)
From:      Barney Cordoba <barney_cordoba@yahoo.com>
To:        freebsd-net@freebsd.org, John Baldwin <jhb@freebsd.org>
Cc:        Adrian Chadd <adrian@freebsd.org>, Luigi Rizzo <rizzo@iet.unipi.it>
Subject:   Re: two problems in dev/e1000/if_lem.c::lem_handle_rxtx()
Message-ID:  <1358610450.75691.YahooMailClassic@web121604.mail.ne1.yahoo.com>
In-Reply-To: <201301181149.42277.jhb@freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
=0A=0A--- On Fri, 1/18/13, John Baldwin <jhb@freebsd.org> wrote:=0A=0A> Fro=
m: John Baldwin <jhb@freebsd.org>=0A> Subject: Re: two problems in dev/e100=
0/if_lem.c::lem_handle_rxtx()=0A> To: freebsd-net@freebsd.org=0A> Cc: "Barn=
ey Cordoba" <barney_cordoba@yahoo.com>, "Adrian Chadd" <adrian@freebsd.org>=
, "Luigi Rizzo" <rizzo@iet.unipi.it>=0A> Date: Friday, January 18, 2013, 11=
:49 AM=0A> On Friday, January 18, 2013 9:30:40=0A> am Barney Cordoba wrote:=
=0A> > =0A> > --- On Thu, 1/17/13, Adrian Chadd <adrian@freebsd.org>=0A> wr=
ote:=0A> > =0A> > > From: Adrian Chadd <adrian@freebsd.org>=0A> > > Subject=
: Re: two problems in=0A> dev/e1000/if_lem.c::lem_handle_rxtx()=0A> > > To:=
 "Barney Cordoba" <barney_cordoba@yahoo.com>=0A> > > Cc: "Luigi Rizzo" <riz=
zo@iet.unipi.it>,=0A> freebsd-net@freebsd.org=0A> > > Date: Thursday, Janua=
ry 17, 2013, 11:48 AM=0A> > > There's also the subtle race=0A> > > conditio=
n in TX and RX handling that=0A> > > re-queuing the taskqueue gets around.=
=0A> > > =0A> > > Which is:=0A> > > =0A> > > * The hardware is constantly r=
eceiving frames ,=0A> right until=0A> > > you blow=0A> > > the FIFO away by=
 filling it up;=0A> > > * The RX thread receives a bunch of frames;=0A> > >=
 * .. and processes them;=0A> > > * .. once it's done processing, the hardw=
are may=0A> have read=0A> > > some more=0A> > > frames in the meantime;=0A>=
 > > * .. and the hardware may have generated a=0A> mitigated=0A> > > inter=
rupt which=0A> > > you're ignoring, since you're processing frames;=0A> > >=
 * So if your architecture isn't 100% paranoid, you=0A> may end=0A> > > up =
having=0A> > > to wait for the next interrupt to handle what's=0A> currentl=
y in=0A> > > the=0A> > > queue.=0A> > > =0A> > > Now if things are done cor=
rect:=0A> > > =0A> > > * The hardware generates a mitigated interrupt=0A> >=
 > * The mask register has that bit disabled, so you=0A> don't end=0A> > > =
up receiving it;=0A> > > * You finish your RX queue processing, and there's=
=0A> more=0A> > > stuff that's=0A> > > appeared in the FIFO (hence why the =
hardware has=0A> generated=0A> > > another=0A> > > mitigated interrupt);=0A=
> > > * You unmask the interrupt;=0A> > > * .. and the hardware immediately=
 sends you the=0A> MSI or=0A> > > signals an interrupt;=0A> > > * .. thus y=
ou re-enter the RX processing thread=0A> almost(!)=0A> > > immediately.=0A>=
 > > =0A> > > However as the poster(s) have said, the interrupt=0A> > > mas=
k/unmask in the=0A> > > intel driver(s) may not be 100% correct, so you're=
=0A> going to=0A> > > end up=0A> > > with situations where interrupts are m=
issed.=0A> > > =0A> > > The reason why this wasn't a big deal in the=0A> de=
ep/distant=0A> > > past is=0A> > > because we didn't used to have kernel pr=
eemption,=0A> or=0A> > > multiple kernel=0A> > > threads running, or an ove=
rly aggressive scheduler=0A> trying=0A> > > to=0A> > > parallelise things a=
s much as possible. A lot of=0A> > > net80211/ath bugs=0A> > > have popped =
out of the woodwork specifically=0A> because of the=0A> > > above=0A> > > c=
hanges to the kernel. They were bugs before, but=0A> people=0A> > > didn't =
hit=0A> > > them.=0A> > > =0A> > =0A> > I don't see the distinction between=
 the rx thread=0A> getting re-scheduled=0A> > "immediately" vs introducing =
another thread. In fact=0A> you increase missed=0A> > interrupts by this me=
thod. The entire point of=0A> interrupt moderation is=0A> > to tune the int=
ervals where a driver is processed.=0A> > =0A> > You might as well just not=
 have a work limit and=0A> process until your done.=0A> > The idea that "ge=
e, I've been taking up too much cpu,=0A> I'd better yield"=0A> > to just qu=
eue a task and continue soon after doesn't=0A> make much sense to =0A> > me=
.=0A> =0A> If there are multiple threads with the same priority then=0A> ba=
tching the work up =0A> into chunks allows the scheduler to round-robin amo=
ng=0A> them.=A0 However, when a =0A> task requeues itself that doesn't actu=
ally work since the=0A> taskqueue thread =0A> will see the requeued task be=
fore it yields the CPU.=A0=0A> Alternatively, if you =0A> force all the rel=
evant interrupt handlers to use the same=0A> thread pool and =0A> instead o=
f requeueing a separate task you requeue your=0A> handler in the ithread =
=0A> pool then you can get the desired round-robin=0A> behavior.=A0 (I have=
 changes to =0A> the ithread stuff that get us part of the way there in tha=
t=0A> handlers can =0A> reschedule themselves and much of the plumbing is i=
n place=0A> for shared thread =0A> pools among different interrupts.)=0A=0A=
I dont see any "round robin" effect here. You have:=0A=0ARepeat:=0A- Proces=
s 100 frames=0Aif (more)=0A- Queue a Task=0A=0Athere's only 1 task at a tim=
e. All its really doing is yielding and=0Arescheduling itself to resume the=
 loop.=0A=0ABC



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1358610450.75691.YahooMailClassic>