Date: Mon, 12 Nov 2012 21:27:28 +0100 From: Andre Oppermann <oppermann@networx.ch> To: Karim <fodillemlinkarim@gmail.com> Cc: freebsd-net@freebsd.org Subject: Re: em/igb if_transmit (drbr) and ALTQ Message-ID: <50A15BB0.60102@networx.ch> In-Reply-To: <50A14AEC.2040303@gmail.com> References: <50A14AEC.2040303@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12.11.2012 20:15, Karim wrote:
> Hi all,
>
> I have been following the current discussions on igb tx/rx locking with great interest and I though
> I would point out (as it was before pointed out in kern/138392) that any driver setting if_start to
> NULL and using if_transmit (multi-queue or not) will effectively break ALTQ.
>
> The reason for this can be found in tbr_timeout() in altq_subr.c where the token bucket regulator
> goes through all ALTQ registered interface and tries to 'kick' them into sending through if_start().
>
> tbr_timeout() :
>
> ...
>
> for (ifp = TAILQ_FIRST(&V_ifnet); ifp;
> ifp = TAILQ_NEXT(ifp, if_list)) {
> /* read from if_snd unlocked */
> if (!TBR_IS_ENABLED(&ifp->if_snd))
> continue;
> active++;
> if (!IFQ_IS_EMPTY(&ifp->if_snd) &&
> ifp->if_start != NULL) /* if_start is NULL if if_transmit is
> used in em/igb driver */
> (*ifp->if_start)(ifp);
> }
>
> ...
>
> As you can see if_start is NULL on those new multi-queue enabled drivers which has for net effect to
> 'break' ALTQ's token bucket regulator.
>
> I am writing this because I am interested in your comments on how this can be fixed properly looking
> forward. The whole range of suggestions; from 'don't compile with EM_MULTIQUEUE defined' to 'here is
> how you can make ALTQ use drbr' will help.
This whole area needs some serious re-consideration for 10.0.
Even without the if_start issue ALTQ is somewhat broken because
the DMA rings nowadays are just too deep.
See this recent message to this list for some thoughts:
http://lists.freebsd.org/pipermail/freebsd-net/2012-November/033780.html
On top of refining the stack/driver boundary I'm thinking of a
dedicated ethernet layer to consolidate all the ethernet extensions,
have a single place to go and to prevent the drivers from re-inventing
the wheel all over every time.
I'm working towards it and leaning into various drivers while trying
to bring in the hybrid interrupt/polling mode with life-lock prevention.
It'll take a few weeks until I'm ready to show a stack/ethernet/driver
prototype for discussion. Parts may surface earlier in my tcp_workqueue
svn branch.
--
Andre
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50A15BB0.60102>