Date: Wed, 26 Jul 2006 06:35:58 -0500 From: Eric Anderson <anderson@centtech.com> To: Murat Balaban <murat@enderunix.org> Cc: freebsd-hackers@freebsd.org Subject: Re: sys/dev/em/if_em.c Message-ID: <44C7539E.5060804@centtech.com> In-Reply-To: <20060726111657.GA22358@enderunix.org> References: <20060726111657.GA22358@enderunix.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 07/26/06 06:16, Murat Balaban wrote:
> Hello hackers,
>
> I have a special-purpose setting where I have a ng_hub like kernel module (ng_lb)
> which I've been coding. The box I'm using has two em(4) adapters, and I've
> hooked em0's lower with my ng_lb's link0, and em1's lower with ng_lb's link1.
>
> Situation looks like this:
>
> lower link0 link1 lower
> em0 ---------------> -------------> ng_lb --------------> ---------------> em1
>
> Every packet that is received by em0 is handed over to my netgraph module and
> after very little modification in the packet ethernet header (changing destination
> mac addresss) I NG_FWD_ITEM() the packet to em1.
>
> I'm generating traffic with a packet generator, and em0 seems to be ok with around
> 910 Mbit/s traffic.
>
> However if I write the packets into em1, em1 seems to drop 40-60 Mbit/s (of 910 Mbit/s)
> data. I digged the problem a bit, and found out that, IFQ_HANDOFF, called deep inside
> from NG_FWD_ITEM was returning ENOBUFS.
>
> A little more investigation proved me that the source of ENOBUFS error was that
> the em1 was running out of Tx descriptors. The relative logic in dev/em/if_em.c
> (em_encap) was that if # of Tx descriptors falls below a threshold, the driver
> tries to clean transmit interrupts once. # of available Tx desc. is again checked
> and if the number is still not incresed ENOBUFS error is returned.
>
> What I'd like to ask is, instead of cleaning the transmit interrupts only once,
> why not do it many times till the number of available tx descriptors increases
> to a moderate level?
>
> The following patch solved my problem, though I wanted to get your opinions about
> it.
What if adapter->num_tx_desc_avail never gets above
EM_TX_CLEANUP_THRESHOLD ? Maybe adding another check (max loop counter)
or something?
> --- if_em_murat.c Wed Jul 26 13:59:22 2006
> +++ if_em.c Wed Jul 26 14:01:11 2006
> @@ -1177,11 +1177,9 @@
> * available hits the threshold
> */
> if (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD) {
> - em_clean_transmit_interrupts(adapter);
> - if (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD) {
> - adapter->no_tx_desc_avail1++;
> - return(ENOBUFS);
> - }
> + do {
> + em_clean_transmit_interrupts(adapter);
> + while (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD);
> }
>
> /*
Eric
--
------------------------------------------------------------------------
Eric Anderson Sr. Systems Administrator Centaur Technology
Anything that works is better than anything that doesn't.
------------------------------------------------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?44C7539E.5060804>