Date: Wed, 26 Jul 2006 12:02:45 -0700 From: Julian Elischer <julian@elischer.org> To: Murat Balaban <murat@enderunix.org> Cc: freebsd-hackers@freebsd.org Subject: Re: sys/dev/em/if_em.c Message-ID: <44C7BC55.6010703@elischer.org> In-Reply-To: <20060726111657.GA22358@enderunix.org> References: <20060726111657.GA22358@enderunix.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Murat Balaban wrote: >Hello hackers, > >I have a special-purpose setting where I have a ng_hub like kernel module (ng_lb) >which I've been coding. The box I'm using has two em(4) adapters, and I've >hooked em0's lower with my ng_lb's link0, and em1's lower with ng_lb's link1. > >Situation looks like this: > > lower link0 link1 lower >em0 ---------------> -------------> ng_lb --------------> ---------------> em1 > >Every packet that is received by em0 is handed over to my netgraph module and >after very little modification in the packet ethernet header (changing destination >mac addresss) I NG_FWD_ITEM() the packet to em1. > > somewhat similar to what ng_bridge does. >I'm generating traffic with a packet generator, and em0 seems to be ok with around >910 Mbit/s traffic. > >However if I write the packets into em1, em1 seems to drop 40-60 Mbit/s (of 910 Mbit/s) >data. I digged the problem a bit, and found out that, IFQ_HANDOFF, called deep inside >from NG_FWD_ITEM was returning ENOBUFS. > >A little more investigation proved me that the source of ENOBUFS error was that >the em1 was running out of Tx descriptors. The relative logic in dev/em/if_em.c >(em_encap) was that if # of Tx descriptors falls below a threshold, the driver >tries to clean transmit interrupts once. # of available Tx desc. is again checked >and if the number is still not incresed ENOBUFS error is returned. > >What I'd like to ask is, instead of cleaning the transmit interrupts only once, >why not do it many times till the number of available tx descriptors increases >to a moderate level? > >The following patch solved my problem, though I wanted to get your opinions about >it. > >Cheers, >Murat >http://www.enderunix.org/murat/ > >PS: Both cards are plugged into a 64-bit 66 Mhz PCI-X bus. I've polling enabled >in both interfaces, and HZ set to 10000. > > >--- if_em_murat.c Wed Jul 26 13:59:22 2006 >+++ if_em.c Wed Jul 26 14:01:11 2006 >@@ -1177,11 +1177,9 @@ > * available hits the threshold > */ > if (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD) { >- em_clean_transmit_interrupts(adapter); >- if (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD) { >- adapter->no_tx_desc_avail1++; >- return(ENOBUFS); >- } >+ do { >+ em_clean_transmit_interrupts(adapter); >+ while (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD); > } > > > please have some limit to the number of times that the loop can be run.. maybe 20 or something. (and maybe something to note that it has hit that limit). > /* >_______________________________________________ >freebsd-hackers@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?44C7BC55.6010703>