Date: Mon, 20 Dec 2010 13:21:35 +0200 From: Shtorm <admin@shtorm.com> To: Eugene Grosbein <egrosbein@rdtc.ru> Cc: net@freebsd.org Subject: Re: lagg/lacp poor traffic distribution Message-ID: <1292844095.1917.136.camel@stormi> In-Reply-To: <4D0CFEFF.3000902@rdtc.ru> References: <4D0CFEFF.3000902@rdtc.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 2010-12-19 at 00:35 +0600, Eugene Grosbein wrote: > Hi! > > I've loaded router using two lagg interfaces in LACP mode. > lagg0 has IP address and two ports (em0 and em1) and carry untagged frames. > lagg1 has no IP address and has two ports (igb0 and igb1) and carry > about 1000 dot-q vlans with lots of hosts in each vlan. > > For lagg1, lagg distributes outgoing traffic over two ports just fine. > For lagg0 (untagged ethernet segment with only 2 MAC addresses) > less than 0.07% (54Mbit/s max) of traffic goes to em0 > and over 99.92% goes to em1, that's bad. > > That's general traffic of several thousands of customers surfing the web, > using torrents etc. I've glanced over lagg/lacp sources if src/sys/net/ > and found nothing suspicious, it should extract and use srcIP/dstIP for hash. > > How do I debug this problem? > > Eugene Grosbein > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" I had this problem with igb driver, and I found, that lagg selects outgoing interface based on packet header flowid field if M_FLOWID field is set. And in the igb driver code flowid is set as #if __FreeBSD_version >= 800000 <------><------><------>rxr->fmp->m_pkthdr.flowid = que->msix; <------><------><------>rxr->fmp->m_flags |= M_FLOWID; #endif The same thing in em driver with MULTIQUEUE That does not give enough number of flows to balance traffic well, so I commented check in if_lagg.c lagg_lb_start(struct lagg_softc *sc, struct mbuf *m) { <------>struct lagg_lb *lb = (struct lagg_lb *)sc->sc_psc; <------>struct lagg_port *lp = NULL; <------>uint32_t p = 0; //<---->if (m->m_flags & M_FLOWID) //<----><------>p = m->m_pkthdr.flowid; //<---->else and with this change I have much better load distribution across interfaces. Hope it helps.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1292844095.1917.136.camel>