From owner-freebsd-net@FreeBSD.ORG  Tue Dec 21 07:41:27 2010
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 56761106564A
	for <net@freebsd.org>; Tue, 21 Dec 2010 07:41:27 +0000 (UTC)
	(envelope-from egrosbein@rdtc.ru)
Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [62.231.161.221])
	by mx1.freebsd.org (Postfix) with ESMTP id B50798FC1D
	for <net@freebsd.org>; Tue, 21 Dec 2010 07:41:26 +0000 (UTC)
Received: from eg.sd.rdtc.ru (localhost [127.0.0.1])
	by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id oBL7fPn6029855;
	Tue, 21 Dec 2010 13:41:25 +0600 (NOVT)
	(envelope-from egrosbein@rdtc.ru)
Message-ID: <4D105A20.306@rdtc.ru>
Date: Tue, 21 Dec 2010 13:41:20 +0600
From: Eugene Grosbein <egrosbein@rdtc.ru>
User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU;
	rv:1.9.1.10) Gecko/20100712 Thunderbird/3.0.5
MIME-Version: 1.0
To: Shtorm <admin@shtorm.com>
References: <4D0CFEFF.3000902@rdtc.ru> <1292844095.1917.136.camel@stormi>
	<4D105800.2070808@rdtc.ru>
In-Reply-To: <4D105800.2070808@rdtc.ru>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Cc: net@freebsd.org
Subject: Re: lagg/lacp poor traffic distribution
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 21 Dec 2010 07:41:27 -0000

On 21.12.2010 13:32, Eugene Grosbein wrote:

>> I had this problem with igb driver, and I found, that lagg selects
>> outgoing interface based on packet header flowid field if M_FLOWID field
>> is set. And in the igb driver code flowid is set as 
>>
>> #if __FreeBSD_version >= 800000
>> <------><------><------>rxr->fmp->m_pkthdr.flowid = que->msix;
>> <------><------><------>rxr->fmp->m_flags |= M_FLOWID;
>> #endif
>>
>> The same thing in em driver with MULTIQUEUE 
>>
>> That does not give enough number of flows to balance traffic well, so I
>> commented check in if_lagg.c
>>
>> lagg_lb_start(struct lagg_softc *sc, struct mbuf *m)
>> {
>> <------>struct lagg_lb *lb = (struct lagg_lb *)sc->sc_psc;
>> <------>struct lagg_port *lp = NULL;
>> <------>uint32_t p = 0;
>>
>> //<---->if (m->m_flags & M_FLOWID)
>> //<----><------>p = m->m_pkthdr.flowid;
>> //<---->else
>>
>> and with this change I have much better load distribution across interfaces.
>>
>> Hope it helps.
>>
> 
> I've made new sysctl net.link.lagg.use_flows to enable/disable
> this code in run time and disabled it. No change.
> 
> --- if_lagg.c.orig	2010-12-20 22:53:21.000000000 +0600
> +++ if_lagg.c	2010-12-20 22:57:50.000000000 +0600

I've just found LACP mode does not use code in place I've patched.
Instead, I should make the change to lacp_select_tx_port()
into ieee8023ad_lacp.c. Sorry for noise, I'll try again.