Date: Tue, 21 Dec 2010 15:41:56 +0100 From: Fabien Thomas <fabien.thomas@netasq.com> To: Eugene Grosbein <egrosbein@rdtc.ru> Cc: net@freebsd.org Subject: Re: lagg/lacp poor traffic distribution Message-ID: <90F64F73-F9A8-4FD6-9303-2FC0D3424751@netasq.com> In-Reply-To: <4D10B2F4.1060404@rdtc.ru> References: <4D0CFEFF.3000902@rdtc.ru> <1292844095.1917.136.camel@stormi> <4D1083D6.6010707@rdtc.ru> <84530C06-AC2E-4E2B-BFD4-693902BB0FA6@netasq.com> <4D10B2F4.1060404@rdtc.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Dec 21, 2010, at 3:00 PM, Eugene Grosbein wrote:
> On 21.12.2010 19:11, Fabien Thomas wrote:
>=20
>>>> I had this problem with igb driver, and I found, that lagg selects
>>>> outgoing interface based on packet header flowid field if M_FLOWID =
field
>>>> is set. And in the igb driver code flowid is set as=20
>>>>=20
>>>> #if __FreeBSD_version >=3D 800000
>>>> <------><------><------>rxr->fmp->m_pkthdr.flowid =3D que->msix;
>>>> <------><------><------>rxr->fmp->m_flags |=3D M_FLOWID;
>>>> #endif
>>>>=20
>>>> The same thing in em driver with MULTIQUEUE=20
>>>>=20
>>>> That does not give enough number of flows to balance traffic well, =
so I
>>>> commented check in if_lagg.c
>>>>=20
>>>> lagg_lb_start(struct lagg_softc *sc, struct mbuf *m)
>>>> {
>>>> <------>struct lagg_lb *lb =3D (struct lagg_lb *)sc->sc_psc;
>>>> <------>struct lagg_port *lp =3D NULL;
>>>> <------>uint32_t p =3D 0;
>>>>=20
>>>> //<---->if (m->m_flags & M_FLOWID)
>>>> //<----><------>p =3D m->m_pkthdr.flowid;
>>>> //<---->else
>>>>=20
>>>> and with this change I have much better load distribution across =
interfaces.
>>>>=20
>>>> Hope it helps.
>>>=20
>>> You are perfectly right. By disabling flow usage I've obtained load =
sharing
>>> close to even (final patch follows). Two questions:
>>>=20
>>> 1. Is it a bug or design problem?
>>=20
>> How many queues have you with igb? If it's one it will explain why =
the flowid is bad for load balancing with lagg.
>=20
> How do I know? I've read igb(4) manual page and found no words
vmstat -i will show the queue (intr for the queue) normally it's the =
number of CPU available.
> about queues within igb, nor I have any knowledge about them.
>=20
>> The problem is that flowid is good if the number of queue is =3D or a =
multiple of the number of lagg ports.
>=20
> Now I see, thanks.
>=20
> Eugene Grosbein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?90F64F73-F9A8-4FD6-9303-2FC0D3424751>