From owner-freebsd-net@FreeBSD.ORG  Sat Feb 25 12:51:29 2012
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id B759A1065670;
	Sat, 25 Feb 2012 12:51:29 +0000 (UTC)
	(envelope-from egrosbein@rdtc.ru)
Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5])
	by mx1.freebsd.org (Postfix) with ESMTP id 01E648FC12;
	Sat, 25 Feb 2012 12:51:28 +0000 (UTC)
Received: from eg.sd.rdtc.ru (localhost [127.0.0.1])
	by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id q1PCpRl8089843;
	Sat, 25 Feb 2012 19:51:27 +0700 (NOVT)
	(envelope-from egrosbein@rdtc.ru)
Message-ID: <4F48D94F.6030803@rdtc.ru>
Date: Sat, 25 Feb 2012 19:51:27 +0700
From: Eugene Grosbein <egrosbein@rdtc.ru>
User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU;
	rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7
MIME-Version: 1.0
References: <4F47C55C.8060006@rdtc.ru>	<4F47D5EF.7060908@rdtc.ru>
	<CAFAOGNQvjJcKb_hhzeoN5ptgm882qimsavzN9b=GW_VgPD8mtw@mail.gmail.com>
	<4F48D14A.7090003@rdtc.ru>
In-Reply-To: <4F48D14A.7090003@rdtc.ru>
Content-Type: text/plain; charset=KOI8-R
Content-Transfer-Encoding: 8bit
Cc: Alexander Motin <mav@FreeBSD.org>, Gleb Smirnoff <glebius@FreeBSD.org>,
	Andrew Thompson <thompsa@FreeBSD.org>, "net@freebsd.org" <net@FreeBSD.org>
Subject: Re: netisr+lagg+fragments=80% packet loss
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 25 Feb 2012 12:51:29 -0000

25.02.2012 19:17, Eugene Grosbein пишет:
> 25.02.2012 02:00, Andrew Thompson пишет:
>> 2012/2/25 Eugene Grosbein <egrosbein@rdtc.ru>:
>>> 25.02.2012 00:14, Eugene Grosbein пишет:
>>>> This problem occurs only when net.isr.direct=0/net.isr.direct_force=0.
>>>> And only when lagg1 has both ports up and running. And when I use oversized pings.
>>>> At the same time, transit oversized pings go through this BRAS just fine,
>>>> no packet loss at all.
>>>
>>> Running two copies of tcpdump for igb0 and igb1 simultaneously,
>>> I see that fragments of the same ICMP echo-reply packet encapsulated within PPPoE
>>> frame always go out through different ports of lagg1. Even when they arrive to client in order,
>>> it seems this depends of switching network in between PPPoE server and client.
>>>
>>
>> If you are running a recent HEAD then you can try setting
>> net.link.lagg.0.use_flowid to zero.
> 
> It helps to set net.link.lagg.1.use_flowid=1, yes (but I use 8.2-STABLE with this sysctl added).
> Perhaps, ip_fragment() does not keep flowid for fragments it creates.

The following patch (by Alexander Chernikov) solves the problem
by preserving flowid for fragments:

--- sys/netinet/ip_output.c.orig	2012-02-25 19:19:15.000000000 +0700
+++ sys/netinet/ip_output.c	2012-02-25 19:32:00.000000000 +0700
@@ -825,6 +825,10 @@
 		}
 		m->m_pkthdr.len = mhlen + len;
 		m->m_pkthdr.rcvif = NULL;
+		if (m0->m_flags & M_FLOWID) {
+			m->m_pkthdr.flowid = m0->m_pkthdr.flowid;
+			m->m_flags |= M_FLOWID;
+		}
 #ifdef MAC
 		mac_netinet_fragment(m0, m);
 #endif