From owner-freebsd-net@FreeBSD.ORG  Fri Jul 18 08:03:58 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id AC4BDDBE;
 Fri, 18 Jul 2014 08:03:58 +0000 (UTC)
Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com
 [IPv6:2607:f8b0:400d:c01::22a])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 5A7E826BA;
 Fri, 18 Jul 2014 08:03:58 +0000 (UTC)
Received: by mail-qc0-f170.google.com with SMTP id c9so3137223qcz.29
 for <multiple recipients>; Fri, 18 Jul 2014 01:03:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:sender:in-reply-to:references:date:message-id:subject
 :from:to:cc:content-type;
 bh=9ZLM3khD5OJzkQ2LG3bNwSSfKcCflqp89lF43c4fpbA=;
 b=NnzsvMkTB1SYl2bk37gIVrnGsQo+vRbGJvE/Cg2mdiFtJLT0/e4vPQz5wVVj3Iv0xB
 KSOVn3GQ9Fyi146+ut5L3wmWtOexUcQMDkHW9ebkD0PV6jtt8dWBvR0AdhIjq0uZ1B4Q
 r6aOBcHtqIKxPjygl8DHT3cu3FPAq2srjRrFaQES/1lb8u5p5YhkZOgkacrkLwNXda3P
 qxeiY7TTHEJIcJZjPF+ovRNo8Yjnmr0LSHqaB5j/TuvdTzDU2GV+mEtkTEQVPgkmL9Vu
 7g1gzSroxJKYRJOiCP/mey1/9VRIcZct9NZ+CaY5a577zEZGR5RnlWQYgOoR1Ejiveya
 cNuQ==
MIME-Version: 1.0
X-Received: by 10.140.38.169 with SMTP id t38mr4954791qgt.3.1405670635347;
 Fri, 18 Jul 2014 01:03:55 -0700 (PDT)
Sender: adrian.chadd@gmail.com
Received: by 10.224.202.193 with HTTP; Fri, 18 Jul 2014 01:03:55 -0700 (PDT)
In-Reply-To: <CAOfEmZhtZCettzD6pKQMHRiQE42nQmBuimOq28cA23R+Yyc13w@mail.gmail.com>
References: <CAOfEmZjmb1bdvn0gR6vD1WeP8o8g7KwXod4TE0iJfa=nicyeng@mail.gmail.com>
 <CAJ-Vmomt2QDXAVBVUk6m8oH4Pa5yErDdG6wWrP3X7+DW137xiA@mail.gmail.com>
 <CAOfEmZja8Tkv_xG8LyR5Nbj+Oga=vvdy=b3pxHqZi0-BBq25Uw@mail.gmail.com>
 <CAJ-VmomY2wP1EyVK4J16sGmMid=sJ9MPZrUY6pgcKGBDXm1T4g@mail.gmail.com>
 <CAOfEmZj5pk7bFB-PBqaJsi+bA73gbsUZzqggs4yEVky3_61NpQ@mail.gmail.com>
 <CAOfEmZhtZCettzD6pKQMHRiQE42nQmBuimOq28cA23R+Yyc13w@mail.gmail.com>
Date: Fri, 18 Jul 2014 01:03:55 -0700
X-Google-Sender-Auth: vvQfWXgrH5GdC8-IuckQYdtUeZU
Message-ID: <CAJ-VmomH6RqK92s1wO8C3w3nZTcV=qsgnOU7GFX2SxDU8uMysA@mail.gmail.com>
Subject: Re: [patch][lagg] - Set a better granularity and distribution on
 roundrobin protocol.
From: Adrian Chadd <adrian@freebsd.org>
To: araujo@freebsd.org
Content-Type: text/plain; charset=UTF-8
Cc: FreeBSD Net <freebsd-net@freebsd.org>
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 18 Jul 2014 08:03:58 -0000

Hi,

I strongly object to having a round-robin method like this. Yes, we
won't get > 1 link of bandwidth out of a single stream, but you're
showing that you can't even get that. There's still something else
weird going on.

I'm sorry, but introducing more out of order possibilities is being a
bad network citizen.



-a


On 18 July 2014 00:49, Marcelo Araujo <araujobsdport@gmail.com> wrote:
> Hello guys,
>
> I made few changes on the lagg(4) patch. Also, I made tests using igb(4),
> ixgbe(4) and em(4); seems everything worked pretty well.
>
> I'm wondering if anyone else could make a review, and what I need to do, to
> see this patch committed.
>
> Best Regards,
>
>
>
>
> 2014-06-24 10:40 GMT+08:00 Marcelo Araujo <araujobsdport@gmail.com>:
>
>>
>>
>> 2014-06-24 6:54 GMT+08:00 Adrian Chadd <adrian@freebsd.org>:
>>
>>> Hi,
>>>
>>> No, don't introduce out of order behaviour. Ever.
>>
>>
>> Yes, it has out of order behavior; with my patch much less. I upload two
>> pcap files and you can see by yourself, if you don't believe in what I'm
>> talking about.
>>
>> Test done using: "iperf -s" and "iperf -c <ip> -i 1 -t 10".
>>
>> 1) Don't change the number of packets(default round robin behavior).
>> http://people.freebsd.org/~araujo/lagg/lagg-nop.cap
>> 8 out of order packets.
>> Several SACKs.
>>
>> 2) Set the number of packets to 50.
>> http://people.freebsd.org/~araujo/lagg/lagg.cap
>> 0 out of order packets.
>> Less SACKs.
>>
>>>
>>> You may not think
>>> it's a problem for TCP, but UDP things and VPN things will start
>>> getting very angry. There are VPN configurations out there that will
>>> drop the VPN if frames are out of order.
>>
>>
>> I'm not thinking that will be a problem for TCP, but, in somehow it will
>> be, less throughput as I showed before, and less SACK. About the VPN,
>> please, tell me which softwares, and let me know where I can get a sample to
>> make a testbed.
>>
>> However to be very honest, I don't believe anyone here when change
>> something at network protocols will make this extensive testbed. It is
>> almost impossible to predict what software it will works or not, and I don't
>> believe anyone here has all these stuff in hands.
>>
>>>
>>>
>>> The ixgbe driver is setting the flowid to the msix queue ID, rather
>>> than a 32 bit unique flow id hash value for the flow. That makes it
>>> hard to do traffic distribution where the flowid is available.
>>
>>
>> Thanks for the explanation.
>>
>>>
>>>
>>> There's an lagg option to re-hash the mbuf rather than rely on the
>>> flowid for outbound port choice - have you looked at using that? Did
>>> that make any difference?
>>
>>
>> Yes, I set to 0 the net.link.lagg.0.use _flowid, it make a little
>> difference to the default round robin implementation, but yet I can't reach
>> more than 5 Gbit/s. With my patch and set the packets to 50, it improved a
>> bit too.
>>
>> So, thank you so much for all review, I don't know if you have time and a
>> testbed to make a real test, as I'm doing. I would be happy if you or more
>> people could make tests on that patch. Also, I have only ixgbe(4) to make
>> tests, would appreciate if this patch could be tested with other NICs too.
>>
>> Best Regards,
>>
>> --
>> Marcelo Araujo            (__)
>> araujo@FreeBSD.org     \\\'',)
>> http://www.FreeBSD.org   \/  \ ^
>> Power To Server.         .\. /_)
>
>
>
>
> --
>
> --
> Marcelo Araujo            (__)
> araujo@FreeBSD.org     \\\'',)
> http://www.FreeBSD.org   \/  \ ^
> Power To Server.         .\. /_)