From owner-freebsd-net@FreeBSD.ORG Fri Jul 18 08:03:58 2014 Return-Path: <owner-freebsd-net@FreeBSD.ORG> Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC4BDDBE; Fri, 18 Jul 2014 08:03:58 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A7E826BA; Fri, 18 Jul 2014 08:03:58 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id c9so3137223qcz.29 for <multiple recipients>; Fri, 18 Jul 2014 01:03:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9ZLM3khD5OJzkQ2LG3bNwSSfKcCflqp89lF43c4fpbA=; b=NnzsvMkTB1SYl2bk37gIVrnGsQo+vRbGJvE/Cg2mdiFtJLT0/e4vPQz5wVVj3Iv0xB KSOVn3GQ9Fyi146+ut5L3wmWtOexUcQMDkHW9ebkD0PV6jtt8dWBvR0AdhIjq0uZ1B4Q r6aOBcHtqIKxPjygl8DHT3cu3FPAq2srjRrFaQES/1lb8u5p5YhkZOgkacrkLwNXda3P qxeiY7TTHEJIcJZjPF+ovRNo8Yjnmr0LSHqaB5j/TuvdTzDU2GV+mEtkTEQVPgkmL9Vu 7g1gzSroxJKYRJOiCP/mey1/9VRIcZct9NZ+CaY5a577zEZGR5RnlWQYgOoR1Ejiveya cNuQ== MIME-Version: 1.0 X-Received: by 10.140.38.169 with SMTP id t38mr4954791qgt.3.1405670635347; Fri, 18 Jul 2014 01:03:55 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Fri, 18 Jul 2014 01:03:55 -0700 (PDT) In-Reply-To: <CAOfEmZhtZCettzD6pKQMHRiQE42nQmBuimOq28cA23R+Yyc13w@mail.gmail.com> References: <CAOfEmZjmb1bdvn0gR6vD1WeP8o8g7KwXod4TE0iJfa=nicyeng@mail.gmail.com> <CAJ-Vmomt2QDXAVBVUk6m8oH4Pa5yErDdG6wWrP3X7+DW137xiA@mail.gmail.com> <CAOfEmZja8Tkv_xG8LyR5Nbj+Oga=vvdy=b3pxHqZi0-BBq25Uw@mail.gmail.com> <CAJ-VmomY2wP1EyVK4J16sGmMid=sJ9MPZrUY6pgcKGBDXm1T4g@mail.gmail.com> <CAOfEmZj5pk7bFB-PBqaJsi+bA73gbsUZzqggs4yEVky3_61NpQ@mail.gmail.com> <CAOfEmZhtZCettzD6pKQMHRiQE42nQmBuimOq28cA23R+Yyc13w@mail.gmail.com> Date: Fri, 18 Jul 2014 01:03:55 -0700 X-Google-Sender-Auth: vvQfWXgrH5GdC8-IuckQYdtUeZU Message-ID: <CAJ-VmomH6RqK92s1wO8C3w3nZTcV=qsgnOU7GFX2SxDU8uMysA@mail.gmail.com> Subject: Re: [patch][lagg] - Set a better granularity and distribution on roundrobin protocol. From: Adrian Chadd <adrian@freebsd.org> To: araujo@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net <freebsd-net@freebsd.org> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/> List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Fri, 18 Jul 2014 08:03:58 -0000 Hi, I strongly object to having a round-robin method like this. Yes, we won't get > 1 link of bandwidth out of a single stream, but you're showing that you can't even get that. There's still something else weird going on. I'm sorry, but introducing more out of order possibilities is being a bad network citizen. -a On 18 July 2014 00:49, Marcelo Araujo <araujobsdport@gmail.com> wrote: > Hello guys, > > I made few changes on the lagg(4) patch. Also, I made tests using igb(4), > ixgbe(4) and em(4); seems everything worked pretty well. > > I'm wondering if anyone else could make a review, and what I need to do, to > see this patch committed. > > Best Regards, > > > > > 2014-06-24 10:40 GMT+08:00 Marcelo Araujo <araujobsdport@gmail.com>: > >> >> >> 2014-06-24 6:54 GMT+08:00 Adrian Chadd <adrian@freebsd.org>: >> >>> Hi, >>> >>> No, don't introduce out of order behaviour. Ever. >> >> >> Yes, it has out of order behavior; with my patch much less. I upload two >> pcap files and you can see by yourself, if you don't believe in what I'm >> talking about. >> >> Test done using: "iperf -s" and "iperf -c <ip> -i 1 -t 10". >> >> 1) Don't change the number of packets(default round robin behavior). >> http://people.freebsd.org/~araujo/lagg/lagg-nop.cap >> 8 out of order packets. >> Several SACKs. >> >> 2) Set the number of packets to 50. >> http://people.freebsd.org/~araujo/lagg/lagg.cap >> 0 out of order packets. >> Less SACKs. >> >>> >>> You may not think >>> it's a problem for TCP, but UDP things and VPN things will start >>> getting very angry. There are VPN configurations out there that will >>> drop the VPN if frames are out of order. >> >> >> I'm not thinking that will be a problem for TCP, but, in somehow it will >> be, less throughput as I showed before, and less SACK. About the VPN, >> please, tell me which softwares, and let me know where I can get a sample to >> make a testbed. >> >> However to be very honest, I don't believe anyone here when change >> something at network protocols will make this extensive testbed. It is >> almost impossible to predict what software it will works or not, and I don't >> believe anyone here has all these stuff in hands. >> >>> >>> >>> The ixgbe driver is setting the flowid to the msix queue ID, rather >>> than a 32 bit unique flow id hash value for the flow. That makes it >>> hard to do traffic distribution where the flowid is available. >> >> >> Thanks for the explanation. >> >>> >>> >>> There's an lagg option to re-hash the mbuf rather than rely on the >>> flowid for outbound port choice - have you looked at using that? Did >>> that make any difference? >> >> >> Yes, I set to 0 the net.link.lagg.0.use _flowid, it make a little >> difference to the default round robin implementation, but yet I can't reach >> more than 5 Gbit/s. With my patch and set the packets to 50, it improved a >> bit too. >> >> So, thank you so much for all review, I don't know if you have time and a >> testbed to make a real test, as I'm doing. I would be happy if you or more >> people could make tests on that patch. Also, I have only ixgbe(4) to make >> tests, would appreciate if this patch could be tested with other NICs too. >> >> Best Regards, >> >> -- >> Marcelo Araujo (__) >> araujo@FreeBSD.org \\\'',) >> http://www.FreeBSD.org \/ \ ^ >> Power To Server. .\. /_) > > > > > -- > > -- > Marcelo Araujo (__) > araujo@FreeBSD.org \\\'',) > http://www.FreeBSD.org \/ \ ^ > Power To Server. .\. /_)