Date: Fri, 18 Jul 2014 15:49:10 +0800 From: Marcelo Araujo <araujobsdport@gmail.com> To: Adrian Chadd <adrian@freebsd.org> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: [patch][lagg] - Set a better granularity and distribution on roundrobin protocol. Message-ID: <CAOfEmZhtZCettzD6pKQMHRiQE42nQmBuimOq28cA23R%2BYyc13w@mail.gmail.com> In-Reply-To: <CAOfEmZj5pk7bFB-PBqaJsi%2BbA73gbsUZzqggs4yEVky3_61NpQ@mail.gmail.com> References: <CAOfEmZjmb1bdvn0gR6vD1WeP8o8g7KwXod4TE0iJfa=nicyeng@mail.gmail.com> <CAJ-Vmomt2QDXAVBVUk6m8oH4Pa5yErDdG6wWrP3X7%2BDW137xiA@mail.gmail.com> <CAOfEmZja8Tkv_xG8LyR5Nbj%2BOga=vvdy=b3pxHqZi0-BBq25Uw@mail.gmail.com> <CAJ-VmomY2wP1EyVK4J16sGmMid=sJ9MPZrUY6pgcKGBDXm1T4g@mail.gmail.com> <CAOfEmZj5pk7bFB-PBqaJsi%2BbA73gbsUZzqggs4yEVky3_61NpQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--f46d043c7e0c28456204fe72fed0 Content-Type: text/plain; charset=UTF-8 Hello guys, I made few changes on the lagg(4) patch. Also, I made tests using igb(4), ixgbe(4) and em(4); seems everything worked pretty well. I'm wondering if anyone else could make a review, and what I need to do, to see this patch committed. Best Regards, 2014-06-24 10:40 GMT+08:00 Marcelo Araujo <araujobsdport@gmail.com>: > > > 2014-06-24 6:54 GMT+08:00 Adrian Chadd <adrian@freebsd.org>: > > Hi, >> >> No, don't introduce out of order behaviour. Ever. > > > Yes, it has out of order behavior; with my patch much less. I upload two > pcap files and you can see by yourself, if you don't believe in what I'm > talking about. > > Test done using: "iperf -s" and "iperf -c <ip> -i 1 -t 10". > > 1) Don't change the number of packets(default round robin behavior). > http://people.freebsd.org/~araujo/lagg/lagg-nop.cap > 8 out of order packets. > Several SACKs. > > 2) Set the number of packets to 50. > http://people.freebsd.org/~araujo/lagg/lagg.cap > 0 out of order packets. > Less SACKs. > > >> You may not think >> it's a problem for TCP, but UDP things and VPN things will start >> getting very angry. There are VPN configurations out there that will >> drop the VPN if frames are out of order. >> > > I'm not thinking that will be a problem for TCP, but, in somehow it will > be, less throughput as I showed before, and less SACK. About the VPN, > please, tell me which softwares, and let me know where I can get a sample > to make a testbed. > > However to be very honest, I don't believe anyone here when change > something at network protocols will make this extensive testbed. It is > almost impossible to predict what software it will works or not, and I > don't believe anyone here has all these stuff in hands. > > >> >> The ixgbe driver is setting the flowid to the msix queue ID, rather >> than a 32 bit unique flow id hash value for the flow. That makes it >> hard to do traffic distribution where the flowid is available. >> > > Thanks for the explanation. > > >> >> There's an lagg option to re-hash the mbuf rather than rely on the >> flowid for outbound port choice - have you looked at using that? Did >> that make any difference? >> > > Yes, I set to 0 the net.link.lagg.0.use _flowid, it make a little > difference to the default round robin implementation, but yet I can't reach > more than 5 Gbit/s. With my patch and set the packets to 50, it improved a > bit too. > > So, thank you so much for all review, I don't know if you have time and a > testbed to make a real test, as I'm doing. I would be happy if you or more > people could make tests on that patch. Also, I have only ixgbe(4) to make > tests, would appreciate if this patch could be tested with other NICs too. > > Best Regards, > > -- > Marcelo Araujo (__) > araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org <http://www.freebsd.org/>; \/ \ ^ > Power To Server. .\. /_) > > -- -- Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org <http://www.freebsd.org/>; \/ \ ^ Power To Server. .\. /_) --f46d043c7e0c28456204fe72fed0 Content-Type: application/octet-stream; name="if_lagg-rr.patch" Content-Disposition: attachment; filename="if_lagg-rr.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hxr7fodu0 SW5kZXg6IGlmX2xhZ2cuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBpZl9sYWdnLmMJKHJldmlzaW9uIDI2ODgz MikKKysrIGlmX2xhZ2cuYwkod29ya2luZyBjb3B5KQpAQCAtMTg3LDYgKzE4NywxMCBAQAogU1lT Q1RMX0lOVChfbmV0X2xpbmtfbGFnZywgT0lEX0FVVE8sIGRlZmF1bHRfZmxvd2lkX3NoaWZ0LCBD VExGTEFHX1JXVFVOLAogICAgICZkZWZfZmxvd2lkX3NoaWZ0LCAwLAogICAgICJEZWZhdWx0IHNl dHRpbmcgZm9yIGZsb3dpZCBzaGlmdCBmb3IgbG9hZCBzaGFyaW5nIik7CitzdGF0aWMgaW50IGxh Z2dfcnJfcGFja2V0cyA9IDA7IC8qIERlZmF1bHQgdmFsdWUgZm9yIHVzaW5nIHJyX3BhY2tldHMg Ki8KK1NZU0NUTF9JTlQoX25ldF9saW5rX2xhZ2csIE9JRF9BVVRPLCBycl9wYWNrZXRzLCBDVExG TEFHX1JXLAorICAgICZsYWdnX3JyX3BhY2tldHMsIDAsCisgICAgIkhvdyBtYW55IHBhY2tldHMg dG8gYmUgc2VuZCBwZXIgaW50ZXJmYWNlIik7CiAKIHN0YXRpYyBpbnQKIGxhZ2dfbW9kZXZlbnQo bW9kdWxlX3QgbW9kLCBpbnQgdHlwZSwgdm9pZCAqZGF0YSkKQEAgLTE2ODcsMTQgKzE2OTEsNzMg QEAKIHsKIAlzdHJ1Y3QgbGFnZ19wb3J0ICpscDsKIAl1aW50MzJfdCBwOworCXVpbnQzMl90IHAy OworCXVpbnQzMl90IHBrdF9zeXNjdGxfY291bnQ7CisJaW50IGlmcF9jb3VudCA9IDE7CiAKIAlw ID0gYXRvbWljX2ZldGNoYWRkXzMyKCZzYy0+c2Nfc2VxLCAxKTsKIAlwICU9IHNjLT5zY19jb3Vu dDsKKworCXAyID0gYXRvbWljX2ZldGNoYWRkXzMyKCZzYy0+c2Nfc2VxLCAxKTsKKwlwMiAlPSBz Yy0+c2NfY291bnQ7CisKIAlscCA9IFNMSVNUX0ZJUlNUKCZzYy0+c2NfcG9ydHMpOwotCXdoaWxl IChwLS0pCi0JCWxwID0gU0xJU1RfTkVYVChscCwgbHBfZW50cmllcyk7CiAKIAkvKgorCSAqIElm IHRoZXJlIGlzIG5vIHJlZmVyZW5jZSBmb3IgdGhlIElGUCwgd2UgbXVzdAorIAkgKiBjb3B5IGl0 IG5vdy4KKwkgKi8KKwlpZiAoc3RybGVuKHNjLT5zY19yZWZfaWZwKSA9PSAwKQorCQlzdHJuY3B5 KHNjLT5zY19yZWZfaWZwLCBscC0+bHBfaWZwLT5pZl94bmFtZSwgc2l6ZW9mKHNjLT5zY19yZWZf aWZwKSk7CisgICAgICAgICAgICAgIAorCS8qCisJICogSWYgaWZwX2NvdW50IHdhcyBub3QgeWV0 IGluaXRpYWxpemVkLCB3ZSBtdXN0CisJICogaW5pdGlhbGl6ZSBub3cuCisJICovCisJaWYgKHNj LT5zY19pZnBfY291bnQgPT0gMCkKKwkJc2MtPnNjX2lmcF9jb3VudCA9IDE7CisKKwkvKgorCSAq IElmIHRoZSBzeXNjdGwgcnJfcGFja2V0cyBpcyBzZXQgdG8gMCwgd2UgbXVzdCB1c2UgdGhlCisJ ICogcm91bmRyb2JpbiBhcyBpdCBpcywgb3Igb3RoZXJ3aXNlLCB3ZSBtdXN0IGFwcGx5IHRoZQor CSAqIGdyYW51bGFyaXR5IGJldHdlZW4gdGhlIGludGVyZmFjZXMgdGhhdCBhcmUgcGFydCBvZiB0 aGUgZ3JvdXAuCisJICovCisJaWYgKCFsYWdnX3JyX3BhY2tldHMpIHsKKwkJd2hpbGUgKHAtLSkK KwkJCWxwID0gU0xJU1RfTkVYVChscCwgbHBfZW50cmllcyk7CisJCWdvdG8gc2VuZF9tYnVmOwor CX0gZWxzZSB7CisJCXBrdF9zeXNjdGxfY291bnQgPSBhdG9taWNfZmV0Y2hhZGRfMzIoJnNjLT5z Y19wa3RfY291bnQsIDEpOworCQlpZiAocGt0X3N5c2N0bF9jb3VudCA9PSBsYWdnX3JyX3BhY2tl dHMpIHsKKwkJCWlmIChzYy0+c2NfaWZwX2NvdW50IDw9IHNjLT5zY19jb3VudCkgeworCQkJCXdo aWxlIChpZnBfY291bnQgPCBzYy0+c2NfaWZwX2NvdW50KSB7CisJCQkJCWxwID0gU0xJU1RfTkVY VChscCwgbHBfZW50cmllcyk7CisJCQkJCWlmcF9jb3VudCsrOworCQkJCX0KKwkJCQlzYy0+c2Nf aWZwX2NvdW50Kys7CisJCQkJaWYgKHNjLT5zY19pZnBfY291bnQgPiBzYy0+c2NfY291bnQpCisJ CQkJCXNjLT5zY19pZnBfY291bnQgPSAwOworCQkJfQorCQkJc3RybmNweShzYy0+c2NfcmVmX2lm cCwgbHAtPmxwX2lmcC0+aWZfeG5hbWUsIHNpemVvZihzYy0+c2NfcmVmX2lmcCkpOworCQkJc2Mt PnNjX3BrdF9jb3VudCA9IDA7CisJCX0KKwl9CisKKwkvKgorCSAqIENoZWNrIGlmIHRoZSBjdXJy ZW50IGludGVyZmFjZSB0byBiZSBlbnF1ZXVlIGlzIG5vdCB0aGUKKwkgKiBzYW1lIHVzZWQgaW4g dGhlIGxhc3Qgcm91bmQuCisJICovCisJbHAgPSBTTElTVF9GSVJTVCgmc2MtPnNjX3BvcnRzKTsK Kwl3aGlsZSAocDItLSkgeworCQlpZiAoc3RyY21wKGxwLT5scF9pZnAtPmlmX3huYW1lLCBzYy0+ c2NfcmVmX2lmcCkgPT0gMCkKKwkJCWJyZWFrOworCQllbHNlCisJCQlscCA9IFNMSVNUX05FWFQo bHAsIGxwX2VudHJpZXMpOworCX0KKwlnb3RvIHNlbmRfbWJ1ZjsKKworc2VuZF9tYnVmOgorCS8q CiAJICogQ2hlY2sgdGhlIHBvcnQncyBsaW5rIHN0YXRlLiBUaGlzIHdpbGwgcmV0dXJuIHRoZSBu ZXh0IGFjdGl2ZQogCSAqIHBvcnQgaWYgdGhlIGxpbmsgaXMgZG93biBvciB0aGUgcG9ydCBpcyBO VUxMLgogCSAqLwpJbmRleDogaWZfbGFnZy5oCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGlmX2xhZ2cuaAkocmV2 aXNpb24gMjY4ODMyKQorKysgaWZfbGFnZy5oCSh3b3JraW5nIGNvcHkpCkBAIC0yMzIsNiArMjMy LDkgQEAKIAlzdHJ1Y3Qgc3lzY3RsX29pZAkJKnNjX29pZDsJLyogc3lzY3RsIHRyZWUgb2lkICov CiAJaW50CQkJCXVzZV9mbG93aWQ7CS8qIHVzZSBNX0ZMT1dJRCAqLwogCWludAkJCQlmbG93aWRf c2hpZnQ7CS8qIHNoaWZ0IHRoZSBmbG93aWQgKi8KKwl1aW50MzJfdAkJCXNjX3BrdF9jb3VudDsg LyogdXNlIGZvciBjb3VudCBwYWNrYXRlcyBwZXIgaWZwICovCisJaW50CQkJCXNjX2lmcF9jb3Vu dDsgLyogY291bnRlciByZWZlcmVuY2Ugb2YgaW50ZXJmYWNlcyBvbiByciAqLworCWNoYXIJCQkJ c2NfcmVmX2lmcFtJRk5BTVNJWl07IC8qIG5hbWUgb2YgdGhlIGlmcCAqLwogfTsKIAogc3RydWN0 IGxhZ2dfcG9ydCB7Cg== --f46d043c7e0c28456204fe72fed0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOfEmZhtZCettzD6pKQMHRiQE42nQmBuimOq28cA23R%2BYyc13w>