From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 02:18:29 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A3941EB for ; Sun, 1 Sep 2013 02:18:29 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm18-vm4.bullet.mail.ne1.yahoo.com (nm18-vm4.bullet.mail.ne1.yahoo.com [98.138.91.178]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 345702FB4 for ; Sun, 1 Sep 2013 02:18:28 +0000 (UTC) Received: from [98.138.90.56] by nm18.bullet.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 02:15:33 -0000 Received: from [98.138.88.232] by tm9.bullet.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 02:15:33 -0000 Received: from [127.0.0.1] by omp1032.mail.ne1.yahoo.com with NNFMP; 01 Sep 2013 02:15:33 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 731413.57168.bm@omp1032.mail.ne1.yahoo.com Received: (qmail 59793 invoked by uid 60001); 1 Sep 2013 02:15:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1378001733; bh=z49vOQO4Dfyl4k79zwsmy1cBF4QGkM84J0GvGYV3bNk=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=INnh61X/+YVyVzysXowwHPUUphlLaZOvdf3lfK3TFEOimvLdbDshHRB4hos9Y4t0bGmFPWkNwz0DbaVqw932QdL3SIyOstXRE8n5Z1XX3ZWehcUiAyDt/R2nnNEJcta/v2/gdqOJX8mVfsRXP7YP9m6KB5fMaJ5A/OgpPAvT5uw= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=kHcvjUBOKTpmZlP+vbvhB5tKQ6gMJ1heq501oZuqwZtueDw1ADfEAknXIO+oq6nFph4MzaMI23JAB7NMMie20lehs2ZimgJqdjJGBQPp+9rsyAnHTdCYhvHQzm7b5XTdtim423fRii5gsmCyTzH/xG8xEGoyij1/rToffUi7RDs=; X-YMail-OSG: kWC2Na4VM1k.J.4rwkrzPOQ.y1Tl1NOLnzdYn80XOhyRJAM TuBf_lFE3.WDn93U.HoZvVkmElu7MqYgF1DazgiafsbwMdi3gIvTUKgDaLw_ anmhwTijDPSy1N13khA5kIpQeeSJeUBj4qkWIxyjd8S_.MH54t_5Repf50or 0l3ctkS1wBBgESXHbKmnOn4I0o5tJGLSCJZaQwMmT7I8h8afcsmWHwye1Dva cxRCed9dy9bIwUatbsZMtBD.poFcFOBiC24_teUjGkfXZzl8wMggg.0LcOjF V6y_3sLExuJHTQiI5lvfiobksbYNkDWXoBICQyskxSzmq4jNB06dC3mrNTGg R_CEydwuL3LzBxlzqGR.hawIo1e7CEchsD2yTgnF3_3zlE0PrsQ_Axtqa535 cnRPL0EvVIo3nr6iOpC96LZs3f3_iKgkmA1KSZXD7X772gqFhsOhpwzXdaj6 tkMLDawV0_mCAGYbFjqmcMAF7eoNFb9XVgEKMhaMYqWfzjyJfRvLshJfd309 2WNmUATEpwd4LLIdEvnMboIJPkMIMJoqqYSViDEmew4ZyPV0K.ElxRnmmmIT bdPRhZ_7PmzLCr1VNjOb_BQ-- Received: from [98.203.118.124] by web121606.mail.ne1.yahoo.com via HTTP; Sat, 31 Aug 2013 19:15:33 PDT X-Rocket-MIMEInfo: 002.001, Tm8sIG5vLiBUaGUgZW50aXJlIHBvaW50IG9mIHRoZSBoYXNoIGlzIHRvIHNlcGFyYXRlIHRoZSAiY29ubmVjdGlvbnMiLiBCdXQgd2hlbiB0ZXN0aW5nIHlvdSBzaG91bGQKdXNlIHJlYWxpc3RpYyBhc3N1bXB0aW9ucy4gWW91J3JlIG5vdCBzcGxpdHRpbmcgcGFja2V0cywgc28gdGhlIGJpZyBwYWNrZXRzIHdpbGwgbWVzcyB1cCB5b3VyIGRpc3RyaWJ1dGlvbgppZiB5b3UgZG9uJ3QgZ2V0IGl0IHJpZ2h0LsKgCgpPZiBjb3Vyc2UgdGhlcmUncyBub3RoaW5nIHJlYWxseSB3cm9uZyB3aXRoIE9PTyBwYWNrZXQBMAEBAQE- X-Mailer: YahooMailWebService/0.8.156.576 References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> , <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> Message-ID: <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> Date: Sat, 31 Aug 2013 19:15:33 -0700 (PDT) From: Barney Cordoba Subject: Re: Flow ID, LACP, and igb To: "T.C. Gubatayao" , Luigi Rizzo , Alan Somers In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Jack F Vogel , "Justin T. Gibbs" , Andre Oppermann , "net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Barney Cordoba List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 02:18:29 -0000 No, no. The entire point of the hash is to separate the "connections". But = when testing you should=0Ause realistic assumptions. You're not splitting p= ackets, so the big packets will mess up your distribution=0Aif you don't ge= t it right.=A0=0A=0AOf course there's nothing really wrong with OOO packets= . We had this discussion before; lots of people=0Ahave round robin dual hom= ing without any ill effects. It's just not an issue.=0A=0A=0ABC=0A=0A______= __________________________=0A From: T.C. Gubatayao =0ATo: Barney Cordoba ; Luigi Rizzo ; Alan Somers =0ACc: Jack F Vogel ; Justin T. Gibbs ; Andre Oppermann ; "net@freebsd.org" =0ASent: Saturday, August 31, 2= 013 9:38 PM=0ASubject: RE: Flow ID, LACP, and igb=0A =0A=0AOn Sat, Aug 31, = 2013 at 8:41 AM, Barney Cordoba wrote:=0A=0A> Al= so, the *most* important thing is distribution with realistic data. The goa= l=0A> should be to use the most trivial function that gives the most balanc= ed=0A> distribution with real numbers. Faster is not better if the result i= s an=0A> unbalanced distribution.=0A=0AAgreed, with a caveat.=A0 It's criti= cal that this distribution be by "flow", so=0Athat out of order packet deli= very is minimized.=0A=0A> Many of your ports will be 80 and 53, and if you'= re going through a router=0A> your ethernets may not be very unique, so why= even bother to include them?=0A> Does getting a good distribution require = that you hash every element=0A> individually, or can you get the same distr= ibution with a faster, simpler way=0A> of creating the seed?=0A>=0A> There'= s also the other consideration of packet size. Packets on port 53 are=0A> l= ikely to be smaller than packets on port 80. What you want is equal=0A> dis= tribution PER PORT on the ports that will carry that vast majority of your= =0A> traffic.=0A=0AUnfortunately, trying to evenly distribute traffic per p= ort based on packet=0Asize will likely result in the reordering of packets,= and bandwidth wasted on=0ATCP retransmissions.=0A=0A> Or better yet, use t= he same number of queues on igb as you have LAGG ports,=0A> and use the que= ue id (or RSS) as the hash, so that your traffic is sync'd=0A> between the = ethernet adapter queues and the LAGG ports. The card has already=0A> done t= he work for you.=0A=0AIsn't this hash for selecting an outbound link?=A0 Th= e ingress adapter hash (RSS)=0Awon't help for packets originating from the = host, or for packets that may have=0Abeen translated or otherwise modified = while traversing the stack.=0A=0AT.C.=0A___________________________________= ____________=0Afreebsd-net@freebsd.org mailing list=0Ahttp://lists.freebsd.= org/mailman/listinfo/freebsd-net=0ATo unsubscribe, send any mail to "freebs= d-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Sep 1 02:27:30 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 26ECF1A9; Sun, 1 Sep 2013 02:27:30 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BDF532FFD; Sun, 1 Sep 2013 02:27:28 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id o14so2863597lbi.32 for ; Sat, 31 Aug 2013 19:27:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Ylc3iq3b1QlBvxdTgYhUgUqLOYU0kK7Dv2264U9ZG1k=; b=aHY+jNZ7CRomOgDJ0DjWOp9DZwfGq0D2aEaI1E7mBMy2aD0MFFw2z7LDjetbuFI/XL zPiq3hDfcDlZLV6jf041gpCCUHWXMWQJfRxc9+f95a63+xRvrWCfd5coqV1a4XJYT9eq +AMuYXwoaY8XUgaVb0STpTXeXu9kYDP1W1eMRAuRa6O0G5+OvRK3H4VgNagW6CmmZDx+ yJ9Yf/h/FWML3NNiO/sYC7CJyLi1Xi323yKSWMkP2dzXqKrDBVWVjbT3xYP1iuRvFg7g STqhbJxOcjFor3tl8NnRLVrJ6nXpU/d3hFl/hpkf11n6bbklV2xSv3vu1sGFr/15Topc YCEg== MIME-Version: 1.0 X-Received: by 10.152.3.42 with SMTP id 10mr15000594laz.22.1378002445948; Sat, 31 Aug 2013 19:27:25 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.165 with HTTP; Sat, 31 Aug 2013 19:27:25 -0700 (PDT) In-Reply-To: <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> References: <521BBD21.4070304@freebsd.org> <521EE8DA.3060107@freebsd.org> <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> <1378001733.36695.YahooMailNeo@web121606.mail.ne1.yahoo.com> Date: Sun, 1 Sep 2013 04:27:25 +0200 X-Google-Sender-Auth: uHUCtpiAk6eN9tIh6b_OBc3skCM Message-ID: Subject: Re: Flow ID, LACP, and igb From: Luigi Rizzo To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Andre Oppermann , Alan Somers , "net@freebsd.org" , Jack F Vogel , "Justin T. Gibbs" , "T.C. Gubatayao" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 02:27:30 -0000 On Sun, Sep 1, 2013 at 4:15 AM, Barney Cordoba wrote: > ... > [your point on testing with realistic assumptions is surely a valid one] > > Of course there's nothing really wrong with OOO packets. We had this > discussion before; lots of people > have round robin dual homing without any ill effects. It's just not an > issue. > It depends on where you are. It may not be an issue if the reordering is not large enough to trigger retransmissions, but even then it is annoying as it causes more work in the endpoint -- it prevents LRO from working, and even on the host stack it takes more work to sort where an out of order segment goes than appending an in-order one to the socket buffer. cheers luigi