From owner-freebsd-net@FreeBSD.ORG Tue Sep 23 00:28:04 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1691338D for ; Tue, 23 Sep 2014 00:28:04 +0000 (UTC) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1991E0F for ; Tue, 23 Sep 2014 00:28:03 +0000 (UTC) Received: by mail-wg0-f43.google.com with SMTP id y10so3790089wgg.2 for ; Mon, 22 Sep 2014 17:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=blDz+QPOzIEOKX/v1lUT4CqKmju6wB37nUFX4PY7ZL0=; b=GYYBia97DzG4+5A8fmGORrvmBS+4jqD6zK9VhKado0nOoiIZOQEmtYYIl+8305Ea8Z KAxpD6nh+4XeW4vcS5Y3VLZ71gOVC/Jowy2J337rhAvPhR2tX1bsbYyl4yQgboOOZ7ZF NtdhfzwMuu+DhwefIWDF4Jpgo5+N1mtHDGEXlUeymzGUjJxZo+enj4Avoxz9pURIRajV zXzfjQDD5J1NhOlgYBps6gwMiMAtLnUKdfNEeT2adOdu7FOC69bzYbjsCv/6GQkQuvQv 3poh/bQEnxVCFIY09vX4RdzIrLK2OLYqiJjX2Bp2pMTBnJO1DPdIR/bk9s525U4SqxF+ c9cw== MIME-Version: 1.0 X-Received: by 10.180.97.98 with SMTP id dz2mr17192684wib.26.1411432081925; Mon, 22 Sep 2014 17:28:01 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.106.199 with HTTP; Mon, 22 Sep 2014 17:28:01 -0700 (PDT) In-Reply-To: <54200A0E-6337-40E1-B5DE-DC42F9CE8CCC@netgate.com> References: <54200A0E-6337-40E1-B5DE-DC42F9CE8CCC@netgate.com> Date: Mon, 22 Sep 2014 17:28:01 -0700 X-Google-Sender-Auth: y9bYq7sjdMZZP-T5-tAyTZlv9qc Message-ID: Subject: Re: How do I balance bandwidth over several virtual NICs? From: Adrian Chadd To: Jim Thompson Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-net@freebsd.org" , Elof Ofel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2014 00:28:04 -0000 On 22 September 2014 15:35, Jim Thompson wrote: > >> On Sep 22, 2014, at 5:15 PM, Adrian Chadd wrote: >> >> On 22 September 2014 13:39, Elof Ofel wrote: >>> Hi Adrian! >>> >>> Now this sounds promising! All my sensors use the ixgbe driver. >>> However, my skills in programming/compiling isn't vast. I know how to p= atch >>> and use poudriere. That's about it. >>> >>> I must admit I don't really understand what you mean with "patch it to = use a >>> symmetric RSS key", but it sounds like the functionality I'm looking fo= r is >>> not yet there in the driver. >> >> A few years ago a couple of researchers figured out you could abuse >> the toeplitz hash to do symmetric RSS hashing: >> >> http://www.ndsl.kaist.edu/~shinae/papers/TR-symRSS.pdf >> >> This means that the same RSS hash value is chosen no matter which >> direction the traffic is going on. >> This is what you need to ensure that the packets going both directions >> in a connection end up in the same NIC hardware receive ring. >> >> So, all you have to do (!) is: >> >> * grab freebsd-head, because the ixgbe driver there has some recent >> bug fixes that you need for this to completely work; >> * look at ixgbe_initialise_rss_mapping() - that's where the RSS key, >> mapping and RSS hash types are configured; >> * patch it to use the example symmetric RSS key that was provided in the= paper; > > possible that XOR-ing the values to be hashed will produce a similar resu= lt. Of course, this is software, not hardware generation of Toeplitz. > >> * patch it to only hash on IPv4 / IPv6 2-tuple, that way you don't end >> up with IPv4/IPv6 fragments in the wrong queue; > > I hope these two aren=E2=80=99t embedded in the code. The coming Intel d= evices both support the symmetric RSS key and > will correctly hash on ipv4/ipv4 4 tuple. > See section 7.1.10.1 and 7.1.10.3 in > http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xl= 710-10-40-controller-datasheet.pdf I think it'll still do the same thing - it'll not be able to 4-tuple IPv4 frames that are fragmented, as the non-first frame doesn't have the TCP/UDP information. (The support for symmetric RSS by XORing the source/destination bits is very nice - we'll have to teach freebsd's RSS code about that at some point.) -a