From owner-freebsd-net@FreeBSD.ORG Mon Sep 22 22:41:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 424FF4AF for ; Mon, 22 Sep 2014 22:41:51 +0000 (UTC) Received: from mail-oa0-f52.google.com (mail-oa0-f52.google.com [209.85.219.52]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 028A52DD for ; Mon, 22 Sep 2014 22:41:50 +0000 (UTC) Received: by mail-oa0-f52.google.com with SMTP id o6so2173016oag.39 for ; Mon, 22 Sep 2014 15:41:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=0Z7hk5j6TQTbbetnAdsf9hRQeyE2XPpQRRNeT1TfKOs=; b=VX44a/7ml2PGdsD9tVQxhaLoVyMHQw5iqZXg9i4cbtswsqEzasOGLOOpYHftmdhsXK 3IV0eRw+U4bTtx5eA5H7rIUGSBJV0z4TjuY/Ph7UHDfwUpsYJKNYZKYDiuYReY7vJZ9s rM2HF2WC23d099YaEt8/Hhor+fTGpuRcqxLlZv6H6ucIfvQ47jei8gjFtPbBhkErO/lX 1+jYdEMRNvR5moZqF64jlFUBxDiYkGyPyCANaSqedUl7dF1D6sjtZhTblyCiEOmZWkiK M2E0cfg1S1osp20RLYzdgkM5obuHa5o1BcL6uAutYdrgNAOtstArJdsI/77dAnAspFCv Zckw== X-Gm-Message-State: ALoCoQkdxEoAUtquRjPGuE2JBX6pd3cWIEndtVoAvq6KWFQUhg2l9uU4bbKimi1Gsxx8IeSzyf7d X-Received: by 10.182.111.229 with SMTP id il5mr29433932obb.3.1411425312021; Mon, 22 Sep 2014 15:35:12 -0700 (PDT) Received: from ?IPv6:2610:160:11:33:5127:735b:4e0e:1c05? ([2610:160:11:33:5127:735b:4e0e:1c05]) by mx.google.com with ESMTPSA id j16sm6928866obe.5.2014.09.22.15.35.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 22 Sep 2014 15:35:11 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1985.3\)) Subject: Re: How do I balance bandwidth over several virtual NICs? From: Jim Thompson In-Reply-To: Date: Mon, 22 Sep 2014 17:35:10 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <54200A0E-6337-40E1-B5DE-DC42F9CE8CCC@netgate.com> References: To: Adrian Chadd X-Mailer: Apple Mail (2.1985.3) Cc: "freebsd-net@freebsd.org" , Elof Ofel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2014 22:41:51 -0000 > On Sep 22, 2014, at 5:15 PM, Adrian Chadd wrote: >=20 > On 22 September 2014 13:39, Elof Ofel wrote: >> Hi Adrian! >>=20 >> Now this sounds promising! All my sensors use the ixgbe driver. >> However, my skills in programming/compiling isn't vast. I know how to = patch >> and use poudriere. That's about it. >>=20 >> I must admit I don't really understand what you mean with "patch it = to use a >> symmetric RSS key", but it sounds like the functionality I'm looking = for is >> not yet there in the driver. >=20 > A few years ago a couple of researchers figured out you could abuse > the toeplitz hash to do symmetric RSS hashing: >=20 > http://www.ndsl.kaist.edu/~shinae/papers/TR-symRSS.pdf >=20 > This means that the same RSS hash value is chosen no matter which > direction the traffic is going on. > This is what you need to ensure that the packets going both directions > in a connection end up in the same NIC hardware receive ring. >=20 > So, all you have to do (!) is: >=20 > * grab freebsd-head, because the ixgbe driver there has some recent > bug fixes that you need for this to completely work; > * look at ixgbe_initialise_rss_mapping() - that's where the RSS key, > mapping and RSS hash types are configured; > * patch it to use the example symmetric RSS key that was provided in = the paper; possible that XOR-ing the values to be hashed will produce a similar = result. Of course, this is software, not hardware generation of = Toeplitz. > * patch it to only hash on IPv4 / IPv6 2-tuple, that way you don't end > up with IPv4/IPv6 fragments in the wrong queue; I hope these two aren=E2=80=99t embedded in the code. The coming Intel = devices both support the symmetric RSS key and will correctly hash on ipv4/ipv4 4 tuple. See section 7.1.10.1 and 7.1.10.3 in=20 = http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xl7= 10-10-40-controller-datasheet.pdf > * configure up say, 4 or 8 rings in /boot/loader.conf: >=20 > hw.ix.num_queues=3D8 > (I think it's hw.ix, it used to be hw.ixgbe..) >=20 > * then, when you use netmap on ixgbe, you just bind to each TX and RX > ring with a separate process or thread. That thread will get packets > in both directions for a given flow. >=20 >> If we assume that someone in the future write and submit the above = into the >> ixgbe driver, could I be so bold as to ask you for a >> commandline/configuration example (a brief guide) of how one would = setup >> netmap and how to configure it to use the RX-queues? >=20 > I don't know of any examples of using netmap in this way from the > command line. I've normally written C code (And when I do, i start > with the bridge example in src/tools/tools/netmap/bridge.c) . >=20 >> That way I can start playing around with netmap and learning it while = I wait >> for the ixgbe driver to be updated... I've got two professional = programmer >> colleagues who've dealt extensively with e.g. the libnids and pfring = source >> code, so if I get a grasp of how to setup netmap, and I find it = interesting, >> it is likely that they can dive into and fix the ixgbe driver and = improve it >> as per above. So please, can you help me with a "netmap guide"? >>=20 >> When I try to find documentation or examples of how to setup netmap I = find >> none. Not even the netmap-enabled pcaplib contain any information as = how to >> use it. I'm no programmer, so showing me different C structs for = delivering >> data is of no use. :-/ >=20 > You mean: >=20 > https://code.google.com/p/netmap-libpcap/ >=20 > ? >=20 > I've not used it before, sorry :( >=20 >=20 >=20 > -a > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"