Date: Mon, 22 Sep 2014 17:35:10 -0500 From: Jim Thompson <jim@netgate.com> To: Adrian Chadd <adrian@freebsd.org> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Elof Ofel <elofu17@hotmail.com> Subject: Re: How do I balance bandwidth over several virtual NICs? Message-ID: <54200A0E-6337-40E1-B5DE-DC42F9CE8CCC@netgate.com> In-Reply-To: <CAJ-Vmo=m9_K96BGr7GYMbEJuqb0x8NyUirKgRMZU71dCRBAGgA@mail.gmail.com> References: <DUB125-W13FDC584F5DF9881CF5FDEBCB30@phx.gbl> <CA%2BP_MZGA_uz_H_QsB%2BdgXEgbXNCjv7w-OToKby=ww%2BvKgnU4_Q@mail.gmail.com> <DUB125-W851F972702452D9809C8E5BCB30@phx.gbl> <CAJ-Vmo=NGGkOkPWQKZ=3gA3vYYyM2kcjd3m85ymdJY3q4ixxLw@mail.gmail.com> <DUB125-W51623651A926D99844D879BCB30@phx.gbl> <CAJ-Vmo=m9_K96BGr7GYMbEJuqb0x8NyUirKgRMZU71dCRBAGgA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Sep 22, 2014, at 5:15 PM, Adrian Chadd <adrian@freebsd.org> wrote: >=20 > On 22 September 2014 13:39, Elof Ofel <elofu17@hotmail.com> wrote: >> Hi Adrian! >>=20 >> Now this sounds promising! All my sensors use the ixgbe driver. >> However, my skills in programming/compiling isn't vast. I know how to = patch >> and use poudriere. That's about it. >>=20 >> I must admit I don't really understand what you mean with "patch it = to use a >> symmetric RSS key", but it sounds like the functionality I'm looking = for is >> not yet there in the driver. >=20 > A few years ago a couple of researchers figured out you could abuse > the toeplitz hash to do symmetric RSS hashing: >=20 > http://www.ndsl.kaist.edu/~shinae/papers/TR-symRSS.pdf >=20 > This means that the same RSS hash value is chosen no matter which > direction the traffic is going on. > This is what you need to ensure that the packets going both directions > in a connection end up in the same NIC hardware receive ring. >=20 > So, all you have to do (!) is: >=20 > * grab freebsd-head, because the ixgbe driver there has some recent > bug fixes that you need for this to completely work; > * look at ixgbe_initialise_rss_mapping() - that's where the RSS key, > mapping and RSS hash types are configured; > * patch it to use the example symmetric RSS key that was provided in = the paper; possible that XOR-ing the values to be hashed will produce a similar = result. Of course, this is software, not hardware generation of = Toeplitz. > * patch it to only hash on IPv4 / IPv6 2-tuple, that way you don't end > up with IPv4/IPv6 fragments in the wrong queue; I hope these two aren=E2=80=99t embedded in the code. The coming Intel = devices both support the symmetric RSS key and will correctly hash on ipv4/ipv4 4 tuple. See section 7.1.10.1 and 7.1.10.3 in=20 = http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xl7= 10-10-40-controller-datasheet.pdf > * configure up say, 4 or 8 rings in /boot/loader.conf: >=20 > hw.ix.num_queues=3D8 > (I think it's hw.ix, it used to be hw.ixgbe..) >=20 > * then, when you use netmap on ixgbe, you just bind to each TX and RX > ring with a separate process or thread. That thread will get packets > in both directions for a given flow. >=20 >> If we assume that someone in the future write and submit the above = into the >> ixgbe driver, could I be so bold as to ask you for a >> commandline/configuration example (a brief guide) of how one would = setup >> netmap and how to configure it to use the RX-queues? >=20 > I don't know of any examples of using netmap in this way from the > command line. I've normally written C code (And when I do, i start > with the bridge example in src/tools/tools/netmap/bridge.c) . >=20 >> That way I can start playing around with netmap and learning it while = I wait >> for the ixgbe driver to be updated... I've got two professional = programmer >> colleagues who've dealt extensively with e.g. the libnids and pfring = source >> code, so if I get a grasp of how to setup netmap, and I find it = interesting, >> it is likely that they can dive into and fix the ixgbe driver and = improve it >> as per above. So please, can you help me with a "netmap guide"? >>=20 >> When I try to find documentation or examples of how to setup netmap I = find >> none. Not even the netmap-enabled pcaplib contain any information as = how to >> use it. I'm no programmer, so showing me different C structs for = delivering >> data is of no use. :-/ >=20 > You mean: >=20 > https://code.google.com/p/netmap-libpcap/ >=20 > ? >=20 > I've not used it before, sorry :( >=20 >=20 >=20 > -a > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54200A0E-6337-40E1-B5DE-DC42F9CE8CCC>