Date: Tue, 1 Jul 2014 11:02:19 -0700 From: Adrian Chadd <adrian@freebsd.org> To: FreeBSD Net <freebsd-net@freebsd.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: [rfc] IP_BINDMULTI and IP_RSSBUCKETID support Message-ID: <CAJ-Vmo=SSEdATcOtrvreaggb88HZ8MjKrbYfgRjJbPJi2iO0tA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi! This patch: http://people.freebsd.org/~adrian/rss/20140701-rss-1.diff Implements a few things: * It introduces IP_BINDMULTI, which allows TCP/UDP sockets to be bound to the same matching address:port. It's intended to be what Linux SO_REUSEADDR has become except I specifically wanted it to be a different option. * .. it only allows the PCBs to be created, it doesn't actually load balance between multiple sockets like this. That requires a bunch more work to happen in the general case. But! * IP_RSSBUCKETID maps an IPv4 TCP (and later UDP, and much later IPv6) socket into a specific RSS bucket rather than in the global PCB list. That way it only receives incoming connections that has to that RSS bucket ID. If userland queries the RSS setup (sysctl net.inet.rss) and creates a worker thread per RSS bucket _AND_ pins it to the correct CPU for the RSS bucket, all the transmit and receive side will stay on the same CPU. The lock contention is almost eliminated in that case. My plan: * get this into the tree and tested/debugged; * add IPv4 UDP extensions to RSS; * teach a couple of popular threaded UDP applications (eg memcached) about this; * bask in what hopefully will be a good set of benefits; * work on IP_BINDMULTI support so multiple threads/processes can listen in the same RSS bucket and get some load balancing; * add IPv6 support; * add RSS rebalancing support; * add multi-socket, multi-NIC RSS awareness. Thanks! -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=SSEdATcOtrvreaggb88HZ8MjKrbYfgRjJbPJi2iO0tA>