From owner-freebsd-net@FreeBSD.ORG Fri Sep 9 12:22:41 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85F861065674 for ; Fri, 9 Sep 2011 12:22:41 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5D6B68FC1F for ; Fri, 9 Sep 2011 12:22:41 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 0B06246B0D; Fri, 9 Sep 2011 08:22:41 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 9C2028A02F; Fri, 9 Sep 2011 08:22:40 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Date: Fri, 9 Sep 2011 08:22:39 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110617; KDE/4.5.5; amd64; ; ) References: <4E678521.7080006@yandex-team.ru> In-Reply-To: <4E678521.7080006@yandex-team.ru> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201109090822.39325.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 09 Sep 2011 08:22:40 -0400 (EDT) Cc: "Alexander V. Chernikov" , jfvogel@gmail.com, melifaro@ipfw.ru Subject: Re: igb/ixgbe RSS/RX queues for non-IP traffic X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 12:22:41 -0000 On Wednesday, September 07, 2011 10:52:17 am Alexander V. Chernikov wrote: > Hello list! > > (CC'in Jack Vogel since this is intel drivers/hardware specific question) > > There are some techniques for assigning network traffic to different NIC > RX queues (bound to different CPUs/cores). > The main problem for all techniques is avoiding any possible packet > reordering in single flow. > > Most obvious (and most used) are described in Microsoft NDIS driver > specification: > Hashing is done on various tuples like: > * IPv46 src/dst + TCP/UDP src/dst port for TCP/UDP > * IPv46 src/dst for general IPv46 traffic > > igb's have a bit more options like assigning selected ethertype or TCP > SYN to different queue (See section 7.1.1 of 82576EB datasheet) > ixgbe (82599) goes even more far: flow director functionality permits 8K > 'perfect match' filters allowing to select any subset in: > * vlan > * IPv46 src/dst > * L4 proto > * TCP/UDP/SCTP protocol ports > and even 'flexible 2-byte tuple anywhere in the first 64 bytes of the > packet' > > There are other possibilities to assign traffic to different queues in > ixbge like using .1q priority filed (DCB functionality) > More information can be found in sections 7.1.2 (RX queues assignment), > 7.1.2.7(Flow Director), 7.7.1 (DCB) of 82599 datasheet > > However, there are many setups where PC can be used as platform for > routing/dispatching non-IP traffic. > PPPoE server is a typical example. > > People have to do some tricks (link aggregation, non-direct ISR) to get > traffic dispatched by more than single CPU/core, but this much less > efficient. > I'm a bit curious: why some generic hashing mechanism based on something > like 'flexible 2-byte tuple anywhere in the first 64 bytes of the > packet' were not added? > It can be (at least from my point of view) easily based on (for example) > flow director functionality. > > Particularly I'm trying to figure out how can I use all this variety of > filters to get MPLS traffic split to different RX queues. > > Maybe someone can point me to the right direction? There is another thread on net@ currently talking a good bit about this. It started with a post with a patch to support the traffic steering on ixgbe adapters using sysctls. -- John Baldwin