From owner-freebsd-hackers@FreeBSD.ORG  Sat Mar 28 22:46:38 2015
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 537E6FDF;
 Sat, 28 Mar 2015 22:46:38 +0000 (UTC)
Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 052A23DC;
 Sat, 28 Mar 2015 22:46:38 +0000 (UTC)
Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD))
 (envelope-from <slw@zxy.spb.ru>)
 id 1YbzUw-0003VX-Rp; Sun, 29 Mar 2015 01:46:35 +0300
Date: Sun, 29 Mar 2015 01:46:34 +0300
From: Slawa Olhovchenkov <slw@zxy.spb.ru>
To: Adrian Chadd <adrian@freebsd.org>
Subject: Re: irq cpu binding
Message-ID: <20150328224634.GH23643@zxy.spb.ru>
References: <20150328183147.GC23643@zxy.spb.ru>
 <CAJ-VmokPdnOSASLcrzHtqs_r6GfUW6Q5ap7r9Tnp1OvKhDh20A@mail.gmail.com>
 <20150328192505.GD23643@zxy.spb.ru>
 <CAJ-Vmo=R5J=LT9ntgZJ4xHZwn8_WEAUC=3wSqPKn3=z8ki_N6A@mail.gmail.com>
 <20150328194959.GE23643@zxy.spb.ru>
 <CAJ-Vmo=1rzB+YNGNuV9s=FnSse7FL7S42OSS4u-PzUs74b850A@mail.gmail.com>
 <20150328201219.GF23643@zxy.spb.ru>
 <CAJ-Vmo=wecgoVYcS14gsOnT86p=HEMdao65aXTi7jLfVVyOELg@mail.gmail.com>
 <20150328221621.GG23643@zxy.spb.ru>
 <CAJ-Vmomd6Z5Ou7cvV1Kg4m=X2907507hqKMWiz6ssZ45Pi_-Dg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAJ-Vmomd6Z5Ou7cvV1Kg4m=X2907507hqKMWiz6ssZ45Pi_-Dg@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: slw@zxy.spb.ru
X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false
Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
 <freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
 <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 28 Mar 2015 22:46:38 -0000

On Sat, Mar 28, 2015 at 03:27:36PM -0700, Adrian Chadd wrote:

> You still have to pin the nginx threads to whatever your limited set
> of RSS CPUs are, or the benefits aren't really there.

nginx already have this ability:

worker_processes  6;
worker_cpu_affinity 000001 000010 000100 001000 010000 100000;

> I'd rather we just get librss defined in FreeBSD-HEAD and then have
> nginx "know" about that for FreeBSD. It already has to know about the
> linux way of doing it and it has to be conditional on it running

I am don't know about nginx support for RSS in Linux.

> Linux. So it's not a big stretch to need to know about FreeBSD.

I am try to read RSS implementation.
I am don't completely understund it, may be.
I found it overingenering and useless in complex setup.

For example: dual NIC, LACP. Uniqueue map NIC queue to CPU core:

NIC 1 que 0 <=> Core 0
NIC 1 que 1 <=> Core 1
NIC 1 que 2 <=> Core 2
NIC 1 que 3 <=> Core 3
NIC 2 que 0 <=> Core 4
NIC 2 que 1 <=> Core 5
NIC 2 que 2 <=> Core 6
NIC 2 que 3 <=> Core 7

ISP network equipment use proper hash for map flow to network link,
you can't predict what NIC got incoming packet (you may assume that
all not-fragment packets dispatch to same NIC, yes).

This is about case of outgoing connections.