From owner-freebsd-net@freebsd.org Thu Nov 7 01:53:21 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A46471A304B for ; Thu, 7 Nov 2019 01:53:21 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 477mdX36Flz3xqV; Thu, 7 Nov 2019 01:53:20 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xA71r31Q065152 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 7 Nov 2019 01:53:05 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: lstewart@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xA71qtF3095699 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 7 Nov 2019 08:52:55 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 10g IPsec ? To: Lawrence Stewart , =?UTF-8?Q?Olivier_Cochard-Labb=c3=a9?= , John-Mark Gurney References: <20191104194637.GA71627@home.opsec.eu> <20191105191514.GG8521@funkthat.com> Cc: freebsd-net@freebsd.org, Kurt Jaeger From: Eugene Grosbein Message-ID: <261b842d-51eb-4522-6ef5-0672e5d1594e@grosbein.net> Date: Thu, 7 Nov 2019 08:52:48 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 477mdX36Flz3xqV X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-2.67 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[grosbein.net]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; IP_SCORE(-1.58)[ip: (-3.93), ipnet: 2a01:4f8::/29(-2.24), asn: 24940(-1.69), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2019 01:53:21 -0000 07.11.2019 8:36, Lawrence Stewart wrote: >>> AES-GCM can run at over 1GB/sec on a single core, so as long as the >>> traffic can be processed by multiple threads (via multiple queues >>> for example), it should be doable. >>> >>> >> I didn't bench this setup (10Gb/s IPSec) but I believe we will have the >> same problem with IPSec as with all VPN setups (like PPPoE or GRE): the >> IPSec tunnel will generate one IP flow preventing load sharing between all >> the NIC's RSS queues. >> I'm not aware of improvement to remove this limitation. > > I never understood why the IPsec SPI couldn't be used to shard > traffic... does anyone know if there is a technical reason why doing so > would be problematic? Generic way do distribute load over CPUs is distinct hardware receive queues of NIC using distinct interrupts to deliver packets to the host while interrupts are bound to distinct CPU cores. It needs hardware capable of splitting packet stream by IPsec SPI and I'm aware of only some 40Gpbs Intel NICs that can be programmed to do so.