From owner-freebsd-net@freebsd.org Thu Nov 7 07:33:17 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7A1521AA54D for ; Thu, 7 Nov 2019 07:33:17 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 477w9n1TKqz4FHR; Thu, 7 Nov 2019 07:33:16 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id xA77WuFq041696 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 6 Nov 2019 23:32:56 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id xA77WtMq041695; Wed, 6 Nov 2019 23:32:55 -0800 (PST) (envelope-from jmg) Date: Wed, 6 Nov 2019 23:32:55 -0800 From: John-Mark Gurney To: Lawrence Stewart Cc: Eugene Grosbein , Olivier =?iso-8859-1?Q?Cochard-Labb=E9?= , freebsd-net@freebsd.org, Kurt Jaeger Subject: Re: 10g IPsec ? Message-ID: <20191107073255.GU8521@funkthat.com> Mail-Followup-To: Lawrence Stewart , Eugene Grosbein , Olivier =?iso-8859-1?Q?Cochard-Labb=E9?= , freebsd-net@freebsd.org, Kurt Jaeger References: <20191104194637.GA71627@home.opsec.eu> <20191105191514.GG8521@funkthat.com> <261b842d-51eb-4522-6ef5-0672e5d1594e@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.0-RELEASE-p7 amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Wed, 06 Nov 2019 23:32:56 -0800 (PST) X-Rspamd-Queue-Id: 477w9n1TKqz4FHR X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2019 07:33:17 -0000 Lawrence Stewart wrote this message on Thu, Nov 07, 2019 at 13:04 +1100: > On 7/11/19 12:52 pm, Eugene Grosbein wrote: > > 07.11.2019 8:36, Lawrence Stewart wrote: > > > >>>> AES-GCM can run at over 1GB/sec on a single core, so as long as the > >>>> traffic can be processed by multiple threads (via multiple queues > >>>> for example), it should be doable. > >>>> > >>>> > >>> I didn't bench this setup (10Gb/s IPSec) but I believe we will have the > >>> same problem with IPSec as with all VPN setups (like PPPoE or GRE): the > >>> IPSec tunnel will generate one IP flow preventing load sharing between all > >>> the NIC's RSS queues. > >>> I'm not aware of improvement to remove this limitation. > >> > >> I never understood why the IPsec SPI couldn't be used to shard > >> traffic... does anyone know if there is a technical reason why doing so > >> would be problematic? > > > > Generic way do distribute load over CPUs is distinct hardware receive queues of NIC > > using distinct interrupts to deliver packets to the host while interrupts are bound > > to distinct CPU cores. It needs hardware capable of splitting packet stream by IPsec SPI > > and I'm aware of only some 40Gpbs Intel NICs that can be programmed to do so. > > Right, a "consumers need to ask for it" issue more so than an inherently > problematic approach. I assumed as much but wasn't sure. Don't we have the option of doing soft re-classification? Where we recalculate the hash, and then do a netisr defer? I mean that'd burn a bunch of extra cpu cycles, but you gotta do what you gotta do. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."