From owner-freebsd-net@freebsd.org Thu Nov 7 02:04:18 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A073E1A35CE for ; Thu, 7 Nov 2019 02:04:18 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [45.63.28.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 477mtB1NgZz3yHB; Thu, 7 Nov 2019 02:04:18 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: by lauren.room52.net (Postfix) with ESMTPSA id C213488F; Thu, 7 Nov 2019 13:04:14 +1100 (AEDT) Subject: Re: 10g IPsec ? To: Eugene Grosbein , =?UTF-8?Q?Olivier_Cochard-Labb=c3=a9?= , John-Mark Gurney Cc: freebsd-net@freebsd.org, Kurt Jaeger References: <20191104194637.GA71627@home.opsec.eu> <20191105191514.GG8521@funkthat.com> <261b842d-51eb-4522-6ef5-0672e5d1594e@grosbein.net> From: Lawrence Stewart Autocrypt: addr=lstewart@freebsd.org; prefer-encrypt=mutual; keydata= mQINBFNq7K8BEADQFhZprR6joPIvqFonlsbZ0M72rkzHkCtGzk+hiE/TZh8df32VaGhXvgAH yP9ictqRai6lYRhO5LyjwR4ysBu4jAZlLCwWWBMY8l2JjuOohsv2+87+hQy+F1nVcPYuNJ4O Eqvqmi/RTU2+kZYGT2kbsSYVWiCUqwzSNWsbVZ8Sw1+ds2e380655Cstm+Ewn3gmX/wXPn3Y 22M+h5KRj3yDn8aJ439lUTcVDQ+Dah/7h4DTn3cXfZdKFSb3HEoiwPh78R3dyQGOQgYnJ3Fp KfKu5gRlXMyB1+6wUBh7G1henvYFrN+HClr+z1fBmsm22Lb7LLs/g6p0FtWslnNxA2CvIC9I Q1nbBoA0bKji/f0S3K7LlAIv/scUqPChfp1EkBvkTOek9N0znzcVCwJTjRjfS0uu6TMWuMXr qpCCrGKonN3gdqKW9pdWmn33kDt0GaESIPOgIRwBk8Ak9/j9Hd/vdtyHab1GKGJTZfIvnJB6 xVy/zwBhUIK/h5dboYqYZds+Ky5g+j+Q4j4bsKdgwjlrRO+eGQTCjRcZoiE0M2PZGK/dt/eS zuHWv0l6r7NkQXn5RBU+5JpdzECyyolKoBrDhHMDdI+Cc3KeQfMSkftKV7UwkScoplI86pLg yKHNxyrmqp3NTE04yxpY1KlEAUv3I/lnkikpj6j1PzGXyReMewARAQABtCpMYXdyZW5jZSBB LiBTdGV3YXJ0IDxsc3Rld2FydEBmcmVlYnNkLm9yZz6JAkAEEwEKACoCGwMFCRLMAwAFCwkI BwMFFQoJCAsFFgMCAQACHgECF4AFAlNq7/0CGQEACgkQmlC8/PQNmwmFSQ/+MpZl3Ysk1D8o FzhmuL5gHyMKWhM6lFGgRVzYW219CyJlxw/twkrdTjZ8YQBTCNnjOLg4ecV+0RQGsRzczfnW 1faBGEOC9CDInwuL00M+sybbiAylotEI54yR+ey0MRcCMuP1+y4LoHLZsCoOOa3PgR5ZrTUq 3Zlq2Zfhq9dq/zrdorWyGxrynlrmZFMqkf1F3ZsHkIvyOkfoJ8+icf/nAcbzsNJP7JMYE1fP PbqbLTR2jRSbHFOu9J1+3yumA5WNbTvHwEgOmxqA20XtGxlpwXXciWWzTN2RJ3EFqCjCiTFC mtYt9/zelkOnLncVQ9s1JHbs6vr4sYNST8bA9LvGOn2BZMDmECoazHfdmM3Q+hcnPYDX8W5h FKqgV049gQZItdZpMuSQ5xgx3K2EkwI5e63XK+SNcC/PUrmbmsAHHhwlrDbBy9LQ6DFQ5uf+ yX3J3+4f4MqSRNQJ4GaLIJXz1lII+CZ3iSEVIJyFMYK2eFlxxo1g26g2USSC0gxxyL4vuz9v GHk90aGPnz6ci0JZiTcBaXr7ObdZoHgDxKjWOq8Mg0juVj5S7Dcwa3vulf5N0Iq0hFffxfoM AAy6Pa+OxKVP6sz1VltTflM73ZqVM+30FLe96iAXzhe/Ku+zI19mJ/87ZKo/5b1lKgDu3II6 dV3IwSQFrV/S0lZEsYBOMw25Ag0EU2rsrwEQALlqsHLKoqF/jfV6O3Nn9MY8r9qeO5tc7jWA lg04uM83itX/E+1ci/V2EqVmsTQFEq2TdGzhxSU1FzoKRxJrsP4dSbazOK54xiXhWMA0fMOL DkM94DXlJwV1+9aamfHshcEHdp6V6nV6YIG7lDD0RDLeTeEtMaK+nSHPWN6beJpRBdLNP+/k 9jtW4Jw/sFXSD1vq7Eb871d9sxY31LijqR+6AJrQ0vJmSH6b8gpr23pslU2f/9utQPFT6tqH H2ZoQoIyGYjyWv6IlYS33/9Idl/A8Zdcsk+aelK/1gzhKwMxvJM7dLCckS8Ir6OTl/72bMZ0 Kb/bFDx0OVPZYhOu53SfUqy8eqhy7KIN6iOgd0mn4rH9+FiJuONq0hMe+Q2NVP7392CRGXXc zgTTwcVc3Nm4zAmEF0h/JpWZWN/AjA9+8zetMnCFjivkr68DGZHLyhbgj+TGan56cTayLLvY Zc/KVfAsI3nGmwMrQhysXZo9JdHPGJwBfTqZOsTDUbrcT4DbU56/1xt3/zpoajnJu+RcfLSd k1TXWzly/BZY60TxewDWxK0f10MzZd7mzidFdvyZKEL6lTkwzZ/kAaqLy4inpqdkicskg020 +kqKxlaCHPFedXLbwfMTthi67jwO8+ZOA7/QWChPL6bmNgZlK7VMHoi7uu+vwCgeh/H3xrMH ABEBAAGJAiUEGAEKAA8FAlNq7K8CGwwFCRLMAwAACgkQmlC8/PQNmwkOWQ//Xxuo/LWfNlqI lsvpnlPgEhVaDQX3vhuru/nMiZ7wq0eTmCD011eHw8wIaF7aOjs6zKM8sf/TpfF30V3GAQSL MpzeebSHWvBfN+XQSgMmwzgrKeQd6kTUf1I501rlzX0vz2NrUqUxktaLMZd6K/1hu7smDHEx fX8SZddz124QpjQJlmCZ0ewFKJ+bMz9W3prdCwV45Gu+glWYnREIXnMd1sXtbQMiu4Z+3CTG VOXmDj14lM3KnO2Vuc6IZTf9LeEeXM5Kvy4JCH4PK3DVInv+1e7KWqFo8KIapUumLxpGHg4c OG+3LTpLg+HTFiuSKqutZU2MDjp49/YdYAGLCEHgQ4rAUWqPNLzva+fbcHIuV/RpjJd6amNC LvtjsizvpE78TzACDtUV8mAHxrVXyrukvHHl+kJVfBH0XpKXxVT1aup+O4eBj5celCNaHF2z m2nKdJW/zjO8lr17Z1q/AxFgHb0blaKNqrpauKC3QUaHjbQEKxTxQXebKrfBdhDg/nG3Qiw9 4NYb/n84KIZMgzXN9OqoOlKgiaLDJ7ijH359GQAhl3UB1Nezfg0BcuxGORlkC063279ilBkE Pw5TRO/8gJH0+We+2unIqUZyHj4P4SLk2xXeyJxqoH+WUNCJBK3IItBQYLb2v0vsutbT193j xwpvFODEWa5OVDC+bcpRp1M= Message-ID: Date: Thu, 7 Nov 2019 13:04:14 +1100 User-Agent: Not your concern MIME-Version: 1.0 In-Reply-To: <261b842d-51eb-4522-6ef5-0672e5d1594e@grosbein.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 477mtB1NgZz3yHB X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-1.87 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-0.87)[-0.871,0]; ASN(0.00)[asn:20473, ipnet:45.63.24.0/21, country:US]; NEURAL_HAM_LONG(-1.00)[-0.999,0] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2019 02:04:18 -0000 On 7/11/19 12:52 pm, Eugene Grosbein wrote: > 07.11.2019 8:36, Lawrence Stewart wrote: > >>>> AES-GCM can run at over 1GB/sec on a single core, so as long as the >>>> traffic can be processed by multiple threads (via multiple queues >>>> for example), it should be doable. >>>> >>>> >>> I didn't bench this setup (10Gb/s IPSec) but I believe we will have the >>> same problem with IPSec as with all VPN setups (like PPPoE or GRE): the >>> IPSec tunnel will generate one IP flow preventing load sharing between all >>> the NIC's RSS queues. >>> I'm not aware of improvement to remove this limitation. >> >> I never understood why the IPsec SPI couldn't be used to shard >> traffic... does anyone know if there is a technical reason why doing so >> would be problematic? > > Generic way do distribute load over CPUs is distinct hardware receive queues of NIC > using distinct interrupts to deliver packets to the host while interrupts are bound > to distinct CPU cores. It needs hardware capable of splitting packet stream by IPsec SPI > and I'm aware of only some 40Gpbs Intel NICs that can be programmed to do so. Right, a "consumers need to ask for it" issue more so than an inherently problematic approach. I assumed as much but wasn't sure. Cheers Lawrence