From owner-freebsd-net@FreeBSD.ORG Thu Nov 6 11:46:25 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 69EB4D46; Thu, 6 Nov 2014 11:46:25 +0000 (UTC) Received: from mx1.sbone.de (bird.sbone.de [46.4.1.90]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0AEE6CE; Thu, 6 Nov 2014 11:46:24 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 5534125D3A82; Thu, 6 Nov 2014 11:46:13 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 703B1C76FDE; Thu, 6 Nov 2014 11:46:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id qaRTM3d2O54a; Thu, 6 Nov 2014 11:46:10 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (orange-tun0-ula.sbone.de [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 13BCCC76FD9; Thu, 6 Nov 2014 11:46:08 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: IPSEC in GENERIC [was: Re: netmap in GENERIC, by default, on HEAD] From: "Bjoern A. Zeeb" In-Reply-To: <03107CAB-445B-4BA9-8F50-69143E360010@neville-neil.com> Date: Thu, 6 Nov 2014 11:46:06 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <65C48D3B-0C08-4F8F-AF19-239238E49E62@FreeBSD.org> References: <92D22BEA-DDE5-4C6E-855C-B8CACB0319AC@neville-neil.com> <545A5C4D.3050603@FreeBSD.org> <03107CAB-445B-4BA9-8F50-69143E360010@neville-neil.com> To: George Neville-Neil X-Mailer: Apple Mail (2.1878.6) Cc: "Alexander V. Chernikov" , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2014 11:46:25 -0000 On 06 Nov 2014, at 01:10 , George Neville-Neil = wrote: > On 5 Nov 2014, at 9:20, Alexander V. Chernikov wrote: >=20 >> On 05.11.2014 19:39, George Neville-Neil wrote: >>> Howdy, >>>=20 >>> Last night (Pacific Time) I committed a change so that GENERIC, on = HEAD has the netmap >>> device enabled. This is to increase the breadth of our testing of = that feature prior >>> to the release of FreeBSD 11. >>>=20 >>> In two weeks I will enable IPSec by default, again in preparation = for 11. >> Please don't. >>=20 >> While I love to be able to use IPSEC features on unmodified GENERIC = kernel, simply enabling >> IPSEC is not the best thing to do in terms of performance. >>=20 >> Current IPSEC locking model is pretty complicated and is not scalable = enough. >> It looks like it requires quite a lot of man-hours/testing to be = reworked to achieve good performance and I'm not sure >> if making it enabled by default will help that. >>=20 >> Current IPv4/IPv6 stack code with some locking modifications is able = to forward 8-10MPPS on something like 2xE2660. >> I'm in process of merging these modification in "proper" way to our = HEAD, progress can be seen in projects/routing. >> While rmlocked radix/lle (and ifa_ref / ifa_unref, and bunch of = other) changes are not there yet, you can probably get >> x2-x4 forwarding/output performance for at least IPv4 traffic (e.g. = 2-3mpps depending on test conditions). >>=20 >> In contrast, I haven't seen IPSEC being able to process more than = 200kpps for any kind of workload. >>=20 >> What we've discussed with glebius@ and jmg@ at EuroBSDCon was to = modify PFIL to be able to monitor/enforce >> hooks ordering and make IPSEC code usual pfil consumer. In that case, = running GENERIC with IPSEC but w/o any SA >> won't influence packet processing path. This also simplifies the = process of making IPSEC loadable. >>=20 >=20 > How soon do you think the pfil patch would be ready? That sounds like = a good first step > towards making all this enabled in HEAD so that we can make sure that = IPSec gets the broad > testing that it needs. I don=92t think making IPsec an pfil consumer is actually feasible but = that=92s a different story. > Also, if folks who know about these problems can send workloads and = test descriptions to the > list that would be very helpful. >=20 > Specific drivers and hardware types would be most helpful as this may = be device related > as well. >=20 > I will turn this on on some machines in the network test lab to see = what I can see. What you would want for the moment is a single read-mostly = (read-lockless, read-non-atomic) integer that tells you if you have any = policy in the system; that=92s for your branch statement. That=92s = probably the closest you can get to enabling it cheaply in GENERIC = without doing much. There=92s a tradeoff in that for a few packets the policy might not be = effective immediately but given the amount of time it takes to =93install=94= the policy thinking about any instant real-time guarantees here is not = feasible anyway. Just my 2cts. Bjoern