Date: Mon, 01 Apr 2019 22:22:48 +0200 From: "Kristof Provost" <kp@FreeBSD.org> To: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net> Cc: rgrimes@freebsd.org, "Andrey V. Elsukov" <bu7cher@yandex.ru>, "Mateusz Guzik" <mjguzik@gmail.com>, freebsd-pf@freebsd.org Subject: Re: svn commit: r345760 - in head: contrib/pf sys/netpfil/pf sbin/pfctl Message-ID: <2CACEDAF-1A73-43ED-92C3-BD7C4558234E@FreeBSD.org> In-Reply-To: <201904011647.x31GlWgv016223@gndrsh.dnsmgr.net> References: <201904011647.x31GlWgv016223@gndrsh.dnsmgr.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 1 Apr 2019, at 18:47, Rodney W. Grimes wrote: > I know for a fact that there is desire, with financials avaliable, > to get our code updated. I do not think there is any specific > criteria desired, other than moved closer to the OpenBSD version. > It’s a good goal, but there are three major issues along the way to importing the latest OpenBSD version. (And I’m sure a whole bunch of smaller ones.) Those are: - scalability - syntax - vimage The scalability issue is the obvious difference: OpenBSD’s pf is still very much single-core oriented, whereas the FreeBSD pf version can cope with multiple cores (somewhat) and is significantly faster on multicore hardware. Our version is by no means perfect, but it’s much faster than OpenBSD’s version. Much of the imperfections we have now is there because pf was designed in a giant locked kernel in the first place. Multi-core scalability was not part of its original design. Adopting OpenBSDs pf would mean redoing all of the locking work Gleb did years ago. Given the differences in OpenBSD’s pf (e.g. they keep states in a tree, not a hash table) it’s not a matter of replaying the previous work on a new pf version. This is a from the ground up introduction of fine grained locking in a code base that assumes a single giant lock. As I understand it the OpenBSD people are working on the problem as well, but I’ve not seen any diffs yet. If they’ve made significant progress we may be able to base our work on theirs. I don’t think it’d be acceptable to not have this, as it’d mean a very large performance regression. For reference, before I did the pfsync work we essentially had a single-threaded pf when pfsync was enabled. On my test hardware this meant a throughput of ~1.1Mpps, rather than the ~3.9Mpps without pfsync. I’d expect OpenBSDs pf to perform at around that ~1.1Mpps number without locking work. The second issue is one of syntax, and that’s what I assume is the main reason people want OpenBSDs pf. We’re still on an older iteration of the pf syntax, but changing that would inevitably mean breaking old configurations. That might be acceptable on a major version update (i.e. going into 13), but would mean the new work could never be backported. That’s probably the only way forward though. I’m playing with importing the ‘match’ keyword and not breaking the ‘scrub’ syntax at the same time, but it’s a non-trivial problem, and that’s only one of the steps along the way. Finally there’s vimage. That’s a FreeBSD-only feature, so naturally OpenBSDs pf is not aware of this. I expect that to be relatively easy to add back in, but it’s another obstacle. As vimage is what lets us have the pf tests we’ve got now I’d be very reluctant to let it go. It’s a supported feature in 12.0, so users will start to rely on it as well. TL;DR: It’s possible, but *hard*. Expect is to be many person-months of effort, and there’s no way it’s going to be a smooth ride. One thing I’ve thought of trying, and that might be an interesting stepping stone, is to create a port (/usr/ports/net/opf or whatever) of OpenBSD’s pf. In that version it’d be acceptable to not fix any of the above issues. It’d still give users to option of getting the new syntax. I’d expect this to be a relatively straightforward exercise. In principle there’s nothing to stop us from doing that same work in base, but we’re **NOT** going to import a fourth firewall. We’re just not. Regards, Kristof From owner-freebsd-pf@freebsd.org Mon Apr 1 21:06:14 2019 Return-Path: <owner-freebsd-pf@freebsd.org> Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7FC23156F193 for <freebsd-pf@mailman.ysv.freebsd.org>; Mon, 1 Apr 2019 21:06:14 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 868A089BEA; Mon, 1 Apr 2019 21:06:09 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x31L66bl017379; Mon, 1 Apr 2019 14:06:06 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x31L66wC017378; Mon, 1 Apr 2019 14:06:06 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net> Message-Id: <201904012106.x31L66wC017378@gndrsh.dnsmgr.net> Subject: Re: svn commit: r345760 - in head: contrib/pf sys/netpfil/pf sbin/pfctl In-Reply-To: <2CACEDAF-1A73-43ED-92C3-BD7C4558234E@FreeBSD.org> To: Kristof Provost <kp@FreeBSD.org> Date: Mon, 1 Apr 2019 14:06:06 -0700 (PDT) CC: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>, rgrimes@FreeBSD.org, "Andrey V. Elsukov" <bu7cher@yandex.ru>, Mateusz Guzik <mjguzik@gmail.com>, freebsd-pf@FreeBSD.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 868A089BEA X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [3.42 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.91)[0.913,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: gndrsh.dnsmgr.net]; NEURAL_SPAM_LONG(0.87)[0.870,0]; NEURAL_SPAM_MEDIUM(0.73)[0.726,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.02)[ip: (0.10), ipnet: 69.59.192.0/19(0.05), asn: 13868(0.03), country: US(-0.06)] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" <freebsd-pf.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-pf>, <mailto:freebsd-pf-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-pf/>; List-Post: <mailto:freebsd-pf@freebsd.org> List-Help: <mailto:freebsd-pf-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-pf>, <mailto:freebsd-pf-request@freebsd.org?subject=subscribe> X-List-Received-Date: Mon, 01 Apr 2019 21:06:14 -0000 > On 1 Apr 2019, at 18:47, Rodney W. Grimes wrote: > > I know for a fact that there is desire, with financials avaliable, > > to get our code updated. I do not think there is any specific > > criteria desired, other than moved closer to the OpenBSD version. > > > It?s a good goal, but there are three major issues along the way to > importing the latest OpenBSD version. (And I?m sure a whole bunch of > smaller ones.) This is a grand start, thank you for providing this input to the process. > Those are: > > - scalability > - syntax > - vimage > > The scalability issue is the obvious difference: OpenBSD?s pf is still > very much single-core oriented, whereas the FreeBSD pf version can cope > with multiple cores (somewhat) and is significantly faster on multicore > hardware. Our version is by no means perfect, but it?s much faster > than OpenBSD?s version. Much of the imperfections we have now is there > because pf was designed in a giant locked kernel in the first place. > Multi-core scalability was not part of its original design. > > Adopting OpenBSDs pf would mean redoing all of the locking work Gleb did > years ago. Given the differences in OpenBSD?s pf (e.g. they keep > states in a tree, not a hash table) it?s not a matter of replaying the > previous work on a new pf version. This is a from the ground up > introduction of fine grained locking in a code base that assumes a > single giant lock. As I understand it the OpenBSD people are working on > the problem as well, but I?ve not seen any diffs yet. If they?ve > made significant progress we may be able to base our work on theirs. > I don?t think it?d be acceptable to not have this, as it?d mean a > very large performance regression. > > For reference, before I did the pfsync work we essentially had a > single-threaded pf when pfsync was enabled. On my test hardware this > meant a throughput of ~1.1Mpps, rather than the ~3.9Mpps without pfsync. > I?d expect OpenBSDs pf to perform at around that ~1.1Mpps number > without locking work. The project funding source is OS agnostic, would it help if the OpenBSD pf implementation was redone in a way that it had fine grained locking. Would it be possible to apply Gleb's work as a upstreaming operation to get this work in both implementations? > The second issue is one of syntax, and that?s what I assume is the > main reason people want OpenBSDs pf. We?re still on an older iteration > of the pf syntax, but changing that would inevitably mean breaking old > configurations. That might be acceptable on a major version update (i.e. > going into 13), but would mean the new work could never be backported. > That?s probably the only way forward though. I?m playing with > importing the ?match? keyword and not breaking the ?scrub? > syntax at the same time, but it?s a non-trivial problem, and that?s > only one of the steps along the way. Isnt there more than just syntax? I thought OpenBSD pf has features that are not present in FreeBSD's pf implementation. > Finally there?s vimage. That?s a FreeBSD-only feature, so naturally > OpenBSDs pf is not aware of this. I expect that to be relatively easy to > add back in, but it?s another obstacle. As vimage is what lets us have > the pf tests we?ve got now I?d be very reluctant to let it go. > It?s a supported feature in 12.0, so users will start to rely on it as > well. > > TL;DR: It?s possible, but *hard*. Expect is to be many person-months > of effort, and there?s no way it?s going to be a smooth ride. So your quantifying this at man months and not man years, that helps a bunch, that is clearly within the scope of the funding. > > One thing I?ve thought of trying, and that might be an interesting > stepping stone, is to create a port (/usr/ports/net/opf or whatever) of > OpenBSD?s pf. In that version it?d be acceptable to not fix any of > the above issues. It?d still give users to option of getting the new > syntax. I?d expect this to be a relatively straightforward exercise. > In principle there?s nothing to stop us from doing that same work in > base, but we?re **NOT** going to import a fourth firewall. We?re > just not. But 4, its nice, its a power of 2, infact it is the second power of 2, such nice round numbers :-) I do like the idea of a first implementation of pf as a port, that being a fast path to update it, but would not want that as a long run solution. > Regards, > Kristof -- Rod Grimes rgrimes@freebsd.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2CACEDAF-1A73-43ED-92C3-BD7C4558234E>