From owner-freebsd-pf@freebsd.org Sun Mar 28 11:47:38 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 18FC85BE2BE for ; Sun, 28 Mar 2021 11:47:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F7YqG01DZz4Wjk for ; Sun, 28 Mar 2021 11:47:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 0063E5BE2BD; Sun, 28 Mar 2021 11:47:38 +0000 (UTC) Delivered-To: pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 003315BE2BC for ; Sun, 28 Mar 2021 11:47:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7YqF6Xdkz4WVL for ; Sun, 28 Mar 2021 11:47:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D3A421F46B for ; Sun, 28 Mar 2021 11:47:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 12SBlbhT093070 for ; Sun, 28 Mar 2021 11:47:37 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 12SBlbWS093069 for pf@FreeBSD.org; Sun, 28 Mar 2021 11:47:37 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 254577] [PATCH] pf: Implement the NAT source port selection of MAP-E Customer Edge Date: Sun, 28 Mar 2021 11:47:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kp@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Mar 2021 11:47:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D254577 Kristof Provost changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kp@freebsd.org --- Comment #3 from Kristof Provost --- I had two minor remarks. Phabricator is easier for code review than bugzill= a, so I posted the patch there: https://reviews.freebsd.org/D29468 If you have (or create) an account you should be able to commandeer the pat= ch and update it when appropriate. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Sun Mar 28 21:00:24 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5F75E5AEB9D for ; Sun, 28 Mar 2021 21:00:24 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F7p5412fjz3NmP for ; Sun, 28 Mar 2021 21:00:24 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id E89605AEB9B; Sun, 28 Mar 2021 21:00:23 +0000 (UTC) Delivered-To: pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E47EF5AED12 for ; Sun, 28 Mar 2021 21:00:23 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7p531Gpnz3NXc for ; Sun, 28 Mar 2021 21:00:23 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8595C26C3E for ; Sun, 28 Mar 2021 21:00:22 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 12SL0MUq092667 for ; Sun, 28 Mar 2021 21:00:22 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 12SL0MYp092666 for pf@FreeBSD.org; Sun, 28 Mar 2021 21:00:22 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202103282100.12SL0MYp092666@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: pf@FreeBSD.org Subject: Problem reports for pf@FreeBSD.org that need special attention Date: Sun, 28 Mar 2021 21:00:22 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Mar 2021 21:00:24 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 203735 | Transparent interception of ipv6 with squid and p Open | 237973 | pf: implement egress keyword to simplify rules ac 2 problems total for which you should take action. From owner-freebsd-pf@freebsd.org Mon Mar 29 14:55:33 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C9B3A5799CC; Mon, 29 Mar 2021 14:55:33 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8Fxd5SKjz3l8N; Mon, 29 Mar 2021 14:55:33 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 948DE29D4B; Mon, 29 Mar 2021 14:55:33 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id C447D1243C; Mon, 29 Mar 2021 16:55:30 +0200 (CEST) From: "Kristof Provost" To: "Cy Schubert" Cc: "FreeBSD pf" , freebsd-arch@freebsd.org Subject: Re: [RFC] pf ioctl changes Date: Mon, 29 Mar 2021 16:55:29 +0200 X-Mailer: MailMate (1.13.2r5673) Message-ID: <18DC1EA9-ABFC-4A06-8710-A3068370EC52@FreeBSD.org> In-Reply-To: <202103291403.12TE3Y2H094131@slippy.cwsent.com> References: <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org> <202103291403.12TE3Y2H094131@slippy.cwsent.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Mar 2021 14:55:33 -0000 On 29 Mar 2021, at 16:03, Cy Schubert wrote: > In message <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org>, > "Kristof > Provost > " writes: >> Hi, >> >> There are several patches in the pipeline that require changes in >> pf’s >> interface between kernel and userspace. >> In the past these have been handled in multiple ways. Either by >> simply >> making the change, breaking binary compatibility, or by introducing a >> v2 >> ioctl (e.g. DIOCADDALTQV1). >> >> While one is better than the other neither is wholly satisfying. New >> versions of calls constitute a maintenance burden after all. >> >> I’d like to change the ioctl interface to use nvlists, which >> would >> make such extensions much easier, because fields can be optional. >> That is, if userspace doesn’t supply the >> ‘shinynewfeature’ field >> the kernel can assume the default value and things just work. >> Similarly, >> if the kernel supplies a ’shinynewfeature’ which userspace >> doesn’t >> know about it’s simply ignored. >> >> The rough plan is to introduce nvlist versions of the get/add rules >> calls for now. Others will follow as the need presents itself. >> As these are new ioctls it is safe to MFC them to stable/12 and >> stable/13. >> The old interface will remain supported in those branches, but >> I’d >> like to remove it from main (and thus FreeBSD 14). >> >> As part of this effort I may end up splitting off the ioctl interface >> code from pfctl into libpfctl, which should make reuse of that code >> easier. >> >> I hope to post preliminary patches in the coming week. >> >> Thoughts? Objections? > > Kernel and userland should be, I'd say must be, kept in sync. We have > many > examples of userland and kernel not being in sync over the years. For > ipfilter, I've made incompatible changes to data structures requiring > userand and kernel be in sync. These are few and far between. > > I've gotten away with this because there is no third party software > that > relies on the ipfilter kernel interfaces. I could be wrong but I doubt > there may be third party software requiring pf ABI compatibility. But > if > there is then verstioned library interfaces are required. > > Given that the advice is to keep kernel and userland in sync there > probably > is no requirement for an UPDATING entry but that would be your call. > There are out-of-tree users of the pf ioctl interface. security/expiretable[1] for example. security/snort2pfcd appears to as well. sysutils/pfstat and sysutils/pftop use the ioctl interface as well, although not the three specific calls of immediate interest. I’m trying to work out how many examples there are, because one way or the other they’re going to have to cope with changes. So, I’d prefer to not just change the definitions of structs, even if we’ve done that in the past. struct pf_rule contains a few peculiarities from historical mistakes that I hope to correct now. Best regards, Kristof [1] Perhaps not the greatest example, because its use of struct pf_state was incorrect, so it couldn’t actually have worked correctly before it stopped building. See PR #253547 for details. From owner-freebsd-pf@freebsd.org Mon Mar 29 15:16:50 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49F2257A6EE; Mon, 29 Mar 2021 15:16:50 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8GQ96cZTz3nvW; Mon, 29 Mar 2021 15:16:49 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.66.148.124]) by shaw.ca with ESMTPA id QtdGlpLzf2SWTQtdHlPeLf; Mon, 29 Mar 2021 09:16:47 -0600 X-Authority-Analysis: v=2.4 cv=fdJod2cF c=1 sm=1 tr=0 ts=6061ef5f a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=xqWC_Br6kY4A:10 a=8nJEP1OIZ-IA:10 a=dESyimp9J3IA:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=EkcXrb_YAAAA:8 a=XSbsiAr4RLdZVirN8WEA:9 a=wPNLvfGTeEIA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=LK5xJRSDVpKd5WXXoEvA:22 a=RBBcRewTFc8P4JkPnay6:22 Received: from slippy.cwsent.com (slippy [IPv6:fc00:1:1:1::5b]) by spqr.komquats.com (Postfix) with ESMTPS id 80AC518A; Mon, 29 Mar 2021 08:16:45 -0700 (PDT) Received: from slippy (localhost [127.0.0.1]) by slippy.cwsent.com (8.16.1/8.16.1) with ESMTP id 12TFGjTi004433; Mon, 29 Mar 2021 08:16:45 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <202103291516.12TFGjTi004433@slippy.cwsent.com> X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: "Kristof Provost" cc: "Cy Schubert" , "FreeBSD pf" , freebsd-arch@freebsd.org Subject: Re: [RFC] pf ioctl changes In-reply-to: <18DC1EA9-ABFC-4A06-8710-A3068370EC52@FreeBSD.org> References: <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org> <202103291403.12TE3Y2H094131@slippy.cwsent.com> <18DC1EA9-ABFC-4A06-8710-A3068370EC52@FreeBSD.org> Comments: In-reply-to "Kristof Provost" message dated "Mon, 29 Mar 2021 16:55:29 +0200." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Mon, 29 Mar 2021 08:16:45 -0700 X-CMAE-Envelope: MS4xfLa1KfqDIepTI3eJZkcB2BRJ9L3xywlsyfx5NBlF8+0/8A8cllRBWXLjC0TG1yEbtQ9Hbu/kbaJWLRbSZa5/CpR7jnd8x4Yoprc4kXljsWfbahB2UmNq +TpQdvwyrHuuUS1UAKbqwBFgF79NdGlKWqIYdDQXJecPfRfKQCaJu3bLK+YE8ygRu9J/GdwsoJ++bs8zu/JEvjFNXIeIlM44tHFLE50/eNS+zkiH9Gur6nBI 6pNrLdDZRa9a+PeVF6st51DzERV2J+7XxoFvwJiDREY= X-Rspamd-Queue-Id: 4F8GQ96cZTz3nvW X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Mar 2021 15:16:50 -0000 In message <18DC1EA9-ABFC-4A06-8710-A3068370EC52@FreeBSD.org>, "Kristof Provost " writes: > On 29 Mar 2021, at 16:03, Cy Schubert wrote: > > In message <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org>, > > "Kristof > > Provost > > " writes: > >> Hi, > >> > >> There are several patches in the pipeline that require changes in > >> pf’s > >> interface between kernel and userspace. > >> In the past these have been handled in multiple ways. Either by > >> simply > >> making the change, breaking binary compatibility, or by introducing a > >> v2 > >> ioctl (e.g. DIOCADDALTQV1). > >> > >> While one is better than the other neither is wholly satisfying. New > >> versions of calls constitute a maintenance burden after all. > >> > >> I’d like to change the ioctl interface to use nvlists, which > >> would > >> make such extensions much easier, because fields can be optional. > >> That is, if userspace doesn’t supply the > >> ‘shinynewfeature’ field > >> the kernel can assume the default value and things just work. > >> Similarly, > >> if the kernel supplies a ’shinynewfeature’ which userspace > >> doesn’t > >> know about it’s simply ignored. > >> > >> The rough plan is to introduce nvlist versions of the get/add rules > >> calls for now. Others will follow as the need presents itself. > >> As these are new ioctls it is safe to MFC them to stable/12 and > >> stable/13. > >> The old interface will remain supported in those branches, but > >> I’d > >> like to remove it from main (and thus FreeBSD 14). > >> > >> As part of this effort I may end up splitting off the ioctl interface > >> code from pfctl into libpfctl, which should make reuse of that code > >> easier. > >> > >> I hope to post preliminary patches in the coming week. > >> > >> Thoughts? Objections? > > > > Kernel and userland should be, I'd say must be, kept in sync. We have > > many > > examples of userland and kernel not being in sync over the years. For > > ipfilter, I've made incompatible changes to data structures requiring > > userand and kernel be in sync. These are few and far between. > > > > I've gotten away with this because there is no third party software > > that > > relies on the ipfilter kernel interfaces. I could be wrong but I doubt > > there may be third party software requiring pf ABI compatibility. But > > if > > there is then verstioned library interfaces are required. > > > > Given that the advice is to keep kernel and userland in sync there > > probably > > is no requirement for an UPDATING entry but that would be your call. > > > There are out-of-tree users of the pf ioctl interface. > security/expiretable[1] for example. > security/snort2pfcd appears to as well. > sysutils/pfstat and sysutils/pftop use the ioctl interface as well, > although not the three specific calls of immediate interest. This complicates things. IMO you'll probably need versioned function calls for at least 13-STABLE EOL. Or, versioning the data structures passed into the kernel such that the new fields are at the tail of the existing structures. > > I’m trying to work out how many examples there are, because one way or > the other they’re going to have to cope with changes. > > So, I’d prefer to not just change the definitions of structs, even if > we’ve done that in the past. struct pf_rule contains a few > peculiarities from historical mistakes that I hope to correct now. Technical debt is difficult to eliminate. We either fix it, paying it off in one lump sum or we pay it off through aggravation and design limitations, with interest, over time. Given that pf uses ioctl, versioned function calls won't help. A new ioctl may be the only answer. If you do choose this, add an identifier and version number to the head of each new struct to future proof pf. > > Best regards, > Kristof > > [1] Perhaps not the greatest example, because its use of struct pf_state > was incorrect, so it couldn’t actually have worked correctly before it > stopped building. See PR #253547 for details. -- Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org The need of the many outweighs the greed of the few. From owner-freebsd-pf@freebsd.org Mon Mar 29 15:25:58 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B306A57AC3E; Mon, 29 Mar 2021 15:25:58 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8Gck4kMlz3p7f; Mon, 29 Mar 2021 15:25:58 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 64F4F2A32F; Mon, 29 Mar 2021 15:25:58 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id C3AE012464; Mon, 29 Mar 2021 17:25:56 +0200 (CEST) From: "Kristof Provost" To: "Cy Schubert" Cc: "FreeBSD pf" , freebsd-arch@freebsd.org Subject: Re: [RFC] pf ioctl changes Date: Mon, 29 Mar 2021 17:25:55 +0200 X-Mailer: MailMate (1.13.2r5673) Message-ID: <75FA4097-ED2A-4B96-9C90-E82F49F7764B@FreeBSD.org> In-Reply-To: <202103291516.12TFGjTi004433@slippy.cwsent.com> References: <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org> <202103291403.12TE3Y2H094131@slippy.cwsent.com> <18DC1EA9-ABFC-4A06-8710-A3068370EC52@FreeBSD.org> <202103291516.12TFGjTi004433@slippy.cwsent.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Mar 2021 15:25:58 -0000 On 29 Mar 2021, at 17:16, Cy Schubert wrote: > In message <18DC1EA9-ABFC-4A06-8710-A3068370EC52@FreeBSD.org>, > "Kristof > Provost > " writes: >> On 29 Mar 2021, at 16:03, Cy Schubert wrote: >>> In message <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org>, >>> "Kristof >>> Provost >>> " writes: >>>> Hi, >>>> >>>> There are several patches in the pipeline that require changes in >>>> pf’s >>>> interface between kernel and userspace. >>>> In the past these have been handled in multiple ways. Either by >>>> simply >>>> making the change, breaking binary compatibility, or by introducing >>>> a >>>> v2 >>>> ioctl (e.g. DIOCADDALTQV1). >>>> >>>> While one is better than the other neither is wholly satisfying. >>>> New >>>> versions of calls constitute a maintenance burden after all. >>>> >>>> I’d like to change the ioctl interface to use nvlists, >>>> which >>>> would >>>> make such extensions much easier, because fields can be optional. >>>> That is, if userspace doesn’t supply the >>>> ‘shinynewfeature’ field >>>> the kernel can assume the default value and things just work. >>>> Similarly, >>>> if the kernel supplies a ’shinynewfeature’ >>>> which userspace >>>> doesn’t >>>> know about it’s simply ignored. >>>> >>>> The rough plan is to introduce nvlist versions of the get/add rules >>>> calls for now. Others will follow as the need presents itself. >>>> As these are new ioctls it is safe to MFC them to stable/12 and >>>> stable/13. >>>> The old interface will remain supported in those branches, but >>>> I’d >>>> like to remove it from main (and thus FreeBSD 14). >>>> >>>> As part of this effort I may end up splitting off the ioctl >>>> interface >>>> code from pfctl into libpfctl, which should make reuse of that code >>>> easier. >>>> >>>> I hope to post preliminary patches in the coming week. >>>> >>>> Thoughts? Objections? >>> >>> Kernel and userland should be, I'd say must be, kept in sync. We >>> have >>> many >>> examples of userland and kernel not being in sync over the years. >>> For >>> ipfilter, I've made incompatible changes to data structures >>> requiring >>> userand and kernel be in sync. These are few and far between. >>> >>> I've gotten away with this because there is no third party software >>> that >>> relies on the ipfilter kernel interfaces. I could be wrong but I >>> doubt >>> there may be third party software requiring pf ABI compatibility. >>> But >>> if >>> there is then verstioned library interfaces are required. >>> >>> Given that the advice is to keep kernel and userland in sync there >>> probably >>> is no requirement for an UPDATING entry but that would be your call. >>> >> There are out-of-tree users of the pf ioctl interface. >> security/expiretable[1] for example. >> security/snort2pfcd appears to as well. >> sysutils/pfstat and sysutils/pftop use the ioctl interface as well, >> although not the three specific calls of immediate interest. > > This complicates things. IMO you'll probably need versioned function > calls > for at least 13-STABLE EOL. Or, versioning the data structures passed > into > the kernel such that the new fields are at the tail of the existing > structures. > That’s essentially the plan. I plan to keep the existing definitions (of both structure and ioctl numbers) in stable/12 and stable/13. They’ll disappear in main (i.e. 14). Alongside we’ll introduce new nvlist variants for those calls, which will have the new features. >> I’m trying to work out how many examples there are, because one >> way or >> the other they’re going to have to cope with changes. >> >> So, I’d prefer to not just change the definitions of structs, >> even if >> we’ve done that in the past. struct pf_rule contains a few >> peculiarities from historical mistakes that I hope to correct now. > > Technical debt is difficult to eliminate. We either fix it, paying it > off > in one lump sum or we pay it off through aggravation and design > limitations, with interest, over time. > Indeed. To take struct pf_rule as an example: it contains counter_u64’s, which don’t really work for userspace, so we’ve added uint64_t versions of those variables. Now the struct has two version of the same field. That can be cleaned up once the ioctls which use the struct have been removed (so on main only). My plan is to remove the struct definition from the kernel’s headers (again, once there are alternative ioctls and only in main), moving it into libpfctl. Then there will be nothing to stop us from removing the counter_u64 versions of those fields, cleaning up the struct. > Given that pf uses ioctl, versioned function calls won't help. A new > ioctl > may be the only answer. If you do choose this, add an identifier and > version number to the head of each new struct to future proof pf. > The nvlist versions will be much more flexible, so embedding a version number seem redundant. Best regards, Kristof From owner-freebsd-pf@freebsd.org Mon Mar 29 14:03:53 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A3818578719; Mon, 29 Mar 2021 14:03:53 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8Dp03SPcz3RCX; Mon, 29 Mar 2021 14:03:51 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.66.148.124]) by shaw.ca with ESMTPA id QsURlPdAJHmS3QsUSlmu8x; Mon, 29 Mar 2021 08:03:47 -0600 X-Authority-Analysis: v=2.4 cv=MaypB7zf c=1 sm=1 tr=0 ts=6061de43 a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=xqWC_Br6kY4A:10 a=8nJEP1OIZ-IA:10 a=dESyimp9J3IA:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=EkcXrb_YAAAA:8 a=6fP0Wl_uBWtMcdXx8xYA:9 a=wPNLvfGTeEIA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=LK5xJRSDVpKd5WXXoEvA:22 a=RBBcRewTFc8P4JkPnay6:22 Received: from slippy.cwsent.com (slippy [IPv6:fc00:1:1:1::5b]) by spqr.komquats.com (Postfix) with ESMTPS id B57298D0; Mon, 29 Mar 2021 07:03:34 -0700 (PDT) Received: from slippy (localhost [127.0.0.1]) by slippy.cwsent.com (8.16.1/8.16.1) with ESMTP id 12TE3Y2H094131; Mon, 29 Mar 2021 07:03:34 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <202103291403.12TE3Y2H094131@slippy.cwsent.com> X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: "Kristof Provost" cc: "FreeBSD pf" , freebsd-arch@freebsd.org Subject: Re: [RFC] pf ioctl changes In-reply-to: <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org> References: <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org> Comments: In-reply-to "Kristof Provost" message dated "Sat, 27 Mar 2021 12:54:28 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Mon, 29 Mar 2021 07:03:34 -0700 X-CMAE-Envelope: MS4xfFKcENIMkskoHVgzpRCsxACpuWR3pzFJNLMWZUc1GHdRyeS/vKuaKb06r2JbgL8YSDKuvnpnFIwwGCNFH7riq6x406XjtiYc7nCpORvSV1RkodTJln3u GffcTr7ud+xIxI2TAOf3u580XkB6Ui1SG9NBZY/NX1RpN5PPMkIOq3Ffy2SGl6e+eWTfMVbMsLevN+uej+6bWXPR3dpvvWmwS2cC6WPCeRc/zLBVDg5QEx6x U4HqtZGHzmKm6hnp0QfB2j0QxKsyC+7t6pTJLpjvOnA= X-Rspamd-Queue-Id: 4F8Dp03SPcz3RCX X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.136.137) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-1.70 / 15.00]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; RWL_MAILSPIKE_GOOD(0.00)[64.59.136.137:from]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[70.66.148.124:received]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[64.59.136.137:from]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[cschubert.com: no valid DMARC record]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[64.59.136.137:from:127.0.2.255]; RCVD_IN_DNSWL_LOW(-0.10)[64.59.136.137:from]; RCVD_TLS_LAST(0.00)[]; R_SPF_NA(0.00)[no SPF record]; MAILMAN_DEST(0.00)[freebsd-arch,freebsd-pf] X-Mailman-Approved-At: Mon, 29 Mar 2021 15:42:19 +0000 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Mar 2021 14:03:53 -0000 In message <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org>, "Kristof Provost " writes: > Hi, > > There are several patches in the pipeline that require changes in pf’s > interface between kernel and userspace. > In the past these have been handled in multiple ways. Either by simply > making the change, breaking binary compatibility, or by introducing a v2 > ioctl (e.g. DIOCADDALTQV1). > > While one is better than the other neither is wholly satisfying. New > versions of calls constitute a maintenance burden after all. > > I’d like to change the ioctl interface to use nvlists, which would > make such extensions much easier, because fields can be optional. > That is, if userspace doesn’t supply the ‘shinynewfeature’ field > the kernel can assume the default value and things just work. Similarly, > if the kernel supplies a ’shinynewfeature’ which userspace doesn’t > know about it’s simply ignored. > > The rough plan is to introduce nvlist versions of the get/add rules > calls for now. Others will follow as the need presents itself. > As these are new ioctls it is safe to MFC them to stable/12 and > stable/13. > The old interface will remain supported in those branches, but I’d > like to remove it from main (and thus FreeBSD 14). > > As part of this effort I may end up splitting off the ioctl interface > code from pfctl into libpfctl, which should make reuse of that code > easier. > > I hope to post preliminary patches in the coming week. > > Thoughts? Objections? Kernel and userland should be, I'd say must be, kept in sync. We have many examples of userland and kernel not being in sync over the years. For ipfilter, I've made incompatible changes to data structures requiring userand and kernel be in sync. These are few and far between. I've gotten away with this because there is no third party software that relies on the ipfilter kernel interfaces. I could be wrong but I doubt there may be third party software requiring pf ABI compatibility. But if there is then verstioned library interfaces are required. Given that the advice is to keep kernel and userland in sync there probably is no requirement for an UPDATING entry but that would be your call. -- Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org The need of the many outweighs the greed of the few. From owner-freebsd-pf@freebsd.org Mon Mar 29 16:16:19 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 77E3157C426; Mon, 29 Mar 2021 16:16:19 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8Hkq0tsRz3sTF; Mon, 29 Mar 2021 16:16:18 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.66.148.124]) by shaw.ca with ESMTPA id QuYflQZRXHmS3QuYglnPLd; Mon, 29 Mar 2021 10:16:17 -0600 X-Authority-Analysis: v=2.4 cv=MaypB7zf c=1 sm=1 tr=0 ts=6061fd51 a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=xqWC_Br6kY4A:10 a=8nJEP1OIZ-IA:10 a=dESyimp9J3IA:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=EkcXrb_YAAAA:8 a=T-UDMwMZcZjofG2IphkA:9 a=wPNLvfGTeEIA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=LK5xJRSDVpKd5WXXoEvA:22 a=RBBcRewTFc8P4JkPnay6:22 Received: from slippy.cwsent.com (slippy [IPv6:fc00:1:1:1::5b]) by spqr.komquats.com (Postfix) with ESMTPS id 0BEBD218; Mon, 29 Mar 2021 09:16:04 -0700 (PDT) Received: from slippy (localhost [127.0.0.1]) by slippy.cwsent.com (8.16.1/8.16.1) with ESMTP id 12TGG3IL005274; Mon, 29 Mar 2021 09:16:03 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <202103291616.12TGG3IL005274@slippy.cwsent.com> X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: "Kristof Provost" cc: "Cy Schubert" , "FreeBSD pf" , freebsd-arch@freebsd.org Subject: Re: [RFC] pf ioctl changes In-reply-to: <75FA4097-ED2A-4B96-9C90-E82F49F7764B@FreeBSD.org> References: <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org> <202103291403.12TE3Y2H094131@slippy.cwsent.com> <18DC1EA9-ABFC-4A06-8710-A3068370EC52@FreeBSD.org> <202103291516.12TFGjTi004433@slippy.cwsent.com> <75FA4097-ED2A-4B96-9C90-E82F49F7764B@FreeBSD.org> Comments: In-reply-to "Kristof Provost" message dated "Mon, 29 Mar 2021 17:25:55 +0200." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Mon, 29 Mar 2021 09:16:03 -0700 X-CMAE-Envelope: MS4xfJOEuTOA14hIp35c38ebBKeIOm9E3yWSo/9VEgmcA3yUmVrSHvFaIMI+xqViwqt7QvzzM7Xx5seMQGnFk4t881DsHynJPAuIc8mxYsMp7NDlHTw+hkKe 8xFClUsiLEKjMaX2lL+4/9fSOGWMA1K68S6hHe8ef6kDmnN158vgP+aATWUkl5xYgc5juyMStNBAAY8d4ZZhYh5PZmZO7EvDETG2oiSN36uShAAFj/ISna27 5L3lIx4cGxwSK9rrqmPnt/0eS7bf2aABIQLGup6lA0E= X-Rspamd-Queue-Id: 4F8Hkq0tsRz3sTF X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Mar 2021 16:16:19 -0000 In message <75FA4097-ED2A-4B96-9C90-E82F49F7764B@FreeBSD.org>, "Kristof Provost " writes: > On 29 Mar 2021, at 17:16, Cy Schubert wrote: > > In message <18DC1EA9-ABFC-4A06-8710-A3068370EC52@FreeBSD.org>, > > "Kristof > > Provost > > " writes: > >> On 29 Mar 2021, at 16:03, Cy Schubert wrote: > >>> In message <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org>, > >>> "Kristof > >>> Provost > >>> " writes: > >>>> Hi, > >>>> > >>>> There are several patches in the pipeline that require changes in > >>>> pf’s > >>>> interface between kernel and userspace. > >>>> In the past these have been handled in multiple ways. Either by > >>>> simply > >>>> making the change, breaking binary compatibility, or by introducing > >>>> a > >>>> v2 > >>>> ioctl (e.g. DIOCADDALTQV1). > >>>> > >>>> While one is better than the other neither is wholly satisfying. > >>>> New > >>>> versions of calls constitute a maintenance burden after all. > >>>> > >>>> I’d like to change the ioctl interface to use nvlists, > >>>> which > >>>> would > >>>> make such extensions much easier, because fields can be optional. > >>>> That is, if userspace doesn’t supply the > >>>> ‘shinynewfeature’ field > >>>> the kernel can assume the default value and things just work. > >>>> Similarly, > >>>> if the kernel supplies a ’shinynewfeature’ > >>>> which userspace > >>>> doesn’t > >>>> know about it’s simply ignored. > >>>> > >>>> The rough plan is to introduce nvlist versions of the get/add rules > >>>> calls for now. Others will follow as the need presents itself. > >>>> As these are new ioctls it is safe to MFC them to stable/12 and > >>>> stable/13. > >>>> The old interface will remain supported in those branches, but > >>>> I’d > >>>> like to remove it from main (and thus FreeBSD 14). > >>>> > >>>> As part of this effort I may end up splitting off the ioctl > >>>> interface > >>>> code from pfctl into libpfctl, which should make reuse of that code > >>>> easier. > >>>> > >>>> I hope to post preliminary patches in the coming week. > >>>> > >>>> Thoughts? Objections? > >>> > >>> Kernel and userland should be, I'd say must be, kept in sync. We > >>> have > >>> many > >>> examples of userland and kernel not being in sync over the years. > >>> For > >>> ipfilter, I've made incompatible changes to data structures > >>> requiring > >>> userand and kernel be in sync. These are few and far between. > >>> > >>> I've gotten away with this because there is no third party software > >>> that > >>> relies on the ipfilter kernel interfaces. I could be wrong but I > >>> doubt > >>> there may be third party software requiring pf ABI compatibility. > >>> But > >>> if > >>> there is then verstioned library interfaces are required. > >>> > >>> Given that the advice is to keep kernel and userland in sync there > >>> probably > >>> is no requirement for an UPDATING entry but that would be your call. > >>> > >> There are out-of-tree users of the pf ioctl interface. > >> security/expiretable[1] for example. > >> security/snort2pfcd appears to as well. > >> sysutils/pfstat and sysutils/pftop use the ioctl interface as well, > >> although not the three specific calls of immediate interest. > > > > This complicates things. IMO you'll probably need versioned function > > calls > > for at least 13-STABLE EOL. Or, versioning the data structures passed > > into > > the kernel such that the new fields are at the tail of the existing > > structures. > > > That’s essentially the plan. I plan to keep the existing definitions > (of both structure and ioctl numbers) in stable/12 and stable/13. > They’ll disappear in main (i.e. 14). > > Alongside we’ll introduce new nvlist variants for those calls, which > will have the new features. > > >> I’m trying to work out how many examples there are, because one > >> way or > >> the other they’re going to have to cope with changes. > >> > >> So, I’d prefer to not just change the definitions of structs, > >> even if > >> we’ve done that in the past. struct pf_rule contains a few > >> peculiarities from historical mistakes that I hope to correct now. > > > > Technical debt is difficult to eliminate. We either fix it, paying it > > off > > in one lump sum or we pay it off through aggravation and design > > limitations, with interest, over time. > > > Indeed. > > To take struct pf_rule as an example: it contains counter_u64’s, which > don’t really work for userspace, so we’ve added uint64_t versions of > those variables. Now the struct has two version of the same field. > That can be cleaned up once the ioctls which use the struct have been > removed (so on main only). My plan is to remove the struct definition > from the kernel’s headers (again, once there are alternative ioctls > and only in main), moving it into libpfctl. > Then there will be nothing to stop us from removing the counter_u64 > versions of those fields, cleaning up the struct. > > > Given that pf uses ioctl, versioned function calls won't help. A new > > ioctl > > may be the only answer. If you do choose this, add an identifier and > > version number to the head of each new struct to future proof pf. > > > The nvlist versions will be much more flexible, so embedding a version > number seem redundant. This is probably the best plan. It of course adds some MFC pain or the requirement to commit directly to -STABLE when fixing serious bugs but it's manageable. -- Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org The need of the many outweighs the greed of the few. From owner-freebsd-pf@freebsd.org Thu Apr 1 11:37:15 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9B9585C9B46 for ; Thu, 1 Apr 2021 11:37:15 +0000 (UTC) (envelope-from f0x0ff@gmail.com) Received: from mail-ot1-x335.google.com (mail-ot1-x335.google.com [IPv6:2607:f8b0:4864:20::335]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FB1PQ5ZWXz3QX8 for ; Thu, 1 Apr 2021 11:37:14 +0000 (UTC) (envelope-from f0x0ff@gmail.com) Received: by mail-ot1-x335.google.com with SMTP id w31-20020a9d36220000b02901f2cbfc9743so1802522otb.7 for ; Thu, 01 Apr 2021 04:37:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=TgkVzRDaS/RHAg8oFvWl8vgG1DrkxXjEUONPqeYe+Ck=; b=QzMcSo3nF221Y5hv/Yr+yh/ZE418rfXiYfCsdAsnqVRUeh020OmOyoaHNp+++FmsTF yb9mUFcCFW0vQmSeICRXUnqfaRsBtc4zR52M5pmujyut1BFzDcVUSjn1eNg3CKDr47v6 NTns0O5BMmVo49Nj7d7KNsM3DA92sVU0+CAcbCYneVL91qESJr+zEgmk1x6986Tgjau2 GyKPbS5Qxq9R5LwtBZVyfGwuHDJYjs/sc08FiP4kwzO/qN7KVoPNpZOTr7Eb3CgQ7LUU IbLP54Bx3XFDQ7inoZltWMy171jHEpcipHWoMl9VfY5LgJFqPcdR30mErN8TN2OfzvNp D1rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=TgkVzRDaS/RHAg8oFvWl8vgG1DrkxXjEUONPqeYe+Ck=; b=f7WCfKM1xAzBIRbFUvQdmV+bWpwCa3achi7c6S7YqQO8qHKqtpxC4ltCPfI7SYz8ea O2bYI881s8z48AEB3ZGKEO7mi2hC5KKPNCdjLM4ozvIMxM3sFc4slNGaEBa4RtAMcUjT yl4tgW3t71+O+aygtJZR2j0qyxOAkq0yJSoryTUeU3ug+zaQ5AbNiiztsIebVmPzqnU3 Mq+VwNGXrSfPb5U8YHpxrjf++DzQQEjaOM5A/8kfd6DuzJakgMkMky4PZkCNU/rimVTC zB3M269ywDha1PZBYWImxzvzi5Se2FSLvceK6Pd6sEfF+oc5ReWNBPpmpjQOipehHLpa PwQw== X-Gm-Message-State: AOAM530D/d74fakeRpL/KEMfzBOcgmBvIvVxf1Xa9UeTg/Ka6GgwYn8T wQINzN2wR24wrR5ZYW8CrPtcTY1uTURVB/Cof5zQO0CAx6vLTQ== X-Google-Smtp-Source: ABdhPJx+r1z2d67FADryI/IRY8lzBpzHPfevVEwspwYTxg1FD6vXRzXhV/J1Xg4SvlUXO+qU8dqWlm5xeDZJSoyW83k= X-Received: by 2002:a9d:550b:: with SMTP id l11mr6366678oth.218.1617277033839; Thu, 01 Apr 2021 04:37:13 -0700 (PDT) MIME-Version: 1.0 From: Plamen Mladenov Date: Thu, 1 Apr 2021 14:37:03 +0300 Message-ID: Subject: pfsync - Active/Active + defer on To: freebsd-pf@freebsd.org X-Rspamd-Queue-Id: 4FB1PQ5ZWXz3QX8 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=QzMcSo3n; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of f0x0ff@gmail.com designates 2607:f8b0:4864:20::335 as permitted sender) smtp.mailfrom=f0x0ff@gmail.com X-Spamd-Result: default: False [-2.02 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::335:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::335:from:127.0.2.255]; NEURAL_SPAM_SHORT(0.98)[0.984]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::335:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-pf] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 11:37:15 -0000 Hello, I'm trying to setup an active-active PF cluster (consists of 2 freebsd hosts:FW1 and FW2) using pfsync and dynamic routing protocol (CARP is not used at this deployment) All works as expected when IN/OUT traffic for a single session is symmetrical (uses either FW1 or FW2). The problem I'm facing is when the traffic is asymmetrical For example: (1) Client ----------TCP SYN ---------> FW1 -------------------------------------> Server | pfsync | (2) Client <--------------------------------- FW2 <------- TCP SYN+ACK------- Server Client is sending TCP segment with SYN flag set which is received and allowed by FW1 and send to the Server. Server is replying with TCP segment with SYC and ACK flag sets (just as per TCP 3 way handshake), but this TCP segment is routed to FW2. At that time FW2 haven't received the SYNC-SENT session (from FW1) yet and therefore it denies that TCP segment. Few miliseconds after that FW2 gets the session from FW1, but the SYN+ACK is already dropped and a TCP re-transmission occurs. I've found that this behavior can be fixed with pfsync "defer" option, however based on my lab and prod tests - this option is not changing anything. As per my understandings, the initial packet should be delayed until session is replicated between both firewalls, but that's not the case. My other concern is that although the "defer" option is there (I can successfully turn it on/off and see it with ifconfig pfsync0) I can't find a word about it in man 4 pfsync on FreeBSD (unlike in OpenBSD documentation) which it makes me think - there is a reason why it's not in the man page. Can someone confirm - is pfsync "defer" option working on FreeBSD? Regard, Plamen Mladenov From owner-freebsd-pf@freebsd.org Fri Apr 2 18:59:55 2021 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0F56B57EF87; Fri, 2 Apr 2021 18:59:55 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FBq9l02BSz3mDY; Fri, 2 Apr 2021 18:59:55 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id CA6551BD1; Fri, 2 Apr 2021 18:59:54 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id A653422ED5; Fri, 2 Apr 2021 20:59:52 +0200 (CEST) From: "Kristof Provost" To: "FreeBSD pf" Cc: freebsd-arch@freebsd.org Subject: Re: [RFC] pf ioctl changes Date: Fri, 02 Apr 2021 20:59:51 +0200 X-Mailer: MailMate (1.13.2r5673) Message-ID: <4E521EB3-7D64-4292-B4F6-8E10AC2B0937@FreeBSD.org> In-Reply-To: <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org> References: <24E09373-EBCD-4ED1-8B59-A44E687F287E@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Apr 2021 18:59:55 -0000 On 27 Mar 2021, at 12:54, Kristof Provost wrote: > I hope to post preliminary patches in the coming week. > - https://reviews.freebsd.org/D29556 - https://reviews.freebsd.org/D29557 - https://reviews.freebsd.org/D29558 - https://reviews.freebsd.org/D29559 - https://reviews.freebsd.org/D29560 - https://reviews.freebsd.org/D29561 - https://reviews.freebsd.org/D29562 Best regards, Kristof