Date: Mon, 8 Jun 2026 10:29:30 +0100 From: Doug Rabson <dfr@rabson.org> To: Kristof Provost <kp@freebsd.org> Cc: freebsd-jail@freebsd.org Subject: Re: Running pfctl inside a jail Message-ID: <CACA0VUhPCX9AzJzaNYF=25PRgU4TeUMPn36CZhBrb8wPDdFX9w@mail.gmail.com> In-Reply-To: <7C23D3B8-1A14-41B7-839A-580DB61E0403@FreeBSD.org> References: <CACA0VUhJ78ES4AGMtLvZOVRJLoK=w=Vot%2BKSbx3Q=ikdC8UkFQ@mail.gmail.com> <96E80293-2013-452F-859C-B725EA7963CF@FreeBSD.org> <CACA0VUhigsCrqxrBySxptLCfh_K6%2BCb%2BT%2BDSJZgHnSMr0i9WOQ@mail.gmail.com> <7C23D3B8-1A14-41B7-839A-580DB61E0403@FreeBSD.org>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Mon, 8 Jun 2026 at 09:37, Kristof Provost <kp@freebsd.org> wrote: > On 8 Jun 2026, at 10:00, Doug Rabson wrote: > > In my smallest test-case, the host and jail use the same root filesystem > > and the host is running 15.0-RELEASE-p8. I haven't tested with stable/15 > > yet. This reproduces the problem for me: > > > > $ sudo pfctl -s nat > > nat on bridge42 inet from <cni-nat> to any -> (bridge42) round-robin > > nat on bridge42 inet6 from <cni-nat> to ! ff00::/8 -> (bridge42) > round-robin > > nat-anchor "cni-rdr/*" all > > rdr-anchor "cni-rdr/*" all > > $ cat jail-pfctl-15 > > #! /bin/sh > > j=$(jail -ic name=pfctl-in-jail15 ip4=inherit ip6=inherit path=/ persist) > > jexec $j pfctl -s nat > > jail -r $j > > $ sudo ./jail-pfctl-15 > > pfctl: DIOCGETRULES: Operation not permitted > > $ freebsd-version -k > > 15.0-RELEASE-p8 > > > > > > Do the pf unit tests cover the case where the jail shares the host vnet? > > > Oh. No, no they do not. That’s just plain not supposed to work. > Historically, though, it has always worked, at least as far back as FreeBSD-13 so this is a regression. > You only ever get to manage your own pf instance, never the one of a > parent jail. > It seems reasonable (to me at least) that if a jail inherits a vnet from its parent, it should be able to manage that vnet. I see some evidence in the history that at least parts of netlink are intended to work for jails which don't have their own vnet (e.g. https://cgit.freebsd.org/src/commit/sys/netlink?id=04f75b980293d517558990a7fda6900445edcac6). I would also like to be able to create interfaces in non-vnet jails but that is another conversation entirely. For what it's worth, this pattern of delegating network management to a privileged container is common on Linux. For instance, the Linux version of kube-proxy as well as the popular Calico cluster networking stack, uses this pattern to manage interfaces and iptable rule sets. Doug. [-- Attachment #2 --] <div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Mon, 8 Jun 2026 at 09:37, Kristof Provost <<a href="mailto:kp@freebsd.org">kp@freebsd.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 8 Jun 2026, at 10:00, Doug Rabson wrote:<br> > In my smallest test-case, the host and jail use the same root filesystem<br> > and the host is running 15.0-RELEASE-p8. I haven't tested with stable/15<br> > yet. This reproduces the problem for me:<br> ><br> > $ sudo pfctl -s nat<br> > nat on bridge42 inet from <cni-nat> to any -> (bridge42) round-robin<br> > nat on bridge42 inet6 from <cni-nat> to ! ff00::/8 -> (bridge42) round-robin<br> > nat-anchor "cni-rdr/*" all<br> > rdr-anchor "cni-rdr/*" all<br> > $ cat jail-pfctl-15<br> > #! /bin/sh<br> > j=$(jail -ic name=pfctl-in-jail15 ip4=inherit ip6=inherit path=/ persist)<br> > jexec $j pfctl -s nat<br> > jail -r $j<br> > $ sudo ./jail-pfctl-15<br> > pfctl: DIOCGETRULES: Operation not permitted<br> > $ freebsd-version -k<br> > 15.0-RELEASE-p8<br> ><br> ><br> > Do the pf unit tests cover the case where the jail shares the host vnet?<br> ><br> Oh. No, no they do not. That’s just plain not supposed to work.<br></blockquote><div><br></div><div>Historically, though, it has always worked, at least as far back as FreeBSD-13 so this is a regression.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">You only ever get to manage your own pf instance, never the one of a parent jail.<br></blockquote><div><br></div><div>It seems reasonable (to me at least) that if a jail inherits a vnet from its parent, it should be able to manage that vnet. I see some evidence in the history that at least parts of netlink are intended to work for jails which don't have their own vnet (e.g. <a href="https://cgit.freebsd.org/src/commit/sys/netlink?id=04f75b980293d517558990a7fda6900445edcac6">https://cgit.freebsd.org/src/commit/sys/netlink?id=04f75b980293d517558990a7fda6900445edcac6</a>). I would also like to be able to create interfaces in non-vnet jails but that is another conversation entirely.</div><div><br></div><div>For what it's worth, this pattern of delegating network management to a privileged container is common on Linux. For instance, the Linux version of kube-proxy as well as the popular Calico cluster networking stack, uses this pattern to manage interfaces and <span class="G8OMXb ng">iptable</span> rule sets.</div><div> </div><div>Doug.</div></div></div>home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACA0VUhPCX9AzJzaNYF=25PRgU4TeUMPn36CZhBrb8wPDdFX9w>