From nobody Wed Oct 16 14:19:40 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XTClr4pTqz5YZ8w for ; Wed, 16 Oct 2024 14:19:52 +0000 (UTC) (envelope-from girgen@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XTClr4F56z4QYY for ; Wed, 16 Oct 2024 14:19:52 +0000 (UTC) (envelope-from girgen@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1729088392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=Mmfy3D+3K7PLLRiIfxMNV0P0cX8ZTTLGf4JevnRDYZs=; b=cpZi/yhkLjc8x+uUGKsY4UyQmPHKP7K6JSp81YLKlUh6csz4mH7BS4imk124+HlRJZ44Yz ZBA2jSuqigi8Un5YUXb7lFhPTJoshAe0EGS6qP9vf1tj7DZ0/tTu53FxZnfrlaKi+D720S Hdn+3SfmcUV2ImfClJLD8f3pJtqZp0Xsmew5eu7V7IRMOEihG+nkYi6ut3zwSQENs4cz9l IkEWNImXExheDUwzwyzz+dyq7Y/X7HGBlLOtjAUBFMibEQkUkOLZsjeNXpx0o9VlL6pd0D C+JDlDTJjTxnhYSBoBkuKwhsP4v1f4WaY7WCzac3N9eo6I58bieJ2fvSmAIe+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1729088392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=Mmfy3D+3K7PLLRiIfxMNV0P0cX8ZTTLGf4JevnRDYZs=; b=og8u1jljlm2V6Mhx5g8swfRxBPUo0L99arzO4tODggpgWI/88Ybqyxn89/GMpCVEnmGAGG RiTdldEU3077eBFnU9G3f7nDgbFyZYmYt/A4U3ESlX0r8gdE0lZicBCailnuFnVCdiuMp6 EybNC3delUdh+Zsc7KUqn8XOG8H0cuQSoJTAcRhMpT7XBP5s1GXlpK2F1tCZAHx9wkL5lp RPt9plCMzUC6XFNpnXRVV63md9ma1L7MnYFELk/mp+4aZkvgZUbZ/czEKqGjOWZwxYvaY6 iS/82VD5gILUgcJMvya1FtwwQ6l9ppelaaD6zBwo6tP95pJuHbWKqS3BGrUUUg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1729088392; a=rsa-sha256; cv=none; b=MdrJWt55HmJHYjLnHpMxoRO/jOzp2FaI6bHanPgbKTbzZ4sCWhzu8nsUcIm1s9SCsM38Ba ejFlpP2am2cScxWeiUGC6amXlMAac8h/OzFc25myiqe0HkPa2YrHwPk+hUF+iyo8cUzMnf Qx4kovlLw/RtC57HeEPvv6biZf9BwERQVw4n2a62UPN5V1svjn0bdQ/VDRS6HP7HKpsYS7 BWI9lP6HzTJkfhAIZ08QjWBGgxIq7dHTShp/ep1h3HUo7FwCImSkfDKKmyfbVUDp88T5Rx A2KtvF/WyZc+WSPNGJHuFudMG1PEBjkj1OlN2j70yZ8pSa8PkTBcvMlH4Hh2kQ== Received: from smtpclient.apple (31-211-209-78.customers.ownit.se [31.211.209.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: girgen/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4XTClr1T5sz1SFd for ; Wed, 16 Oct 2024 14:19:52 +0000 (UTC) (envelope-from girgen@FreeBSD.org) From: Palle Girgensohn Content-Type: multipart/alternative; boundary="Apple-Mail=_AC6D5B8C-A6E6-413A-BF25-E00942506C27" List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3818.100.11.1.3\)) Subject: pf for netgraph jails? Message-Id: <7D5BD9CC-8A08-4C74-B2E6-E0437235F3B1@FreeBSD.org> Date: Wed, 16 Oct 2024 16:19:40 +0200 To: "freebsd-net@freebsd.org" X-Mailer: Apple Mail (2.3818.100.11.1.3) --Apple-Mail=_AC6D5B8C-A6E6-413A-BF25-E00942506C27 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi! Using FreeBSD-14.1, I have a rather simple setup with jails using = netgraph (using the `/usr/share/examples/jails/jng` script and "model"). The host machine has two interfaces: bnxt0: (external, has no IP#) bnxt1: 192.168.1.79/24 jail.conf: -- host.hostname =3D "$name.example.com "; # = hostname path =3D "/jails/$name"; exec.clean; exec.system_user =3D "root"; exec.jail_user =3D "root"; vnet; # netgraph vnet.interface =3D ng0_$name, ng1_$name; # vnet interface(s) exec.prestart +=3D "jng bridge $name bnxt0 bnxt1"; # bridge = interface(s) exec.poststop +=3D "jng shutdown $name"; # destroy interface(s) exec.start +=3D "/bin/sh /etc/rc"; exec.stop =3D "/bin/sh /etc/rc.shutdown jail"; exec.consolelog =3D "/var/log/jail_$name.log"; mount.devfs; # mount devfs mount.fdescfs; devfs_ruleset=3D5; allow.mlock=3D1; mount.fstab=3D"/etc/fstab.$name"; fw {} -- which creates a single jail `fw'. /jails/fw/etc/rc.conf: -- hostname=3Dfw.example.com ifconfig_ng0_fw=3D"inet 1.2.3.4/26" ifconfig_ng1_fw=3D"inet 192.168.1.212/24" defaultrouter=3D"1.2.3.1" sshd_enable=3D"yes" -- $ sudo ngctl list There are 8 total nodes: Name: ngctl69965 Type: socket ID: 00000021 Num hooks: 0 Name: bnxt0 Type: ether ID: 00000001 Num hooks: 2 Name: bnxt1 Type: ether ID: 00000002 Num hooks: 2 Name: ue0 Type: ether ID: 00000003 Num hooks: 0 Name: bnxt0bridge Type: bridge ID: 00000009 Num hooks: 3 Name: ng0_fw Type: eiface ID: 0000000e Num hooks: 1 Name: bnxt1bridge Type: bridge ID: 00000016 Num hooks: 3 Name: ng1_fw Type: eiface ID: 0000001b Num hooks: 1 I plan to create a reasonably large number of jails this way, by just = adding jname {} to the jail.conf file. Now, I would like to have a simple generic setup with pf filtering out = unwanted ports from incoming traffic. I tried this simplistic setup: -- ext_if =3D "bnxt0" int_if =3D "bnxt1" block in on $ext_if dns_servers =3D "{ 192.168.1.194, 1.2.3.9, 8.8.8.8, 1.1.1.1 }" pass in on $ext_if proto { tcp udp } from $dns_servers to any port 53 pass in on $ext_if proto tcp from any to any port { 80 443 22 } -- but nothing happens, everything is passed directly into the jail: nc -l 4444 (inside the jail) and I can just telnet 1.2.3.4 4444 I assume I'm doing some simple mistake here, but find very little = information wrt the combo of netgraph, pf and jails. Any tips? I tried = configuring pf to work on the bridge interface but no difference. What = am I missing here? Palle= --Apple-Mail=_AC6D5B8C-A6E6-413A-BF25-E00942506C27 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi!
Using = FreeBSD-14.1, I = have a rather simple setup with jails using netgraph (using the = `/usr/share/examples/jails/jng` script and "model").

The host = machine has two interfaces:
bnxt0: = (external, has no IP#)
bnxt1: = 192.168.1.79/24

jail.conf:

--
host.hostname =3D "$name.example.com";    # hostname

path =3D = "/jails/$name";

exec.clean;
exec.system_user =3D "root";
exec.jail_user =3D "root";

vnet;

# = netgraph
vnet.interface =3D ng0_$name, = ng1_$name;          # vnet = interface(s)
exec.prestart +=3D "jng bridge = $name bnxt0 bnxt1";        # bridge = interface(s)
exec.poststop +=3D "jng shutdown = $name";  # destroy interface(s)

exec.start +=3D "/bin/sh = /etc/rc";
exec.stop =3D "/bin/sh = /etc/rc.shutdown jail";
exec.consolelog =3D "/var/log/jail_$name.log";
mount.devfs;    # mount devfs

mount.fdescfs;
devfs_ruleset=3D5;

allow.mlock=3D1;

mount.fstab=3D"/etc/fstab.$name";

fw = {}
--

which creates a single jail = `fw'.

/jails/fw/etc/rc.conf:
--
hostname=3Dfw.example.com
ifconfig_ng0_fw=3D"inet 1.2.3.4/26"
ifconfig_ng1_fw=3D"inet 192.168.1.212/24"
defaultrouter=3D"1.2.3.1"

sshd_enable=3D"yes"
--


$ sudo ngctl = list
There are 8 total nodes:
 Name: ngctl69965 =      Type: socket =          ID: 00000021 =   Num hooks: 0
 Name: = bnxt0           Type: = ether           ID: = 00000001   Num hooks: 2
 Name: = bnxt1           Type: = ether           ID: = 00000002   Num hooks: 2
 Name: = ue0 =             Ty= pe: ether =           ID: 00000003 =   Num hooks: 0
 Name: = bnxt0bridge     Type: bridge =          ID: 00000009 =   Num hooks: 3
 Name: = ng0_fw          Type: = eiface          ID: = 0000000e   Num hooks: 1
 Name: = bnxt1bridge     Type: bridge =          ID: 00000016 =   Num hooks: 3
 Name: = ng1_fw          Type: = eiface          ID: = 0000001b   Num hooks: 1

I plan to create a reasonably = large number of jails this way, by just adding jname {} to the jail.conf = file.

Now, I would like to have a simple generic setup with pf = filtering out unwanted ports from incoming traffic.

I tried this = simplistic setup:
--
ext_if =3D = "bnxt0"
int_if =3D "bnxt1"


block in on $ext_if


dns_servers =3D "{ 192.168.1.194, = 1.2.3.9, 8.8.8.8, 1.1.1.1 }"


pass in on $ext_if proto { tcp udp } from $dns_servers = to any port 53
pass in on $ext_if proto tcp from = any to any port { 80 443 22 }
--

but nothing = happens, everything is passed directly into the jail:

nc -l 4444 =   (inside the jail)

and I can just telnet 1.2.3.4 = 4444

I assume I'm doing some simple mistake here, but find = very little information wrt the combo of netgraph, pf and jails. Any = tips? I tried configuring pf to work  on the bridge interface but = no difference. What am I missing here?

Palle= --Apple-Mail=_AC6D5B8C-A6E6-413A-BF25-E00942506C27--