Date: Sun, 13 Jul 2025 16:55:45 -0400 From: Michael Sierchio <kudzu@tenebras.com> To: Christos Chatzaras <chris@cretaforce.gr> Cc: freebsd-net <freebsd-net@freebsd.org>, FreeBSD Questions Mailing List <freebsd-questions@freebsd.org> Subject: Re: Issues with IPFW skipto Rule and Whitelisting Logic Message-ID: <CAHu1Y71cuy5sHEMMrZsAve%2B2RAn7ndQqf2jtm-gyFBS-PDEEiA@mail.gmail.com> In-Reply-To: <3A01EF48-EBE8-48C3-9C66-6A250A240341@cretaforce.gr> References: <3A01EF48-EBE8-48C3-9C66-6A250A240341@cretaforce.gr>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000008599610639d5c6e5
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
I haven't had a chance to read this in detail, but
what about UDP? Most DNS traffic is UDP.
And these lines are subtly wrong:
*$cmd 10031 allow tcp from me to any dst-port 443 out via $pif setup
keep-state$cmd 10033 allow tcp from any to me dst-port 443 in via $pif
setup keep-state*
because 'via' causes these rules to catch packets twice as they're
processed by the kernel. IMHO these should be
$cmd 10031 allow tcp from me to any dst-port 443 out xmit $pif setup
keep-state
$cmd 10033 allow tcp from any to me dst-port 443 in recv $pif setup
keep-state
I'll have more comments when I get a chance to peruse fully.
On Sun, Jul 13, 2025 at 4:41=E2=80=AFPM Christos Chatzaras <chris@cretaforc=
e.gr>
wrote:
> I am using ipfw with these rules:
>
> ----------------
> #!/bin/sh
>
> # Set rules command prefix
> cmd=3D"ipfw -q add "
> cmd2=3D"ipfw -q "
>
> # Public interface
> pif=3D`ifconfig -l | awk '{ print $1 }'`
>
> # Flush all rules
> ipfw -q -f flush
>
> # Flush all tables
> $cmd2 table 1 flush
> $cmd2 table 3 flush
>
> # Allow loopback and deny loopback spoofing
> $cmd 00010 allow ip from any to any via lo0
> $cmd 00020 deny ip from any to 127.0.0.0/8
> $cmd 00030 deny ip from 127.0.0.0/8 to any
>
> # Catch spoofing from outside.
> $cmd 00031 deny ip from any to any not antispoof via $pif
>
> # Checks stateful rules
> $cmd 00050 check-state
> $cmd 00060 deny tcp from any to any established
>
> # ALLOW WHITELIST - IGNORE RULE 00100
> $cmd2 00070 add skipto 00101 ip from 'table(3)' to any
>
> # DENY INCOMING LIST
> $cmd 00100 reset ip from 'table(1)' to any
>
> # ICMP
> $cmd 01010 allow icmp from any to any out via $pif keep-state
> $cmd 01011 allow icmp from any to any in via $pif
>
> # WWW
> $cmd 10031 allow tcp from me to any dst-port 443 out via $pif setup
> keep-state
> $cmd 10033 allow tcp from any to me dst-port 443 in via $pif setup
> keep-state
>
> # Deny everything else, and log it
> $cmd 56599 deny log all from any to any
> ----------------
>
> And ipfw list includes:
>
> ----------------
> 00070 skipto 101 ip from table(3) to any
> 00100 reset ip from table(1) to any
> ----------------
>
> Currently, table(1) holds about 1.9 million entries (both individual IPs
> and subnets), while table(3) contains about 10,000 entries (also a mix of
> single IPs and subnets).
>
> These tables are populated using this script few times per day:
>
> ----------------
> #!/bin/sh
>
> tempdir=3D$(mktemp -d /tmp/ipfw.XXXXXX)
> trap "rm -rf $tempdir" EXIT
>
> fetch -q -o "$tempdir/allow.txt" https://example.com/ipfw/allow.txt ||
> exit 1
> fetch -q -o "$tempdir/deny.txt" https://example.com/ipfw/deny.txt || exit
> 1
>
> update_table() {
> table=3D$1
> file=3D$2
> current_file=3D"$tempdir/current_table_$table.txt"
> ipfw -q table "$table" list | awk '{print $1}' | sed 's/\/32$//' |
> sort > "$current_file"
> cat "$file" | sed 's/\/32$//' | sort | uniq >
> "$tempdir/new_table_$table.txt"
>
> comm -13 "$tempdir/new_table_$table.txt" "$current_file" | while read
> -r ip; do
> [ -n "$ip" ] && ipfw -q table "$table" delete "$ip"
> done
>
> comm -23 "$tempdir/new_table_$table.txt" "$current_file" | while read
> -r ip; do
> [ -n "$ip" ] && ipfw -q table "$table" add "$ip"
> done
> }
>
> update_table 3 "$tempdir/allow.txt"
> update_table 1 "$tempdir/deny.txt"
> ----------------
>
> My intended logic is that any IP present in table(3) should always be
> allowed, even if it or its subnet also appears in table(1).
>
> For instance, 175.178.167.241 is in table(3), while 175.178.0.0/16 is
> present in table(1).
>
> After rebooting the server and populating the tables by running the updat=
e
> script for the first time, access from 175.178.167.241 works correctly.
> However, after subsequent runs of the update script - which only updates
> unrelated entries and does not modify 175.178.167.241 or 175.178.0.0/16 -
> access from 175.178.167.241 is no longer permitted.
>
> Additionally, when this issue arises, adding 175.178.0.0/16 to table(3)
> allows access again. Even after removing that entry, as long as
> 175.178.167.241 remains in table(3) and I wait for any active sessions to
> clear, access continues to work.
>
> Does anyone have any ideas about what could be causing this behavior?
>
--0000000000008599610639d5c6e5
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">I haven't had a chance to read this in detail, but <br=
><br>what about UDP?=C2=A0 Most DNS traffic is UDP.<br><br>And these lines =
are subtly wrong:<br><br><i>$cmd 10031 allow tcp from me to any dst-port 44=
3 out via $pif setup keep-state<br>$cmd 10033 allow tcp from any to me dst-=
port 443 in via $pif setup keep-state</i><div><i><br></i></div><div>because=
'via' causes these rules to catch packets twice as they're pro=
cessed by the kernel.=C2=A0 IMHO these should be<br><br>$cmd 10031 allow tc=
p from me to any dst-port 443 out xmit $pif setup keep-state<br>$cmd 10033 =
allow tcp from any to me dst-port 443 in recv $pif setup keep-state<br></di=
v><div><br></div><div>I'll have more comments when I get a chance to pe=
ruse fully.</div><div><br></div></div><br><br><div class=3D"gmail_quote gma=
il_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Sun, Jul 13, 2=
025 at 4:41=E2=80=AFPM Christos Chatzaras <<a href=3D"mailto:chris@creta=
force.gr">chris@cretaforce.gr</a>> wrote:<br></div><blockquote class=3D"=
gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20=
4,204,204);padding-left:1ex">I am using ipfw with these rules:<br>
<br>
----------------<br>
#!/bin/sh<br>
<br>
# Set rules command prefix<br>
cmd=3D"ipfw -q add "<br>
cmd2=3D"ipfw -q "<br>
<br>
# Public interface<br>
pif=3D`ifconfig -l | awk '{ print $1 }'`<br>
<br>
# Flush all rules<br>
ipfw -q -f flush<br>
<br>
# Flush all tables<br>
$cmd2 table 1 flush<br>
$cmd2 table 3 flush<br>
<br>
# Allow loopback and deny loopback spoofing<br>
$cmd 00010 allow ip from any to any via lo0<br>
$cmd 00020 deny ip from any to <a href=3D"http://127.0.0.0/8" rel=3D"norefe=
rrer" target=3D"_blank">127.0.0.0/8</a><br>
$cmd 00030 deny ip from <a href=3D"http://127.0.0.0/8" rel=3D"noreferrer" t=
arget=3D"_blank">127.0.0.0/8</a> to any<br>
<br>
# Catch spoofing from outside.<br>
$cmd 00031 deny ip from any to any not antispoof via $pif<br>
<br>
# Checks stateful rules<br>
$cmd 00050 check-state<br>
$cmd 00060 deny tcp from any to any established<br>
<br>
# ALLOW WHITELIST - IGNORE RULE 00100<br>
$cmd2 00070 add skipto 00101 ip from 'table(3)' to any<br>
<br>
# DENY INCOMING LIST<br>
$cmd 00100 reset ip from 'table(1)' to any<br>
<br>
# ICMP<br>
$cmd 01010 allow icmp from any to any out via $pif keep-state<br>
$cmd 01011 allow icmp from any to any in via $pif<br>
<br>
# WWW<br>
$cmd 10031 allow tcp from me to any dst-port 443 out via $pif setup keep-st=
ate<br>
$cmd 10033 allow tcp from any to me dst-port 443 in via $pif setup keep-sta=
te<br>
<br>
# Deny everything else, and log it<br>
$cmd 56599 deny log all from any to any<br>
----------------<br>
<br>
And ipfw list includes:<br>
<br>
----------------<br>
00070 skipto 101 ip from table(3) to any<br>
00100 reset ip from table(1) to any<br>
----------------<br>
<br>
Currently, table(1) holds about 1.9 million entries (both individual IPs an=
d subnets), while table(3) contains about 10,000 entries (also a mix of sin=
gle IPs and subnets).<br>
<br>
These tables are populated using this script few times per day:<br>
<br>
----------------<br>
#!/bin/sh<br>
<br>
tempdir=3D$(mktemp -d /tmp/ipfw.XXXXXX)<br>
trap "rm -rf $tempdir" EXIT<br>
<br>
fetch -q -o "$tempdir/allow.txt" <a href=3D"https://example.com/i=
pfw/allow.txt" rel=3D"noreferrer" target=3D"_blank">https://example.com/ipf=
w/allow.txt</a> || exit 1<br>
fetch -q -o "$tempdir/deny.txt" <a href=3D"https://example.com/ip=
fw/deny.txt" rel=3D"noreferrer" target=3D"_blank">https://example.com/ipfw/=
deny.txt</a> || exit 1<br>
<br>
update_table() {<br>
=C2=A0 =C2=A0 table=3D$1<br>
=C2=A0 =C2=A0 file=3D$2<br>
=C2=A0 =C2=A0 current_file=3D"$tempdir/current_table_$table.txt"<=
br>
=C2=A0 =C2=A0 ipfw -q table "$table" list | awk '{print $1}&#=
39; | sed 's/\/32$//' | sort > "$current_file"<br>
=C2=A0 =C2=A0 cat "$file" | sed 's/\/32$//' | sort | uniq=
> "$tempdir/new_table_$table.txt"<br>
<br>
=C2=A0 =C2=A0 comm -13 "$tempdir/new_table_$table.txt" "$cur=
rent_file" | while read -r ip; do<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 [ -n "$ip" ] && ipfw -q table=
"$table" delete "$ip"<br>
=C2=A0 =C2=A0 done<br>
<br>
=C2=A0 =C2=A0 comm -23 "$tempdir/new_table_$table.txt" "$cur=
rent_file" | while read -r ip; do<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 [ -n "$ip" ] && ipfw -q table=
"$table" add "$ip"<br>
=C2=A0 =C2=A0 done<br>
}<br>
<br>
update_table 3 "$tempdir/allow.txt"<br>
update_table 1 "$tempdir/deny.txt"<br>
----------------<br>
<br>
My intended logic is that any IP present in table(3) should always be allow=
ed, even if it or its subnet also appears in table(1).<br>
<br>
For instance, 175.178.167.241 is in table(3), while <a href=3D"http://175.1=
78.0.0/16" rel=3D"noreferrer" target=3D"_blank">175.178.0.0/16</a> is prese=
nt in table(1).<br>
<br>
After rebooting the server and populating the tables by running the update =
script for the first time, access from 175.178.167.241 works correctly. How=
ever, after subsequent runs of the update script - which only updates unrel=
ated entries and does not modify 175.178.167.241 or <a href=3D"http://175.1=
78.0.0/16" rel=3D"noreferrer" target=3D"_blank">175.178.0.0/16</a> - access=
from 175.178.167.241 is no longer permitted.<br>
<br>
Additionally, when this issue arises, adding <a href=3D"http://175.178.0.0/=
16" rel=3D"noreferrer" target=3D"_blank">175.178.0.0/16</a> to table(3) all=
ows access again. Even after removing that entry, as long as 175.178.167.24=
1 remains in table(3) and I wait for any active sessions to clear, access c=
ontinues to work.<br>
<br>
Does anyone have any ideas about what could be causing this behavior?<br>
</blockquote></div>
--0000000000008599610639d5c6e5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHu1Y71cuy5sHEMMrZsAve%2B2RAn7ndQqf2jtm-gyFBS-PDEEiA>