From nobody Sun Jul 13 20:55:45 2025 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bgHmy0Dbhz61TL7 for ; Sun, 13 Jul 2025 20:56:34 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bgHmx4ZzSz42tm for ; Sun, 13 Jul 2025 20:56:33 +0000 (UTC) (envelope-from kudzu@tenebras.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-31329098ae8so2725967a91.1 for ; Sun, 13 Jul 2025 13:56:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tenebras-com.20230601.gappssmtp.com; s=20230601; t=1752440182; x=1753044982; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=603aDwfk1omtRIuo81a1stpFKCsyjB/kNN7wevOk6yM=; b=GzPd4RXqomv6ZuT2U/451sHIdK6LMnWwguzM3ldRAPn+Xe2/cNYU2IqQRfHJFp47wI eaApApx/CfKKOXOU4o7SJ3dQ/1H0/psCZzOMXYRcp/C7STBduLEhmhEyA/zbelX5X2cv HcomWQ2Huy/1s+oV4XSR0aAJa3oJb/CHhHD9I/glPKfFJdYxtg62UFa60wcZ77CbwBKk 4VZE7gINHP+HXSmIv4ICqaXYdzEra2uSWBD12dUBIprHtYij88YNokHaKk1GaC/+UhSm fqJGQefhKyKNLh8X8r/fdejFPPHPPfMPOaWUyZqXVXc0A9UJJ77DUzqFo/RLe1jB+K88 jnZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752440182; x=1753044982; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=603aDwfk1omtRIuo81a1stpFKCsyjB/kNN7wevOk6yM=; b=s22aOLw1DgH4EgJwjYrus2yS8PXzeRShe374WSapXFAB9uEY9MXM97Lj4Xus/qmS0l /UIIWyTBFTgOm3HaFNUYcGZ7sLdiXoPP3ezyM7ZMDKICW4gOWpMzdBOGuELYhmXbNr/R 8SLhPpSGxi+WjSwx+DchZ5BcnJEi1w5T2sE+XbSPBuKefTSiA+/XDJx+21Kp4cbg/6Ct s4QGG5fq9JX/ajS0meExTxLmpb9M6FaVl27fmrUJQqHEYkBB23jS5Brny9wr0be4k5xg X/tNncSgmlOT5pLvKwgTckdsUrU1ahRoaL/xBzpqnQhohGJ8yo3BXOSwqlPb861NhCao MEKg== X-Gm-Message-State: AOJu0YzsOahxiaeg/2N7w91tu7hJq6NBJxLphfI17sH8qyMlpKdiVDTU OpRYnxAodFWbkGv66TGr2n3XtdkdTeQaoOUvU8NAVY+ZR27BAkBeKt6tdGDr2wNLh9sb6FVNX7m J2oln7YzHnKq9/7+PSTuIBPujdxJZikqZ7o9phSFknYf9FYw2NcRlfls= X-Gm-Gg: ASbGncvKFR2DE1MevXRI16AsZqc22CDXXpiYr81LyTUxXjpebkPaMotYEX3kLLbIJOh eCekcJU6zQWzVo53RdblIWw5v6HerAcW9rpWVqnwaSMIS2H2W/HS+hZ9KDTFHbVRH2FsmryIYdk FeS8XwMRfqariauwNT4MJOTLJhjS3Z7+R9/e6BoZACSP8kqD6g/3dbWwQINWrMDvtxSu70N+cmG jtQ6p+rWESnEacSA08+CZqrNZ5n8l9SLbPEWSVZTg== X-Google-Smtp-Source: AGHT+IG+ip3lOJy+JqkYxIjd3s/VLmVClt6owFLYgtqiAqWA7kaQ/NWbfOrRJOPrrydcQoLwL1BR3uSY1sDCMwQJ8OE= X-Received: by 2002:a17:90b:3c8f:b0:315:9624:37db with SMTP id 98e67ed59e1d1-31c4d30d1e2mr15686680a91.3.1752440181717; Sun, 13 Jul 2025 13:56:21 -0700 (PDT) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 References: <3A01EF48-EBE8-48C3-9C66-6A250A240341@cretaforce.gr> In-Reply-To: <3A01EF48-EBE8-48C3-9C66-6A250A240341@cretaforce.gr> From: Michael Sierchio Date: Sun, 13 Jul 2025 16:55:45 -0400 X-Gm-Features: Ac12FXyUPtbi-6eyG0BoEiEwTAlPmfToMkZRl4Lpw0UpqGl2k3DSGC6pHfipCzo Message-ID: Subject: Re: Issues with IPFW skipto Rule and Whitelisting Logic To: Christos Chatzaras Cc: freebsd-net , FreeBSD Questions Mailing List Content-Type: multipart/alternative; boundary="0000000000008599610639d5c6e5" X-Rspamd-Queue-Id: 4bgHmx4ZzSz42tm X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --0000000000008599610639d5c6e5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I haven't had a chance to read this in detail, but what about UDP? Most DNS traffic is UDP. And these lines are subtly wrong: *$cmd 10031 allow tcp from me to any dst-port 443 out via $pif setup keep-state$cmd 10033 allow tcp from any to me dst-port 443 in via $pif setup keep-state* because 'via' causes these rules to catch packets twice as they're processed by the kernel. IMHO these should be $cmd 10031 allow tcp from me to any dst-port 443 out xmit $pif setup keep-state $cmd 10033 allow tcp from any to me dst-port 443 in recv $pif setup keep-state I'll have more comments when I get a chance to peruse fully. On Sun, Jul 13, 2025 at 4:41=E2=80=AFPM Christos Chatzaras wrote: > I am using ipfw with these rules: > > ---------------- > #!/bin/sh > > # Set rules command prefix > cmd=3D"ipfw -q add " > cmd2=3D"ipfw -q " > > # Public interface > pif=3D`ifconfig -l | awk '{ print $1 }'` > > # Flush all rules > ipfw -q -f flush > > # Flush all tables > $cmd2 table 1 flush > $cmd2 table 3 flush > > # Allow loopback and deny loopback spoofing > $cmd 00010 allow ip from any to any via lo0 > $cmd 00020 deny ip from any to 127.0.0.0/8 > $cmd 00030 deny ip from 127.0.0.0/8 to any > > # Catch spoofing from outside. > $cmd 00031 deny ip from any to any not antispoof via $pif > > # Checks stateful rules > $cmd 00050 check-state > $cmd 00060 deny tcp from any to any established > > # ALLOW WHITELIST - IGNORE RULE 00100 > $cmd2 00070 add skipto 00101 ip from 'table(3)' to any > > # DENY INCOMING LIST > $cmd 00100 reset ip from 'table(1)' to any > > # ICMP > $cmd 01010 allow icmp from any to any out via $pif keep-state > $cmd 01011 allow icmp from any to any in via $pif > > # WWW > $cmd 10031 allow tcp from me to any dst-port 443 out via $pif setup > keep-state > $cmd 10033 allow tcp from any to me dst-port 443 in via $pif setup > keep-state > > # Deny everything else, and log it > $cmd 56599 deny log all from any to any > ---------------- > > And ipfw list includes: > > ---------------- > 00070 skipto 101 ip from table(3) to any > 00100 reset ip from table(1) to any > ---------------- > > Currently, table(1) holds about 1.9 million entries (both individual IPs > and subnets), while table(3) contains about 10,000 entries (also a mix of > single IPs and subnets). > > These tables are populated using this script few times per day: > > ---------------- > #!/bin/sh > > tempdir=3D$(mktemp -d /tmp/ipfw.XXXXXX) > trap "rm -rf $tempdir" EXIT > > fetch -q -o "$tempdir/allow.txt" https://example.com/ipfw/allow.txt || > exit 1 > fetch -q -o "$tempdir/deny.txt" https://example.com/ipfw/deny.txt || exit > 1 > > update_table() { > table=3D$1 > file=3D$2 > current_file=3D"$tempdir/current_table_$table.txt" > ipfw -q table "$table" list | awk '{print $1}' | sed 's/\/32$//' | > sort > "$current_file" > cat "$file" | sed 's/\/32$//' | sort | uniq > > "$tempdir/new_table_$table.txt" > > comm -13 "$tempdir/new_table_$table.txt" "$current_file" | while read > -r ip; do > [ -n "$ip" ] && ipfw -q table "$table" delete "$ip" > done > > comm -23 "$tempdir/new_table_$table.txt" "$current_file" | while read > -r ip; do > [ -n "$ip" ] && ipfw -q table "$table" add "$ip" > done > } > > update_table 3 "$tempdir/allow.txt" > update_table 1 "$tempdir/deny.txt" > ---------------- > > My intended logic is that any IP present in table(3) should always be > allowed, even if it or its subnet also appears in table(1). > > For instance, 175.178.167.241 is in table(3), while 175.178.0.0/16 is > present in table(1). > > After rebooting the server and populating the tables by running the updat= e > script for the first time, access from 175.178.167.241 works correctly. > However, after subsequent runs of the update script - which only updates > unrelated entries and does not modify 175.178.167.241 or 175.178.0.0/16 - > access from 175.178.167.241 is no longer permitted. > > Additionally, when this issue arises, adding 175.178.0.0/16 to table(3) > allows access again. Even after removing that entry, as long as > 175.178.167.241 remains in table(3) and I wait for any active sessions to > clear, access continues to work. > > Does anyone have any ideas about what could be causing this behavior? > --0000000000008599610639d5c6e5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I haven't had a chance to read this in detail, but
what about UDP?=C2=A0 Most DNS traffic is UDP.

And these lines = are subtly wrong:

$cmd 10031 allow tcp from me to any dst-port 44= 3 out via $pif setup keep-state
$cmd 10033 allow tcp from any to me dst-= port 443 in via $pif setup keep-state

because= 'via' causes these rules to catch packets twice as they're pro= cessed by the kernel.=C2=A0 IMHO these should be

$cmd 10031 allow tc= p from me to any dst-port 443 out xmit $pif setup keep-state
$cmd 10033 = allow tcp from any to me dst-port 443 in recv $pif setup keep-state

I'll have more comments when I get a chance to pe= ruse fully.



On Sun, Jul 13, 2= 025 at 4:41=E2=80=AFPM Christos Chatzaras <chris@cretaforce.gr> wrote:
I am using ipfw with these rules:

----------------
#!/bin/sh

# Set rules command prefix
cmd=3D"ipfw -q add "
cmd2=3D"ipfw -q "

# Public interface
pif=3D`ifconfig -l | awk '{ print $1 }'`

# Flush all rules
ipfw -q -f flush

# Flush all tables
$cmd2 table 1 flush
$cmd2 table 3 flush

# Allow loopback and deny loopback spoofing
$cmd 00010 allow ip from any to any via lo0
$cmd 00020 deny ip from any to 127.0.0.0/8
$cmd 00030 deny ip from 127.0.0.0/8 to any

# Catch spoofing from outside.
$cmd 00031 deny ip from any to any not antispoof via $pif

# Checks stateful rules
$cmd 00050 check-state
$cmd 00060 deny tcp from any to any established

# ALLOW WHITELIST - IGNORE RULE 00100
$cmd2 00070 add skipto 00101 ip from 'table(3)' to any

# DENY INCOMING LIST
$cmd 00100 reset ip from 'table(1)' to any

# ICMP
$cmd 01010 allow icmp from any to any out via $pif keep-state
$cmd 01011 allow icmp from any to any in via $pif

# WWW
$cmd 10031 allow tcp from me to any dst-port 443 out via $pif setup keep-st= ate
$cmd 10033 allow tcp from any to me dst-port 443 in via $pif setup keep-sta= te

# Deny everything else, and log it
$cmd 56599 deny log all from any to any
----------------

And ipfw list includes:

----------------
00070 skipto 101 ip from table(3) to any
00100 reset ip from table(1) to any
----------------

Currently, table(1) holds about 1.9 million entries (both individual IPs an= d subnets), while table(3) contains about 10,000 entries (also a mix of sin= gle IPs and subnets).

These tables are populated using this script few times per day:

----------------
#!/bin/sh

tempdir=3D$(mktemp -d /tmp/ipfw.XXXXXX)
trap "rm -rf $tempdir" EXIT

fetch -q -o "$tempdir/allow.txt" https://example.com/ipf= w/allow.txt || exit 1
fetch -q -o "$tempdir/deny.txt" https://example.com/ipfw/= deny.txt || exit 1

update_table() {
=C2=A0 =C2=A0 table=3D$1
=C2=A0 =C2=A0 file=3D$2
=C2=A0 =C2=A0 current_file=3D"$tempdir/current_table_$table.txt"<= br> =C2=A0 =C2=A0 ipfw -q table "$table" list | awk '{print $1}&#= 39; | sed 's/\/32$//' | sort > "$current_file"
=C2=A0 =C2=A0 cat "$file" | sed 's/\/32$//' | sort | uniq= > "$tempdir/new_table_$table.txt"

=C2=A0 =C2=A0 comm -13 "$tempdir/new_table_$table.txt" "$cur= rent_file" | while read -r ip; do
=C2=A0 =C2=A0 =C2=A0 =C2=A0 [ -n "$ip" ] && ipfw -q table= "$table" delete "$ip"
=C2=A0 =C2=A0 done

=C2=A0 =C2=A0 comm -23 "$tempdir/new_table_$table.txt" "$cur= rent_file" | while read -r ip; do
=C2=A0 =C2=A0 =C2=A0 =C2=A0 [ -n "$ip" ] && ipfw -q table= "$table" add "$ip"
=C2=A0 =C2=A0 done
}

update_table 3 "$tempdir/allow.txt"
update_table 1 "$tempdir/deny.txt"
----------------

My intended logic is that any IP present in table(3) should always be allow= ed, even if it or its subnet also appears in table(1).

For instance, 175.178.167.241 is in table(3), while 175.178.0.0/16 is prese= nt in table(1).

After rebooting the server and populating the tables by running the update = script for the first time, access from 175.178.167.241 works correctly. How= ever, after subsequent runs of the update script - which only updates unrel= ated entries and does not modify 175.178.167.241 or 175.178.0.0/16 - access= from 175.178.167.241 is no longer permitted.

Additionally, when this issue arises, adding 175.178.0.0/16 to table(3) all= ows access again. Even after removing that entry, as long as 175.178.167.24= 1 remains in table(3) and I wait for any active sessions to clear, access c= ontinues to work.

Does anyone have any ideas about what could be causing this behavior?
--0000000000008599610639d5c6e5--