From nobody Sun Jul 16 10:44:31 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R3hfm0Pl5z2tv2P; Sun, 16 Jul 2023 10:44:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R3hfl73mhz42sM; Sun, 16 Jul 2023 10:44:31 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689504272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=wmjW9ZXYupZAtmHNKJDKY8EiIR4F4XEjO3BXnLAqE8Q=; b=gtJ8Yl6kcG3JOZCfaYv9VreEMIRzqsFOZ8gQBCorb1eEl6sISokMjS+FTP+p9zDjPTVayZ 6DtNXiY/T2osLJ0vFJqL2Q+OBO20bxVCii0ZfMuoBeSagNpi4S4OVbENihZRlKnkLdEQ18 H3C5F7aYJ2b7UKZtWS+CbySrnUzb7aWr/c31TrxRx3jr2gWenBTzMJn0dOsJJKkj1COqsB xf8JNh+9OOMLByGaeyiOkRJZZ0pMW3oJcxANGbSOwr9InVLp25ItE124rvN9Ip34ha1nzm 8f9V04Rr/2W3/LHOiK3KY5ojPrJoQD6+CxZgp1GuiurvbQ8EkMaFO+3n164KSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689504272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=wmjW9ZXYupZAtmHNKJDKY8EiIR4F4XEjO3BXnLAqE8Q=; b=F61W+yyRnomyre3i5UtPCeB2r+2lAZb/O7mdBuX9Fat1hHSCR3Pu0PQN6DYHNMTFqV+cD6 ggpEPqaCLEexhVT9ZaLRNMq5XEsg0LHeq7gQI+CvXvrgovfabEeytySxT2/Nn34oHVfhab pLA7w1tBC8iwKHdqouW005KnQl2cfbkTq0WJ0VvRpP2tU6ndnj9l/tH7cT1muuVg45oANl PyKaz3hxB0im13uuj1UxzCQteUXj6kiCR5bbWOST64pbw1adMi1t6gfVwvxeaLfVWfTuDA EX3HejYonuQ78jrY3Sbj3pOeLEOh0p8FYaokIZ7vifVPKLRM1LfXo1ytRxenFA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1689504272; a=rsa-sha256; cv=none; b=iKtytlNMNX2jXRHF0aoJXv0AlTt//s7A3oDoMUxNblmK8M3901ZlS6/9Yhy5EzCdU5Y+9q NhTJU6aWYzKaGhkD9hCwYYhef59G7iSY+Gi7DrzNYv0Djg5kMIe6HWISRkOwNJB/9gt2AL 7sp847jT5RxD8aFjsO8FBteauPgxFbi7twf8KOIzqsU/9vOk74IMAxL2jEYRm307+h3F7x 8Hl9V7YR7J3h8Y0qRmwRBMVuvn75YUHEKeLkxHI+yzdA78iLs1KaJW87OAWg8+oGR4DPf9 rllJlxzoOkmKgbCUttT/BBtPf1iNdB7eq5k6CG2P0jorYAZyA5PSOhfPT5FVwA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4R3hfl674zzfmr; Sun, 16 Jul 2023 10:44:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 36GAiV4i098669; Sun, 16 Jul 2023 10:44:31 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 36GAiVm5098668; Sun, 16 Jul 2023 10:44:31 GMT (envelope-from git) Date: Sun, 16 Jul 2023 10:44:31 GMT Message-Id: <202307161044.36GAiVm5098668@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Doug Rabson Subject: git: b22299c457b2 - stable/13 - netinet*: Fix redirects for connections from localhost List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dfr X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: b22299c457b21d77fc5770b9f1a9043487b25ed9 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by dfr: URL: https://cgit.FreeBSD.org/src/commit/?id=b22299c457b21d77fc5770b9f1a9043487b25ed9 commit b22299c457b21d77fc5770b9f1a9043487b25ed9 Author: Doug Rabson AuthorDate: 2023-05-24 13:11:37 +0000 Commit: Doug Rabson CommitDate: 2023-07-14 10:07:58 +0000 netinet*: Fix redirects for connections from localhost Redirect rules use PFIL_IN and PFIL_OUT events to allow packet filter rules to change the destination address and port for a connection. Typically, the rule triggers on an input event when a packet is received by a router and the destination address and/or port is changed to implement the redirect. When a reply packet on this connection is output to the network, the rule triggers again, reversing the modification. When the connection is initiated on the same host as the packet filter, it is initially output via lo0 which queues it for input processing. This causes an input event on the lo0 interface, allowing redirect processing to rewrite the destination and create state for the connection. However, when the reply is received, no corresponding output event is generated; instead, the packet is delivered to the higher level protocol (e.g. tcp or udp) without reversing the redirect, the reply is not matched to the connection and the packet is dropped (for tcp, a connection reset is also sent). This commit fixes the problem by adding a second packet filter call in the input path. The second call happens right before the handoff to higher level processing and provides the missing output event to allow the redirect's reply processing to perform its rewrite. This extra processing is disabled by default and can be enabled using pfilctl: pfilctl link -o pf:default-out inet-local pfilctl link -o pf:default-out6 inet6-local PR: 268717 Reviewed-by: kp, melifaro MFC-after: 2 weeks Differential Revision: https://reviews.freebsd.org/D40256 (cherry picked from commit 5ab151574c8a1824c6cd8eded28506cb983284bc) --- sys/netinet/ip_input.c | 22 ++++++++- sys/netinet/ip_var.h | 4 ++ sys/netinet6/ip6_input.c | 19 ++++++++ sys/netinet6/ip6_var.h | 4 ++ tests/sys/netpfil/common/Makefile | 1 + tests/sys/netpfil/{pf => common}/rdr.sh | 84 +++++++++++++++++++++++++++++---- tests/sys/netpfil/common/utils.subr | 4 ++ tests/sys/netpfil/pf/Makefile | 1 - 8 files changed, 127 insertions(+), 12 deletions(-) diff --git a/sys/netinet/ip_input.c b/sys/netinet/ip_input.c index 2cfd3c544c72..50b5b5a00a96 100644 --- a/sys/netinet/ip_input.c +++ b/sys/netinet/ip_input.c @@ -140,7 +140,9 @@ SYSCTL_INT(_net_inet_ip, OID_AUTO, check_interface, CTLFLAG_VNET | CTLFLAG_RW, &VNET_NAME(ip_checkinterface), 0, "Verify packet arrives on correct interface"); -VNET_DEFINE(pfil_head_t, inet_pfil_head); /* Packet filter hooks */ +/* Packet filter hooks */ +VNET_DEFINE(pfil_head_t, inet_pfil_head); +VNET_DEFINE(pfil_head_t, inet_local_pfil_head); static struct netisr_handler ip_nh = { .nh_name = "ip", @@ -328,6 +330,10 @@ ip_init(void) args.pa_headname = PFIL_INET_NAME; V_inet_pfil_head = pfil_head_register(&args); + args.pa_flags = PFIL_OUT; + args.pa_headname = PFIL_INET_LOCAL_NAME; + V_inet_local_pfil_head = pfil_head_register(&args); + if (hhook_head_register(HHOOK_TYPE_IPSEC_IN, AF_INET, &V_ipsec_hhh_in[HHOOK_IPSEC_INET], HHOOK_WAITOK | HHOOK_HEADISINVNET) != 0) @@ -816,6 +822,20 @@ ours: return; #endif /* IPSTEALTH */ + /* + * We are going to ship the packet to the local protocol stack. Call the + * filter again for this 'output' action, allowing redirect-like rules + * to adjust the source address. + */ + if (PFIL_HOOKED_OUT(V_inet_local_pfil_head)) { + if (pfil_run_hooks(V_inet_local_pfil_head, &m, V_loif, PFIL_OUT, NULL) != + PFIL_PASS) + return; + if (m == NULL) /* consumed by filter */ + return; + ip = mtod(m, struct ip *); + } + /* * Attempt reassembly; if it succeeds, proceed. * ip_reass() will return a different mbuf. diff --git a/sys/netinet/ip_var.h b/sys/netinet/ip_var.h index 77b6ee88507a..79a530bca67b 100644 --- a/sys/netinet/ip_var.h +++ b/sys/netinet/ip_var.h @@ -248,6 +248,10 @@ VNET_DECLARE(struct pfil_head *, inet_pfil_head); #define V_inet_pfil_head VNET(inet_pfil_head) #define PFIL_INET_NAME "inet" +VNET_DECLARE(struct pfil_head *, inet_local_pfil_head); +#define V_inet_local_pfil_head VNET(inet_local_pfil_head) +#define PFIL_INET_LOCAL_NAME "inet-local" + void in_delayed_cksum(struct mbuf *m); /* Hooks for ipfw, dummynet, divert etc. Most are declared in raw_ip.c */ diff --git a/sys/netinet6/ip6_input.c b/sys/netinet6/ip6_input.c index 2423b61e0b36..e77bb9b68e5d 100644 --- a/sys/netinet6/ip6_input.c +++ b/sys/netinet6/ip6_input.c @@ -194,6 +194,7 @@ SYSCTL_PROC(_net_inet6_ip6, IPV6CTL_INTRDQMAXLEN, intr_direct_queue_maxlen, #endif VNET_DEFINE(pfil_head_t, inet6_pfil_head); +VNET_DEFINE(pfil_head_t, inet6_local_pfil_head); VNET_PCPUSTAT_DEFINE(struct ip6stat, ip6stat); VNET_PCPUSTAT_SYSINIT(ip6stat); @@ -233,6 +234,10 @@ ip6_init(void) args.pa_headname = PFIL_INET6_NAME; V_inet6_pfil_head = pfil_head_register(&args); + args.pa_flags = PFIL_OUT; + args.pa_headname = PFIL_INET6_LOCAL_NAME; + V_inet6_local_pfil_head = pfil_head_register(&args); + if (hhook_head_register(HHOOK_TYPE_IPSEC_IN, AF_INET6, &V_ipsec_hhh_in[HHOOK_IPSEC_INET6], HHOOK_WAITOK | HHOOK_HEADISINVNET) != 0) @@ -895,6 +900,20 @@ passin: return; } + /* + * We are going to ship the packet to the local protocol stack. Call the + * filter again for this 'output' action, allowing redirect-like rules + * to adjust the source address. + */ + if (PFIL_HOOKED_OUT(V_inet6_local_pfil_head)) { + if (pfil_run_hooks(V_inet6_local_pfil_head, &m, V_loif, PFIL_OUT, + NULL) != PFIL_PASS) + return; + if (m == NULL) /* consumed by filter */ + return; + ip6 = mtod(m, struct ip6_hdr *); + } + /* * Tell launch routine the next header */ diff --git a/sys/netinet6/ip6_var.h b/sys/netinet6/ip6_var.h index de7a938a3289..632c3f9774e7 100644 --- a/sys/netinet6/ip6_var.h +++ b/sys/netinet6/ip6_var.h @@ -329,6 +329,10 @@ VNET_DECLARE(struct pfil_head *, inet6_pfil_head); #define V_inet6_pfil_head VNET(inet6_pfil_head) #define PFIL_INET6_NAME "inet6" +VNET_DECLARE(struct pfil_head *, inet6_local_pfil_head); +#define V_inet6_local_pfil_head VNET(inet6_local_pfil_head) +#define PFIL_INET6_LOCAL_NAME "inet6-local" + #ifdef IPSTEALTH VNET_DECLARE(int, ip6stealth); #define V_ip6stealth VNET(ip6stealth) diff --git a/tests/sys/netpfil/common/Makefile b/tests/sys/netpfil/common/Makefile index 5ea3e7d9b687..99c81de3462e 100644 --- a/tests/sys/netpfil/common/Makefile +++ b/tests/sys/netpfil/common/Makefile @@ -9,6 +9,7 @@ ATF_TESTS_SH+= \ dummynet \ pass_block \ nat \ + rdr \ tos \ fragments \ forward diff --git a/tests/sys/netpfil/pf/rdr.sh b/tests/sys/netpfil/common/rdr.sh similarity index 51% rename from tests/sys/netpfil/pf/rdr.sh rename to tests/sys/netpfil/common/rdr.sh index 983abe054a35..0c2169ce6b28 100644 --- a/tests/sys/netpfil/pf/rdr.sh +++ b/tests/sys/netpfil/common/rdr.sh @@ -26,17 +26,19 @@ # SUCH DAMAGE. . $(atf_get_srcdir)/utils.subr +. $(atf_get_srcdir)/runner.subr -atf_test_case "basic" "cleanup" basic_head() { - atf_set descr 'Basic rdr test' + atf_set descr 'Basic IPv4 NAT test' atf_set require.user root } basic_body() { - pft_init + firewall=$1 + firewall_init $firewall + nat_init $firewall epair=$(vnet_mkepair) @@ -48,10 +50,13 @@ basic_body() jexec alcatraz ifconfig ${epair}b 192.0.2.1/24 up jexec alcatraz sysctl net.inet.ip.forwarding=1 - # Enable pf! - jexec alcatraz pfctl -e - pft_set_rules alcatraz \ - "rdr pass on ${epair}b proto tcp from any to 198.51.100.0/24 port 1234 -> 192.0.2.1 port 4321" + # Enable redirect filter rule + firewall_config alcatraz ${firewall} \ + "pf" \ + "rdr pass on ${epair}b proto tcp from any to 198.51.100.0/24 port 1234 -> 192.0.2.1 port 4321" \ + "ipfnat" \ + "rdr ${epair}b from any to 198.51.100.0/24 port = 1234 -> 192.0.2.1 port 4321 tcp" + echo "foo" | jexec alcatraz nc -N -l 4321 & sleep 1 @@ -64,10 +69,69 @@ basic_body() basic_cleanup() { - pft_cleanup + firewall=$1 + firewall_cleanup $firewall } -atf_init_test_cases() +local_redirect_head() { - atf_add_test_case "basic" + atf_set descr 'Redirect local traffic test' + atf_set require.user root } + +local_redirect_body() +{ + firewall=$1 + firewall_init $firewall + nat_init $firewall + + bridge=$(vnet_mkbridge) + ifconfig ${bridge} 192.0.2.1/24 up + + epair1=$(vnet_mkepair) + epair2=$(vnet_mkepair) + + vnet_mkjail first ${epair1}b + ifconfig ${epair1}a up + ifconfig ${bridge} addm ${epair1}a + jexec first ifconfig ${epair1}b 192.0.2.2/24 up + jexec first ifconfig lo0 127.0.0.1/8 up + + vnet_mkjail second ${epair2}b + ifconfig ${epair2}a up + ifconfig ${bridge} addm ${epair2}a + jexec second ifconfig ${epair2}b 192.0.2.3/24 up + jexec second ifconfig lo0 127.0.0.1/8 up + jexec second sysctl net.inet.ip.forwarding=1 + + # Enable redirect filter rule + firewall_config second ${firewall} \ + "pf" \ + "rdr pass proto tcp from any to 192.0.2.3/24 port 1234 -> 192.0.2.2 port 4321" \ + "ipfnat" \ + "rdr '*' from any to 192.0.2.3/24 port = 1234 -> 192.0.2.2 port 4321 tcp" + + echo "foo" | jexec first nc -N -l 4321 & + sleep 1 + + # Verify that second can use its rule to redirect local connections to first + result=$(jexec second nc -N -w 3 192.0.2.3 1234) + if [ "$result" != "foo" ]; then + atf_fail "Redirect failed" + fi +} + +local_redirect_cleanup() +{ + firewall=$1 + firewall_cleanup $firewall +} + +setup_tests \ + basic \ + pf \ + ipfnat \ + local_redirect \ + pf \ + ipfnat + diff --git a/tests/sys/netpfil/common/utils.subr b/tests/sys/netpfil/common/utils.subr index 722271981af4..d0028e663c45 100644 --- a/tests/sys/netpfil/common/utils.subr +++ b/tests/sys/netpfil/common/utils.subr @@ -58,12 +58,16 @@ firewall_config() jexec ${jname} pfctl -e jexec ${jname} pfctl -F all jexec ${jname} pfctl -f $cwd/pf.rule + jexec ${jname} pfilctl link -o pf:default-out inet-local + jexec ${jname} pfilctl link -o pf:default-out6 inet6-local elif [ ${fw} == "ipf" ]; then jexec ${jname} ipf -E jexec ${jname} ipf -Fa -f $cwd/ipf.rule elif [ ${fw} == "ipfnat" ]; then jexec ${jname} service ipfilter start jexec ${jname} ipnat -CF -f $cwd/ipfnat.rule + jexec ${jname} pfilctl link -o ipfilter:default-ip4 inet-local + jexec ${jname} pfilctl link -o ipfilter:default-ip6 inet6-local else atf_fail "$fw is not a valid firewall to configure" fi diff --git a/tests/sys/netpfil/pf/Makefile b/tests/sys/netpfil/pf/Makefile index d0a17ec40f43..8f6b57797e23 100644 --- a/tests/sys/netpfil/pf/Makefile +++ b/tests/sys/netpfil/pf/Makefile @@ -21,7 +21,6 @@ ATF_TESTS_SH+= altq \ pass_block \ pfsync \ proxy \ - rdr \ ridentifier \ route_to \ rules_counter \