From owner-freebsd-pf@freebsd.org Tue Dec 3 19:26:07 2019 Return-Path: Delivered-To: freebsd-pf@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CF8D01BAEA8 for ; Tue, 3 Dec 2019 19:26:07 +0000 (UTC) (envelope-from artem@viklenko.net) Received: from alf.viklenko.net (alf.viklenko.net [IPv6:2001:470:71:d72::61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "www.viklenko.net", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47SBmG12w5z3MYv for ; Tue, 3 Dec 2019 19:26:05 +0000 (UTC) (envelope-from artem@viklenko.net) Received: from [IPv6:2001:470:71:d72:1208:b1ff:fe93:6f45] ([IPv6:2001:470:71:d72:1208:b1ff:fe93:6f45]) (authenticated bits=0) by alf.viklenko.net (8.15.2/8.15.2) with ESMTPSA id xB3JPsHq035917 (version=TLSv1.2 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 3 Dec 2019 21:25:58 +0200 (EET) (envelope-from artem@viklenko.net) Subject: Re: pf's states To: freebsd-pf@freebsd.org References: <20191202025642.GA99174@admin.sibptus.ru> <7a5b77d9-29d2-4fb4-b82c-3e6a194baf6e@tuxpowered.net> <20191202152543.GA16128@admin.sibptus.ru> <20191203034903.GA33853@admin.sibptus.ru> <20191203094911.GF40372@admin.sibptus.ru> From: Artem Viklenko Message-ID: Date: Tue, 3 Dec 2019 21:25:54 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: <20191203094911.GF40372@admin.sibptus.ru> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Language: uk-UA Content-Transfer-Encoding: 8bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (alf.viklenko.net [IPv6:2001:470:71:d72:0:0:0:61]); Tue, 03 Dec 2019 21:25:58 +0200 (EET) X-Rspamd-Queue-Id: 47SBmG12w5z3MYv X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.64 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[viklenko.net:s=alf-mail]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-pf@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DKIM_TRACE(0.00)[viklenko.net:+]; DMARC_POLICY_ALLOW(-0.50)[viklenko.net,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-1.64)[ipnet: 2001:470::/32(-4.64), asn: 6939(-3.53), country: US(-0.05)]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Dec 2019 19:26:07 -0000 Had to build test lab.... I still not 100% sure about state-policy - can't check it now. But it definitelly can influence on the final result. First of all check output of pfctl -vvsr - it will show full ruleset. Next - after sending traffic - check pfctl -vvss - it will show exact states and by which rule it was created. Third - use tcpdump on pflog0 interface to see logs but you need rules with log keyword for this. In short - as you don't use quick keyword - the last rule wins as Max already noted. Simple ruleset: block drop log inet all label "BLOCK-ALL" pass inet proto icmp from 192.168.32.0/26 to 192.168.34.0/26 keep state pass inet proto tcp from 192.168.32.0/26 to 192.168.34.0/26 flags S/SA keep state # pfctl -vvsr @0 block drop log inet all label "BLOCK-ALL" [ Evaluations: 19 Packets: 17 Bytes: 1301 States: 0 ] [ Inserted: uid 0 pid 1242 State Creations: 0 ] @1 pass inet proto icmp from 192.168.32.0/26 to 192.168.34.0/26 keep state [ Evaluations: 19 Packets: 68 Bytes: 5712 States: 2 ] [ Inserted: uid 0 pid 1242 State Creations: 2 ] @2 pass inet proto tcp from 192.168.32.0/26 to 192.168.34.0/26 flags S/SA keep state [ Evaluations: 19 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1242 State Creations: 0 ] Ping from 192.168.32.9 to 192.168.34.2 produce two states: # pfctl -vvss all icmp 192.168.34.2:8 <- 192.168.32.9:8 0:0 age 00:00:20, expires in 00:00:10, 20:20 pkts, 1680:1680 bytes, rule 1 id: 000000005de6aaaa creatorid: 60d9c3f7 all icmp 192.168.32.9:8 -> 192.168.34.2:8 0:0 age 00:00:20, expires in 00:00:10, 20:20 pkts, 1680:1680 bytes, rule 1 id: 000000005de6aaab creatorid: 60d9c3f7 One for each action - receive via one interface and transmit via another. As state-policy is folating - no interface names was shown. Traffic coming in the system was inspected by pf rules and first state was created. Then traffic going out to destination via another interface was inspected by pf again and second state was created by the same rule #1. ICMP replies going in reverse direction pass due to these states. Now while host 192.168.32.9 continues to ping 192.168.34.2 and states active, I've added another rule: block drop inet proto icmp from 192.168.34.2 to 192.168.32.9 # pfctl -vvsr @0 block drop log inet all label "BLOCK-ALL" [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1330 State Creations: 0 ] @1 pass inet proto icmp from 192.168.32.0/26 to 192.168.34.0/26 keep state [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1330 State Creations: 0 ] @2 pass inet proto tcp from 192.168.32.0/26 to 192.168.34.0/26 flags S/SA keep state [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1330 State Creations: 0 ] @3 block drop inet proto icmp from 192.168.34.2 to 192.168.32.9 [ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1330 State Creations: 0 ] But already established states allows traffic to pass # pfctl -vvss all icmp 192.168.34.2:9 <- 192.168.32.9:9 0:0 age 00:02:46, expires in 00:00:10, 163:163 pkts, 13692:13692 bytes, rule 1 id: 000000005de6aaac creatorid: 60d9c3f7 all icmp 192.168.32.9:9 -> 192.168.34.2:9 0:0 age 00:02:46, expires in 00:00:10, 163:163 pkts, 13692:13692 bytes, rule 1 id: 000000005de6aaad creatorid: 60d9c3f7 # pfctl -vvss all icmp 192.168.34.2:9 <- 192.168.32.9:9 0:0 age 00:02:47, expires in 00:00:10, 164:164 pkts, 13776:13776 bytes, rule 1 id: 000000005de6aaac creatorid: 60d9c3f7 all icmp 192.168.32.9:9 -> 192.168.34.2:9 0:0 age 00:02:47, expires in 00:00:10, 164:164 pkts, 13776:13776 bytes, rule 1 id: 000000005de6aaad creatorid: 60d9c3f7 Now stop ping and let states to expire. Start ping again and... it still successfull. # pfctl -vvss all icmp 192.168.34.2:10 <- 192.168.32.9:10 0:0 age 00:00:06, expires in 00:00:10, 7:7 pkts, 588:588 bytes, rule 1 id: 000000005de6aaae creatorid: 8908ecf5 all icmp 192.168.32.9:10 -> 192.168.34.2:10 0:0 age 00:00:06, expires in 00:00:10, 7:7 pkts, 588:588 bytes, rule 1 id: 000000005de6aaaf creatorid: 8908ecf5 Because states was again created by rule #1. A bit changed ruleset (in keyword added): # pfctl -vvsr @0 block drop log inet all label "BLOCK-ALL" [ Evaluations: 21 Packets: 20 Bytes: 1680 States: 0 ] [ Inserted: uid 0 pid 1458 State Creations: 0 ] @1 pass in inet proto icmp from 192.168.32.0/26 to 192.168.34.0/26 keep state [ Evaluations: 21 Packets: 40 Bytes: 2800 States: 0 ] [ Inserted: uid 0 pid 1458 State Creations: 1 ] @2 pass in inet proto tcp from 192.168.32.0/26 to 192.168.34.0/26 flags S/SA keep state [ Evaluations: 1 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1458 State Creations: 0 ] @3 block drop in inet proto icmp from 192.168.34.2 to 192.168.32.9 [ Evaluations: 1 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1458 State Creations: 0 ] lead to: PING 192.168.34.2 (192.168.34.2) 56(84) bytes of data. From 192.168.32.1 icmp_seq=1 Destination Host Unreachable From 192.168.32.1 icmp_seq=2 Destination Host Unreachable From 192.168.32.1 icmp_seq=3 Destination Host Unreachable only one state: # pfctl -vvss all icmp 192.168.34.2:14 <- 192.168.32.9:14 0:0 age 00:00:02, expires in 00:00:10, 3:3 pkts, 252:168 bytes, rule 1 id: 000000005de6aab7 creatorid: 9dc6ab26 and: # tcpdump -fnet -s0 -p -i pflog0 tcpdump: WARNING: pflog0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 65535 bytes rule 0..16777216/0(match): block out on fxp1: 192.168.32.9 > 192.168.34.2: ICMP echo request, id 14, seq 15, length 64 rule 0..16777216/0(match): block out on fxp1: 192.168.32.9 > 192.168.34.2: ICMP echo request, id 14, seq 16, length 64 Rule #0 blocked traffic while firewall tried to forward packet and transmit it via fxp1 interface. Again changing ruleset: # pfctl -vvsr @0 pass in inet proto icmp from 192.168.32.0/26 to 192.168.34.0/26 keep state [ Evaluations: 44 Packets: 17 Bytes: 1428 States: 0 ] [ Inserted: uid 0 pid 1533 State Creations: 1 ] @1 pass in inet proto tcp from 192.168.32.0/26 to 192.168.34.0/26 flags S/SA keep state [ Evaluations: 6 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 0 pid 1533 State Creations: 0 ] @2 block drop in log inet proto icmp from 192.168.34.2 to 192.168.32.9 [ Evaluations: 23 Packets: 17 Bytes: 1428 States: 0 ] [ Inserted: uid 0 pid 1533 State Creations: 0 ] start to ping... no response: PING 192.168.34.2 (192.168.34.2) 56(84) bytes of data. ^C --- 192.168.34.2 ping statistics --- 17 packets transmitted, 0 received, 100% packet loss, time 16209ms # tcpdump -fnet -s0 -p -i pflog0 tcpdump: WARNING: pflog0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 65535 bytes rule 2..16777216/0(match): block in on fxp1: 192.168.34.2 > 192.168.32.9: ICMP echo reply, id 16, seq 8, length 64 rule 2..16777216/0(match): block in on fxp1: 192.168.34.2 > 192.168.32.9: ICMP echo reply, id 16, seq 9, length 64 ^C 2 packets captured 3 packets received by filter 0 packets dropped by kernel only one state: # pfctl -vvss all icmp 192.168.34.2:16 <- 192.168.32.9:16 0:0 age 00:00:13, expires in 00:00:09, 13:0 pkts, 1092:0 bytes, rule 0 id: 000000005de6aab9 creatorid: 4ac4eac6 while traffic leaves firewall, reaches destination and it replies: # tcpdump -fnet -s 0 -p -i fxp1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on fxp1, link-type EN10MB (Ethernet), capture size 65535 bytes 00:17:65:f5:ed:f1 > 00:16:d4:bf:71:45, ethertype IPv4 (0x0800), length 98: 192.168.32.9 > 192.168.34.2: ICMP echo request, id 17, seq 1, length 64 00:16:d4:bf:71:45 > 00:17:65:f5:ed:f1, ethertype IPv4 (0x0800), length 98: 192.168.34.2 > 192.168.32.9: ICMP echo reply, id 17, s eq 1, length 64 00:17:65:f5:ed:f1 > 00:16:d4:bf:71:45, ethertype IPv4 (0x0800), length 98: 192.168.32.9 > 192.168.34.2: ICMP echo request, id 17, seq 2, length 64 00:16:d4:bf:71:45 > 00:17:65:f5:ed:f1, ethertype IPv4 (0x0800), length 98: 192.168.34.2 > 192.168.32.9: ICMP echo reply, id 17, s eq 2, length 64 This is because by default pf allows traffic but not create states. You can start pf with empty ruleset and see no states while traffic passing firewall. So then traffic came back it was blocked by last matched rule with keyword in which is rule #2 in this case. To summarise: You should carefully build ruleset and check what is going on with traffic all the way. Use log keyword to send data to pflog interface where it can be checked. pfctl -vvsr, pfctl -vvss showes which state was created by whic rule. That's all! 03.12.19 11:49, Victor Sudakov пише: > Morgan Wesström wrote: >>> Do you mean to say that a state checks not only address:port pairs, but >>> also TCP flags? This is a new notion for me. What would be a "pass" rule >>> to create a "catch all" state with no regard for TCP flags? >> >> For TCP it checks the flags when the state is created. From man pf.conf > > Forget TCP for now, let's explain the ICMP ping case I posted earlier. > > [dd] > >>> I'm afraid this is an incorrect assumption. According to man pf.conf, by >>> default "state-policy=floating" and state is not bound to interfaces. >>> The output of "pfctl -s state" does not indicate any interfaces either, >>> just protocols, addresses and ports. >>> >> >> This is weird. My state tables clearly shows the interface name first on >> the line instead of "all" but I use state-policy if-bound. I have no >> experience with floating mode, thus my assumptions earlier. I apologize >> if I was wrong. > > You need not apologize, my lab runs a very basic pf configuration where > state-policy=floating by default. > -- Regards!