From owner-freebsd-questions@freebsd.org Tue Jan 29 19:32:10 2019 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A14E114C8E0D for ; Tue, 29 Jan 2019 19:32:10 +0000 (UTC) (envelope-from asv@inhio.net) Received: from cz-prg-mx-01.inhio.net (mail.inhio.net [178.238.36.226]) by mx1.freebsd.org (Postfix) with ESMTP id B562D963D3; Tue, 29 Jan 2019 19:32:07 +0000 (UTC) (envelope-from asv@inhio.net) Received: from titanio (titanio.inhio.net [10.0.0.21]) by cz-prg-mx-01.inhio.net (Postfix) with ESMTPSA id F337521D1A; Tue, 29 Jan 2019 20:31:57 +0100 (CET) Message-ID: Subject: Re: PF issue since 11.2-RELEASE From: ASV To: Kristof Provost Cc: questions list Date: Tue, 29 Jan 2019 20:31:53 +0100 In-Reply-To: <51F0845A-2BB3-4BC9-977D-BB0E6C305ED3@FreeBSD.org> References: <989e79372513e9769c6857b531f14df8ce0b6f3a.camel@inhio.net> <51F0845A-2BB3-4BC9-977D-BB0E6C305ED3@FreeBSD.org> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-UU1uDKv5/tllAjBdc87J" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 X-Rspamd-Queue-Id: B562D963D3 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asv@inhio.net designates 178.238.36.226 as permitted sender) smtp.mailfrom=asv@inhio.net X-Spamd-Result: default: False [-4.67 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[inhio.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(0.01)[country: CZ(0.03)]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[mail.inhio.net]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.88)[-0.876,0]; SIGNED_PGP(-2.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:24971, ipnet:178.238.32.0/20, country:CZ]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jan 2019 19:32:10 -0000 --=-UU1uDKv5/tllAjBdc87J Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable OK, I understand. Here it follows my pf.conf: ext_if=3D"lagg0" tun0_if=3D"tun0" B01=3D"172.16.3.2" K01=3D"172.16.3.3" W01=3D"172.16.3.4" W03=3D"172.16.3.5" K02=3D"172.16.3.6" W02=3D"172.16.3.7" set skip on lo set limit table-entries 700000 scrub in all #nat on $ext_if from $B01 to any -> ($ext_if:0) #nat on $ext_if from $K01 to any -> ($ext_if:0) #nat on $ext_if from $K02 to any -> ($ext_if:0) nat on $ext_if from $W03 to any -> ($ext_if:0) nat on $ext_if from $W01 to any -> ($ext_if:0) nat on $ext_if from $W02 to any -> ($ext_if:0) rdr on $ext_if proto tcp from any to 170.100.0.124 port 80 -> 172.16.3.4 po= rt 80 rdr on $ext_if proto tcp from any to 170.100.0.124 port 443 -> 172.16.3.4 p= ort 443 rdr on $ext_if proto tcp from any to 170.100.0.126 port 587 -> 170.100.0.12= 6 port 25 rdr on $tun0_if proto tcp from any to 170.100.0.126 port 587 -> 170.100.0.1= 26 port 25 rdr on $ext_if proto { tcp, udp } from any to 170.100.0.131 port 53 -> 172.= 17.2.8 rdr on $ext_if proto tcp from any to 170.100.0.124 port 10009 -> 172.16.3.6 rdr on $ext_if proto tcp from any to 170.100.0.124 port 10012 -> 172.16.3.7 antispoof quick for { lo $ext_if } block in log (all) all pass out log (all) on $ext_if pass in on $tun0_if pass out on $tun0_if table <0_KNOWN_SPAMMERS> persist file "/etc/0_KNOWN_SPAMMERS" table persist anchor "ftp-proxy/*" anchor f2b { anchor sip { block in quick log to any } } block in quick on $ext_if from <0_KNOWN_SPAMMERS> pass in quick log on $tun0_if from 10.0.0.0/26 tag ZONE1 modulate state pass in quick log on $tun0_if from 172.17.2.0/28 tag ZONE2 modulate state pass in quick log on $tun0_if from 172.16.0.0/27 tag ZONE3 modulate state pass in quick on $ext_if proto icmp from any to { 170.100.0.124, 170.100.0.= 126, 170.100.0.127, 170.100.0.128, 170.100.0.129, 170.100.0.130, 170.100.0.= 131 } keep state icmp-type echoreq pass in log on $ext_if proto tcp from any to 170.100.0.124 port 10000 synpr= oxy state \ (max-src-conn 6, max-src-conn-rate 3/1, overload f= lush global) pass in log on $ext_if proto tcp from any to 170.100.0.125 port 10001 synpr= oxy state \ (max-src-conn 6, max-src-conn-rate 3/1, overload f= lush global) pass in log on $ext_if proto tcp from any to 170.100.0.126 port 10003 synpr= oxy state \ (max-src-conn 6, max-src-conn-rate 3/1, overload f= lush global) pass in log on $ext_if proto tcp from any to 170.100.0.127 port 10004 synpr= oxy state \ (max-src-conn 6, max-src-conn-rate 3/1, overload f= lush global) #pass in log on $ext_if proto tcp from any to 170.100.0.128 port 10007 synp= roxy state \ # (max-src-conn 6, max-src-conn-rate 3/1, overload = flush global) #pass in log on $ext_if proto tcp from any to 170.100.0.129 port 10008 synp= roxy state \ # (max-src-conn 6, max-src-conn-rate 3/1, overload = flush global) pass in log on $ext_if proto tcp from any to 170.100.0.130 port 10011 synpr= oxy state \ (max-src-conn 6, max-src-conn-rate 3/1, overload f= lush global) pass in log on $ext_if proto tcp from any to 172.16.3.6 port 10009 synproxy= state \ (max-src-conn 6, max-src-conn-rate 3/1, overload f= lush global) pass in log on $ext_if proto tcp from any to 172.16.3.7 port 10012 synproxy= state \ (max-src-conn 6, max-src-conn-rate 3/1, overload f= lush global) pass in log on $ext_if proto udp from any to 170.100.0.125 port 5060 keep s= tate pass in log on $ext_if proto tcp from any to 170.100.0.125 port 5061 keep s= tate pass in on $ext_if proto udp from any to 170.100.0.125 port 10000:30000 kee= p state pass in log on $ext_if proto tcp from any to 170.100.0.126 port { 25, 143, = 587 } synproxy state pass in log on $ext_if proto { tcp, udp } from any to 170.100.0.127 port 53= synproxy state pass in log on $ext_if proto { tcp, udp } from { 72.45.24.0/20, 80.0.0.100 = } to 170.100.0.124 port 1194 pass in log on $ext_if proto tcp from any to 172.16.3.4 port { 80, 443 } sy= nproxy state pass in log on $ext_if proto tcp from any to 170.100.0.128 port { 80, 443 }= synproxy state pass in log on $ext_if proto tcp from any to 170.100.0.130 port { 80, 443 }= synproxy state pass in quick log on $ext_if from any to 172.17.2.8 synproxy state On Mon, 2019-01-28 at 09:28 +0100, Kristof Provost wrote: > On 27 Jan 2019, at 19:03, ASV wrote: > > On Sun, 2019-01-27 at 16:08 +0100, Kristof Provost wrote: > > > On 26 Jan 2019, at 17:00, ASV wrote: > > > > since I've upgraded to 11.2 (from 11.1) I've observed that > > > > anytime > > > > I > > > > change something on pf.conf and reload (pfctl -f /etc/pf.conf) > > > > I > > > > partially loose connectivity. Partially means that I still am > > > > connected > > > > to the server but the server cannot connect anywhere or ping > > > > anything > > > > (no hosts no IPs) also the jails instantly suffers from the > > > > same. > > >=20 > > > That sounds like your established connection continues (because > > > it > > > keeps > > > using the old rules), and something is wrong with the new rules. > >=20 > > Hi and thanks for your reply! > > This is not the case as I'm modifying NAT rules so I'm not > > expecting > > anything to start being blocked but the reverse. > >=20 > > > The logical debugging steps would be: > > > - check the ruleset matches what you expect (pfctl -s rules) > > > - check the state table (pfctl -s states) > > > - use pflog to determine what rule causes traffic to be dropped > > >=20 > > > > The quickest fix is to revert the PF configuration to the > > > > previous > > > > one > > > > and reload. Everything starts working again. > > > >=20 > > >=20 > > > What do you mean by =E2=80=98previous one=E2=80=99? Do you have two r= ulesets? > > > What > > > are the two rulesets? > >=20 > > The configuration hasn't been changed in ages and when I need to > > upgrade the ports of a couple of jails which are NOT routed to the > > internet I simply un-comment few NAT lines and reload the pf conf. > > I've > > been doing this specific action for almost 7 years, never a > > problem. > > Therefore there is no problem in the rules. > >=20 > > For previous ruleset I mean that since the jails start losing > > connectivity (as long as I "push" the new ruleset) with internet > > and > > with each other, I re-comment these lines and reload. Sometime it > > works > > sometime it doesn't and I need to: > >=20 > > service pf restart > >=20 > > which obviously forces me to re-login. > >=20 > > > > I've been trying to find the root cause of this without > > > > success. > > > > Did I > > > > miss some major change on the PF port on FreeBSD? I've never > > > > seen > > > > this > > > > serious issue before nor on FreeBSD neither on OpenBSD. > > >=20 > > > It=E2=80=99s very difficult to debug this with the extremely limited > > > information you=E2=80=99ve included. > > > Please post, at the very least, your pf ruleset and a full > > > description > > > of what you=E2=80=99re doing when things break and how you recover. > >=20 > > #nat on $ext_if from $SRV01 to any -> ($ext_if:0) > > #nat on $ext_if from $SRV02 to any -> ($ext_if:0) > > #nat on $ext_if from $SRV03 to any -> ($ext_if:0) > >=20 > > not much really. But the same happens with other rules. Basically > > whatever I modify there now requires a full pf restart, which is > > not > > very practical as it kicks me out. > >=20 > > I'm also having plenty of issues using fail2ban, just to mention > > another as it is somehow related (even though a broader topic), > > where > > rules are in place but aren't enforced. > >=20 > > # pfctl -a f2b/asterisk-udp -t f2b-asterisk-udp -Ts > > > > > > > >=20 > > # pfctl -a f2b/asterisk-udp -t f2b-asterisk-udp -s rules > > block drop quick proto udp from to any port =3D > > sip > > block drop quick proto udp from to any port =3D > > sip-tls > >=20 > > then killing the state (if any) and check: > > # pfctl -k ; tcpdump udp -nettt -i pflog0 > > port 5060 and host > > killed 1 states from 1 sources and 0 destinations > > tcpdump: verbose output suppressed, use -v or -vv for full protocol > > decode > > listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture > > size 262144 bytes > > 00:00:00.000000 rule 24/0(match): pass in on lagg0: > address>.6175 > .5060: SIP: REGISTER sip: > ip address> SIP/2.0 > > 00:01:07.344401 rule 24/0(match): pass in on lagg0: > address>.6115 > .5060: SIP: REGISTER sip: > ip address> SIP/2.0 > > 00:00:39.690851 rule 24/0(match): pass in on lagg0: > address>.6158 > .5060: SIP: REGISTER sip: > ip address> SIP/2.0 > > 00:00:43.058753 rule 24/0(match): pass in on lagg0: > address>.6179 > .5060: SIP: REGISTER sip: > ip address> SIP/2.0 > > 00:00:43.912680 rule 24/0(match): pass in on lagg0: > address>.6119 > .5060: SIP: REGISTER sip: > ip address> SIP/2.0 > >=20 > > it is clearly not enforcing the rules. > >=20 >=20 > Please share the full pf.conf. I=E2=80=99m sure you understand that I can= not > debug your problem without *full* information. >=20 > Regards, > Kristof --=-UU1uDKv5/tllAjBdc87J Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEE5dE8BwbhhcQw2TsezaQsUNd+zIkFAlxQqikACgkQzaQsUNd+ zImgtQf9EYMNlBa7LCRgW1DtI3Jixrucp9Yi5MDV6o2a2qgsrOtdS63Za6jrOLIc /su/Qp9voOhzs5rPtfZ7jNHEBapErnytmlZ/GzkQ2aIwAM8/rUK7sb3w9L66NAnl AH7zXh2lEEQS3i9ptEGemq//gEn7UbLnbsHskCMPLRLBPNFAVAn6JP1u8n5eiBkh 3LubHaNvteeB4Cn2uYKic3vZMQZnH1naPOu4jBMoQE0xEgQlEfMPVKlKEjENgBLA 1kwhHFIaim6gjK9MoueeiF9vYGc7Rrx+gpLnSHykUfRGoaFIUdCCLYx6NI/6NhXq qqqinzX4nNTW5mDAXPfwVnROrwU5Pg== =kYB/ -----END PGP SIGNATURE----- --=-UU1uDKv5/tllAjBdc87J--