From owner-freebsd-questions@freebsd.org Mon Aug 16 21:54:20 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7B120655C2E for ; Mon, 16 Aug 2021 21:54:20 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from kicp.uchicago.edu (kicp.uchicago.edu [128.135.20.70]) by mx1.freebsd.org (Postfix) with ESMTP id 4GpScC5mWTz3lFX for ; Mon, 16 Aug 2021 21:54:19 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from point.uchicago.edu (point.uchicago.edu [128.135.52.6]) (Authenticated sender: galtsev) by kicp.uchicago.edu (Postfix) with ESMTPSA id 6F6EC4E498 for ; Mon, 16 Aug 2021 16:54:19 -0500 (CDT) Subject: Re: Jails: pf blocks access to localhost of host system To: freebsd-questions@freebsd.org References: <8f6aa4f4-f361-e6eb-985b-291b7be631e0@kicp.uchicago.edu> From: Valeri Galtsev Message-ID: <8a330aeb-fccf-f8bd-1154-15288e3f0a9d@kicp.uchicago.edu> Date: Mon, 16 Aug 2021 16:54:17 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4GpScC5mWTz3lFX X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=uchicago.edu (policy=none); spf=none (mx1.freebsd.org: domain of galtsev@kicp.uchicago.edu has no SPF policy when checking 128.135.20.70) smtp.mailfrom=galtsev@kicp.uchicago.edu X-Spamd-Result: default: False [-0.91 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_SPAM_SHORT(0.99)[0.992]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:160, ipnet:128.135.0.0/16, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-questions]; DMARC_POLICY_SOFTFAIL(0.10)[uchicago.edu : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Aug 2021 21:54:20 -0000 On 8/16/21 4:41 PM, Matthew Seaman wrote: > On 16/08/2021 22:31, Valeri Galtsev wrote: >> Dear Experts in pf and jails. >> >> I was recent refugee from to pf from ipfw which I was happily using >> for decade (why is different story). I seem to be able to configure >> all I need from pf, except one thing, which is jail access to >> localhost of host system. >> >> I configure jails "by the book" (i.e FreeBSD handbook), I do not use >> any scripts facilitating simple enough for me process. My jail >> configuration is like the following: >> >> # cat /etc/jail.conf >> >> jailname { >>      host.hostname = "hostname.domainname"; >>      ip4.addr = X.Y.Z.W; # address on public address space >>      devfs_ruleset=7;  // integer number of devfsrules_jail_bpf ruleset >>                        // in /etc/devfs.rules >>      persist;        // Required because there are no processes >>      exec.start = "/etc/rc.d/cron start; /etc/rc.d/syslogd start; >> /usr/local/etc/rc.d/apache24 start"; >>      exec.stop = "/usr/local/etc/rc.d/apache24 stop; /etc/rc.d/syslogd >> stop; /etc/rc.d/cron stop"; >> } >> >> As you see, there is no lo0 configured in jail. (I do have lo0 >> configured in jails on some machines, but I do not feel it necessary >> in this case). Now if script from apache in that jail connects port 25 >> on localhost to send email, the connection goes ho postfix I run on >> host system. >> >> While I was using ipfw having the rule allowing all traffic on lo0 to >> pass (I figure) at the top of rules did the trick. I can not achieve >> the same with pf. Searching on we web didn't help either. >> >> Here if my simplified to necessary minimum for debugging pf.conf: >> >> >> # cat /etc/pf.conf >> ##################### BEGIN >> # macro name(s) for external interface(s). >> ext_if = "bce0" >> int_if = "bce1" >> >> services = "{ 80 443 3306 5432 9102 }" >> >> icmp_types = "{ echoreq unreach }" >> >> # set a default deny everything policy. >> block all >> >> # Allow all traffic on trusted interfaces >> pass quick from { lo0 $int_if } to any keep state >> >> # keep state on any outbound tcp, udp or icmp traffic. >> # modulate the isn of >> # outgoing packets. (initial sequence number) broken operating systems >> # sometimes don’t randomize this number, making it guessable. >> pass out on $ext_if proto { tcp, udp, icmp } from any to any \ >>      modulate state >> >> # set a rule that allows inbound ssh traffic >> pass in on $ext_if proto tcp to port { 22 } \ >>      keep state >> >> # On workstation we will pass all outgoing traffic >> pass out all keep state >> >> #  We pass what comes to our services (but DHCP taken care os above) >> pass in on $ext_if proto { tcp udp } to port $services >> >> # and icmp types we want >> pass inet proto icmp icmp-type $icmp_types >> >> ##################### END >> >> >> If I am inside jail the command >> >> telnet localhost 25 >> >> successfully connects to postfix run on host system. When I enable pf >> with rules as above, similar connection from inside jail just hangs. >> >> I was assuming the rule: >> >> pass quick from { lo0 $int_if } to any keep state >> >> will do the trick, but no, the connection from inside jail to host >> system localhost port 25 is blocked by pf. >> >> >> Can someone give me any pointers for this particular case? >> >> Thanks a lot in advance for your pointers! >> >> Valeri >> >> >> PS I know, I can do the following and it will work: configure lo0 in >> jail (splitting part of 127.0.0.0/8 away from host to guest/jail) and >> run postfix is jail. But being able to tell pf what I want it to [not] >> do will be preferable. >> > > Try: > > set skip on { lo0 $int_if } > Tried that (replaced what I had for lo0 with): set skip on { lo0 $int_if } Does not help: the connection from inside jail to localhost:25 [on host system] still hangs after I enable pf. I figure, the order of statements in pf.conf shouldn't matter, right? Thanks anyway. I have a feeling that _I_ do something gravely wrong, but... Valeri > (and delete any filtering rules on lo0 or $int_if) > > which says to pf not to touch any packets traversing those interfaces. > >     Cheers, > >     Matthew > > -- ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++