From owner-freebsd-net@freebsd.org Fri Feb 22 18:33:08 2019 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F18614F62A4 for ; Fri, 22 Feb 2019 18:33:08 +0000 (UTC) (envelope-from olgeni@olgeni.com) Received: from hub.olgeni.com (hub.olgeni.com [31.171.246.156]) by mx1.freebsd.org (Postfix) with ESMTP id 382588144C for ; Fri, 22 Feb 2019 18:33:06 +0000 (UTC) (envelope-from olgeni@olgeni.com) Received: from [192.168.0.80] (unknown [82.84.91.88]) by hub.olgeni.com (Postfix) with ESMTPSA id 5ED8CD7916 for ; Fri, 22 Feb 2019 19:33:05 +0100 (CET) Date: Fri, 22 Feb 2019 19:33:04 +0100 (CET) From: Giacomo Olgeni X-X-Sender: olgeni@macbook.local To: freebsd-net@FreeBSD.org Subject: Re: IPFW NAT in VNET jail In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (OSX 202 2017-01-01) X-OpenPGP-KeyID: 0x90B7A98E6450AE47 X-OpenPGP-Fingerprint: 7133 AB4D DFC8 0A0D F891 B0D2 90B7 A98E 6450 AE47 X-OpenPGP-URL: http://hub.olgeni.com/~olgeni/pgp/olgeni@olgeni.com MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 382588144C X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of olgeni@olgeni.com designates 31.171.246.156 as permitted sender) smtp.mailfrom=olgeni@olgeni.com X-Spamd-Result: default: False [-1.70 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.79)[-0.792,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.99)[-0.992,0]; DMARC_NA(0.00)[olgeni.com]; NEURAL_SPAM_SHORT(0.29)[0.292,0]; MX_GOOD(-0.01)[cached: hub.olgeni.com]; IP_SCORE(0.01)[country: CH(0.03)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:50837, ipnet:31.171.244.0/22, country:CH]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Feb 2019 18:33:08 -0000 On Fri, 22 Feb 2019, Giacomo Olgeni wrote: > 00100 0 0 nat 1 ip from any to any recv epair0b > 00200 74 4080 nat 1 ip from any to any xmit epair0b > 00300 0 0 check-state :default > 00400 6 360 allow tcp from any to any out xmit epair0b setup keep-state :default > 00500 0 0 allow udp from any to any out xmit epair0b setup > 65535 113149 11125118 allow ip from any to any More testing with a know good configuration, that I got by setting a few things in rc.conf. On the VNET jail, which doesn't work, I have: rc.conf ======= defaultrouter="10.64.23.33" firewall_enable="YES" firewall_nat_enable="YES" firewall_nat_interface="epair0b" firewall_type="open" Rules ===== 00050 18 860 nat 123 ip4 from any to any via epair0b 00100 0 0 allow ip from any to any via lo0 00200 0 0 deny ip from any to 127.0.0.0/8 00300 0 0 deny ip from 127.0.0.0/8 to any 00400 0 0 deny ip from any to ::1 00500 0 0 deny ip from ::1 to any 00600 1 76 allow ipv6-icmp from :: to ff02::/16 00700 0 0 allow ipv6-icmp from fe80::/10 to fe80::/10 00800 0 0 allow ipv6-icmp from fe80::/10 to ff02::/16 00900 0 0 allow ipv6-icmp from any to any icmp6types 1 01000 0 0 allow ipv6-icmp from any to any icmp6types 2,135,136 65000 175 16234 allow ip from any to any 65535 15 1209 allow ip from any to any On a bhyve VM, same host, same bridge(4) interface, I have... rc.conf ======= defaultrouter="10.64.23.33" firewall_enable="YES" firewall_nat_enable="YES" firewall_nat_interface="vtnet1" firewall_type="open" ifconfig_vtnet0="inet 192.168.3.202/24" ifconfig_vtnet1="inet 10.64.23.34/28" Rules ===== 00050 10 839 nat 123 ip4 from any to any via vtnet1 00100 64 6304 allow ip from any to any via lo0 00200 0 0 deny ip from any to 127.0.0.0/8 00300 0 0 deny ip from 127.0.0.0/8 to any 00400 0 0 deny ip from any to ::1 00500 0 0 deny ip from ::1 to any 00600 0 0 allow ipv6-icmp from :: to ff02::/16 00700 0 0 allow ipv6-icmp from fe80::/10 to fe80::/10 00800 0 0 allow ipv6-icmp from fe80::/10 to ff02::/16 00900 0 0 allow ipv6-icmp from any to any icmp6types 1 01000 0 0 allow ipv6-icmp from any to any icmp6types 2,135,136 65000 34 2767 allow ip from any to any 65535 0 0 allow ip from any to any And the NAT works correctly. So, there's apparently something going on with in-kernel NAT and VNET, but I don't have the faintest clue where to look... Problem not solved but cheated around :)