From owner-freebsd-net@FreeBSD.ORG Mon Oct 20 16:18:24 2014 Return-Path: <owner-freebsd-net@FreeBSD.ORG> Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0454FD65 for <freebsd-net@freebsd.org>; Mon, 20 Oct 2014 16:18:24 +0000 (UTC) Received: from mx1.shrew.net (mx1.shrew.net [38.97.5.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0B783B2 for <freebsd-net@freebsd.org>; Mon, 20 Oct 2014 16:18:23 +0000 (UTC) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx1.shrew.net (8.14.7/8.14.7) with ESMTP id s9KGHG9V026323 for <freebsd-net@freebsd.org>; Mon, 20 Oct 2014 11:17:16 -0500 (CDT) (envelope-from mgrooms@shrew.net) Received: from [10.16.32.30] (rrcs-50-84-127-134.sw.biz.rr.com [50.84.127.134]) by mail.shrew.net (Postfix) with ESMTPSA id 667BC18A7D7 for <freebsd-net@freebsd.org>; Mon, 20 Oct 2014 11:17:11 -0500 (CDT) Message-ID: <544535C2.9020301@shrew.net> Date: Mon, 20 Oct 2014 11:18:10 -0500 From: Matthew Grooms <mgrooms@shrew.net> User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Broken IPsec + enc +pf/ipfw Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mx1.shrew.net [10.24.10.10]); Mon, 20 Oct 2014 11:17:16 -0500 (CDT) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/> List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Mon, 20 Oct 2014 16:18:24 -0000 All, There appears to be an issue with FreeBSD 10.x when using enc device to filter inbound traffic on the receive path. After searching the mailing lists, I see two different people reporting the issue ... https://lists.freebsd.org/pipermail/freebsd-stable/2014-January/076900.html https://lists.freebsd.org/pipermail/freebsd-net/2014-February/037951.html The second report had tracked it down to a specific commit ... http://svnweb.freebsd.org/base?view=revision&revision=254519 There appears to be quite a bit of info in both email threads, but I'll do my best to sum it up again. When an ESP packet arrives at a host for processing, the packet is seen on the enc0 interface but isn't seen by the pf firewall. This prevents the firewall from creating a state entry so returned packets get blocked by the firewall. IMO, this degrades the security of IPsec as makes it pretty much impossible to have a stateful firewall protect inbound traffic that arrives from an IPsec peer unless you are willing to write rules that let private network traffic inbound via your public interface after disabling enc ... # enc0 device options net.enc.in.ipsec_bpf_mask=2 net.enc.in.ipsec_filter_mask=2 net.enc.out.ipsec_bpf_mask=1 net.enc.out.ipsec_filter_mask=1 NOTE: These are the recommended settings from the ENC(4) man page. It should cause the inbound and outbound packets to be visible on the enc device without the outer header. [root@fw1 ~]# tcpdump -nei enc0 src or dst x.x.x.x tcpdump: WARNING: enc0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enc0, link-type ENC (OpenBSD encapsulated IP), capture size 65535 bytes capability mode sandbox enabled 11:06:42.256172 (authentic,confidential): SPI 0x032a11d7: x.x.x.x > y.y.y.y: ICMP echo request, id 12350, seq 0, length 64 11:06:43.257731 (authentic,confidential): SPI 0x032a11d7: x.x.x.x > y.y.y.y: ICMP echo request, id 12350, seq 1, length 64 11:06:44.260129 (authentic,confidential): SPI 0x032a11d7: x.x.x.x > y.y.y.y: ICMP echo request, id 12350, seq 2, length 64 11:06:45.263452 (authentic,confidential): SPI 0x032a11d7: x.x.x.x > y.y.y.y: ICMP echo request, id 12350, seq 3, length 64 11:06:46.264225 (authentic,confidential): SPI 0x032a11d7: x.x.x.x > y.y.y.y: ICMP echo request, id 12350, seq 4, length 64 11:06:47.258684 (authentic,confidential): SPI 0x032a11d7: x.x.x.x > y.y.y.y: ICMP echo request, id 12350, seq 5, length 64 [root@fw1 ~]# tcpdump -nei pflog0 src or dst x.x.x.x tcpdump: WARNING: pflog0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 65535 bytes capability mode sandbox enabled 11:06:42.257911 rule 0..16777216/0(match): block in on vlan20: y.y.y.y > x.x.x.x: ICMP echo reply, id 12350, seq 0, length 64 11:06:43.258224 rule 0..16777216/0(match): block in on vlan20: y.y.y.y > x.x.x.x: ICMP echo reply, id 12350, seq 1, length 64 11:06:44.260459 rule 0..16777216/0(match): block in on vlan20: y.y.y.y > x.x.x.x: ICMP echo reply, id 12350, seq 2, length 64 11:06:45.263674 rule 0..16777216/0(match): block in on vlan20: y.y.y.y > x.x.x.x: ICMP echo reply, id 12350, seq 3, length 64 11:06:46.264364 rule 0..16777216/0(match): block in on vlan20: y.y.y.y > x.x.x.x: ICMP echo reply, id 12350, seq 4, length 64 11:06:47.259121 rule 0..16777216/0(match): block in on vlan20: y.y.y.y > x.x.x.x: ICMP echo reply, id 12350, seq 5, length 64 I can confirm this is a regression between 9.x to 10.x as I upgraded a set of firewalls last night from 9.2 to 10.0. They use exactly the same configuration with the exception of some tweaks to the carp interface config ( changed significantly between 9.x and 10.x ). I also have two other sets of firewalls that run the exact same type of configuration on 9.x that are working exactly as expected. Lastly, I tried to locate a relevant PR but didn't find anything concrete. Is this related to the issue? And if so, can it be MFCd? https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=110959 Thanks, -Matthew