From owner-freebsd-net@FreeBSD.ORG Tue Oct 21 16:06:49 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E9888A8 for ; Tue, 21 Oct 2014 16:06:49 +0000 (UTC) Received: from mail.1970jan1-epo.ch (mail.1970jan1-epo.ch [IPv6:2a02:2770:13::1a24:0:11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A4CB140 for ; Tue, 21 Oct 2014 16:06:48 +0000 (UTC) Received: from 1970jan1-epo.ch (c-174-57-248-44.hsd1.pa.comcast.net [174.57.248.44]) by mail.1970jan1-epo.ch (Postfix) with ESMTPSA id 8710B52D; Tue, 21 Oct 2014 16:06:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=1970jan1-epo.ch; s=1970; t=1413907605; bh=vQBbNTTwC9zUthgeKI6I/kp4wpvJq6APEK2rnSquDZQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=udFRh/AB8HAZDSGbKO+eoM6bgEUAByt2aFCItpPl5vkLXyaMmWwo5lrPyV9nWhDJ1 VZrvUUWLgykbKf63V7+UWHTO6LX4CuQMvWTYeO6J/Yqm/STpd6n7HUynBoujE9fHpr gVMcGxBawEn8EOjxw0DfI3eej0FVnQor6tEYvtaE= Date: Tue, 21 Oct 2014 12:06:43 -0400 From: Kyle Williams To: "Andrey V. Elsukov" Subject: Re: Broken IPsec + enc +pf/ipfw Message-ID: <20141021160643.GB2787@1970jan1-epo.ch> References: <544535C2.9020301@shrew.net> <544566D2.40303@FreeBSD.org> <544569CF.2060905@shrew.net> <54457599.4060102@yandex.ru> <54458001.6000507@shrew.net> <544611F8.9070403@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <544611F8.9070403@yandex.ru> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-net@freebsd.org, Matthew Grooms X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Oct 2014 16:06:49 -0000 Hello, I'm currently using 10.0, IPSEC, racoon, enc, and pf between two remote hosts without NATT. The gif tunnel is ipv4 only, host A is ipv4 only, host B is ipv4/ipv6. I use IPSEC to route traffic between jails on both hosts, with the jails using cloned lo1 and 10.0.0.0/8 addresses. I'm testing the posted patches on host A with the following pf.conf: block all pass all I'm using the recommended sysctl's: net.enc.in.ipsec_bpf_mask=2 net.enc.in.ipsec_filter_mask=2 net.enc.out.ipsec_bpf_mask=1 net.enc.out.ipsec_filter_mask=1 On Tue Oct 21 00:50:33 2014, Andrey V. Elsukov wrote: > On 20.10.2014 20:18, Matthew Grooms wrote: >> Lastly, I tried to locate a relevant PR but didn't find anything >> concrete. Is this related to the issue? And if so, can it be MFCd? >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=110959 > > Did you try the patch from last PR? It is small and should be applicable > to stable/10. On 10/20/2014 3:50 PM, Andrey V. Elsukov wrote: >This commit fixes similar problem with ipfw in 11.0-CURRENT. But I think >it won't help you with pf in 10. I guess r266800 is what you need. On Tue Oct 21 11:57:44 2014, Andrey V. Elsukov wrote: >It also changes places from where pfil consumers are called. You may use >dtrace script to see where is the problem. Try this: > >> kldload dtraceall >> cat > ~/ipsec.d >#!/usr/sbin/dtrace -s > >fbt::ipsec_filter:entry >{ > m = *(struct mbuf **)arg0; > ip = (struct ip *)m->m_hdr.mh_data; > printf("%s: %s: %s->%s proto %d", > (arg1 & 1) ? "in": "out", > (arg2 & 1) ? "before": "after", > inet_ntoa(&ip->ip_src.s_addr), > inet_ntoa(&ip->ip_dst.s_addr), > ip->ip_p); >} >^D >> chmod +x ~/ipsec.d >> ~/ipsec.d > >This script will print messages when ipsec_filter function will be >invoked. Can you show what it will print for your case? IPSEC traffic is still blocked after applying r272695. Here is the output of the above script during an incoming wget: 2 26761 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 2 26761 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 2 26761 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 ... With pf.conf only containing `pass all', traffic is passed (IP's munged): 2 26761 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 2 26761 ipsec_filter:entry out: after: 1.1.1.1->2.2.2.2 proto 4 1 26761 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 1 26761 ipsec_filter:entry out: after: 1.1.1.1->2.2.2.2 proto 4 1 26761 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 1 26761 ipsec_filter:entry out: after: 1.1.1.1->2.2.2.2 proto 4 2 26761 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 2 26761 ipsec_filter:entry out: after: 1.1.1.1->2.2.2.2 proto 4 1 26761 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 1 26761 ipsec_filter:entry out: after: 1.1.1.1->2.2.2.2 proto 4 ... Applying r266800 failed on sys/netipsec/ipsec_output.c on the last chunk, so I copied ipsec_output.c from head. r272695 is included as well. Traffic is still blocked with `block all' and `pass all' in pf.conf. Here is the dtrace output: 0 26759 ipsec_filter:entry in: after: 10.0.4.2->10.0.5.10 proto 6 3 26759 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 0 26759 ipsec_filter:entry in: after: 10.0.4.2->10.0.5.10 proto 6 3 26759 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 0 26759 ipsec_filter:entry in: after: 10.0.4.2->10.0.5.10 proto 6 3 26759 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 0 26759 ipsec_filter:entry in: after: 10.0.4.2->10.0.5.10 proto 6 ... And finally with only `pass all', traffic is passed: 0 26759 ipsec_filter:entry in: after: 10.0.4.2->10.0.5.10 proto 6 3 26759 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 3 26759 ipsec_filter:entry out: after: 1.1.1.1->2.2.2.2 proto 4 0 26759 ipsec_filter:entry in: after: 10.0.4.2->10.0.5.10 proto 6 3 26759 ipsec_filter:entry in: after: 10.0.4.2->10.0.5.10 proto 6 3 26759 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 3 26759 ipsec_filter:entry out: after: 1.1.1.1->2.2.2.2 proto 4 0 26759 ipsec_filter:entry in: after: 10.0.4.2->10.0.5.10 proto 6 0 26759 ipsec_filter:entry out: before: 10.0.5.10->10.0.4.2 proto 6 0 26759 ipsec_filter:entry out: after: 1.1.1.1->2.2.2.2 proto 4 ... I'm willing to test more kernel patches, but I can't install head. -- Kyle Williams (541) 250 0314 Kyle@1970Jan1-epo.ch PGP key: 0xD1E5BADF