From owner-freebsd-net@freebsd.org Tue Jul 25 08:25:47 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9DEADBDB46 for ; Tue, 25 Jul 2017 08:25:47 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward3j.cmail.yandex.net (forward3j.cmail.yandex.net [5.255.227.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 460E46D5A4 for ; Tue, 25 Jul 2017 08:25:46 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [IPv6:2a02:6b8:0:801::ac]) by forward3j.cmail.yandex.net (Yandex) with ESMTP id C325620F0F; Tue, 25 Jul 2017 11:25:37 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id A63BF3EC0F7B; Tue, 25 Jul 2017 11:25:28 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id EyNVhkFY9q-PRYaVqfn; Tue, 25 Jul 2017 11:25:27 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1500971127; bh=ABN8dew9bwGWNdZwWrA+j1zve89a+7S7NE/rBuqU9V8=; h=Subject:To:References:From:Message-ID:Date:In-Reply-To; b=qD9GvXQEOZtNvhIujGt/r+Vs7wlSdM1cccRRq2DxuErWlXGeKD2PCnCHV+N8dLUg3 xVReXHiTahC31QyKYTisFrQZA46gy9i25LG8DdKnWS9EL4N6Dqt28E0dK1y3kXzTbI zp8yiX16TbwVhFZElBICNRhAhQuz/UHOcs4luyM0= Authentication-Results: smtp2j.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0,1 0 Subject: Re: NAT before IPSEC - reply packets stuck at enc0 To: "Muenz, Michael" , freebsd-net@freebsd.org References: <459d59f7-2895-8aed-d547-be46a0fbb918@spam-fetish.org> <1c0de616-91ff-a6f9-d946-f098bc1a709f@spam-fetish.org> <911903d1-f353-d5d6-d400-d86150f88136@yandex.ru> <2d607e1a-a2c0-0f85-1530-c478962a76cd@spam-fetish.org> <3344e189-cdf0-a2c9-3a2a-645460866f2d@yandex.ru> <1279753e-9ad1-2c02-304e-5001e2bbc82f@spam-fetish.org> <15e6eb38-ef0c-7bfd-5f2c-d2acc8ea1af4@yandex.ru> <63e80fcf-915e-2dd5-d8c9-1904c8261c6f@yandex.ru> <1c91cd8f-105d-e886-3126-67505c6c3900@spam-fetish.org> <1e889acf-49d1-b70f-7097-82e6e4dfabb6@spam-fetish.org> <454ed1b7-a80f-b096-cfa1-3c32d1e60f7d@yandex.ru> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Message-ID: <5dfdfbb3-1046-5abe-b23a-b62c215b5d08@yandex.ru> Date: Tue, 25 Jul 2017 11:22:49 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="I64HfSqiGlCeOfli5nrFgB1En4jU6eN5I" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2017 08:25:47 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --I64HfSqiGlCeOfli5nrFgB1En4jU6eN5I Content-Type: multipart/mixed; boundary="8T1qC2HHavVhSkEFa4RFAU8NsGeTl98Am"; protected-headers="v1" From: "Andrey V. Elsukov" To: "Muenz, Michael" , freebsd-net@freebsd.org Message-ID: <5dfdfbb3-1046-5abe-b23a-b62c215b5d08@yandex.ru> Subject: Re: NAT before IPSEC - reply packets stuck at enc0 References: <459d59f7-2895-8aed-d547-be46a0fbb918@spam-fetish.org> <1c0de616-91ff-a6f9-d946-f098bc1a709f@spam-fetish.org> <911903d1-f353-d5d6-d400-d86150f88136@yandex.ru> <2d607e1a-a2c0-0f85-1530-c478962a76cd@spam-fetish.org> <3344e189-cdf0-a2c9-3a2a-645460866f2d@yandex.ru> <1279753e-9ad1-2c02-304e-5001e2bbc82f@spam-fetish.org> <15e6eb38-ef0c-7bfd-5f2c-d2acc8ea1af4@yandex.ru> <63e80fcf-915e-2dd5-d8c9-1904c8261c6f@yandex.ru> <1c91cd8f-105d-e886-3126-67505c6c3900@spam-fetish.org> <1e889acf-49d1-b70f-7097-82e6e4dfabb6@spam-fetish.org> <454ed1b7-a80f-b096-cfa1-3c32d1e60f7d@yandex.ru> In-Reply-To: --8T1qC2HHavVhSkEFa4RFAU8NsGeTl98Am Content-Type: multipart/mixed; boundary="------------A5B3023F169B70506BD3C49B" Content-Language: en-US This is a multi-part message in MIME format. --------------A5B3023F169B70506BD3C49B Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 25.07.2017 10:36, Muenz, Michael wrote: > Am 24.07.2017 um 19:01 schrieb Andrey V. Elsukov: >> >> .1.1: ICMP echo reply, id 33347, seq 28416, length 8 >> This does not match with what I expected to see. The reply here should= >> be something like "10.24.66.25 > 10.26.2.N: ICMP echo reply". >> >> It seems the problem is with ipfw_nat, that for both directions thinks= >> that packets are inbound and this leads to incorrect translation. >> >> Can you modify your IPsec security policies, so outgoing packets from >> 10.26.2.0/24 will go through the same tunnel? Then you need to modify >> nat rule: >> >> ipfw nat 1 config ip 10.26.1.1 >> ipfw add 179 nat 1 log ip from 10.26.2.0/24 to 10.24.66.0/24 out xmit >> enc0 >> ipfw add 179 nat 1 log ip from 10.24.66.0/24 to 10.26.1.1 in recv enc0= >> >=20 > Hi, >=20 > when I change it to >=20 > out xmit enc0 >=20 > nothing happens because the packets have to math the IPSEC SA before > entering the tunnel (and enc0 I guess). > So it has to be >=20 > in recv vtnet1 ICMP request should be matched by outbound IPsec policy. Looking to your tcpdump, you use tunnel IPsec mode. So, how this should work: * 10.26.2.N sends ICMP request to 10.24.66.25 * 10.26.1.1 handles it by tunnel mode IPsec security policy, something li= ke: spdadd -4 10.26.2.0/24 10.24.66.0/24 any -P out ipsec \ esp/tunnel/213.244.192.191-81.24.74.3/require; * IPsec code does lookup for IPsec SA and uses something like: add 213.244.192.191 81.24.74.3 esp 0x2478d746 -m tunnel -E ...; * Then if_enc(4) pfil handler is called and now you need to do address translation (with net.enc.out.ipsec_filter_mask=3D1), and in tcpdump you should see not yet translated packet: 10.26.2.N > 10.24.66.25: ICMP echo request .... * Then IPsec code does IP encapsulation and you will see: 213.244.192.191 > 81.24.74.3: IPIP ... 10.26.1.1 > 10.24.66.25: ICMP echo request (in my previous patch was a bug, and you did not see outbound packet two times in tcpdump, I attached new patch where it is fixed) * Then IPsec sends encrypted packet to 81.24.74.3. * The second host sends ICMP reply to 10.26.1.1. * 213.244.192.191 recieves encrypted packet and does IPsec SA lookup, it should have something like: add 81.24.74.3 213.244.192.191 esp 0xce702ac1 -m tunnel -E ...; * IPsec code does decryption and if_enc(4) hook is called, and you will see in the tcpdump: 81.24.74.3 > 213.244.192.191: IPIP ... 10.24.66.25 > 10.26.1.1: ICMP echo reply * Since we use net.enc.out.ipsec_filter_mask=3D2, this packet will not be= handled by firewall and IPsec will do IP decapsultion, and then will pass decapsulated packet to the pfil where it will be translated: 10.24.66.25 > 10.26.2.N: ICMP echo reply .... * Then this packet goes to ip_input() -> ip_forward() processing. To avoid extra IPsec processing you should have no security policy that matches "10.24.66.25 > 10.26.2.N" packet, or this should be policy that requires "none" IPsec processing. * Then ip_forward() sends this packet to 10.26.2.N. --=20 WBR, Andrey V. Elsukov --------------A5B3023F169B70506BD3C49B Content-Type: text/x-patch; name="if_enc.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="if_enc.diff" Index: sys/net/if_enc.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/net/if_enc.c (revision 321414) +++ sys/net/if_enc.c (working copy) @@ -223,10 +223,11 @@ enc_hhook(int32_t hhook_type, int32_t hhook_id, vo if (ctx->af !=3D hhook_id) return (EPFNOSUPPORT); =20 - if (((hhook_type =3D=3D HHOOK_TYPE_IPSEC_IN && - (ctx->enc & V_bpf_mask_in) !=3D 0) || + if ((ctx->enc & IPSEC_ENC_BEFORE) !=3D 0 && ( + (hhook_type =3D=3D HHOOK_TYPE_IPSEC_IN && + (V_bpf_mask_in & IPSEC_ENC_BEFORE) !=3D 0) || (hhook_type =3D=3D HHOOK_TYPE_IPSEC_OUT && - (ctx->enc & V_bpf_mask_out) !=3D 0)) && + (V_bpf_mask_out & IPSEC_ENC_BEFORE) !=3D 0)) && bpf_peers_present(ifp->if_bpf) !=3D 0) { hdr.af =3D ctx->af; hdr.spi =3D ctx->sav->spi; @@ -247,7 +248,7 @@ enc_hhook(int32_t hhook_type, int32_t hhook_id, vo (*ctx->mp)->m_pkthdr.len); } if ((ctx->enc & V_filter_mask_in) =3D=3D 0) - return (0); /* skip pfil processing */ + goto handle_bpf; /* skip pfil processing */ pdir =3D PFIL_IN; break; case HHOOK_TYPE_IPSEC_OUT: @@ -258,7 +259,7 @@ enc_hhook(int32_t hhook_type, int32_t hhook_id, vo (*ctx->mp)->m_pkthdr.len); } if ((ctx->enc & V_filter_mask_out) =3D=3D 0) - return (0); /* skip pfil processing */ + goto handle_bpf; /* skip pfil processing */ pdir =3D PFIL_OUT; break; default: @@ -280,7 +281,7 @@ enc_hhook(int32_t hhook_type, int32_t hhook_id, vo ph =3D NULL; } if (ph =3D=3D NULL || !PFIL_HOOKED(ph)) - return (0); + goto handle_bpf; /* Make a packet looks like it was received on enc(4) */ rcvif =3D (*ctx->mp)->m_pkthdr.rcvif; (*ctx->mp)->m_pkthdr.rcvif =3D ifp; @@ -290,6 +291,24 @@ enc_hhook(int32_t hhook_type, int32_t hhook_id, vo return (EACCES); } (*ctx->mp)->m_pkthdr.rcvif =3D rcvif; + +handle_bpf: + if ((ctx->enc & IPSEC_ENC_AFTER) !=3D 0 && ( + (hhook_type =3D=3D HHOOK_TYPE_IPSEC_IN && + (V_bpf_mask_in & IPSEC_ENC_AFTER) !=3D 0) || + (hhook_type =3D=3D HHOOK_TYPE_IPSEC_OUT && + (V_bpf_mask_out & IPSEC_ENC_AFTER) !=3D 0)) && + bpf_peers_present(ifp->if_bpf) !=3D 0) { + hdr.af =3D ctx->af; + hdr.spi =3D ctx->sav->spi; + hdr.flags =3D 0; + if (ctx->sav->alg_enc !=3D SADB_EALG_NONE) + hdr.flags |=3D M_CONF; + if (ctx->sav->alg_auth !=3D SADB_AALG_NONE) + hdr.flags |=3D M_AUTH; + bpf_mtap2(ifp->if_bpf, &hdr, sizeof(hdr), *ctx->mp); + } + return (0); } =20 --------------A5B3023F169B70506BD3C49B-- --8T1qC2HHavVhSkEFa4RFAU8NsGeTl98Am-- --I64HfSqiGlCeOfli5nrFgB1En4jU6eN5I Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAll2/9kACgkQAcXqBBDI oXoB/wf/Y7/5VpSG7v8h8pACw6Iyh/ZGCUoqGtsSLEp9WRFjW7kgP5rz+rFQrblj igHaPHFtUIRc30JV7BH3jLklXcjmm+2LELs77zbYl8SeOF0qQ5c6nBsjloBZpF8K gFou4alGdfCoCOn852KYs3tNB16Zg1gvJqeHxJSMS4mCtzslqUfuS0nvc6fu1kX7 61V3ONhEaAg2yze0WZtqX+lzlqWQwFjSwVwkOTiGUlBlPGaToWWUF+HfAcV6iGkD ePLrgJ4iuurNlC2919CuqXXZvX/cowfuFWxvfZ0Jf/nZd3yW3hGo/+uOdq7gioXx XhuWnqz4uJvsX8bTx+g8icdjMA9kJg== =W92W -----END PGP SIGNATURE----- --I64HfSqiGlCeOfli5nrFgB1En4jU6eN5I--