From owner-freebsd-net@freebsd.org Mon Jul 24 11:21:11 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F614C7B1D0 for ; Mon, 24 Jul 2017 11:21:11 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward5o.cmail.yandex.net (forward5o.cmail.yandex.net [IPv6:2a02:6b8:0:1a72::28a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C147564A20 for ; Mon, 24 Jul 2017 11:21:10 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp1o.mail.yandex.net (smtp1o.mail.yandex.net [37.140.190.26]) by forward5o.cmail.yandex.net (Yandex) with ESMTP id 5851F20E92; Mon, 24 Jul 2017 14:20:58 +0300 (MSK) Received: from smtp1o.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp1o.mail.yandex.net (Yandex) with ESMTP id 587FB130050E; Mon, 24 Jul 2017 14:20:56 +0300 (MSK) Received: by smtp1o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id A7qBK1zAyD-KuNOGA5M; Mon, 24 Jul 2017 14:20:56 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1500895256; bh=u6IXQjAuX5PwIobjVgfjnmnWs7c+fRMkarpMGaYClhg=; h=Subject:To:References:From:Message-ID:Date:In-Reply-To; b=ZBPOcxTduSnpmeUyUM112swBmOZ6GkQPW8eOIeCwKxLOQjohlaRM+bDoHLz7iUDSo LTfPGMeqXwpTo0lvo3EMYT9NKCV0e2zl6orQDE8emhV0/yKRXy4l/LUDi3MTZjhMbd V9AFo4zulAGF6jL5Up5JNV62Ele0D6LGJD0jepuI= Authentication-Results: smtp1o.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0,1 0 Subject: Re: NAT before IPSEC - reply packets stuck at enc0 To: "Muenz, Michael" , freebsd-net@freebsd.org References: <459d59f7-2895-8aed-d547-be46a0fbb918@spam-fetish.org> <1c0de616-91ff-a6f9-d946-f098bc1a709f@spam-fetish.org> <911903d1-f353-d5d6-d400-d86150f88136@yandex.ru> <2d607e1a-a2c0-0f85-1530-c478962a76cd@spam-fetish.org> <3344e189-cdf0-a2c9-3a2a-645460866f2d@yandex.ru> <1279753e-9ad1-2c02-304e-5001e2bbc82f@spam-fetish.org> <15e6eb38-ef0c-7bfd-5f2c-d2acc8ea1af4@yandex.ru> <63e80fcf-915e-2dd5-d8c9-1904c8261c6f@yandex.ru> <1c91cd8f-105d-e886-3126-67505c6c3900@spam-fetish.org> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Message-ID: Date: Mon, 24 Jul 2017 14:18:25 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: <1c91cd8f-105d-e886-3126-67505c6c3900@spam-fetish.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="Qcuaj3v39TmoBVLO1d3NrWsCV9xmACmkx" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2017 11:21:11 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Qcuaj3v39TmoBVLO1d3NrWsCV9xmACmkx Content-Type: multipart/mixed; boundary="rqv4iNSLN231c0avbcsNJ0E7XVn7fl75k"; protected-headers="v1" From: "Andrey V. Elsukov" To: "Muenz, Michael" , freebsd-net@freebsd.org Message-ID: Subject: Re: NAT before IPSEC - reply packets stuck at enc0 References: <459d59f7-2895-8aed-d547-be46a0fbb918@spam-fetish.org> <1c0de616-91ff-a6f9-d946-f098bc1a709f@spam-fetish.org> <911903d1-f353-d5d6-d400-d86150f88136@yandex.ru> <2d607e1a-a2c0-0f85-1530-c478962a76cd@spam-fetish.org> <3344e189-cdf0-a2c9-3a2a-645460866f2d@yandex.ru> <1279753e-9ad1-2c02-304e-5001e2bbc82f@spam-fetish.org> <15e6eb38-ef0c-7bfd-5f2c-d2acc8ea1af4@yandex.ru> <63e80fcf-915e-2dd5-d8c9-1904c8261c6f@yandex.ru> <1c91cd8f-105d-e886-3126-67505c6c3900@spam-fetish.org> In-Reply-To: <1c91cd8f-105d-e886-3126-67505c6c3900@spam-fetish.org> --rqv4iNSLN231c0avbcsNJ0E7XVn7fl75k Content-Type: multipart/mixed; boundary="------------7672A0159611E8D5F8F8B955" Content-Language: en-US This is a multi-part message in MIME format. --------------7672A0159611E8D5F8F8B955 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 22.07.2017 08:36, Muenz, Michael wrote: > Am 21.07.2017 um 13:08 schrieb Andrey V. Elsukov: >> >> With ipfw(4) it should work, at least on FreeBSD. pfsense/opensense ha= ve >> their own patches, so I don't know what can be wrong there. >> >=20 > I also tried 11.0 and 11.1RC3 vanilla kernels, no luck. > Will build a test setup with the OPNsense devs. >=20 > I'm still positive that this can't be a huge issue. >=20 > Thanks for your efforts Andrey! Ok, let's try to debug the problem. Please, use 11.1-RC, it has significantly changed IPsec stack. Apply attached patch to if_enc(4), it makes if_enc a bit useful for debugging your problem. You need to rebuild and reinstall sys/modules/if_enc. Now enable verbose BPF logging: net.enc.out.ipsec_bpf_mask=3D3 net.enc.in.ipsec_bpf_mask=3D3 According your tcpdump output, you need to set net.enc.out.ipsec_filter_mask=3D2 Show what you will see in the `tcpdump -nvi enc0` with such config options. Also, show what you have in the `sysctl net.inet.ip.fw` and `ipfw show` output. --=20 WBR, Andrey V. Elsukov --------------7672A0159611E8D5F8F8B955 Content-Type: text/x-patch; name="if_enc.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="if_enc.diff" Index: sys/net/if_enc.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/net/if_enc.c (revision 321414) +++ sys/net/if_enc.c (working copy) @@ -223,10 +223,11 @@ enc_hhook(int32_t hhook_type, int32_t hhook_id, vo if (ctx->af !=3D hhook_id) return (EPFNOSUPPORT); =20 - if (((hhook_type =3D=3D HHOOK_TYPE_IPSEC_IN && - (ctx->enc & V_bpf_mask_in) !=3D 0) || + if ((ctx->enc & IPSEC_ENC_BEFORE) !=3D 0 && ( + (hhook_type =3D=3D HHOOK_TYPE_IPSEC_IN && + (V_bpf_mask_in & IPSEC_ENC_BEFORE) !=3D 0) || (hhook_type =3D=3D HHOOK_TYPE_IPSEC_OUT && - (ctx->enc & V_bpf_mask_out) !=3D 0)) && + (V_bpf_mask_out & IPSEC_ENC_BEFORE) !=3D 0)) && bpf_peers_present(ifp->if_bpf) !=3D 0) { hdr.af =3D ctx->af; hdr.spi =3D ctx->sav->spi; @@ -290,6 +291,23 @@ enc_hhook(int32_t hhook_type, int32_t hhook_id, vo return (EACCES); } (*ctx->mp)->m_pkthdr.rcvif =3D rcvif; + + if ((ctx->enc & IPSEC_ENC_AFTER) !=3D 0 && ( + (hhook_type =3D=3D HHOOK_TYPE_IPSEC_IN && + (V_bpf_mask_in & IPSEC_ENC_AFTER) !=3D 0) || + (hhook_type =3D=3D HHOOK_TYPE_IPSEC_OUT && + (V_bpf_mask_out & IPSEC_ENC_AFTER) !=3D 0)) && + bpf_peers_present(ifp->if_bpf) !=3D 0) { + hdr.af =3D ctx->af; + hdr.spi =3D ctx->sav->spi; + hdr.flags =3D 0; + if (ctx->sav->alg_enc !=3D SADB_EALG_NONE) + hdr.flags |=3D M_CONF; + if (ctx->sav->alg_auth !=3D SADB_AALG_NONE) + hdr.flags |=3D M_AUTH; + bpf_mtap2(ifp->if_bpf, &hdr, sizeof(hdr), *ctx->mp); + } + return (0); } =20 --------------7672A0159611E8D5F8F8B955-- --rqv4iNSLN231c0avbcsNJ0E7XVn7fl75k-- --Qcuaj3v39TmoBVLO1d3NrWsCV9xmACmkx Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAll114IACgkQAcXqBBDI oXojwQgAvjPXA3LLKwKQBB3UCePbSz+0llmXBNgycbuLpKdYNPm6G0Z9DoYq7O2a 60rLI35J4rht+pevxn1Sl/n1OXY1QfwCsvuWrHYwOSB5yLzBea2WLmTb5czl/Ao/ RWswjEwjkey6cykQvY0zDiG3dXyS10Srw5kS9CKrTU/jEABHdbeuq6+qsxSupHUN Kpnk6Sjfu+X2+uvudE7NmnecRTseCylN9TF5inoUFor6kbkdrZf1HEZMa/D/IhqZ JEnZUfuWPAxCMs761Xn9x7TkyrdT7Zc1rF/OyWQp1F3gvK+hwuJ7yOe3Zmu3ROOl ChRwPoqD2Mfa9wX+0fDhcjD006CbOw== =Jf5b -----END PGP SIGNATURE----- --Qcuaj3v39TmoBVLO1d3NrWsCV9xmACmkx--