From owner-freebsd-ipfw@freebsd.org Sat Dec 10 03:23:16 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22B70C6F11B for ; Sat, 10 Dec 2016 03:23:16 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EAC4E1959 for ; Sat, 10 Dec 2016 03:23:15 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id EECFC66275 for ; Fri, 9 Dec 2016 21:23:07 -0600 (CST) Subject: Re: IPFW problem with passing IPSEC through in-kernel NAT To: freebsd-ipfw@freebsd.org References: <099203a1-f601-bb79-548d-27c62fcbf556@denninger.net> <005b34c8-2217-fa06-5584-6999022481a3@denninger.net> <156E272C-0EFA-4A15-8544-C580AAEB6033@obsigna.com> <01fbc965-f5bc-0f62-eb89-02e097e03cf7@denninger.net> From: Karl Denninger Message-ID: Date: Fri, 9 Dec 2016 21:23:01 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <01fbc965-f5bc-0f62-eb89-02e097e03cf7@denninger.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms090906050805080905000008" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2016 03:23:16 -0000 This is a cryptographically signed message in MIME format. --------------ms090906050805080905000008 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 12/9/2016 07:16, Karl Denninger wrote: > On 12/9/2016 06:18, Dr. Rolf Jansen wrote: >>> Am 09.12.2016 um 02:11 schrieb Karl Denninger : >>> ... >>> Some more information on this issue.... I suspect that something is >>> getting mangled somewhere in the IP stack, perhaps related to hardwar= e >>> checksumming or similar -- or in the ipfw code. >> I had always ran into IPsec-NAT-UDP checksumming issues since I starte= d working with FreeBSD, that tim v8.0. With a rather simple change in the= respective kernel source file at least my issue can be resolved. This ma= y be related to your issue or even not, anyway, I guess it is worth to gi= ve it a try. >> >> I am now running FreeBSD 11-RELEASE-p5. On line 462 of file /usr/src/s= ys/netinet/udp_usrreq.c, I replaced: >> >> if (uh->uh_sum) { >> >> with: >> >> if (uh->uh_sum && >> uh->uh_dport !=3D htons(1701) && >> uh->uh_dport !=3D htons(4500)) { >> >> This effectively skips extended UDP checksumming for certain UDP ports= -- here the L2TP and IPsec-NAT-T ports. When I investigated the issue, I= found in one related RFC, that IPsec-NAT-T isn't supposed to do UDP chec= ksumming on the encapsulated packets anyway, and my patch enforces this b= ehaviour. >> >> Best regards >> >> Rolf >> > In this case is that I never get to the use of port 4500 (there are no > packets emitted on that port that I can find); the initial key exchange= > on port 500 is failing, and in-kernel NAT appears to be involved in som= e > fashion because I'm getting inside addresses that are (in some cases) > not being NATted at all despite the fact that as far as I can tell they= > *should* be. > > I'm going to spend some time refactoring the IPFW rule set to > compartmentalize the various paths through it more-fully. Perhaps that= > will shed some more light on the problem, or at least make > more-reasonable an attempt to trace it. > I have completely re-factored the IPFW rule set that I am using here (it was formerly built on top of the "Simple" config in /etc/rc.firewall) to be completely stand-alone and the problem has disappeared. Bottom line -- this appears to have been some sort of problem with the rule set rather than ipfw itself. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms090906050805080905000008 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEyMTAwMzIzMDFaME8GCSqGSIb3DQEJBDFCBEDI 1bOF8mB0n7DNPZyg5ujC8xWVTch0GfMWpUzQ/C1gsgFFDQicXtaR/w9VW2Qkb1qJu43Bmz5g GpRC4jEtCjjfMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAAX1uLFpi 2scUfsKL9d3xcwtmbeKb2Nsb4PIKR9tm9HnmvqPG1oZvGkxKXvySqBnMvKxpPJo1oY0Nd7uS s6Gw6GXV57Eczh7TH/GhE1KE85UEugwN7QWXL4bwyKQonJmjKrU62wkpW4y3M9BTu0JQAmEL SIuCokQM86Og2W3L7PZgIk5BHxWk3XMURTdAz4aIkOk9HlTi3C68Wf/DP+qM/enFxmdrZQVj t3eoqS7GmNfzbIMSVDNglvYg1bI/P7oVPHAp41OL82Ch0sGeCrTiU2YSA3wqzpDt9IIb4YuV RK2Z0LWhipBLg15AUiEoTbRFWEtMYxXZ9df+dJ0cA/lOJwuKJ1P1iAqUGTW+xL4fQj7tGi/h DyRlKzlAnJzR6M0HlwLazVELaohLTyHzYMW6t9VkT8YwzHa8I63tJ279wVfVdbm/mZrgo1KV KWe1RbwsR5+jk/4WZaCbCJzD+EkOCyKWmRMqmWAA0LgSyrJpsLBd2M79zjo/AB8ZsQGAijl3 ekYWVUHQbGlvCfcyM/VuSmKg+MfZdkocxKtLWPXnmhvBsIJ2zsgaAWjLLb8GTLMYe0G3BoZZ kB0xdwBilfoJ3sJILCLVoG8ElpwE6hDVaA+68ileZHvMoEa9K3X1jheo+zstZK3QPRjUTi4S cutMRWbp+aSt43iiV5XaHW97JQ0AAAAAAAA= --------------ms090906050805080905000008--