From owner-freebsd-net@freebsd.org Sun Dec 15 16:15:37 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 556421C817D for ; Sun, 15 Dec 2019 16:15:37 +0000 (UTC) (envelope-from john@saltant.com) Received: from twaddle.saltant.net (twaddle.saltant.net [72.78.188.147]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47bTyw1tzFz4fcW for ; Sun, 15 Dec 2019 16:15:35 +0000 (UTC) (envelope-from john@saltant.com) Received: from dither.saltant.net (dither.saltant.net [IPv6:2001:470:8d6f:1001::3]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by twaddle.saltant.net (Postfix) with ESMTPSA id 4E71D2AEBA; Sun, 15 Dec 2019 11:15:34 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=saltant.com; s=twaddle; t=1576426534; bh=U1fpFHNwewl7k7V2Jtf0S/X08VtlED6sIpezMbO6pJY=; h=Subject:To:References:From:Date:In-Reply-To; b=FAOSDhBWZ2t0//JlpRrW1YhkGJzcBrlco6fLQJBMP266/4xvSbKdGpxfjANcWq778 aj//Us2ZaIFb8W+uZQe+4UiqMzy22jjK8eLrvvajJ7wdVnGtm5SO0YkPknOYNK6zbx kGq5gPwuB1QgxY1sNjufUHaph7Bmvym0ibbGi4TlEoXZIxD9x11adyqrJat4ESYlj5 t0XjxArfcvtdqByOB9DiTfCY5mD/pascjJbVmz+nJLzcsgNGqIWJhHcqZ3k9sBnXtr igV22+r9yGOq+eAge0ek4Qja5lGmnoWHSYoIKNna35rQBvw6I3586OCwxAJFpdwRMe U3t/Ge+vVHRNA== Subject: Re: NAT64 return traffic vanishes after successful de-alias To: "Andrey V. Elsukov" , FreeBSD Networking References: <9f3ee846-1357-0b73-cc0f-e001ea74b15c@saltant.com> <52463470-973e-aa5f-73f5-dd9ba39edf79@yandex.ru> From: "John W. O'Brien" Autocrypt: addr=john@saltant.com; prefer-encrypt=mutual; keydata= mQINBFpcMG0BEACeAEQ0ZTUEH+6B8XIBid2H8g1yY+niHxVphqz8JwnQtYX+bS+Kl3vr783F HH81DEbfPtYgHY53NF9FjSzCyj13lXVnEGQOdxXzZVKsN1nyuXCN2hDOFH7Yc5yQ8h85T4Hv sqPIGIXOztu4MX14iUAcTgLhfibNQBeKDeNI+BBeaE9lPuNVeiM+xsI4JYcjmDbjFzAHRpBo ull0koUFh6RZAKE7u17yLej1pTIQQVjQpWdK37BAq4hdkLwjGDY8mDGo3ZwGdNibxIAxv/wi KU6u2DfUg8+kLHIhOqk/+kFQ/uK5YA1azsyD5eIbNAs4W7LglA6SkiGBglTwkP0VCrkPdD14 6sx3U7uFgexDWbVuhLIkcPQ0SRmnjgUKHgk7px/jMvAPKSKoL0JQNdP/+pnO9CDLGmoHx9gE 5kVr5dQK8c/WauEfimAdE9qLuN6vb0Iei73q3e3OOHAUusR5wC5SwXt4iilbaK4r04NKXyfb SB3+qWST07F9cmMscfEStSBhpez3awB+1jz8gr40tkEGsFZGvD2KKAgZdKpoxv6IrZepclWz HpqHF01SRFORYMsd1d83XlEu/S1/Z9YJ87RoCdZuYCkjnoRPtpTi9d+JD/u3ZiQFwLUz/Ne3 VqiGKvY66EGcO3tvANMg6GWD9sqlnBDp9Lls0ChEY3dgDYd6DQARAQABtCJKb2huIFcuIE8n QnJpZW4gPGpvaG5Ac2FsdGFudC5jb20+iQJUBBMBCAA+AhsDBQsJCAcCBhUKCQgLAgQWAgMB Ah4BAheAFiEENPkbBr3zmPAVSH2HM8TWS4ldvzsFAlpcMTMFCQX2qcYACgkQM8TWS4ldvztT xQ//eHb1mgd40Z0fN2GnJti6/9uJ771IO6slFQ02GZcXZI+FIQo8Yd1dHe0e0Codu78qvJNr ggUtqdxH6SVp7K1AWHeLH5S0PF6iG5B+YUux080wEv/Mr8PPMgAD8gS3wiPDDgB/kUXO52bn DC3Fc0dUrFE/JAOByVEEDL5nLF6SQNpAtIUnaAIIuhKxi0d40LMcLUwuJ6jExynw8Iu7OVtu Y1PRAH5ESt6wYZq8ro8ukh4rMOxiWtT1yNEgHgnq3N4jKErVo87YJijHSSj80IKxUiKb/T6K tGTEBTKiSUV3OFj0ZoPxcbUmhIg2sBCNHaUCiI0KabqN1NyK2glKtcK6NpWy3JIHvtr3+VL1 /tvQTwlVUIacmsuxkGzm5vJPs/i2RtwsJXEXPmIRNgJ1EwZgpg5VqqEUDlmSyRLb48QcDrdv utKLA1MKLib1fD+0XmxZTbCMlFMlvJjAoBlVq60mvB/Jnv1TTnZ2eN6DKMWoxHKmPICh5F1q esmT/aJRIUoCiAgcChi4Ol4XmW3dM7ypjKCGHzyr6emCky5pjqSQZyFzg0RN5UjUQBISAGmJ E8hCFZIy7tf8meqIDbtkONh+JShN6u3t02JrnzSOQjZCh5WQW9Pnu7unJlIsYB10aZ6rvuAK YjghT8QLG8QVgJj/U9oeVG1Ag60fmLZdOFjRGmm5Ag0EWlwwbQEQANebvidw1D5SKSmG3Ut8 p9vngBi5HjYe4FSYcfz0NgYa893RiScQ6yjOwuEf/fEoBgvpVnhcbu0JsaYvDNNzFGzPQcj0 CFhkr5s7REWNLGmmFCxCaGieTxIQdYsLxwn72mops8bsrL0a++8NDE+l7X4K3EUyp9GP7pIq 4l9jeIJ/RnX3yySRlXxcM3P+DV9ltXsnQ9pC/qEVVyK18C1zoiskhxmAY9cv9TJOaANHtA7R 7+hM5TyppIz7kqiwiCf6XfVFqKH0I0srdamb0KTnAZpmyx7iNKYl60PdIfEwkwck8fcGwOSA lwE9CLkHLwKMjx/gF3xRag5xjOdP/Out0cQ/pXv8DWnKblWbiGZheB4xUqhOT9Cj/8u/tKtC 51C9wID26hsrhtSAMJPUwQoo/SwLNEd1JpkqUP1njOdlV8FmM1EozHLPSvwlTm6oWwubkkY6 QkUHqXuO+2VdNhyDfx23fQhd0UPhQ0ceDRnjaSB9ycWqpktBP5iNQajYbx5Ktt8fC2Y+Ztjo u1KY7wJSUzqh7uZgR1TqIOVZp7bdPLBGHW5eNEf0Awq17utGe6d9i4hPmeNqELUz71hjmABm bIQJ+VgqYcQ0T/PrjwhzHv5g3jn67/ftW91nlTNpbhwm8suIdPA1hF6vgnZ3B4+JsevnevLG yU6YCb0OOKleP6pZABEBAAGJAjwEGAEIACYCGwwWIQQ0+RsGvfOY8BVIfYczxNZLiV2/OwUC WlwxTQUJBBV2YAAKCRAzxNZLiV2/O2PnD/wMKz/rzYbf0SaTvgae4jqryrcWRta56dcnVe7W KPuUu4Q/WBGhXKeCfPrlr399bILxZGw5TXuGMjS8gEoMd81PEMcWaMpgg3F569Cxd9GN6AZd LXXrZa0aM7dvZkz98ymILEnqHMpF74sLvZY2PrsOwo2gKXNqhtCJ2ph8OUKhG+NHvAomjMu9 lPQMkXJ4HRV0OljawqAe4y+IFu2K4abWwZw1mdniTCb5al8V2umzf26QL0DgeFp3banlfjYW Dn5cRuDBQqIoR/6cQaKdFKTJYiTVK3p3WRWiJQniYi39S8CR646w+zVi7ax1shSB0r0lxIFo CZu285HcMd7HsHH+T2ZI45ilayUoyoZvxPPlwhiRzyYZ6qqAAXKDihhda7uNApUqLwoSn5FW njmx6KdlVPF9ycCdf+in5k6nVlHWG15ogF/Y96K+/Q1Iuod9rzWqT4bz9a5olY8r++QE3V1b H3z803wXEUAJg+WGTkYXFNw7w6RhSSEhBRzupDoCROSkRhe3vQGy5FLG+BMV9n9nevhj5sBx CM1BbNBdB5H/2RcXh0wSb6zjewgs3UAbBvCQOdMAMo8XpYM5SLBqtaY7oalBElTxtFnwSNJm hMbahYE/wHbkmMqalrzGyQxbSUdrmE64CIX8xmv47fnjRoTZMzKim/02MRH+Ss1M+rLzp7kB DQRaXDCyAQgAyaQWiyazOcbV1JVndXG3JbeWom0Ros4RgjliRNLTm4rLefgk4mtvQpsGvTX7 bsiNRkxu2KdDo8zEG95e7FqbftxOFlptaEnJlrfrod6a5GX7E4cW74RgMHU9yj0IYijInENP FDf5yok1NvQ4IdS7Wqetta8X3hb2+iAXVkwDOhC9HTxEKZSWpsuZSs3eh2B2ypowa/12B4Dj ZXZ0ImUeLXqjL/ze5HmwcrQ1wqvo1pxc5NTA8vmwP4d9bnuKV6C7OIqw1Bw/VCxmNjX31gL3 a8K1eTMWu6TBkZ8z798eidmpU6gHB4zqE7NhBpHvNPePbQodXsMH40b5W82B3CRNDwARAQAB iQNyBBgBCAAmAhsCFiEENPkbBr3zmPAVSH2HM8TWS4ldvzsFAlpcMU0FCQQVdhsBQMB0IAQZ AQgAHRYhBCqRB5JEaEg4iCZEDlj7SueqT/5uBQJaXDCyAAoJEFj7SueqT/5u3SEH/21Wd0DD DVDx9jW6j7AlYSaJI9FZQVBZq0AakK3DgzWoyppb0NgNIWCRkghYmeni7ZyufmJg8mqzoWJT E8SeS9CYBhtmT3VO2N+w6x988GBplC69nhqoQBvHf81REZlWC72k5DIxfHJHWLI/9/aWc3ND wwifSdIjuGwfytqDp1RcAlCgx79ej8oodEII+PIBsLV6C7S9QV6kfJ1OXHE/lqbBV62Ywu/Y xHhvWgCOR8mz41NMrDz/K0otILUVwoDcE5tMOx5j6GFQEItFi/GFKogssV+4Tk9COmPS8ka7 ZFEnjjdoCiL3OveN2P4mBqG2Mh/0HAA/0v2DP6jqKHmaINkJEDPE1kuJXb872swP/3Ftis9+ 285gWUT7sKMbHkLxwwc/4Ga0vkBFyp9xRprlkvd7ivq2DP1gWvVds/V28BGFQ7SoRA5rLO+K BP7a2JJCk0025W4M8D6rp2mYj7iHLoxCNb5bScPYmBMnhKH4fg9QJWZozHik7wXrQNmrRb3A e+L0XfQ83tviuQhQsi+JtupQgf9d2a2Yza5bppdPYKialrJre3LIh/T4g4kJeoa4IQPwkXe8 httQa48571xINK2vtNkIjc4iG7mM4bAFCjZLx7AM8Dc3vVcZNbd21o5mhxe0WN9nICG8oKk5 9KwJKu6ul6TR0BxzvzpgcQyZGsDfhETsI/z0G7TVUXnRbZIgJHYH7DOVycjZLHAxQ5KweHkA bincQlaI0HMFf7FGtYnrUy3voTZ70xYQoYH1Gh/MeuELnscsTNBvYgOI2xYPOYilcFA4D3ZP p7p7ou7eZRkBLD6HHnrTgZB/Hn6FIklwll8jev3KBYWjSGKKcJQMK38OvJHDwHe1Wue+xpPl tFGoX7KCLFxe+VDmFjhfcgmoPJYBBq6D2s5AUj7cjTZUhb727ROSsK6KFCQhW25j8MJF+qGT RcRcWqgTQZoxWNqr5Foyeu3KoUY5ywBcPjqBMyqod27wOS8iQmHskLf7v9UrOR3/zLWASFyX MaAD/5Af9kIDAmJcwLvO0Mz9HDQB Organization: Saltant Solutions Message-ID: <15ce6744-91f1-e755-22c7-0c5355686d90@saltant.com> Date: Sun, 15 Dec 2019 11:15:30 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:68.0) Gecko/20100101 Thunderbird/68.3.0 MIME-Version: 1.0 In-Reply-To: <52463470-973e-aa5f-73f5-dd9ba39edf79@yandex.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="IglUFb5dkvL3RpZtKTPLgvLWWSoUEBUfL" X-Rspamd-Queue-Id: 47bTyw1tzFz4fcW X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=saltant.com header.s=twaddle header.b=FAOSDhBW; dmarc=none; spf=pass (mx1.freebsd.org: domain of john@saltant.com designates 72.78.188.147 as permitted sender) smtp.mailfrom=john@saltant.com X-Spamd-Result: default: False [-5.42 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[saltant.com:s=twaddle]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:72.78.188.144/29]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; DMARC_NA(0.00)[saltant.com]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[saltant.com:+]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; FREEMAIL_TO(0.00)[yandex.ru]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; IP_SCORE(-0.82)[ipnet: 72.78.0.0/16(-4.87), asn: 701(0.81), country: US(-0.05)]; ASN(0.00)[asn:701, ipnet:72.78.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2019 16:15:37 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --IglUFb5dkvL3RpZtKTPLgvLWWSoUEBUfL Content-Type: multipart/mixed; boundary="OsYedDgdivxpdAXCHQbXQlDunjkTOjnBk" --OsYedDgdivxpdAXCHQbXQlDunjkTOjnBk Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2019/12/15 05:44, Andrey V. Elsukov wrote: > On 14.12.2019 22:54, John W. O'Brien wrote: >> Hello FreeBSD Networking, >> >> As the subject summarizes, I have a mostly-working NAT64 rig, but retu= rn >> traffic is disappearing, and I haven't been able to figure out why. I >> observe the post-translation (4-to-6) packets via ipfwlog0, but a simp= le >> ipfw counter rule ipfw matches nothing. >=20 > I suspect you have disabled IPv6 on the interface, where IPv4 address i= s > configured. Check that IFDISABLED flag is not set on the IPv4 side > interface. >=20 > When NAT64 does translation, by default it reschedules a packet again o= n > the same interface, but from another address family, so if you have > disabled IPv6, a packet will be just dropped by ip6_input. > You can enable IPv6 by the following command: >=20 > # ifconfig igb0 inet6 -ifdisabled Yes, this is exactly the problem. Thank you very much! The reason it was working in the EC2 case is because the FreeBSD AMIs set ipv6_activate_all_interfaces=3D"YES". It helps me quite a lot to learn the concept of "reschedules a packet again on the same interface". That fills in a gap that I am sure will come in handy when trying to reason about behavior in the future. Incidentally, where are those drops counted? I did start looking at "netstat -i" and "netstat -s" for clues, and even now that I know what to look for, I'm not sure I know what I'm seeing. Is it "ip6: output packets discarded due to no route"? --=20 John W. O'Brien OpenPGP keys: 0x33C4D64B895DBF3B --OsYedDgdivxpdAXCHQbXQlDunjkTOjnBk-- --IglUFb5dkvL3RpZtKTPLgvLWWSoUEBUfL Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEKpEHkkRoSDiIJkQOWPtK56pP/m4FAl32XCIACgkQWPtK56pP /m5AvAgAlCos7ED2TYRMBXuk6jQXMXv1hmSu48rsVbTp1werlLCCXbprdARlPK3Q NKLRTIIpYMJE/0Otqpna/EcLCRlarpRR5iLwnOc0O5guwdKG6BKcmFZcaV1S7pNq +VECPi0GuyolAWlwA1ZahsGiSYLAxpOGDpwPHpQYRMqdryrw1M/ElXT5cM2UE9qP rU2m2IUy7BnOqgSPnWXm4UCRt+Z69tstQteLBmGq1mCGpb0ORQtQ3bIgH9yhS9LS G/ilplKy4XbZKxn0ZI5SsuzRhP4QzqeL8ANoCE4cAJI0wuBW6TDlQap/+7vJ1jkx TzbfZimr5i2fPsreDh2WYBGx6vSqMA== =88Dp -----END PGP SIGNATURE----- --IglUFb5dkvL3RpZtKTPLgvLWWSoUEBUfL--