From owner-freebsd-net@freebsd.org Sat Dec 14 22:35:45 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8BF4B1D89E9 for ; Sat, 14 Dec 2019 22:35:45 +0000 (UTC) (envelope-from john@saltant.com) Received: from twaddle.saltant.net (twaddle.saltant.net [72.78.188.147]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47b2S020Cjz3L6S for ; Sat, 14 Dec 2019 22:35:43 +0000 (UTC) (envelope-from john@saltant.com) Received: from statler.priv.n.saltant.net (unknown [IPv6:2001:470:8d6f:0:e89c:86ea:6d81:3ec7]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by twaddle.saltant.net (Postfix) with ESMTPSA id 007222AC15; Sat, 14 Dec 2019 17:35:42 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=saltant.com; s=twaddle; t=1576362943; bh=q9t/HTLCIfb1T6/8jv0A28u1Trn4OOLLmnPvwiEeTDk=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=ANVsJ+4infhOur+M8TSTvy9V8Gr/7P/LETttSuHmCVVCiyl5am4eS6sW/ZaKVbNVo gABLQkNrEMt185tLEMKHbOuLW4Bd6B6aUlfZv3r5ZF7/taTqwF262LqU6nmQa4O6JW pyA7mdVIE0ULuJ5JWfII5tsaAQok+iSoosvIFnZccqmacLx2uX/xMOh1H0IPniodSM BFHKWRGeVAYntq7O0X9lrwgCjwcBs55qxJEKBctvdTUYiAPdksg94YhqNJ4+Yq4S6P IZoDBJLdIQ19TkEXO4GlQ5AVZH2rCG2vmJk/Efeofe95gEG/4ObQhu3ui9DsbNCHAo feJgEN2b3NMuw== Subject: Re: NAT64 return traffic vanishes after successful de-alias To: bsd-lists@BSDforge.com Cc: FreeBSD Networking References: <2401399a05f75fa4b78f4d66c67c9e97@udns.ultimatedns.net> From: "John W. O'Brien" Autocrypt: addr=john@saltant.com; prefer-encrypt=mutual; keydata= mQINBFpcMG0BEACeAEQ0ZTUEH+6B8XIBid2H8g1yY+niHxVphqz8JwnQtYX+bS+Kl3vr783F HH81DEbfPtYgHY53NF9FjSzCyj13lXVnEGQOdxXzZVKsN1nyuXCN2hDOFH7Yc5yQ8h85T4Hv sqPIGIXOztu4MX14iUAcTgLhfibNQBeKDeNI+BBeaE9lPuNVeiM+xsI4JYcjmDbjFzAHRpBo ull0koUFh6RZAKE7u17yLej1pTIQQVjQpWdK37BAq4hdkLwjGDY8mDGo3ZwGdNibxIAxv/wi KU6u2DfUg8+kLHIhOqk/+kFQ/uK5YA1azsyD5eIbNAs4W7LglA6SkiGBglTwkP0VCrkPdD14 6sx3U7uFgexDWbVuhLIkcPQ0SRmnjgUKHgk7px/jMvAPKSKoL0JQNdP/+pnO9CDLGmoHx9gE 5kVr5dQK8c/WauEfimAdE9qLuN6vb0Iei73q3e3OOHAUusR5wC5SwXt4iilbaK4r04NKXyfb SB3+qWST07F9cmMscfEStSBhpez3awB+1jz8gr40tkEGsFZGvD2KKAgZdKpoxv6IrZepclWz HpqHF01SRFORYMsd1d83XlEu/S1/Z9YJ87RoCdZuYCkjnoRPtpTi9d+JD/u3ZiQFwLUz/Ne3 VqiGKvY66EGcO3tvANMg6GWD9sqlnBDp9Lls0ChEY3dgDYd6DQARAQABtCJKb2huIFcuIE8n QnJpZW4gPGpvaG5Ac2FsdGFudC5jb20+iQJUBBMBCAA+AhsDBQsJCAcCBhUKCQgLAgQWAgMB Ah4BAheAFiEENPkbBr3zmPAVSH2HM8TWS4ldvzsFAlpcMTMFCQX2qcYACgkQM8TWS4ldvztT xQ//eHb1mgd40Z0fN2GnJti6/9uJ771IO6slFQ02GZcXZI+FIQo8Yd1dHe0e0Codu78qvJNr ggUtqdxH6SVp7K1AWHeLH5S0PF6iG5B+YUux080wEv/Mr8PPMgAD8gS3wiPDDgB/kUXO52bn DC3Fc0dUrFE/JAOByVEEDL5nLF6SQNpAtIUnaAIIuhKxi0d40LMcLUwuJ6jExynw8Iu7OVtu Y1PRAH5ESt6wYZq8ro8ukh4rMOxiWtT1yNEgHgnq3N4jKErVo87YJijHSSj80IKxUiKb/T6K tGTEBTKiSUV3OFj0ZoPxcbUmhIg2sBCNHaUCiI0KabqN1NyK2glKtcK6NpWy3JIHvtr3+VL1 /tvQTwlVUIacmsuxkGzm5vJPs/i2RtwsJXEXPmIRNgJ1EwZgpg5VqqEUDlmSyRLb48QcDrdv utKLA1MKLib1fD+0XmxZTbCMlFMlvJjAoBlVq60mvB/Jnv1TTnZ2eN6DKMWoxHKmPICh5F1q esmT/aJRIUoCiAgcChi4Ol4XmW3dM7ypjKCGHzyr6emCky5pjqSQZyFzg0RN5UjUQBISAGmJ E8hCFZIy7tf8meqIDbtkONh+JShN6u3t02JrnzSOQjZCh5WQW9Pnu7unJlIsYB10aZ6rvuAK YjghT8QLG8QVgJj/U9oeVG1Ag60fmLZdOFjRGmm5Ag0EWlwwbQEQANebvidw1D5SKSmG3Ut8 p9vngBi5HjYe4FSYcfz0NgYa893RiScQ6yjOwuEf/fEoBgvpVnhcbu0JsaYvDNNzFGzPQcj0 CFhkr5s7REWNLGmmFCxCaGieTxIQdYsLxwn72mops8bsrL0a++8NDE+l7X4K3EUyp9GP7pIq 4l9jeIJ/RnX3yySRlXxcM3P+DV9ltXsnQ9pC/qEVVyK18C1zoiskhxmAY9cv9TJOaANHtA7R 7+hM5TyppIz7kqiwiCf6XfVFqKH0I0srdamb0KTnAZpmyx7iNKYl60PdIfEwkwck8fcGwOSA lwE9CLkHLwKMjx/gF3xRag5xjOdP/Out0cQ/pXv8DWnKblWbiGZheB4xUqhOT9Cj/8u/tKtC 51C9wID26hsrhtSAMJPUwQoo/SwLNEd1JpkqUP1njOdlV8FmM1EozHLPSvwlTm6oWwubkkY6 QkUHqXuO+2VdNhyDfx23fQhd0UPhQ0ceDRnjaSB9ycWqpktBP5iNQajYbx5Ktt8fC2Y+Ztjo u1KY7wJSUzqh7uZgR1TqIOVZp7bdPLBGHW5eNEf0Awq17utGe6d9i4hPmeNqELUz71hjmABm bIQJ+VgqYcQ0T/PrjwhzHv5g3jn67/ftW91nlTNpbhwm8suIdPA1hF6vgnZ3B4+JsevnevLG yU6YCb0OOKleP6pZABEBAAGJAjwEGAEIACYCGwwWIQQ0+RsGvfOY8BVIfYczxNZLiV2/OwUC WlwxTQUJBBV2YAAKCRAzxNZLiV2/O2PnD/wMKz/rzYbf0SaTvgae4jqryrcWRta56dcnVe7W KPuUu4Q/WBGhXKeCfPrlr399bILxZGw5TXuGMjS8gEoMd81PEMcWaMpgg3F569Cxd9GN6AZd LXXrZa0aM7dvZkz98ymILEnqHMpF74sLvZY2PrsOwo2gKXNqhtCJ2ph8OUKhG+NHvAomjMu9 lPQMkXJ4HRV0OljawqAe4y+IFu2K4abWwZw1mdniTCb5al8V2umzf26QL0DgeFp3banlfjYW Dn5cRuDBQqIoR/6cQaKdFKTJYiTVK3p3WRWiJQniYi39S8CR646w+zVi7ax1shSB0r0lxIFo CZu285HcMd7HsHH+T2ZI45ilayUoyoZvxPPlwhiRzyYZ6qqAAXKDihhda7uNApUqLwoSn5FW njmx6KdlVPF9ycCdf+in5k6nVlHWG15ogF/Y96K+/Q1Iuod9rzWqT4bz9a5olY8r++QE3V1b H3z803wXEUAJg+WGTkYXFNw7w6RhSSEhBRzupDoCROSkRhe3vQGy5FLG+BMV9n9nevhj5sBx CM1BbNBdB5H/2RcXh0wSb6zjewgs3UAbBvCQOdMAMo8XpYM5SLBqtaY7oalBElTxtFnwSNJm hMbahYE/wHbkmMqalrzGyQxbSUdrmE64CIX8xmv47fnjRoTZMzKim/02MRH+Ss1M+rLzp7kB DQRaXDCyAQgAyaQWiyazOcbV1JVndXG3JbeWom0Ros4RgjliRNLTm4rLefgk4mtvQpsGvTX7 bsiNRkxu2KdDo8zEG95e7FqbftxOFlptaEnJlrfrod6a5GX7E4cW74RgMHU9yj0IYijInENP FDf5yok1NvQ4IdS7Wqetta8X3hb2+iAXVkwDOhC9HTxEKZSWpsuZSs3eh2B2ypowa/12B4Dj ZXZ0ImUeLXqjL/ze5HmwcrQ1wqvo1pxc5NTA8vmwP4d9bnuKV6C7OIqw1Bw/VCxmNjX31gL3 a8K1eTMWu6TBkZ8z798eidmpU6gHB4zqE7NhBpHvNPePbQodXsMH40b5W82B3CRNDwARAQAB iQNyBBgBCAAmAhsCFiEENPkbBr3zmPAVSH2HM8TWS4ldvzsFAlpcMU0FCQQVdhsBQMB0IAQZ AQgAHRYhBCqRB5JEaEg4iCZEDlj7SueqT/5uBQJaXDCyAAoJEFj7SueqT/5u3SEH/21Wd0DD DVDx9jW6j7AlYSaJI9FZQVBZq0AakK3DgzWoyppb0NgNIWCRkghYmeni7ZyufmJg8mqzoWJT E8SeS9CYBhtmT3VO2N+w6x988GBplC69nhqoQBvHf81REZlWC72k5DIxfHJHWLI/9/aWc3ND wwifSdIjuGwfytqDp1RcAlCgx79ej8oodEII+PIBsLV6C7S9QV6kfJ1OXHE/lqbBV62Ywu/Y xHhvWgCOR8mz41NMrDz/K0otILUVwoDcE5tMOx5j6GFQEItFi/GFKogssV+4Tk9COmPS8ka7 ZFEnjjdoCiL3OveN2P4mBqG2Mh/0HAA/0v2DP6jqKHmaINkJEDPE1kuJXb872swP/3Ftis9+ 285gWUT7sKMbHkLxwwc/4Ga0vkBFyp9xRprlkvd7ivq2DP1gWvVds/V28BGFQ7SoRA5rLO+K BP7a2JJCk0025W4M8D6rp2mYj7iHLoxCNb5bScPYmBMnhKH4fg9QJWZozHik7wXrQNmrRb3A e+L0XfQ83tviuQhQsi+JtupQgf9d2a2Yza5bppdPYKialrJre3LIh/T4g4kJeoa4IQPwkXe8 httQa48571xINK2vtNkIjc4iG7mM4bAFCjZLx7AM8Dc3vVcZNbd21o5mhxe0WN9nICG8oKk5 9KwJKu6ul6TR0BxzvzpgcQyZGsDfhETsI/z0G7TVUXnRbZIgJHYH7DOVycjZLHAxQ5KweHkA bincQlaI0HMFf7FGtYnrUy3voTZ70xYQoYH1Gh/MeuELnscsTNBvYgOI2xYPOYilcFA4D3ZP p7p7ou7eZRkBLD6HHnrTgZB/Hn6FIklwll8jev3KBYWjSGKKcJQMK38OvJHDwHe1Wue+xpPl tFGoX7KCLFxe+VDmFjhfcgmoPJYBBq6D2s5AUj7cjTZUhb727ROSsK6KFCQhW25j8MJF+qGT RcRcWqgTQZoxWNqr5Foyeu3KoUY5ywBcPjqBMyqod27wOS8iQmHskLf7v9UrOR3/zLWASFyX MaAD/5Af9kIDAmJcwLvO0Mz9HDQB Organization: Saltant Solutions Message-ID: Date: Sat, 14 Dec 2019 17:35:37 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:68.0) Gecko/20100101 Thunderbird/68.3.0 MIME-Version: 1.0 In-Reply-To: <2401399a05f75fa4b78f4d66c67c9e97@udns.ultimatedns.net> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="lUZEI2Au6YMZk3rx4mk0MSNsKSkamfFn4" X-Rspamd-Queue-Id: 47b2S020Cjz3L6S X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=saltant.com header.s=twaddle header.b=ANVsJ+4i; dmarc=none; spf=pass (mx1.freebsd.org: domain of john@saltant.com designates 72.78.188.147 as permitted sender) smtp.mailfrom=john@saltant.com X-Spamd-Result: default: False [-4.44 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[saltant.com:s=twaddle]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:72.78.188.144/29]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; DMARC_NA(0.00)[saltant.com]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[saltant.com:+]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; IP_SCORE(0.15)[asn: 701(0.83), country: US(-0.05)]; ASN(0.00)[asn:701, ipnet:72.78.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2019 22:35:45 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --lUZEI2Au6YMZk3rx4mk0MSNsKSkamfFn4 Content-Type: multipart/mixed; boundary="3gUBQze92IdzxAX9WNuhPK2Mcl6sMjicJ" --3gUBQze92IdzxAX9WNuhPK2Mcl6sMjicJ Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2019/12/14 17:15, Chris wrote: > On Sat, 14 Dec 2019 14:54:26 -0500 John W. OBrien john@saltant.com said= >=20 >> Hello FreeBSD Networking, >> >> As the subject summarizes, I have a mostly-working NAT64 rig, but retu= rn >> traffic is disappearing, and I haven't been able to figure out why. I >> observe the post-translation (4-to-6) packets via ipfwlog0, but a simp= le >> ipfw counter rule ipfw matches nothing. >> >> My attempt to develop a minimum reproducible example failed in the sen= se >> that I did not reproduce the problem. Of course, this implies that one= >> of the many differences between the simplified test (EC2 instance, two= >> jails) and the problem rig (physical server, lagg, vlans, other things= >> going on) is the cause. >> >> What I am hoping this list can help me with is being smart about what = I >> try next. Otherwise, I would probably just try to brute force a soluti= on >> by thinking of ways to permute the config that would rule each possibl= e >> difference in or out. >> >> So far my main troubleshooting tools have been ipfw for its rule >> counters and nat64lsn stats output, netstat to look at fibs, and tcpdu= mp >> pointed at real and diagnostic interfaces. What debugging tools and >> techniques should I employ to do better than brute force? >> >> If it would help, I would gladly share the working, EC2/jail demo >> configs on the list. Sharing the non-working configs I would prefer to= >> do privately or not at all. >> >> This is on 12.1-RELEASE. >> >> Thank you, >=20 > pf(4) is pretty close to metal, and would probably be a good candidate = for > acquiring the type of statistics your hoping to find; pfctl(8), pfctl -= s, > and pfctl -T are a few examples. Hi Chris, Thank you for the suggestion. I think I need a little help understanding how I would put it into practice though. The nat64lsn module is part of the ipfw firewall, and pf in FreeBSD hasn't yet picked up a NAT64 capability, so I cannot abandon ipfw in this case. Is the idea to run ipfw and pf at the same time? --=20 John W. O'Brien OpenPGP keys: 0x33C4D64B895DBF3B --3gUBQze92IdzxAX9WNuhPK2Mcl6sMjicJ-- --lUZEI2Au6YMZk3rx4mk0MSNsKSkamfFn4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEKpEHkkRoSDiIJkQOWPtK56pP/m4FAl31Y7oACgkQWPtK56pP /m7/2Af7BHL+F9yLrieF3i3AQRPHvGz/QtkkOAdXg1nn2v4IsmJY2mIsfvQLb12u mcZhmSW25TkQptur/L4U3p/+7K4EhseH1rzMbstalzI01M7KVkmlu3dVM/QFL6kf b0kuQ0XOYNGn9WQMGvmZD1Z46hRRhNLxUZ8bRJoXRCnLNAKoF7qsn56lyL8sf6jB oEE17Zj6YCi9tRfF0KOepjenI0fxuCS2Fcn6Rled8E8wj7WDd6q27dQ41HM0U15y p6coOUNHd/rncnFoQAX8oGJHEheIkjI3vAmWYtmwND3HOKaEAZE5dNWhVAmvW3Uj cQH/sYA5wKwTqaWwn64g/X0V66y60g== =79JF -----END PGP SIGNATURE----- --lUZEI2Au6YMZk3rx4mk0MSNsKSkamfFn4--