From nobody Sun Sep 18 09:56:12 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MVjqz5zXWz4cbj9 for ; Sun, 18 Sep 2022 09:56:15 +0000 (UTC) (envelope-from driesm.michiels@gmail.com) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MVjqy2NkWz3wkV; Sun, 18 Sep 2022 09:56:14 +0000 (UTC) (envelope-from driesm.michiels@gmail.com) Received: by mail-wr1-x432.google.com with SMTP id n12so2365565wrx.9; Sun, 18 Sep 2022 02:56:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=thread-index:content-language:content-transfer-encoding :mime-version:message-id:date:subject:in-reply-to:references:cc:to :from:from:to:cc:subject:date; bh=Jp5Ei23Ajmmo0CPWLiFIrCP7jET9y8hJo0ZDA/we70o=; b=JJq1/GMbd8DrQZdE9T4A6scJfGedKOMlWHrmFUguuvN66ikU/wvw4O8RBUZE7cx+Fz ktBUZ5VgFDnA/QAlgWAIFmbTjLfLbmQf48gVxBgrBcRyKu8kLoQavLMniyVSv9bVIgGQ 0MLFHkFlkJ1k1hHVzFRTVtCtAQL1L8gdCnaYCIpYkHfglIAG28fzTUM/Mk0fVQodhKnZ 6mEGMv7q7h02Wp7oA663iBl3AgLTRz0wInaKCOKNpEfji0Wy8qSevK8EPzhH8xaVsmjV UvQP2aw6W7Dvd+G/hFbpE0SmA3lQ9R56FvjHhspZd49iimdiBkAodrkKzTvhDH+m5ex3 L1SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=thread-index:content-language:content-transfer-encoding :mime-version:message-id:date:subject:in-reply-to:references:cc:to :from:x-gm-message-state:from:to:cc:subject:date; bh=Jp5Ei23Ajmmo0CPWLiFIrCP7jET9y8hJo0ZDA/we70o=; b=ZgKnPZ8rr4KWXVhcKYlWJZMi2YKu8f28opNRBv19hBxS8ZPTLgt/3sB5ScA8YlEzcN RAqZTV3CeytlfIGgSUkDz+k41yUq0KIUTyBBYd5efIZOFwiYjXKpoVDP3QDFwHW+3k5U sh74BFpJIe0nRRiQJnW/+rkTbboSRH5rjW+EkyvjVuA0EeRtSrK8N9478tsh/QDXH94C KRksnbTxR7zTs6aKH+sUchEAKdBXVRVZQlvfWh5Cm+f/6Ui6LZOowvbCsgb/SPzAvw8y /c2EPUpvd78RghhUZuoJ3LQftuos8csWUIJRVBhKb6lpXsuABRHgWFHR7yZSja2O1eYI PGaw== X-Gm-Message-State: ACrzQf3pGu+ofMCrAQpbBMVNEk9H51fuQQ/zocst4PLVDL+wHjfR2DJU 2tkWn0WQb8+ZgdNYjmKB7E/WDrwZgpE= X-Google-Smtp-Source: AMsMyM6NlRPvlYSOVRf0h9EcFz8mh6B+m20CMsA9h0zrXka5SNxvp/JcEi8M3DRxrhzBLpXrMtdKVg== X-Received: by 2002:adf:ec03:0:b0:228:76bd:76fc with SMTP id x3-20020adfec03000000b0022876bd76fcmr7639488wrn.533.1663494972581; Sun, 18 Sep 2022 02:56:12 -0700 (PDT) Received: from DRIESPC ([2a02:a03f:8254:de01:a86b:4ba9:3f66:fe35]) by smtp.gmail.com with ESMTPSA id x6-20020adfffc6000000b0022abd7d57b1sm10093386wrs.115.2022.09.18.02.56.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 18 Sep 2022 02:56:12 -0700 (PDT) From: To: , "'Evilham'" Cc: References: <004701d8c9ea$63e10090$2ba301b0$@FreeBSD.org> <007d01d8c9eb$87a45120$96ecf360$@FreeBSD.org> <6b34c0b352f1e1a9d5787b948dc4262bcdc6@yggdrasil.evilham.com> <002a01d8ca6d$ff31cf10$fd956d30$@FreeBSD.org> In-Reply-To: <002a01d8ca6d$ff31cf10$fd956d30$@FreeBSD.org> Subject: RE: PPPoE (mpd5) IPv6 issues Date: Sun, 18 Sep 2022 11:56:12 +0200 Message-ID: <002701d8cb44$e2de3e20$a89aba60$@gmail.com> List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Content-Language: en-be Thread-Index: AQJZulkgwP+QrU8Xsk0/1jKwHaM6NgIDH3JKAUokN+wBPEgEB6y+xdXw X-Rspamd-Queue-Id: 4MVjqy2NkWz3wkV X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b="JJq1/GMb"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of driesm.michiels@gmail.com designates 2a00:1450:4864:20::432 as permitted sender) smtp.mailfrom=driesm.michiels@gmail.com X-Spamd-Result: default: False [-3.24 / 15.00]; NEURAL_HAM_LONG(-0.92)[-0.922]; NEURAL_HAM_SHORT(-0.86)[-0.857]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_MEDIUM(-0.46)[-0.458]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; MIME_GOOD(-0.10)[text/plain]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; FROM_NO_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::432:from]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; TAGGED_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N > -----Original Message----- > From: driesm@FreeBSD.org > Sent: Saturday, 17 September 2022 10:18 > To: 'Evilham' > Cc: freebsd-net@freebsd.org > Subject: RE: PPPoE (mpd5) IPv6 issues >=20 > > -----Original Message----- > > From: Evilham > > Sent: Friday, 16 September 2022 22:00 > > To: driesm@FreeBSD.org > > Cc: freebsd-net@freebsd.org > > Subject: Re: PPPoE (mpd5) IPv6 issues > > > > Hey, > > > > On dv., set. 16 2022, driesm@FreeBSD.org wrote: > > > > > Hmm this mail was not finished, sorry about that. > > > > > > I will include the link that fell-off, IPv6 PPPoE MSS incorrect > > > | Netgate Forum > > > > > > > > > > > > Any help or pointers are greatly appreciated! > > > > > > > > > > > > PS: I use IPFW as my firewall, if this has anything to do with it. = I > > > do have a =E2=80=9Creassemble all in=E2=80=9D rule at the top of = my ruleset > > > > > > > > > > > > From: driesm@FreeBSD.org > > > Sent: Friday, 16 September 2022 18:36 > > > To: freebsd-net@freebsd.org > > > Subject: PPPoE (mpd5) IPv6 issues > > > > > > > > > > > > Hi freebsd-net! > > > > > > > > > > > > After a lot of google and tinkering I have hit a brick wall trying > > > to set-up my new ISP which uses PPPoE. > > > > > > I=E2=80=99m at the point where IPv4 works fine 100% for all = websites. > > > But from the moment I turn on IPv6 all sites that prefer IPv6 stop > > > working, I have confirmed that routing table etc all look good. So > > > I=E2=80=99m more suspicious to a MTU / MSS issue. > > > > > > Although I cant find that much info about it. I have tcpmss fix > > > turned on in my mpd.conf > > > > > > > > > > > > There seems to be a few sites like google.com and youtube.com that > > > keep working. > > > > > > > > > > > > I have googled thoroughly but I was unable to find a fix for my > > > problem. The one close to my issue seems this issue from 2020: > > > > > > Some things to take into account are: MTU of the physical interface > > and ICMP blackholes, see below. > > > > For the MTU of the physical interface, take into account that if a > > VLAN and PCP (vlan(4)+ifconfig(8)) are required by the wholesale > > provider, those consume 4 bytes each and taking PPPoE's 8 bytes into > > account as well, your physical interface should have a higher MTU (4 > > bytes for VLAN + 4 bytes for PCP + 8 bytes for PPPoE =3D 16 bytes > > overhead on physical interface) than the connection (I'd guess one = of > > 1500, 1492 or 1484 depending on the wholesale provider and the ISP). > > > > > > Basically, from the router if the interface created by mpd5 has an = MTU > > of 1500, you should be able to: > > > > ping -6D -s 1452 ipv6.icmp.host > > > > With -6D being "do not fragment" > > And 1452 bytes being the payload =3D > > 1500 bytes - 40 bytes from IPv6 headers - 8 bytes for ICMP > > header > > > > > > And you **should** receive a "ping: sendmsg: Message too long" > > with: > > > > ping -6D -s 1453 ipv6.icmp.host > > > > > > When in doubt, try: > > ping -6D -s 1232 ipv6.icmp.host > > This should work since 1280 bytes is the minimum MTU for IPv6 and > > gradually go up to establish what the maximum MTU for your = connection > > is being. >=20 > Thanks a lot for the troubleshooting steps! >=20 > After trying this I identified the max ping size to be 1444 bytes. > 1445 bytes results in message too long. This was both the case on the = router > as one of the clients (windows), so consistency seems good here. >=20 > This was with ng0 configured on the standard 1492 bytes MTU. > All my other physical interfaces are on the standard 1500 MTU. >=20 > Now how do I translate this info to getting IPv6 to work reliably? > Which MTU should I tweak? LAN side or WAN side? >=20 > Dries Okay so after some more tinkering I did found a solution. I disabled tcpmssfix in mpd.conf, and moved it to IPFW with one rule: tcp-setmss 1432 tcp from any to any out Seems like 1492 - 40 -20 works for both IPv4 and IPv6 stacks. I'm not sure which value is being used by mpd5 when setting up = ng_tcpmss. >=20 > > > > > > If you don't receive the "Message too long" with way too large > > packets, there might be a bit of an ICMP blackhole on the ISP side = or > > on the route to the ipv6.icmp.host. > > > > This is usually not too tragic for IPv4 because routers do the > > fragmenting magic, but it is terrible for IPv6 since it is the = client > > that has to fragment, and for that it needs to receive the "Message > > too long" ICMP response. > > > > By using tcpdump on the router, convince yourself truly that it is = not > > *you* dropping all incoming ICMP! If you see the ICMP "message too > > long" packets arrive and they are not forwarded, the issue is on = your > > side, time to check the firewall. > > > > If you are 100% sure your side is right, it might be on the ISP = side; > > we actually had this issue until recently in our associative ISP and > > we have seen large commercial providers have issues with this as = well > > (IPv6 not widely adopted around here). > > > > A workaround was to announce locally a lower MTU for IPv6 with > > rtadvd(8). > > Over time that proved to be an issue on Windows hosts, which didn't > > seem to handle well a different MTU for IPv4 and IPv6, so I ended up > > announcing the lower MTU internally for both stacks temporarily = (using > > DHCP for v4). > > > > Hope that helps and wasn't too all over the place, this is memory > > talking here :-D. > > > > Cheers, > > -- > > Evilham