From nobody Fri Sep 16 20:00:23 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MTlLF5G51z4d3VC for ; Fri, 16 Sep 2022 20:00:37 +0000 (UTC) (envelope-from contact@evilham.com) Received: from yggdrasil.evilham.com (yggdrasil.evilham.com [IPv6:2a02:2770::216:3eff:fee1:cf9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4MTlLD5Bgjz3KG3; Fri, 16 Sep 2022 20:00:36 +0000 (UTC) (envelope-from contact@evilham.com) From: Evilham DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=evilham.com; s=mail; t=1663358427; bh=j29aDadxHJqGB6q4GtC+ukzShfRHQCP4pFqRjVIi2xs=; h=From:To:Cc:Subject:References:In-reply-to:Date; b=NJ8b1V1zTvc8TNq4APciLz+O+751ajTz2UA1wClLtdL5WW/v7RRn/xR2vcWpFbjqP ghc2shqdvhAFBMfoJp3YeBZ57A/FljavKCZ/l/aQI3MkSOb1ZOOrXQjTqUoR6NcEOq MeaAw0tK5wDi5wTBFVxsRa1LBlTTEwTM3zX+Souc= To: driesm@FreeBSD.org Cc: freebsd-net@freebsd.org Subject: Re: PPPoE (mpd5) IPv6 issues References: <004701d8c9ea$63e10090$2ba301b0$@FreeBSD.org> <007d01d8c9eb$87a45120$96ecf360$@FreeBSD.org> In-reply-to: <007d01d8c9eb$87a45120$96ecf360$@FreeBSD.org> Date: Fri, 16 Sep 2022 22:00:23 +0200 Message-ID: <6b34c0b352f1e1a9d5787b948dc4262bcdc6@yggdrasil.evilham.com> List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4MTlLD5Bgjz3KG3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=evilham.com header.s=mail header.b=NJ8b1V1z; dmarc=pass (policy=quarantine) header.from=evilham.com; spf=pass (mx1.freebsd.org: domain of contact@evilham.com designates 2a02:2770::216:3eff:fee1:cf9 as permitted sender) smtp.mailfrom=contact@evilham.com X-Spamd-Result: default: False [-3.78 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-0.996]; NEURAL_HAM_LONG(-0.91)[-0.913]; NEURAL_HAM_MEDIUM(-0.87)[-0.871]; DMARC_POLICY_ALLOW(-0.50)[evilham.com,quarantine]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[evilham.com:s=mail]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; RCPT_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROMTLD(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[evilham.com:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; ASN(0.00)[asn:196752, ipnet:2a02:2770::/32, country:NL] X-ThisMailContainsUnwantedMimeParts: N Hey, On dv., set. 16 2022, driesm@FreeBSD.org wrote: > Hmm this mail was not finished, sorry about that. > > I will include the link that fell-off, IPv6 PPPoE MSS incorrect=20 > | Netgate Forum > > > > Any help or pointers are greatly appreciated! > > > > PS: I use IPFW as my firewall, if this has anything to do with=20 > it. I do have a =E2=80=9Creassemble all in=E2=80=9D rule at the top of my= =20 > ruleset > > > > From: driesm@FreeBSD.org > Sent: Friday, 16 September 2022 18:36 > To: freebsd-net@freebsd.org > Subject: PPPoE (mpd5) IPv6 issues > > > > Hi freebsd-net! > > > > After a lot of google and tinkering I have hit a brick wall=20 > trying to set-up my new ISP which uses PPPoE. > > I=E2=80=99m at the point where IPv4 works fine 100% for all websites.=20 > But from the moment I turn on IPv6 all sites that prefer IPv6=20 > stop working, I > have confirmed that routing table etc all look good. So I=E2=80=99m more= =20 > suspicious to a MTU / MSS issue. > > Although I cant find that much info about it. I have tcpmss fix=20 > turned on in my mpd.conf > > > > There seems to be a few sites like google.com and youtube.com=20 > that keep working. > > > > I have googled thoroughly but I was unable to find a fix for my=20 > problem. The one close to my issue seems this issue from 2020: Some things to take into account are: MTU of the physical=20 interface and ICMP blackholes, see below. For the MTU of the physical interface, take into account that if a=20 VLAN and PCP (vlan(4)+ifconfig(8)) are required by the wholesale=20 provider, those consume 4 bytes each and taking PPPoE's 8 bytes=20 into account as well, your physical interface should have a higher=20 MTU (4 bytes for VLAN + 4 bytes for PCP + 8 bytes for PPPoE =3D 16=20 bytes overhead on physical interface) than the connection (I'd=20 guess one of 1500, 1492 or 1484 depending on the wholesale=20 provider and the ISP). Basically, from the router if the interface created by mpd5 has an=20 MTU of 1500, you should be able to: ping -6D -s 1452 ipv6.icmp.host With -6D being "do not fragment" And 1452 bytes being the payload =3D 1500 bytes - 40 bytes from IPv6 headers - 8 bytes for ICMP=20 header And you **should** receive a "ping: sendmsg: Message too long"=20 with: ping -6D -s 1453 ipv6.icmp.host When in doubt, try: ping -6D -s 1232 ipv6.icmp.host This should work since 1280 bytes is the minimum MTU for IPv6 and=20 gradually go up to establish what the maximum MTU for your=20 connection is being. If you don't receive the "Message too long" with way too large=20 packets, there might be a bit of an ICMP blackhole on the ISP side=20 or on the route to the ipv6.icmp.host. This is usually not too tragic for IPv4 because routers do the=20 fragmenting magic, but it is terrible for IPv6 since it is the=20 client that has to fragment, and for that it needs to receive the=20 "Message too long" ICMP response. By using tcpdump on the router, convince yourself truly that it is=20 not *you* dropping all incoming ICMP! If you see the ICMP "message=20 too long" packets arrive and they are not forwarded, the issue is=20 on your side, time to check the firewall. If you are 100% sure your side is right, it might be on the ISP=20 side; we actually had this issue until recently in our associative=20 ISP and we have seen large commercial providers have issues with=20 this as well (IPv6 not widely adopted around here). A workaround was to announce locally a lower MTU for IPv6 with=20 rtadvd(8). Over time that proved to be an issue on Windows hosts, which=20 didn't seem to handle well a different MTU for IPv4 and IPv6, so I=20 ended up announcing the lower MTU internally for both stacks=20 temporarily (using DHCP for v4). Hope that helps and wasn't too all over the place, this is memory=20 talking here :-D. Cheers, -- Evilham