Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 18 Sep 2022 11:56:12 +0200
From:      <driesm.michiels@gmail.com>
To:        <driesm@FreeBSD.org>, "'Evilham'" <contact@evilham.com>
Cc:        <freebsd-net@freebsd.org>
Subject:   RE: PPPoE (mpd5) IPv6 issues
Message-ID:  <002701d8cb44$e2de3e20$a89aba60$@gmail.com>
In-Reply-To: <002a01d8ca6d$ff31cf10$fd956d30$@FreeBSD.org>
References:  <004701d8c9ea$63e10090$2ba301b0$@FreeBSD.org> <007d01d8c9eb$87a45120$96ecf360$@FreeBSD.org> <6b34c0b352f1e1a9d5787b948dc4262bcdc6@yggdrasil.evilham.com> <002a01d8ca6d$ff31cf10$fd956d30$@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help


> -----Original Message-----
> From: driesm@FreeBSD.org <driesm@FreeBSD.org>
> Sent: Saturday, 17 September 2022 10:18
> To: 'Evilham' <contact@evilham.com>
> Cc: freebsd-net@freebsd.org
> Subject: RE: PPPoE (mpd5) IPv6 issues
>=20
> > -----Original Message-----
> > From: Evilham <contact@evilham.com>
> > Sent: Friday, 16 September 2022 22:00
> > To: driesm@FreeBSD.org
> > Cc: freebsd-net@freebsd.org
> > Subject: Re: PPPoE (mpd5) IPv6 issues
> >
> > Hey,
> >
> > On dv., set. 16 2022, driesm@FreeBSD.org wrote:
> >
> > > Hmm this mail was not finished, sorry about that.
> > >
> > > I will include the link that fell-off, IPv6 PPPoE MSS incorrect
> > > | Netgate Forum
> > >
> > >
> > >
> > > Any help or pointers are greatly appreciated!
> > >
> > >
> > >
> > > PS: I use IPFW as my firewall, if this has anything to do with it. =
I
> > > do have a =E2=80=9Creassemble all in=E2=80=9D rule at the top of =
my ruleset
> > >
> > >
> > >
> > > From: driesm@FreeBSD.org <driesm@FreeBSD.org>
> > > Sent: Friday, 16 September 2022 18:36
> > > To: freebsd-net@freebsd.org
> > > Subject: PPPoE (mpd5) IPv6 issues
> > >
> > >
> > >
> > > Hi freebsd-net!
> > >
> > >
> > >
> > > After a lot of google and tinkering I have hit a brick wall trying
> > > to set-up my new ISP which uses PPPoE.
> > >
> > > I=E2=80=99m at the point where IPv4 works fine 100% for all =
websites.
> > > But from the moment I turn on IPv6 all sites that prefer IPv6 stop
> > > working, I have confirmed that routing table etc all look good. So
> > > I=E2=80=99m more suspicious to a MTU / MSS issue.
> > >
> > > Although I cant find that much info about it. I have tcpmss fix
> > > turned on in my mpd.conf
> > >
> > >
> > >
> > > There seems to be a few sites like google.com and youtube.com that
> > > keep working.
> > >
> > >
> > >
> > > I have googled thoroughly but I was unable to find a fix for my
> > > problem. The one close to my issue seems  this issue from 2020:
> >
> >
> > Some things to take into account are: MTU of the physical interface
> > and ICMP blackholes, see below.
> >
> > For the MTU of the physical interface, take into account that if a
> > VLAN and PCP (vlan(4)+ifconfig(8)) are required by the wholesale
> > provider, those consume 4 bytes each and taking PPPoE's 8 bytes into
> > account as well, your physical interface should have a higher MTU (4
> > bytes for VLAN + 4 bytes for PCP + 8 bytes for PPPoE =3D 16 bytes
> > overhead on physical interface) than the connection (I'd guess one =
of
> > 1500, 1492 or 1484 depending on the wholesale provider and the ISP).
> >
> >
> > Basically, from the router if the interface created by mpd5 has an =
MTU
> > of 1500, you should be able to:
> >
> > ping -6D -s 1452 ipv6.icmp.host
> >
> > With -6D being "do not fragment"
> > And 1452 bytes being the payload =3D
> >   1500 bytes - 40 bytes from IPv6 headers - 8 bytes for ICMP
> >   header
> >
> >
> > And you **should** receive a "ping: sendmsg: Message too long"
> > with:
> >
> > ping -6D -s 1453 ipv6.icmp.host
> >
> >
> > When in doubt, try:
> > ping -6D -s 1232 ipv6.icmp.host
> > This should work since 1280 bytes is the minimum MTU for IPv6 and
> > gradually go up to establish what the maximum MTU for your =
connection
> > is being.
>=20
> Thanks a lot for the troubleshooting steps!
>=20
> After trying this I identified the max ping size to be 1444 bytes.
> 1445 bytes results in message too long. This was both the case on the =
router
> as one of the clients (windows), so consistency seems good here.
>=20
> This was with ng0 configured on the standard 1492 bytes MTU.
> All my other physical interfaces are on the standard 1500 MTU.
>=20
> Now how do I translate this info to getting IPv6 to work reliably?
> Which MTU should I tweak?  LAN side or WAN side?
>=20
> Dries

Okay so after some more tinkering I did found a solution.
I disabled tcpmssfix in mpd.conf, and moved it to IPFW with one rule:
	tcp-setmss 1432 tcp from any to any out

Seems like 1492 - 40 -20 works for both IPv4 and IPv6 stacks.
I'm not sure which value is being used by mpd5 when setting up =
ng_tcpmss.

>=20
> >
> >
> > If you don't receive the "Message too long" with way too large
> > packets, there might be a bit of an ICMP blackhole on the ISP side =
or
> > on the route to the ipv6.icmp.host.
> >
> > This is usually not too tragic for IPv4 because routers do the
> > fragmenting magic, but it is terrible for IPv6 since it is the =
client
> > that has to fragment, and for that it needs to receive the "Message
> > too long" ICMP response.
> >
> > By using tcpdump on the router, convince yourself truly that it is =
not
> > *you* dropping all incoming ICMP! If you see the ICMP "message too
> > long" packets arrive and they are not forwarded, the issue is on =
your
> > side, time to check the firewall.
> >
> > If you are 100% sure your side is right, it might be on the ISP =
side;
> > we actually had this issue until recently in our associative ISP and
> > we have seen large commercial providers have issues with this as =
well
> > (IPv6 not widely adopted around here).
> >
> > A workaround was to announce locally a lower MTU for IPv6 with
> > rtadvd(8).
> > Over time that proved to be an issue on Windows hosts, which didn't
> > seem to handle well a different MTU for IPv4 and IPv6, so I ended up
> > announcing the lower MTU internally for both stacks temporarily =
(using
> > DHCP for v4).
> >
> > Hope that helps and wasn't too all over the place, this is memory
> > talking here :-D.
> >
> > Cheers,
> > --
> > Evilham





Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?002701d8cb44$e2de3e20$a89aba60$>