Date: Wed, 4 Jun 2025 21:06:39 +0200 From: Michael Tuexen <michael.tuexen@lurchi.franken.de> To: Christos Chatzaras <chris@cretaforce.gr> Cc: questions@freebsd.org, freebsd-net <freebsd-net@freebsd.org> Subject: Re: Problem with net.inet.tcp.path_mtu_discovery=1 Message-ID: <C36F3F3E-F6B2-47B7-BED7-CEE4DAF11354@lurchi.franken.de> In-Reply-To: <D9DE01A2-96DF-4804-875C-2424BEF733F3@cretaforce.gr> References: <9728060D-2C02-426B-BACE-F2D2F651A62F@cretaforce.gr> <bf557c42-625f-4b8a-b5df-7a45c84e40ee@app.fastmail.com> <D9DE01A2-96DF-4804-875C-2424BEF733F3@cretaforce.gr>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 4. Jun 2025, at 19:29, Christos Chatzaras <chris@cretaforce.gr> = wrote: >=20 >=20 >=20 >> On 4 Jun 2025, at 19:36, Dave Cottlehuber <dch@skunkwerks.at> wrote: >>=20 >> On Wed, 4 Jun 2025, at 16:36, Christos Chatzaras wrote: >>> Hello, >>>=20 >>> I manage some servers hosting websites. >>=20 >> What does tcpdump/wireshark show for traffic, particularly icmp? = Wireshark is very helpful in explaining some issues. >>=20 >> What is the actual MTU on the working net vs the failing one? >>=20 >> Is there a local MTU where the failing websites start working again? >>=20 >> see ping(8) and use -v -D -s =E2=80=A6. together to find a working = MTU and cross check with tcpdump to find where things seem to break. >>=20 >> On a recent cloud environment I needed to add =E2=80=98 set = reassemble yes no-df=E2=80=99 to my pf.conf to address MTU issues = between VNET jails and the internet. >>=20 >> Happy hunting >> Dave >>=20 >=20 > First, I reverted the server settings to their defaults: > sysctl net.inet.tcp.path_mtu_discovery=3D1 > sysctl net.inet.tcp.pmtud_blackhole_detection=3D0 >=20 > Next, I set the MTU on my local computer to 1460 and everything worked = as expected: > tcpdump: listening on en0, link-type EN10MB (Ethernet), snapshot = length 524288 bytes > 20:15:05.651375 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto = TCP (6), length 64) > 192.168.2.18.65322 > 94.130.217.87.443: Flags [S], cksum 0x293e = (correct), seq 3503095669, win 65535, options [mss 1420,nop,wscale = 6,nop,nop,TS val 639376397 ecr 0,sackOK,eol], length 0 > 20:15:05.705913 IP (tos 0x0, ttl 54, id 0, offset 0, flags [DF], proto = TCP (6), length 60) > 94.130.217.87.443 > 192.168.2.18.65322: Flags [S.], cksum 0x9c22 = (correct), seq 3647364942, ack 3503095670, win 65535, options [mss = 1460,nop,wscale 6,sackOK,TS val 1782053626 ecr 639376397], length 0 >=20 > However, when I set my local computer=E2=80=99s MTU back to 1500 (the = default), the issue reappeared: > tcpdump: listening on en0, link-type EN10MB (Ethernet), snapshot = length 524288 bytes > 20:17:45.662993 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto = TCP (6), length 64) > 192.168.2.18.65333 > 94.130.217.87.443: Flags [S], cksum 0x4a07 = (correct), seq 3674289142, win 65535, options [mss 1460,nop,wscale = 6,nop,nop,TS val 681359835 ecr 0,sackOK,eol], length 0 > 20:17:45.726988 IP (tos 0x0, ttl 54, id 0, offset 0, flags [DF], proto = TCP (6), length 60) > 94.130.217.87.443 > 192.168.2.18.65333: Flags [S.], cksum 0x9b1d = (correct), seq 1443843488, ack 3674289143, win 65535, options [mss = 1460,nop,wscale 6,sackOK,TS val 2890559459 ecr 681359835], length 0 >=20 > So, with local computer MTU 1460, everything works, but with MTU 1500, = the problem persists. The difference is that you announce a smaller MSS in SYN segment you sent. This means that the peer can only send you smaller TCP segments. So there seems to be a problem if the peer sends too large TCP segments. That means that the peer must do PMTUD or TCP blackhole detection, not the local node. Best regards Michael
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C36F3F3E-F6B2-47B7-BED7-CEE4DAF11354>