Date: Thu, 1 May 2014 14:52:47 +0200 From: Hellmuth Michaelis <hm@hellmuth-michaelis.de> To: freebsd-net@FreeBSD.org Subject: pf and ipv6 pmtu discovery not working Message-ID: <5B8E8711-24A5-46AD-893A-12C087117672@hellmuth-michaelis.de>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Hi, for some time now i'm trying to nail down a phenomenon of hanging ipv6 transfers, it looks to me like there is a problem of getting ipv6 path mtu discovery right when using FreeBSD and pf. My setup is an outside router (pisix, an rpi running freebsd connecting to sixxs) connected via a firewall to a "freebsd server" (ernie). The "outside router" has an MTU of 1280 bytes, everything after it to the inside has an MTU of 1500 bytes. On the server i have pf running. Its a 10-stable as of 28th april amd64 box. Now when the "freebsd server" sends a packet via the "outside router" which is larger than 1280 Bytes to the internet, the "outside router" sends an ICMP packet too big to the server, which in turn lowers its MTU as shown in this tcpdump (which was taken with "set skip on fxp0" in /etc/pf.conf, where fxp0 is the interface where this takes place): 10:45:16.071634 IP6 ernie.30711 > ber01s09-in-x18.1e100.net.https: Flags [P.], seq 518:1849, ack 167, win 2053, options [nop,nop,TS val 915142 ecr 2428082038], length 1331 10:45:16.073732 IP6 pisix > ernie: ICMP6, packet too big, mtu 1280, length 1240 10:45:16.073989 IP6 ernie.30711 > ber01s09-in-x18.1e100.net.https: Flags [.], seq 518:1726, ack 167, win 2053, options [nop,nop,TS val 915144 ecr 2428082038], length 1208 10:45:16.074119 IP6 ernie.30711 > ber01s09-in-x18.1e100.net.https: Flags [P.], seq 1726:1849, ack 167, win 2053, options [nop,nop,TS val 915144 ecr 2428082038], length 123 When the "set skip on fxp0" line in pf.conf is removed, the above described pmtu scenario does not work anymore: 14:02:11.106566 IP6 ernie.17576 > ber01s09-in-x18.1e100.net.https: Flags [.], seq 1890:3288, ack 13944, win 2053, options [nop,nop,TS val 12730177 ecr 2439884647], length 1398 14:02:11.109301 IP6 pisix > ernie: ICMP6, packet too big, mtu 1280, length 1240 14:02:11.788529 IP6 ernie.17576 > ber01s09-in-x18.1e100.net.https: Flags [.], seq 1890:3288, ack 13944, win 2053, options [nop,nop,TS val 12730859 ecr 2439884647], length 1398 14:02:11.791271 IP6 pisix > ernie: ICMP6, packet too big, mtu 1280, length 1240 14:02:12.965534 IP6 ernie.17576 > ber01s09-in-x18.1e100.net.https: Flags [.], seq 1890:3288, ack 13944, win 2053, options [nop,nop,TS val 12732036 ecr 2439884647], length 1398 14:02:12.968302 IP6 pisix > ernie: ICMP6, packet too big, mtu 1280, length 1240 14:02:15.093370 IP6 ernie.17576 > ber01s09-in-x18.1e100.net.https: Flags [.], seq 1890:3288, ack 13944, win 2053, options [nop,nop,TS val 12734164 ecr 2439884647], length 1398 I found a similar thread of a FreeBSD bug in this area and tried to disable net.inet.tcp.rfc1323, this makes no difference. All sorts of rearranging rules or stripping ruled down to a bare minimum in pf.conf made no difference. Either disable pf at all or add "set skip on fxp0" make pmtu work. Finally i added some printf's in /usr/src/sys/netinet6/icmp6.c, but when pf is enabled, the icmp's don't even get there, so they must be filtered out earlier. At this point i found out about pfctl -x loud. For the pf enabled state, the console said for a similar situation: pf: BAD ICMP 2:0 2001:xxxx:yyyy:100::253 -> 2001:xxxx:yyyy:200::1 state: TCP out wire: 2a00:1450:4008:801::101f[443] 2001:xxxx:yyyy:200::1[58987] stack: - [lo=2065898190 high=2065912306 win=2053 modulator=0 wscale=7] [lo=2904051605 high=2904314389 win=240 modulator=0 wscale=6] 4:4 seq=1059571332 Then i made the following modification to the file /usr/src/sys/netpfil/pf/pf.c: 4665c4665 < if (!((*state)->state_flags & PFSTATE_SLOPPY) && --- > if ((icmptype != ICMP6_PACKET_TOO_BIG) && (!((*state)->state_flags & PFSTATE_SLOPPY) && 4667c4667 < !SEQ_GEQ(seq, src->seqlo - (dst->max_win << dws)))) { --- > !SEQ_GEQ(seq, src->seqlo - (dst->max_win << dws))))) { and voila, pmtu works with pf enabled. It seems the behaviour must have somthing to do with the if statement if (!((*state)->state_flags & PFSTATE_SLOPPY) && (!SEQ_GEQ(src->seqhi, seq) || !SEQ_GEQ(seq, src->seqlo - (dst->max_win << dws)))) in /usr/src/sys/netpfil/pf/pf.c Setting "keep state (sloppy)" in a line in pf.conf letting v6 icmp’s thru made no difference. Same with adding „no state“. I'm a bit stuck here right now. Is there perhaps someone with a bit more insight in this type of pf-mechanics who might want to have a look at this ? Hellmuth -- Hellmuth Michaelis Hallstraße 20 25462 Rellingen Mobil: +49-160-9645 5696 Telefon: +49-4101-85299-20 Telefax: +49-4101-85299-21 web: www.hellmuth-michaelis.de mail: hm@hellmuth-michaelis.de [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJTYkOfAAoJEDc16e0fOQ8BGQAH/0yRpq+W4Fq08ZRnN0+YkO5Y b+03XkW80DpOg5rYDPkPJ8TgkN7EmpUM/n8VeJhnndc8L+gtZpaBOx6s/7qnYI1o vzuEog4e9fI0abGdME6WEocGExRUQMLtmGwj//rBAAtg7QbNQMqSmm1LFDbsXa+x vtMc3yYqp45CI5GfWsAmdVdNYfCiTqLeIqL+evJB4/2U9el1yuKU1L3yKTveLAns 0un3cI0iCO0qVAIOTduQQwSxNxRrcPHRaZn8ctbDI0x0CYc5/rW8z1XS3QzQ6LMR 7YIPirc23HlWS5Hwnv+0VKE3W5xmVkcawHB62MlHWYCTH5UdhabfpMmL5fmrino= =5v0D -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5B8E8711-24A5-46AD-893A-12C087117672>