From owner-freebsd-net@FreeBSD.ORG Thu May 14 22:29:33 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 280C9106564A for ; Thu, 14 May 2009 22:29:33 +0000 (UTC) (envelope-from oberman@es.net) Received: from mailgw.es.net (mail4.es.net [IPv6:2001:400:6000:6::2]) by mx1.freebsd.org (Postfix) with ESMTP id D138C8FC1C for ; Thu, 14 May 2009 22:29:32 +0000 (UTC) (envelope-from oberman@es.net) Received: from ptavv.es.net (ptavv.es.net [IPv6:2001:400:910::29]) by mailgw.es.net (8.14.3/8.14.3) with ESMTP id n4EMTVB4011495 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 14 May 2009 15:29:31 -0700 Received: from ptavv.es.net (ptavv.es.net [127.0.0.1]) by ptavv.es.net (Tachyon Server) with ESMTP id D71611CC0B; Thu, 14 May 2009 15:29:30 -0700 (PDT) To: sthaug@nethelp.no In-reply-to: Your message of "Fri, 15 May 2009 00:09:02 +0200." <20090515.000902.74658525.sthaug@nethelp.no> Date: Thu, 14 May 2009 15:29:30 -0700 From: "Kevin Oberman" Message-Id: <20090514222930.D71611CC0B@ptavv.es.net> Cc: freebsd-net@freebsd.org Subject: Re: IPv6 fragmentation weirdness X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 22:29:33 -0000 > Date: Fri, 15 May 2009 00:09:02 +0200 (CEST) > From: sthaug@nethelp.no > > > First, why is the kernel fragmenting this at all as it fits in the > > interface MTU? > > Good question, I definitely disagree with this behavior and would say > that it breaks POLA. But it's documented (see the ping6 -m option). > > > Can anyone fetch anything from ftp.funet.fi via IPv6? I suspect it is > > something in the path that is blocking my traffic, so others may not see > > this, but I think the root issues is the kernel fragmenting packets way > > below MTU size. > > I just picked up a copy of the 7.2 bootonly ISO image using IPv6. Slow > but usable. My path (from Oslo, Norway) is: > > sthaug@lab1% traceroute6 ftp.funet.fi > traceroute6 to ftp.funet.fi (2001:708:10:9::20:1) from 2001:8c0:8b00:1::2, 64 hops max, 12 byte packets > 1 ge-0-0-9-515.br1.fn3.no.catchbone.net 0.254 ms 4.917 ms 0.203 ms > 2 c10G-ge-5-1-0.cr2.osls.no.catchbone.net 0.485 ms 0.408 ms 0.399 ms > 3 c10G-xe-4-1-0.br1.osls.no.catchbone.net 0.364 ms 0.351 ms 0.361 ms > 4 2001:2000:3083:6::1 9.006 ms 8.848 ms 8.966 ms > 5 s-ipv6-b1-link.ipv6.telia.net 19.481 ms 19.590 ms 19.412 ms > 6 2001:2000:3080:d::2 110.907 ms 109.056 ms 119.495 ms > 7 helsinki0-rtr.funet.fi 116.305 ms 123.534 ms 119.472 ms > 8 csc0-x0000-helsinki0.ipv6.funet.fi 118.873 ms 117.439 ms 116.054 ms > 9 ftp.funet.fi 115.777 ms 116.087 ms 117.735 ms > > Note that the IPv6 transit from Telia is tunnelled, and the RTT is awful > compared to IPv4 (IPv4 RTT to ftp.funet.fi from the same box is around > 17 ms). > > Steinar Haug, Nethelp consulting, sthaug@nethelp.no > Thanks, Steinar. I just re-read the man page and I had misunderstood what it was saying. That still leave me baffled as to what is happening. My path is, as would be expected, very different. traceroute6 to ftp.funet.fi (2001:708:10:9::20:1) from 2001:400:0:40::200:101, 64 hops max, 12 byte packets 1 esnet.rt1.ams.nl.geant2.net (2001:798:29:10aa::9) 83.998 ms 115.099 ms 85.969 ms 2 so-7-0-0.rt2.cop.dk.geant2.net (2001:798:cc:1501:2201::1) 101.692 ms 96.955 ms 96.868 ms 3 nordunet-gw.rt2.cop.dk.geant2.net (2001:798:15:10aa::2) 179.931 ms 205.407 ms 195.268 ms 4 dk-uni.nordu.net (2001:948:0:f055::2) 210.468 ms se-fre.nordu.net (2001:948:0:f03f::1) 187.479 ms dk-uni.nordu.net (2001:948:0:f055::2) 190.578 ms 5 se-tug.nordu.net (2001:948:0:f049::2) 188.170 ms 213.538 ms se-tug.nordu.net (2001:948:0:f056::1) 183.273 ms 6 helsinki0-rtr.funet.fi (2001:948:0:f035::2) 188.114 ms 189.214 ms 192.192 ms 7 csc0-x0000-helsinki0.ipv6.funet.fi (2001:708:0:f000::1:2) 186.166 ms 190.181 ms 186.669 ms 8 ftp.funet.fi (2001:708:10:9::20:1) 186.251 ms 198.591 ms 205.987 ms This is exactly the same as my IPv4 path. Something along this path is silently refusing to pass a packet at the start of the transfer and that screams MTU. If I ping from a Juniper router, I can get 1482 byte packets through, so I suspect that there is a tunnel somewhere. But FreeBSD boxes die at the lower limit. Does the kernel fragmentation only affect ICMP or are TCP packet also fragmented at 1280 bytes? -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751