From owner-freebsd-net@FreeBSD.ORG Tue Jul 10 08:29:12 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D4B4D106566C for ; Tue, 10 Jul 2012 08:29:12 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (106-30.3-213.fix.bluewin.ch [213.3.30.106]) by mx1.freebsd.org (Postfix) with ESMTP id 418308FC08 for ; Tue, 10 Jul 2012 08:29:05 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost.benzedrine.cx [127.0.0.1]) by insomnia.benzedrine.cx (8.14.1/8.13.4) with ESMTP id q6A8MX7O008318 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Tue, 10 Jul 2012 10:22:33 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.1/8.12.10/Submit) id q6A8MWNY017764; Tue, 10 Jul 2012 10:22:32 +0200 (MEST) Date: Tue, 10 Jul 2012 10:22:32 +0200 From: Daniel Hartmeier To: Eugene Grosbein Message-ID: <20120710082232.GA9145@insomnia.benzedrine.cx> References: <4FFAA917.8020700@rdtc.ru> <4FFABCF1.6070403@rdtc.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4FFABCF1.6070403@rdtc.ru> User-Agent: Mutt/1.5.12-2006-07-14 Cc: "net@freebsd.org" Subject: Re: ip_reass() fails to reassemble fragmented out-of-order traffic X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jul 2012 08:29:12 -0000 On Mon, Jul 09, 2012 at 06:13:53PM +0700, Eugene Grosbein wrote: > tcpdump shows no errors in fragment's checksums. > Still, they were not reassembled. I fed your two fragments (with libdnet) to ip_input.c ip_reass() with debug printfs added, it reassembles them fine, even when in reversed order: Jul 10 09:06:03 testA kernel: ip_reass() called ip_id 0x87EE Jul 10 09:06:03 testA kernel: ip_reass: no queue Jul 10 09:06:03 testA kernel: ip_reass: IP_MF clear Jul 10 09:06:03 testA kernel: ip_reass: created queue Jul 10 09:06:03 testA kernel: ip_reass: returning NULL Jul 10 09:06:03 testA kernel: ip_reass() called ip_id 0x87EE Jul 10 09:06:03 testA kernel: ip_reass: found queue Jul 10 09:06:03 testA kernel: ip_reass: IP_MF set Jul 10 09:06:03 testA kernel: ip_reass: no previous segment Jul 10 09:06:03 testA kernel: ip_reass: complete? ip_off 0, next 0 Jul 10 09:06:03 testA kernel: ip_reass: complete? ip_off 1480, next 1480 Jul 10 09:06:03 testA kernel: ip_reass: complete! Jul 10 09:06:03 testA kernel: ip_reass: concated Jul 10 09:06:03 testA kernel: ip_reass: m_fixhdr() 1500 -> 1501 Jul 10 09:06:03 testA kernel: ip_reass: returning non-NULL, m_len 1500, ip_len 1501 Jul 10 09:06:03 testA kernel: mbuf: 0xc4bb3c00 len: 1500, next: 0xc4fdb000, 803, Jul 10 09:06:03 testA kernel: mbuf: 0xc4fdb000 len: 1, next: 0, 3 Jul 10 09:06:03 testA kernel: ip_input: ip_reass returned m_len 1500, ip_len 1501, hlen 20 And I see the ping with tcpdump on gif0 testA# ifconfig gif0 create testA# ifconfig gif0 tunnel outer-me outer-peer testA# ifconfig gif0 inet 192.168.50.10 192.168.50.9 netmask 255.255.255.0 up testA# tcpdump -s 1600 -nvvvi gif0 tcpdump: listening on gif0, link-type NULL (BSD loopback), capture size 1600 bytes 10:12:14.028436 IP (tos 0x0, ttl 128, id 61059, offset 0, flags [DF], proto ICMP (1), length 1481) 192.168.50.10 > 192.168.50.9: ICMP echo request, id 39570, seq 0, length 1461 ^C netstat -s diff: - 4 fragments received + 6 fragments received - 2 packets reassembled ok + 3 packets reassembled ok Input histogram: - echo: 1 - 1 message response generated + echo: 2 + 2 message responses generated Output histogram: - echo reply: 1 + echo reply: 2 Are any netstat -s counters increasing that show an error (fragments dropped, smaller than minimum)? Do you have lots of fragments in flight at the same time, hitting the fragment cache limit? testA# sysctl -a | grep maxfrag net.inet.ip.maxfragpackets: 800 net.inet.ip.maxfragsperpacket: 16 Are you running any pfil consumers (ipfilter, ipfw, pf), maybe unintentionally (with empty ruleset)? If so, can you try disabling them? Daniel