From owner-freebsd-net@FreeBSD.ORG Fri Mar 30 22:01:26 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B4BBD106566C; Fri, 30 Mar 2012 22:01:26 +0000 (UTC) (envelope-from hunter@comsys.com.ua) Received: from mail.ice-tech.com.ua (mail.ice-tech.com.ua [77.120.117.100]) by mx1.freebsd.org (Postfix) with ESMTP id 503548FC14; Fri, 30 Mar 2012 22:01:26 +0000 (UTC) Received: from 94.244.147.77.nash.net.ua ([94.244.147.77] helo=hunters-MacBook-Pro.local) by mail.ice-tech.com.ua with esmtpa (Exim 4.77 (FreeBSD)) (envelope-from ) id 1SDjsl-0003L4-2b; Sat, 31 Mar 2012 01:01:19 +0300 Message-ID: <4F762D11.1080604@comsys.com.ua> Date: Sat, 31 Mar 2012 01:00:49 +0300 From: Sergey Smitienko User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0) Gecko/20120327 Thunderbird/11.0.1 MIME-Version: 1.0 To: Andre Oppermann References: <4F7463CF.8010206@comsys.com.ua> <4F759DB3.2060706@freebsd.org> <4F75AF4D.1040203@comsys.com.ua> <4F75CD8A.2050509@freebsd.org> In-Reply-To: <4F75CD8A.2050509@freebsd.org> X-SA-Exim-Connect-IP: 94.244.147.77 X-SA-Exim-Mail-From: hunter@comsys.com.ua X-SA-Exim-Scanned: No (on mail.ice-tech.com.ua); SAEximRunCond expanded to false Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: FreeBSD 9.0 generates incorrect SEQ/ACK numbers under load X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Mar 2012 22:01:26 -0000 30.03.12 18:13, Andre Oppermann wrote: > On 30.03.2012 15:04, Sergey Smitienko wrote: >> Here you go, two sessions, one with win set in Syn/Ack packet and other >> with separate "windows open" Ack packet. > > Thanks for the tcpdumps. The window update issue seems to be separate > from the seq#ack# problem. No, it's not. You gave me an idea. I have pf running on the server. It's has basic ruleset. We have table with 4k+ networks of our usual visitors. pf rules looks like this: pass in quick from to port 80 keep state pass in quick from any to port 80 synproxy state. So, in case of synproxy pf anwers Syn packet with Syn/Ack without knowledge of window size, and then passes connection to the kernel tcp stack and generates "window open" Ack packet. I've replaced "synproxy state" with usual "keep state" in pf and I don't see any Syn/Ack packets with zero window size. >From the over side, I have 20Gb of tcpdump files with 10^8 packets recorded. I've wrote a simple parser, which can detect sessions with incorrect sec/ack numbers. Then I've checked all IP addresses with failed TCP sessions and non of them was from set. So, 100% of failed sessions was comming through pf synproxy state. Synproxy state also include modulate state function, which is basicky an addition of random number to seq/ack numbers. So, I think there is a case, then tcp comming from kernel is not properly modulated/demodulated by pf and this causes generation of incorrect seq/ack numbers. > Why do set the recvspace to the very low value of 8192? 8K is big enough for usual GET request or for POST with login or comment. -- Sergey Smitienko