From owner-freebsd-bugs@FreeBSD.ORG Fri Mar 30 22:10:13 2012 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9AEF7106566C for ; Fri, 30 Mar 2012 22:10:13 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7C0BC8FC14 for ; Fri, 30 Mar 2012 22:10:13 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q2UMADQe035771 for ; Fri, 30 Mar 2012 22:10:13 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q2UMADM1035770; Fri, 30 Mar 2012 22:10:13 GMT (envelope-from gnats) Date: Fri, 30 Mar 2012 22:10:13 GMT Message-Id: <201203302210.q2UMADM1035770@freefall.freebsd.org> To: freebsd-bugs@FreeBSD.org From: Sergey Smitienko Cc: Subject: Re: kern/166501: FreeBSD 9.0 generates incorrect SEQ/ACK numbers under load X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Sergey Smitienko List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Mar 2012 22:10:13 -0000 The following reply was made to PR kern/166501; it has been noted by GNATS. From: Sergey Smitienko To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/166501: FreeBSD 9.0 generates incorrect SEQ/ACK numbers under load Date: Sat, 31 Mar 2012 01:07:40 +0300 This is a multi-part message in MIME format. --------------080707000708040403050503 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit I have pf running on the server. It's has basic ruleset. I have table with 4k+ networks of our web site usual visitors. pf rules looks like this: pass in quick from to port 80 keep state pass in quick from any to port 80 synproxy state. In the tcpdump in report you can see Syn/Ack packet with 0 window size. And then one more packet with 8K tcp window. This packet is generated by pf synproxy. Pf anwers Syn packet with Syn/Ack without knowledge of window size, and then passes connection to the kernel tcp stack and generates "window open" Ack packet. From the over side, I have 20Gb of tcpdump files with 10^8 packets recorded. I've wrote a simple parser, which can detect sessions with incorrect seq/ack numbers. Then I've checked all IP addresses with failed TCP sessions and non of them was from set. So, 100% of failed sessions was comming through pf synproxy state. Synproxy state includes modulate state function, which is basicky an addition of strong random number to seq/ack numbers. So, I think there is a case, then tcp comming from kernel is not properly modulated/demodulated by pf and this causes generation of incorrect seq/ack numbers. -- Sergey Smitienko --------------080707000708040403050503 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit 
I have pf running on the server.  It's has basic ruleset.
 I have table <trusted> with 4k+ networks of our web site usual visitors.
 pf rules looks like this:
 
 pass in quick from <trusted> to <me> port 80 keep state
 pass in quick from any to <me> port 80 synproxy state.
 
 In the tcpdump in report you can see Syn/Ack packet with 0 window size.
 And then one more packet with 8K tcp window.
 
 This packet is generated by pf synproxy. Pf anwers Syn packet with Syn/Ack 
 without knowledge of window size, and then passes connection to the kernel 
 tcp stack and generates "window open" Ack packet.
 
 From the over side, I have 20Gb of tcpdump files with 10^8 packets recorded.
 I've wrote a simple parser, which can detect sessions with incorrect
 seq/ack numbers. Then I've checked all IP addresses with failed TCP sessions 
 and non of them was from <trusted> set.
 So, 100% of failed sessions was comming through pf synproxy state.
 Synproxy state includes modulate state function, which is basicky an addition 
 of strong random number to seq/ack numbers.
 So, I think there is a case, then tcp comming from kernel is not
 properly modulated/demodulated
 by pf and this causes generation of incorrect seq/ack numbers.
 
 -- 
 Sergey Smitienko
--------------080707000708040403050503--