From owner-freebsd-net@FreeBSD.ORG Fri Feb 1 23:09:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C07966F6 for ; Fri, 1 Feb 2013 23:09:19 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 1349A31F for ; Fri, 1 Feb 2013 23:09:18 +0000 (UTC) Received: (qmail 49823 invoked from network); 2 Feb 2013 00:28:53 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 2 Feb 2013 00:28:53 -0000 Message-ID: <510C4B17.4040509@freebsd.org> Date: Sat, 02 Feb 2013 00:09:11 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Kevin Day Subject: Re: Syncookies break with Windows 8 References: <510C4424.4030701@networx.ch> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Feb 2013 23:09:19 -0000 On 01.02.2013 23:54, Kevin Day wrote: > > On Feb 1, 2013, at 4:39 PM, Andre Oppermann wrote: >> >> This is not true. FreeBSD uses bits in the timestamp to encode all recognized TCP options >> including window scaling. >> > > Sorry, you are correct here. Reading through a half dozen TCP implementations in the last day > trying to figure out what was happening and it all turns into a blur. :) I can imagine... :) >>> This doesn't affect Linux because it uses timestamp options to stuff the client's wscale, so >>> it gets re-learned on the ACK. OpenBSD and OS X don't have syncookies. NetBSD seems to have >>> the same problem if it's new syncookie implementation gets turned on. >> >> This can't be because of the lack of timestamps. Linux must be encoding the scale in the ISS >> taking away bits from the cookie. >> >> I haven't looked into how Linux actually does it recently. >> > > In Linux, it appears if timestamps are not enabled, either because they're turned off by sysctl > or the server didn't see one from the client, it disables scaling. > > http://fxr.watson.org/fxr/source/net/ipv4/syncookies.c?v=linux-2.6#L235 > > wscale_ok defaults to 0. It's set to '1' only if the server saw_tstamp, sysctl_tcp_timestamps > isn't turned off, etc. OK. >>> Any thoughts? Was there a reason why we're forcing the use of wscale on syncookie >>> connections? >> >> We can change the behavior of syncookies in a couple of ways to deal with this problem: >> >> 1/ send syncookies only when the syncache overflows and set wscale to 0 in the SYN-ACK when >> timestamps are not active. >> > > That sounds similar to Linux, if I'm reading the code correctly. > >> At the moment we send syncookies on every SYN-ACK and bump the oldest entry from the syncache >> when it is full. That results in potentially every segment degrading to syncookie only. The >> default values are insufficient for such high loads. >> >> In general at 10,000 connections per second you should significantly increase the size of your >> syncache to 3 * conn/sec at least. >> >> In the loader you can set these tunables: >> >> net.inet.tcp.syncache.hashsize = 2048 net.inet.tcp.syncache.bucketlimit = 32 >> net.inet.tcp.syncache.cachelimit = 65536 >> >> These settings are a bit more complicated than they should be. >> > > We're doing similar. During a DDoS we tried changing to syncookies_only and somehow that got left > that way, which is how we ran into this problem more reliably. Turning that off, and using higher > syncache sizes helped hide this greatly, but it's probably still happening here and elsewhere > occasionally. I'm working on a solution. Have to make sure that the chance to crack a reduced cookie during its 30 seconds lifetime isn't too high. That means involving our resident crypto experts for verification. -- Andre