Date: Sat, 02 Feb 2013 00:09:11 +0100 From: Andre Oppermann <andre@freebsd.org> To: Kevin Day <kevin@your.org> Cc: freebsd-net@freebsd.org Subject: Re: Syncookies break with Windows 8 Message-ID: <510C4B17.4040509@freebsd.org> In-Reply-To: <E45CF5BB-39A2-45F0-B0A8-DD01D8299585@your.org> References: <CA61E725-8370-4ED2-BBA7-F6FAFF93A553@your.org> <510C4424.4030701@networx.ch> <E45CF5BB-39A2-45F0-B0A8-DD01D8299585@your.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 01.02.2013 23:54, Kevin Day wrote: > > On Feb 1, 2013, at 4:39 PM, Andre Oppermann <oppermann@networx.ch> wrote: >> >> This is not true. FreeBSD uses bits in the timestamp to encode all recognized TCP options >> including window scaling. >> > > Sorry, you are correct here. Reading through a half dozen TCP implementations in the last day > trying to figure out what was happening and it all turns into a blur. :) I can imagine... :) >>> This doesn't affect Linux because it uses timestamp options to stuff the client's wscale, so >>> it gets re-learned on the ACK. OpenBSD and OS X don't have syncookies. NetBSD seems to have >>> the same problem if it's new syncookie implementation gets turned on. >> >> This can't be because of the lack of timestamps. Linux must be encoding the scale in the ISS >> taking away bits from the cookie. >> >> I haven't looked into how Linux actually does it recently. >> > > In Linux, it appears if timestamps are not enabled, either because they're turned off by sysctl > or the server didn't see one from the client, it disables scaling. > > http://fxr.watson.org/fxr/source/net/ipv4/syncookies.c?v=linux-2.6#L235 > > wscale_ok defaults to 0. It's set to '1' only if the server saw_tstamp, sysctl_tcp_timestamps > isn't turned off, etc. OK. >>> Any thoughts? Was there a reason why we're forcing the use of wscale on syncookie >>> connections? >> >> We can change the behavior of syncookies in a couple of ways to deal with this problem: >> >> 1/ send syncookies only when the syncache overflows and set wscale to 0 in the SYN-ACK when >> timestamps are not active. >> > > That sounds similar to Linux, if I'm reading the code correctly. > >> At the moment we send syncookies on every SYN-ACK and bump the oldest entry from the syncache >> when it is full. That results in potentially every segment degrading to syncookie only. The >> default values are insufficient for such high loads. >> >> In general at 10,000 connections per second you should significantly increase the size of your >> syncache to 3 * conn/sec at least. >> >> In the loader you can set these tunables: >> >> net.inet.tcp.syncache.hashsize = 2048 net.inet.tcp.syncache.bucketlimit = 32 >> net.inet.tcp.syncache.cachelimit = 65536 >> >> These settings are a bit more complicated than they should be. >> > > We're doing similar. During a DDoS we tried changing to syncookies_only and somehow that got left > that way, which is how we ran into this problem more reliably. Turning that off, and using higher > syncache sizes helped hide this greatly, but it's probably still happening here and elsewhere > occasionally. I'm working on a solution. Have to make sure that the chance to crack a reduced cookie during its 30 seconds lifetime isn't too high. That means involving our resident crypto experts for verification. -- Andre
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?510C4B17.4040509>