Date: Fri, 1 Feb 2013 16:54:43 -0600 From: Kevin Day <kevin@your.org> To: Andre Oppermann <oppermann@networx.ch> Cc: freebsd-net@freebsd.org Subject: Re: Syncookies break with Windows 8 Message-ID: <E45CF5BB-39A2-45F0-B0A8-DD01D8299585@your.org> In-Reply-To: <510C4424.4030701@networx.ch> References: <CA61E725-8370-4ED2-BBA7-F6FAFF93A553@your.org> <510C4424.4030701@networx.ch>
next in thread | previous in thread | raw e-mail | index | archive | help
On Feb 1, 2013, at 4:39 PM, Andre Oppermann <oppermann@networx.ch> = wrote: >=20 > This is not true. FreeBSD uses bits in the timestamp to encode all > recognized TCP options including window scaling. >=20 Sorry, you are correct here. Reading through a half dozen TCP = implementations in the last day trying to figure out what was happening = and it all turns into a blur. :) >> This doesn't affect Linux because it uses timestamp options to stuff = the client's wscale, so it >> gets re-learned on the ACK. OpenBSD and OS X don't have syncookies. = NetBSD seems to have the same >> problem if it's new syncookie implementation gets turned on. >=20 > This can't be because of the lack of timestamps. Linux must be > encoding the scale in the ISS taking away bits from the cookie. >=20 > I haven't looked into how Linux actually does it recently. >=20 In Linux, it appears if timestamps are not enabled, either because = they're turned off by sysctl or the server didn't see one from the = client, it disables scaling. http://fxr.watson.org/fxr/source/net/ipv4/syncookies.c?v=3Dlinux-2.6#L235 wscale_ok defaults to 0. It's set to '1' only if the server saw_tstamp, = sysctl_tcp_timestamps isn't turned off, etc.=20 >> Any thoughts? Was there a reason why we're forcing the use of wscale = on syncookie connections? >=20 > We can change the behavior of syncookies in a couple of ways to > deal with this problem: >=20 > 1/ send syncookies only when the syncache overflows and set wscale > to 0 in the SYN-ACK when timestamps are not active. >=20 That sounds similar to Linux, if I'm reading the code correctly. > At the moment we send syncookies on every SYN-ACK and bump the oldest > entry from the syncache when it is full. That results in potentially > every segment degrading to syncookie only. The default values are > insufficient for such high loads. >=20 > In general at 10,000 connections per second you should significantly > increase the size of your syncache to 3 * conn/sec at least. >=20 > In the loader you can set these tunables: >=20 > net.inet.tcp.syncache.hashsize =3D 2048 > net.inet.tcp.syncache.bucketlimit =3D 32 > net.inet.tcp.syncache.cachelimit =3D 65536 >=20 > These settings are a bit more complicated than they should be. >=20 We're doing similar. During a DDoS we tried changing to syncookies_only = and somehow that got left that way, which is how we ran into this = problem more reliably. Turning that off, and using higher syncache sizes = helped hide this greatly, but it's probably still happening here and = elsewhere occasionally. -- Kevin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E45CF5BB-39A2-45F0-B0A8-DD01D8299585>