Date: Sun, 11 Feb 2001 05:30:16 -0800 From: Peter Wemm <peter@netplex.com.au> To: Alfred Perlstein <bright@wintelcom.net> Cc: net@FreeBSD.ORG, jlemon@FreeBSD.ORG Subject: Re: somaxconn and foot removal Message-ID: <200102111330.f1BDUGU36650@mobile.wemm.org> In-Reply-To: <20010211015949.K3274@fw.wintelcom.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Alfred Perlstein wrote: > * Alfred Perlstein <bright@wintelcom.net> [010211 01:55] wrote: > > The sysctl for somaxconn is an int, however the queue limits in the > > socket structures are 'short' this can cause some bad behavior if > > one sets somaxconn to more than 32k. > > > > A) So, do we bump the sockets to use 'int' for so->so_qlimit? > > B) Do we fix solisten() to compensate? > > C) Or de we fix the sysctl (patch below)? > > > > I have patches for A and C. > > Also note that I'd like to MFC this shortly which is why i'm a bit > worried about mucking with socket/xsocket. I could use A for -current > and C for -stable though. Let me know what sounds the best, also > for C should I log(LOG_NOTICE or something? EINVAL is a bit cryptic. For what it's worth, we found (at Yahoo) that excessively large listen queues tend to cause more problems than they solve. The circumstances are probably different, but we found that on one particular application, a queue of 10 was better than the queue of 1024 that they had been using. This particular application is probably quite different to yours, but we found that it was generally bad to accept more than about a second or two's worth of connections. ie: this particular group of systems were processing 7-8 connections per second, so a queue depth of 1024 was about 140 seconds. Most of them would time out when they waited that long (30 or 60 second protocol timeout) so when the machine was overloaded and backing up, it was being made worse by accepting all these connections, doing processing to get them in the listen queue, then timing out. What we ended up with was a LOT of races where sockets would get to the head of the queue right as the remote was in the process of initiating a timeout, so we got large numbers of 'connection reset by peer' type problems being reported by accept and getsockname()/getpeername() etc. It was also bad because the userland app then wasted time processing a dying connection, thus contributing further to the overload. Anyway, just be careful, ok? larger listen queues are not a magic solution for all problems. At 100 connections per second, the current limit is about 327 seconds worth of delay. at 500 per second, it is 65 seconds delay. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200102111330.f1BDUGU36650>