Date: Wed, 03 Oct 2012 22:15:32 +0200 From: Andre Oppermann <andre@freebsd.org> To: Adrian Chadd <adrian@freebsd.org> Cc: freebsd@chrysalisnet.org, freebsd-current@freebsd.org Subject: Re: sysctl kern.ipc.somaxconn limit 65535 why? Message-ID: <506C9CE4.6080400@freebsd.org> In-Reply-To: <CAJ-Vmo=CtC1SpsedP3nHJsrApTLzktGrjopeV0vXShr0FOUsmA@mail.gmail.com> References: <03e101cda197$326dc240$974946c0$@org> <CAJ-Vmo=CtC1SpsedP3nHJsrApTLzktGrjopeV0vXShr0FOUsmA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 03.10.2012 22:03, Adrian Chadd wrote: > Hi, > > somaxconn is the connection queue depth. If it's sitting at a couple > hundred thousand then something else is going crazily wrong. > > I understand your frustration, but there's a lot of instances where > the application just isn't doing things "right" and the OS tries to > hide it as much as psosible. Blowing out somaxconn to chew up a whole > lot of resources seems a bit silly. I'd rather investigate why the > userland application is not servicing the connect queue often enough. > > I've written network services that supported tens of thousands of new > TCP connections a second on a LAN and I never once had to bump > somaxconn past 32767. I'm not saying that it won't apply to your > scenario, I'm just trying to explain that there's likely more going > on. I guess the problem is rather kern.ipc.maxsockets which is only 25600. The name somaxconn is confusing as it specifies the listen queue limit instead of the maximum number of connections as the it suggests. This would also explain the persisting problems Chris described. maxsockets should be bumped up quite a bit by default IMO. How far needs some analysis because there are some dependencies and memory requirements. -- Andre > Adrian > > On 3 October 2012 11:45, <freebsd@chrysalisnet.org> wrote: >> Hi everyone. >> >> Actually 65k sockets is incredibly easy to reach. >> >> I manage some servers for a very large website, it currently has several >> http servers clustered to handle daily traffic and this is only dynamic >> content, static has its own servers, databases also have own servers. >> >> We recently started using memcached to cache some queries and we found on >> default server limits sockets were saturated extremely quickly, on the linux >> servers bumping it to a few hundred K solved the issue, wasnt possible on >> bsd. We did manage to get it down to only needing about 20k by setting >> extremely low timeout values. >> >> In addition we had to migrate all our mysql servers from freebsd to debian >> because they were hitting some arbitary OS limit but I could never figure >> out what, sys% usage went through the roof when this limit was hit, issue >> didnt occur on debian. I feel recently freebsd is more focused on desktop's >> and as such developer's never develop for a heavy server usage scenario, and >> I keep coming across hardcoded low limits. As rightly pointed out default >> values now days are useless 128 for somaxconn? maybe ok for a desktop. >> >> Freebsd also has issues with high numbers of cpu cores, very high locking >> overheads, the problem is the modern hardware is focusing on more and more >> cores and as such much more work on a server so a OS has to handle more and >> more processing and data, this includes more bigger connection queues and so >> on. FreeBSD just isnt scaling with more powerful hardware. Also forgetting >> as well DDOS attack scenarios where a high queue size is useful. >> >> I have used FreeBSD since the 4.x days and have been forced to migrate OS on >> several servers as FreeBSD to be blunt couldnt handle the load on those >> servers. It does seem as if developers of the OS are getting out of touch >> in respect to modern hosting enviromnents. I cant tell app developers to >> fix their apps to work on FreeBSD, they dont care, if it works fine on >> windows and linux then the app isnt broken as far as they are concerned. >> The more FreeBSD dev's have this aatitude of not adapting the less machines >> will run the OS. I hate many things about debian but at the end of the day >> I have to use what can handle the load or I lose my job. >> >> I do accept its entirely possible some tunables could fix any issues I have >> come across but believe me I have spent 100s of man hours on issues, and >> tuned everything I could find. >> >> To me this 32767 limit on somaxconn seems restrive and I vote for it been >> bumped to at least 8x that amount as a minimum. A more suitable default >> would probably be around 1024 as well instead of 128. Dead lingering >> connections in timewait etc. all use one of these, and that can very easily >> get into the 1000s. >> >> Chris >> >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?506C9CE4.6080400>