Date: Sun, 29 Aug 2010 21:10:42 +0200 From: Andre Oppermann <andre@freebsd.org> To: Mike Tancsa <mike@sentex.net> Cc: freebsd-stable@freebsd.org Subject: Re: syncache errors Message-ID: <4C7AB0B2.9020003@freebsd.org> In-Reply-To: <201008291716.o7THGxcF083744@lava.sentex.ca> References: <201008291409.o7TE9QCJ082862@lava.sentex.ca> <4C7A7DD4.9020803@freebsd.org> <201008291716.o7THGxcF083744@lava.sentex.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On 29.08.2010 19:16, Mike Tancsa wrote: > At 11:33 AM 8/29/2010, Andre Oppermann wrote: >> On 29.08.2010 16:09, Mike Tancsa wrote: >>> After upgrading to a recent STABLE, I have been seeing the following sporadic errors >>> >>> Aug 28 04:15:15 smarthost2 kernel: TCP: [xx.yy.165.120]:53617 to [xx.yy.164.50]:25; syncache_socket: >>> Socket create failed due to limits or memory shortage >>> >>> this is with >>> FreeBSD 8.1-STABLE #7: Wed Aug 25 15:32:05 EDT 2010 >>> and the previous kernel was from July 20th. >>> >>> The odd thing is that the error is triggered from a RELENG_6 host only it would seem. I noticed >>> there are a number of syncache and tcp updates to RELENG_8, is it possible this is related ? I dont >>> have any specific tweaks in /etc/sysctl.conf >>> >>> other than >>> net.inet.tcp.log_in_vain: 1 >>> net.inet.udp.log_in_vain: 1 >> >> The log_in_vain sysctl would cause the logging of syncache errors >> as well (net.inet.tcp.log_debug). This was a POLA violation and >> I've separated them on August 27 on 8-stable. So if you update >> you won't see them anymore. That doesn't change the fact that >> the socket create failed though. >> >> If it only happens from a RELENG_6 box it probably is a problem >> with port re-usage by RELENG_6. The difficulty is that sonewconn() >> fails and doesn't return an error code. Hence it may be one of >> listen queue limits reached, memory shortage or a problem with >> inserting the inpcb into the hash lists. > > Actually, I think I might have found the issue. I was focusing on the "memory" part not the limits. > The 'RELENG_6 only' is just a fluke as thats a box that sends a lot of email to this particular > server. It turns out, sendmail was rate limiting the server > sm-mta[25923]: deferring connections on daemon IPv4: 8 per second > and somehow syncache is aware/logs this now where as it did not before ? When sendmail is deferring the connections it should not show up in the syncache logs. The accept() by sendmail is much later than when the socket for a new connection is created in syncache. Here it points to the limits in the listen queue. Maybe sendmail is getting behind in accepting new connections and the listen queue is overflowing. -- Andre
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C7AB0B2.9020003>