From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 11 19:49:48 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7A9C104; Tue, 11 Feb 2014 19:49:48 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AD6CE19A5; Tue, 11 Feb 2014 19:49:48 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A6242B98A; Tue, 11 Feb 2014 14:49:47 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: The sonewconn listen queue overflow issue Date: Tue, 11 Feb 2014 13:30:54 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <52EFEEF7.5010704@xs4all.nl> In-Reply-To: <52EFEEF7.5010704@xs4all.nl> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402111330.54758.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 11 Feb 2014 14:49:47 -0500 (EST) Cc: hackers@freebsd.org, Photo stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Feb 2014 19:49:48 -0000 On Monday, February 03, 2014 2:33:11 pm Photo stuff wrote: > L.S. > > I came across a lot of messages in the log since, I think, about 9.1, > that sometimes fill it up so much that all else is made invisible, of > the type: > > sonewconn: pcb 0xyyyyyyyyyyyyyyyy: Listen queue overflow: 8 already in > queue awaiting acceptance > > I searched a bit on the web and came across recommendations to try > netstat -nAa to find out which program this came from. > > Well, running netstat -nAa |grep pcb 0xyyyyyyyyyyyyyyyy in a loop didn't > work, it didn't give any output even though the messages kept coming in > the log during that time. > > I forgot about it then recently searched a bit more, and I found this page: > > http://lawrencechen.net/2014/sonewconn-pcb-0xfffffe006acd9310-listen-queue > > So a listen queue overflow of 8 is caused by a listen value of 5 (QLEN > > 3 * (QLIM / 2)) > > Doing: > netstat -LaAn | grep '/5 ' > > resulted in > ffffzzzzzzzzzzzz tcp4 0/0/5 127.0.0.1:8000 > > The ffffzzz value was not the value in the message log, > but the connection means in my case it had to be the junkbuster proxy > (which I still use, still works well :) ), as I didn't run anything else > locally. > > So I looked into junkbuster's cource and in bind.c I changed 2 instances of: > > while (listen(fd, 5) == -1) { > > to > > while (listen(fd, 10) == -1) { /* 10 instead of 5, this fixes dmesg > spamming of the type 'sonewconn: pcb 0xyyyyyyyyyyyyyyyy: Listen queue > overflow: 8 already in queue awaiting acceptance'? */ > > This improved the situation, but still gave the issue of the log filling > up too much. So then I changed it to 20, which gave me silence :) > > Checking the latest log I couldn't find anything in the last 10 days. > > So for people who have this issue I recommend calculating which value > the listen-value that the overflow-value corresponds to, then checking > as above and then it should be possible to find the daemon causing the > issue. And modify that program... > > But while this removes the errors, what do these messages really > signify? I mean which didn't this happen before in earlier versions of > FreeBSD? Maybe earlier versions just dropped the connections without logging a message? The message means that connections are arriving faster than the userland program can accept them. There is a stat for that in 'netstat -s -p tcp' and you can see if it was increasing on older versions of FreeBSD (if you still have a machine with that around) to make sure the only change is the additional log message. > Btw., at the moment (running 10.0 RC2) my message log now gets filled up > with something else :) > > hwpstate0: set freq failed, err 6 > > This happened long ago already in 9.1 with my AMD3500+ and still now on > my AMD FX6100... I am not familiar with how hwpstate works, sorry. -- John Baldwin