From owner-freebsd-stable@FreeBSD.ORG Fri May 4 00:08:42 2007 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3C33A16A404 for ; Fri, 4 May 2007 00:08:42 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.240]) by mx1.freebsd.org (Postfix) with ESMTP id EEC6F13C459 for ; Fri, 4 May 2007 00:08:41 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by an-out-0708.google.com with SMTP id c24so717141ana for ; Thu, 03 May 2007 17:08:41 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=mIQIViSm8TLcHlN0IM1Vi/gRKiLDmgNs9ppU4n742vpW6EZPK/Mh1YXSCnaXvwC3DXSgroyFkL+fuzMsViz5SKYBGtqz6vKi9M6kYFU7Tia9UmcLQXZ7UwCTU4jAFHihXfJrg/cMrwWTYbB2s+unlglEfkJt17/dX09fip3pimM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=lSHo6JEF1oSwJxU4z0ulAw/z3Hz/DK9umzEFMiDm9Lr4Us/z86zVS2q6FgxNl4gS+5YOqNO8mlwknNVoarEdW4D+LAOEa35o+9CooInKI8rb0Njs6JEsUsxxNdtpb0DAVGDSrmskEtjHuMInjW4v+k+fEbA5JVq3fZLNqj/Paa8= Received: by 10.100.144.11 with SMTP id r11mr2170226and.1178237321100; Thu, 03 May 2007 17:08:41 -0700 (PDT) Received: by 10.100.197.3 with HTTP; Thu, 3 May 2007 17:08:40 -0700 (PDT) Message-ID: Date: Fri, 4 May 2007 08:08:40 +0800 From: "Adrian Chadd" Sender: adrian.chadd@gmail.com To: "Marc G. Fournier" In-Reply-To: <5C07C592A1A1F97E055965A6@ganymede.hub.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <5C07C592A1A1F97E055965A6@ganymede.hub.org> X-Google-Sender-Auth: 1aab13a3e1fdb215 Cc: freebsd-stable@freebsd.org Subject: Re: What triggers "No Buffer Space Available"? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2007 00:08:42 -0000 On 04/05/07, Marc G. Fournier wrote: > 'k, all I'm looking at right now is the Unix Domain Sockets, and the output of > netstat -> sockstat is growing since I first started counting both .. Hm! What about graphing them? It shouldn't be hard to write an mrtg shell script data source to graph these things on your different servers to compare/contrast. You could graph different chunks of the netstat -m output along with the stuff below. Ever played with munin, btw? The exercise would be to find other stuff on the system to correlate against your apparent unbounded socket growth and then investigate possible causation. Adrian > > This was shortly after reboot: > > mars# netstat -A | grep stream | wc -l ; sockstat -u | wc -l > 2705 > 2981 > > - From your explanation above, I'm guessing that the higher sockstat #s is where > you were talking about one socket being used by multiple processes? But, right > now: > > mars# netstat -nA | grep stream | wc -l ; sockstat -u | wc -l > 5025 > 2905 > > sockstat -u #s are *down*, but netstat -na is almost double ... > > Again, based on what you state above: "Sockets can also exist without any > referencing process (if the application closes, but there is still data > draining on an open socket)." > > Now, looking at another 6-STABLE server, but one that has been running for 2 > months now, I'm seeing numbers more consistent with what mars looks like > shortly after all the jails start up: > > venus# netstat -nA | grep stream | wc -l ; sockstat -u | wc -l > 2126 > 2209 > > So, if those sockets on mars are 'still draining on an open socket', is there > some way of finding out where? If I'm understanding what you've said above, > these 'draining sockets' don't have any processes associated with them anymore? > So, its not like I can just kill off a process, correct? > > > - ---- > Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) > Email . scrappy@hub.org MSN . scrappy@hub.org > Yahoo . yscrappy Skype: hub.org ICQ . 7615664 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (FreeBSD) > > iD8DBQFGOlh34QvfyHIvDvMRApSUAJ9jPszXBw83hXPRLbczimNWFtn6WwCgpijT > nDWi/kW4Gt8/J2a4U3n2prk= > =IQCW > -----END PGP SIGNATURE----- > > -- Adrian Chadd - adrian@freebsd.org