From owner-freebsd-current@FreeBSD.ORG Mon Nov 30 00:53:06 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A3B11065676; Mon, 30 Nov 2009 00:53:06 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-qy0-f174.google.com (mail-qy0-f174.google.com [209.85.221.174]) by mx1.freebsd.org (Postfix) with ESMTP id 304C48FC15; Mon, 30 Nov 2009 00:53:05 +0000 (UTC) Received: by qyk4 with SMTP id 4so1374806qyk.7 for ; Sun, 29 Nov 2009 16:53:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=TxyPgAoSRjnOiWjgoz2NnDLb+yCogjKNN3q7eWgE6XI=; b=jrzPQu3Ltxg0B3zeFTiRzjX/ghfuAWCBT6/vKSU2wntKZZlMGdkeMYqqGva1a2F6Dg ZMjmwU0HafIJvuADz4xfxHPciZt7B3sf6q1gdU94LdUuEvS/W86z//xh1bc/kn5KVCKp PYUXUA8RKXPAra+ZLPeJvWMXXVLs1rtreTxKY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; b=gCQVXhugz6G0XOj12czRZsiIyqMuVRTLcsnlhBvBx5xgOHijR61EcxW0hZvCuFZk/m Jn9VKoOqwYVEk9FwHyHZ/qy9wbwfPJfPLV4K6Y5i01A012p7IoDk4y3LBAfm8MWfG8hF qkZirE+U1KXtnXXxbPaoGeLtKr0erdNsaks7o= Received: by 10.224.62.233 with SMTP id y41mr1808969qah.11.1259542384288; Sun, 29 Nov 2009 16:53:04 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 6sm10118752qwk.11.2009.11.29.16.53.02 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 29 Nov 2009 16:53:03 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Sun, 29 Nov 2009 16:52:36 -0800 From: Pyun YongHyeon Date: Sun, 29 Nov 2009 16:52:36 -0800 To: Eirik =?iso-8859-1?Q?=D8verby?= Message-ID: <20091130005236.GC1123@michelle.cdnetworks.com> References: <20091129013026.GA1355@michelle.cdnetworks.com> <74BFE523-4BB3-4748-98BA-71FBD9829CD5@anduin.net> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: Gavin Atkinson , weldon@excelsusphoto.com, freebsd-current@freebsd.org, Robert Watson Subject: Re: FreeBSD 8.0 - network stack crashes? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2009 00:53:06 -0000 On Mon, Nov 30, 2009 at 12:21:16AM +0100, Eirik ??verby wrote: > On 29. nov. 2009, at 15.29, Robert Watson wrote: > > > On Sun, 29 Nov 2009, Eirik Øverby wrote: > > > >> I just did that (-rxcsum -txcsum -tso), but the numbers still keep rising. I'll wait and see if it goes down again, then reboot with those values to see how it behaves. But right away it doesn't look too good .. > > > > It would be interesting to know if any of the counters in the output of netstat -s grow linearly with the allocation count in netstat -m. Often times leaks are associated with edge cases in the stack (typically because if they are in common cases the bug is detected really quickly!) -- usually error handling, where in some error case the unwinding fails to free an mbuf that it should free. These are notoriously hard to track down, unfortunately, but the stats output (especially where delta alloc is linear to delta stat) may inform the situation some more. > > From what I can tell, all that goes up with mbuf usage is traffic/packet counts. I can't say I see anything fishy in there. > If system exhausted all available mbufs it still should not crash the box. Use -d option of netstat(1) to see whether packet drop counter still goes up when you know system can't receive any frames. AFAIK em(4) was carefully written to recover from Rx resource shortage such that it just drops incoming frames when it can't get new mbuf. This may result in dropping incoming connection request but it means it still tries to recover from the resource exhaustion. It's not clear where mbuf leak comes from, though. > From the last few samples in > http://anduin.net/~ltning/netstat.log 404 > you can see the host stops receiving any packets, but does a few retransmits before the session where this script ran timed out. > By chance do you use pf/ipfw/ipf? > /Eirik >