From owner-freebsd-net@FreeBSD.ORG Tue Dec 16 10:42:17 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D89E1065675 for ; Tue, 16 Dec 2008 10:42:17 +0000 (UTC) (envelope-from yonyossef.lists@gmail.com) Received: from ik-out-1112.google.com (ik-out-1112.google.com [66.249.90.176]) by mx1.freebsd.org (Postfix) with ESMTP id DEDE18FC25 for ; Tue, 16 Dec 2008 10:42:16 +0000 (UTC) (envelope-from yonyossef.lists@gmail.com) Received: by ik-out-1112.google.com with SMTP id c21so816887ika.3 for ; Tue, 16 Dec 2008 02:42:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references:subject :date:message-id:mime-version:content-type:content-transfer-encoding :x-mailer:thread-index:x-mimeole:in-reply-to; bh=SX2uFWLMjO6NT4mW3O+swSlGCQAHM8cjfPEjFTokI2s=; b=lVSUM6LsT85SZ5l6kuapKJTouceAkzY4SZk3Uhbkbs+vkMqctc/dVKZgmE8YE9Jka3 1D1zGm8g5szak2ct0d72jk+zZTuwMltCp2S0vmhQOI/e4JEqB6D1TTqiinRylKDHoHhW fOGuJGJmPLu7wSRUghdPtL2n+7tbtLVYzZJiI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:subject:date:message-id:mime-version :content-type:content-transfer-encoding:x-mailer:thread-index :x-mimeole:in-reply-to; b=ILT4DC1OHuxJar3A6NIMwmjrhNHy6ltNAT23YGQ46BN8038PIGUCbFw97jQaep185L IdW3fWQ7vvgnsKd4XJQKoEzg+uyGX43XT4gd3iSMMHh8TBPTjxPm/lqcW5mVzVVwuaBo 6mDqfc4mpVH1ir9WOOicIu8VFnXBXqCH71cN8= Received: by 10.66.239.16 with SMTP id m16mr2580434ugh.36.1229424135394; Tue, 16 Dec 2008 02:42:15 -0800 (PST) Received: from mtllpt03 (DSL212-235-20-133.bb.netvision.net.il [212.235.20.133]) by mx.google.com with ESMTPS id y6sm1305945uge.28.2008.12.16.02.42.13 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 16 Dec 2008 02:42:14 -0800 (PST) From: "Yony Yossef" To: "'Robert Watson'" , "Yony Yossef" References: <000501c95ebe$709bddb0$220f000a@mtl.com> Date: Tue, 16 Dec 2008 12:42:11 +0200 Message-ID: <000c01c95f6a$f52d0430$220f000a@mtl.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: Acle54ZAlKUBKty+QMiP/4EuFNTF0gAgoNzw X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350 In-Reply-To: Cc: freebsd-net@freebsd.org Subject: RE: About netstat -m: What is "mbuf+clusters out of packet secondary zone in use" ? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Dec 2008 10:42:17 -0000 > > I'm testing an Ethernet driver on FreeBSD 6.3. > > > > Running netstat -m during an ethernt stress test I see that the > > "mbuf+clusters out of packet secondary zone in use" number > is growing > > gradualy. Problem is it never goes down after I stop the > test, so it's > > pushing the "mbufs in use" up until the following stress test > > iterations reach the OS limit. What does this number mean? > > It seems likely that one of two things is happening: > > (1) a leak of mbuf + clusters > (2) a bug in stats on mbuf + clusters > > Could you paste the output of "vmstat -z | grep -i mbuf" into > an e-mail? > That's the underlying vm stats from the zone used to generate > netstat -m output, and might shed more light on things. Thanks Robert. Here's the dump. In the beginning of the test this is the status: Tue Dec 16 10:34:15 UTC 2008 4353/1152/5505 mbufs in use (current/cache/total) 4352/706/5058/65536 mbuf clusters in use (current/cache/total/max) 256/640 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 9792K/1700K/11492K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines ITEM SIZE LIMIT USED FREE REQUESTS FAILURES mbuf_packet: 256, 0, 256, 640, 12466, 0 mbuf: 256, 0, 4097, 512, 26854, 0 mbuf_cluster: 2048, 65536, 4992, 66, 4992, 0 mbuf_jumbo_pagesize: 4096, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 0, 0, 0, 0, 0 After a while the mbuf+clusters is growing. This output was taken a long time after the test ended: Tue Dec 16 10:38:58 UTC 2008 12873/62142/75015 mbufs in use (current/cache/total) 6780/58756/65536/65536 mbuf clusters in use (current/cache/total/max) 2337/542 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 16778K/133047K/149825K bytes allocated to network (current/cache/total) 0/77/15 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines ITEM SIZE LIMIT USED FREE REQUESTS FAILURES mbuf_packet: 256, 0, 2337, 542, 1939590, 15 mbuf: 256, 0, 10536, 61600, 10221115, 0 mbuf_cluster: 2048, 65536, 7322, 58214, 3316319, 77 mbuf_jumbo_pagesize: 4096, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 0, 0, 0, 0, 0