From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 25 11:59:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9499CB01; Wed, 25 Dec 2013 11:59:48 +0000 (UTC) Received: from golf.bouncedomains.com (golf.bouncedomains.com [103.18.252.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4DC1C102C; Wed, 25 Dec 2013 11:59:47 +0000 (UTC) Received: from cpe-124-183-171-215.lns17.ken.bigpond.net.au ([124.183.171.215]:53747 helo=rmbp.bigpond) by golf.bouncedomains.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1VvmEc-0006B2-Pj; Wed, 25 Dec 2013 22:02:42 +1100 From: Mark van der Meulen Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: vmstat buckets Message-Id: <4260012E-9911-47AB-88B9-FDCD6748E6EE@fivenynes.com> Date: Wed, 25 Dec 2013 22:02:42 +1100 To: "freebsd-net@freebsd.org" , freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1822\)) X-Mailer: Apple Mail (2.1822) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - golf.bouncedomains.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - fivenynes.com X-Get-Message-Sender-Via: golf.bouncedomains.com: authenticated_id: mark@fivenynes.com X-Source: X-Source-Args: X-Source-Dir: X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Dec 2013 11:59:48 -0000 Hi All, I have question about buckets in vmstat -z output which I haven=92t been = able to find an answer for online=85 I have 6 FreeBSD 9 boxes running as routers and they are seeing varying = results in vmstat -z. I=92m interested in understanding what the Buckets = represent/mean and how I can influence usage of them. Some of the routers show 0 free buckets under 64 or 128 Bucket and so I = did some research and haven=92t been able to find out what it means - = most people who have asked online have received an answer that it is = nothing to worry about, which is meaningless to me. It seems that the higher the usage of the router, especially when it is = also running userspace applications the less Bucket availability there = is. Also I found someone speculating it was related to free kernel = memory so that was updated on some routers however I saw little = difference(although not sure if adding it in /etc/sysctl.conf actually = does anything as some haven=92t had the vm.kmem_size value change at = all, despite reboots). Can anyone explain what the buckets (16,32,64,128) do and what they = represent? Also if you know anything about troubleshooting bucket = failures? There doesn=92t appear to be anything useful online despite = many searches. I am aware that it may not have anything to do with the problems I am = seeing, but I would still like to understand. Thanks, Mark = --------------------------------------------------------------------------= ----- Background: I have a number of FreeBSD 9 boxes running as routers in a testing = environment where I am seeing odd latency issues. Most of them have = different hardware and configurations now just to rule out configuration = or hardware issues. What I am seeing is occasional latency increases when routing traffic so = that on the =93client=94 end the experience is like the link has hung = for a split second - this happens for both L2TP/PPPoE clients and plain = ethernet connected clients. I haven=92t been able to identify anything = obvious like fragmentation, mtu issues, etc. The setup is: 3 x Dual Quad Core, 8GB RAM running as LAC & LNS w/ Quagga(BGP, OSPF), = MPD5, Netflow, IPFW Stateless(recently introduced and my problem existed = before filtering was introduced). 3 x Dual Quad Core, 8GB RAM running as Routers w/ Quagga(BGP, OSPF) Router 1 =3D=3D=3D=3D=3D=3D=3D=3D ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP 16 Bucket: 152, 0, 198, 2, 198, 0, 0 32 Bucket: 280, 0, 291, 3, 291, 16, 0 64 Bucket: 536, 0, 221, 3, 221, 58, 0 128 Bucket: 1048, 0, 7993, 2, 7993, 505, 0 vm.kmem_size: 4127952896 Router 2(LNS) =3D=3D=3D=3D=3D=3D=3D=3D ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP 16 Bucket: 152, 0, 199, 1, 199, 0, 0 32 Bucket: 280, 0, 329, 7, 333, 0, 0 64 Bucket: 536, 0, 241, 4, 248, 57, 0 128 Bucket: 1048, 0, 7960, 2, 9479,1002, 0 vm.kmem_size: 2147483648 Router 3(LNS) =3D=3D=3D=3D=3D=3D=3D=3D ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP 16 Bucket: 152, 0, 194, 6, 194, 0, 0 32 Bucket: 280, 0, 325, 11, 325, 20, 0 64 Bucket: 536, 0, 278, 2, 278, 57, 0 128 Bucket: 1048, 0, 8031, 0, 8031, 955, 0 vm.kmem_size: 4127952896 Router 4(LNS) =3D=3D=3D=3D=3D=3D=3D=3D ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP 16 Bucket: 152, 0, 197, 3, 197, 0, 0 32 Bucket: 280, 0, 265, 1, 265, 0, 0 64 Bucket: 536, 0, 227, 4, 227, 57, 0 128 Bucket: 1048, 0, 7824, 0, 7824, 721, 0 vm.kmem_size: 2147483648 Router 5 =3D=3D=3D=3D=3D=3D=3D=3D ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP 16 Bucket: 152, 0, 204, 21, 204, 0, 0 32 Bucket: 280, 0, 313, 9, 313, 1, 0 64 Bucket: 536, 0, 268, 5, 268, 57, 0 128 Bucket: 1048, 0, 9204, 0, 9204,5723, 0 vm.kmem_size: 4127952896 Router 6 =3D=3D=3D=3D=3D=3D=3D=3D ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP 16 Bucket: 152, 0, 80, 20, 80, 0, 0 32 Bucket: 280, 0, 86, 12, 86, 0, 0 64 Bucket: 536, 0, 49, 0, 49, 57, 0 128 Bucket: 1048, 0, 8697, 0, 8697,5604, 0 vm.kmem_size: 3108118528