From owner-freebsd-fs@FreeBSD.ORG Tue Jun 26 16:09:08 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3D74D1065670; Tue, 26 Jun 2012 16:09:08 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2822C8FC17; Tue, 26 Jun 2012 16:09:06 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA04238; Tue, 26 Jun 2012 19:08:51 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4FE9DE92.3040700@FreeBSD.org> Date: Tue, 26 Jun 2012 19:08:50 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120610 Thunderbird/13.0 MIME-Version: 1.0 To: Mark Felder , John Baldwin References: <201206251443.41768.jhb@freebsd.org> <4FE9CC00.9090501@FreeBSD.org> <4FE9DB60.1030905@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: [PATCH] Simple ARC stats in top X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jun 2012 16:09:08 -0000 on 26/06/2012 18:57 Mark Felder said the following: > On Tue, 26 Jun 2012 10:55:12 -0500, Andriy Gapon wrote: > >> Does your system also has L2 ARC? >> If so, could you please show a value of kstat.zfs.misc.arcstats.l2_hdr_size? >> Otherwise, it's hard for me to explain the huge difference between Max Size and >> ARC Size. > > Yes, it does. > > zfs2# sysctl kstat.zfs.misc.arcstats.l2_hdr_size > kstat.zfs.misc.arcstats.l2_hdr_size: 6775072568 Aha, so no mystery, those 6G in ARC are actually used by L2ARC headers. I missed the L2 headers because I haven't used L2 yet. I've updated arc-sizes.sh script now to account for these headers. So the correct ARC size breakdown is: hdr_size + data_size + other_size + l2_hdr_size, where data_size can further be broken down to mfu_size + mru_size + anon_size. I think that users of zfs-stats / arc_summary.pl could benefit from this knowledge too. (I will repeat just in case that zfs-stats mistreats kstat.zfs.misc.arcstats.p as current MRU size whereas it is actually target MRU size) > zfs2# zfs-stats -L > > ------------------------------------------------------------------------ > ZFS Subsystem Report Tue Jun 26 10:56:54 2012 > ------------------------------------------------------------------------ > > L2 ARC Summary: (HEALTHY) > Passed Headroom: 61 > Tried Lock Failures: 252.97k > IO In Progress: 0 > Low Memory Aborts: 0 > Free on Write: 382.20k > Writes While Full: 11.08k > R/W Clashes: 270 > Bad Checksums: 0 > IO Errors: 0 > SPA Mismatch: 0 > > L2 ARC Size: (Adaptive) 225.09 GiB > Header Size: 2.80% 6.31 GiB > > L2 ARC Breakdown: 25.52m > Hit Ratio: 87.46% 22.32m > Miss Ratio: 12.54% 3.20m > Feeds: 3.99m > > L2 ARC Buffer: > Bytes Scanned: 42.67 TiB > Buffer Iterations: 3.99m > List Iterations: 255.38m > NULL List Iterations: 219.99m > > L2 ARC Writes: > Writes Sent: 100.00% 913.05k > > ------------------------------------------------------------------------ -- Andriy Gapon