From owner-freebsd-fs@freebsd.org Tue Sep 8 07:43:08 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E32773E66A9; Tue, 8 Sep 2020 07:43:08 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Blxvw5Xsrz4Nk9; Tue, 8 Sep 2020 07:43:08 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f16b9007cacd1235d1cc1e5.dip0.t-ipconnect.de [IPv6:2003:cd:5f16:b900:7cac:d123:5d1c:c1e5]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 88A9921D76; Tue, 8 Sep 2020 07:43:07 +0000 (UTC) (envelope-from se@freebsd.org) To: FreeBSD CURRENT , freebsd-fs@freebsd.org From: Stefan Esser Subject: OpenZFS and L2ARC Message-ID: Date: Tue, 8 Sep 2020 09:43:03 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.2.1 MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Sep 2020 07:43:09 -0000 OpenZFS seems to work quite well for me, in general, but I have questions regarding the L2ARC statistics. The system uses a 3 * 6 TB raidz1 (plus further ZFS volumes that are not relevant here, since without level 2 ARC) and an 1 TB M.2 SSD with a 256 GB partition for the L2ARC (and most of it currently unused, else). The L2ARC seems to have filled to the limit of 256 GB, but after several reboots, sysctl reports a L2ARC size of nearly twice the allocated space: kstat.zfs.misc.arcstats.l2_size: 534620858880 That is 497 GiB, and might be possible with a lz4 compression factor of 2 - if the value reported is not the space allocated, but the actual (uncompressed) data held by the L2ARC. The sysutils/zfs-stats port reports the following values for this system, BTW: ------------------------------------------------------------------------ ZFS Subsystem Report Tue Sep 8 09:02:46 2020 ------------------------------------------------------------------------ L2 ARC Summary: (HEALTHY) Passed Headroom: 0 Tried Lock Failures: 0 IO In Progress: 0 Low Memory Aborts: 7 Free on Write: 123 Writes While Full: 0 R/W Clashes: 0 Bad Checksums: 0 IO Errors: 0 SPA Mismatch: 0 L2 ARC Size: (Adaptive) 497.91 GiB Header Size: 0.11% 558.83 MiB L2 ARC Evicts: Lock Retries: 6 Upon Reading: 0 L2 ARC Breakdown: 5.75 m Hit Ratio: 81.94% 4.71 m Miss Ratio: 18.06% 1.04 m Feeds: 235.04 k L2 ARC Buffer: Bytes Scanned: 0 Bytes Buffer Iterations: 0 List Iterations: 0 NULL List Iterations: 0 L2 ARC Writes: Writes Sent: 100.00% 22.67 k ------------------------------------------------------------------------ With the FreeBSD ZFS (without persistent L2ARC) I never got more than 20% hit ratio on the L2ARC between reboots. Quite a number of sysctl variable names have changed, and the port needs to be adapted to the new names (therefore there are lots of 0 values in the -L output). The following names used by zfs-stats do not exist in OpenZFS: kstat.zfs.misc.arcstats.recycle_miss kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned kstat.zfs.misc.arcstats.l2_write_buffer_iter kstat.zfs.misc.arcstats.l2_write_buffer_list_iter kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter kstat.zfs.misc.arcstats.l2_write_full kstat.zfs.misc.arcstats.l2_write_in_l2 kstat.zfs.misc.arcstats.l2_write_io_in_progress kstat.zfs.misc.arcstats.l2_write_not_cacheable kstat.zfs.misc.arcstats.l2_write_passed_headroom kstat.zfs.misc.arcstats.l2_write_pios kstat.zfs.misc.arcstats.l2_write_spa_mismatch kstat.zfs.misc.arcstats.l2_write_trylock_fail kstat.zfs.misc.arcstats.l2_writes_hdr_miss vfs.zfs.vdev.cache.size The existence of vfs.zfs.vdev.cache.size vs vfs.zfs.vdev.cache_size can be used to detect OpenZFS, and is easily fixed. But the above listed L2ARC values seem to have been removed from or have never existed in OpenZFS, and I did not find any substitutes. Are there any plans to re-create them in OpenZFS on FreeBSD or are they gone for good? I'd like to update the zfs-stats port for compatibilíty with OpenZFS ...