From owner-freebsd-stable@freebsd.org Fri May 14 12:27:07 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4D02D645895 for ; Fri, 14 May 2021 12:27:07 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FhST71Zqwz3R25 for ; Fri, 14 May 2021 12:27:07 +0000 (UTC) (envelope-from se@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 3487C645754; Fri, 14 May 2021 12:27:07 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3451464557A for ; Fri, 14 May 2021 12:27:07 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FhST70yvKz3R4V; Fri, 14 May 2021 12:27:07 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-449.fritz.box (p200300cd5f135c00dc1848bba4ed8617.dip0.t-ipconnect.de [IPv6:2003:cd:5f13:5c00:dc18:48bb:a4ed:8617]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id A32B724858; Fri, 14 May 2021 12:27:06 +0000 (UTC) (envelope-from se@freebsd.org) To: Pete French References: From: Stefan Esser Cc: stable@freebsd.org Subject: Re: Does FreeBSD 13 disable the VEV cache in ZFS ? Message-ID: <44b0243a-45b6-d098-948b-a6f3d6a53ba7@freebsd.org> Date: Fri, 14 May 2021 14:27:05 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="0mdWVhfS9OcRv2Gkf4Yd27ESgE5UauhwN" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 May 2021 12:27:07 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --0mdWVhfS9OcRv2Gkf4Yd27ESgE5UauhwN Content-Type: multipart/mixed; boundary="LpRLxRJNo09ZcGQLiIj1QqFlTNRKMRbGx"; protected-headers="v1" From: Stefan Esser To: Pete French Cc: stable@freebsd.org Message-ID: <44b0243a-45b6-d098-948b-a6f3d6a53ba7@freebsd.org> Subject: Re: Does FreeBSD 13 disable the VEV cache in ZFS ? References: In-Reply-To: --LpRLxRJNo09ZcGQLiIj1QqFlTNRKMRbGx Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Am 14.05.21 um 10:34 schrieb Pete French: >=20 > Am just upgrading my machiens, and have noticed an oddity. > This is on a machine runnign 12.2 >=20 > # zfs-stats -D >=20 > -----------------------------------------------------------------------= - > ZFS Subsystem Report Fri May 14 08:30:50 2021 > -----------------------------------------------------------------------= - >=20 > VDEV Cache Summary: 88.31 m > Hit Ratio: 31.20% 27.55 m > Miss Ratio: 68.48% 60.47 m > Delegations: 0.32% 284.97 k >=20 > -----------------------------------------------------------------------= - >=20 >=20 > This is on a machine running 13.0 >=20 > # zfs-stats -D >=20 > -----------------------------------------------------------------------= - > ZFS Subsystem Report Fri May 14 08:32:18 2021 > -----------------------------------------------------------------------= - >=20 > VDEV cache is disabled >=20 > -----------------------------------------------------------------------= - >=20 > Same config on both. So, is it really disabled, or is zfs-stats getting= this > wrong for some reason ? I cant find any refernbce to this in the releas= e notes > or by googling. Hi Pete, zfs-stats has last been modified by me to make it work with the sysctl na= mes changed by OpenZFS. (But I might have missed a few ...) Could you check the values of the following sysctl variables: vfs.zfs.vdev.cache_size vfs.zfs.vdev.cache_bshift vfs.zfs.vdev.cache_max kstat.zfs.misc.vdev_cache_stats.misses kstat.zfs.misc.vdev_cache_stats.hits kstat.zfs.misc.vdev_cache_stats.delegations On my system, vfs.zfs.vdev.cache_size is always 0 - and that results in t= he message "VDEV cache is disabled" that you got, too. And I'd guess that th= e kstat variables will all report 0, too (not surprising if the size is 0).= A quick web search reveals: https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%= 20Parameters.html#zfs-vdev-cache-size Quote: Note: with the current ZFS code, the vdev cache is not helpful and in som= e cases actually harmful. Thus it is disabled by setting the zfs_vdev_cache= _size =3D 0 You may want to try setting "vfs.zfs.vdev.cache_size" (name changed, it u= sed to be cache.size instead of cache_size) in /boot/loader.conf and perform = a few performance tests. (I have not verified that the current implementati= on actually uses and supports a value specified that way, though.) Regards, STefan --LpRLxRJNo09ZcGQLiIj1QqFlTNRKMRbGx-- --0mdWVhfS9OcRv2Gkf4Yd27ESgE5UauhwN Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAmCebJkFAwAAAAAACgkQR+u171r99UQ7 NggAg7Srre02gZmUnUfeENKedoDwpiespWqrDzsEHTbninv+ddnSZvjQoyat9Mp6U5pcexwaxtwl eYLMt3LuIEP9TUJXtG1UIdCMnXrhkW/yEoi3d8z+QvTvSnYQhLovP4R9ssIzI+GqEtzk2WMw+YXQ /VTHYb2UVejJ3mRB1biKbQlvctOweDRP+2NB7FN+Th/OfROJIPXTs+SZWeGdbuCYdG6qBFpIDlvU CMprzW7z5b4ctIeC3/N5FF2DOI2Ibmotm2E0EvcmPXw5zWEJonqAROXm/uMeuKymh3kpo82LknWF iPAyKJO5dw1wJ+x8QrwQZi3Eaoh4yt7cMcescbMzQw== =joTH -----END PGP SIGNATURE----- --0mdWVhfS9OcRv2Gkf4Yd27ESgE5UauhwN--