From owner-dev-commits-doc-all@freebsd.org Wed Dec 30 18:48:02 2020 Return-Path: Delivered-To: dev-commits-doc-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 90C614CC29D for ; Wed, 30 Dec 2020 18:48:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D5gJy3fqbz4WnN; Wed, 30 Dec 2020 18:48:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6FF7D22D35; Wed, 30 Dec 2020 18:48:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 0BUIm2hu040216; Wed, 30 Dec 2020 18:48:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 0BUIm2qX040215; Wed, 30 Dec 2020 18:48:02 GMT (envelope-from git) Date: Wed, 30 Dec 2020 18:48:02 GMT Message-Id: <202012301848.0BUIm2qX040215@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Allan Jude Subject: git: b5882a38ef - main - Add a section about Zstandard compression to the ZFS handbook MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: allanjude X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: b5882a38ef4fefccf1f74b77947a35d276d25958 Auto-Submitted: auto-generated X-BeenThere: dev-commits-doc-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Commit messages for all branches of the doc repository." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Dec 2020 18:48:02 -0000 The branch main has been updated by allanjude: URL: https://cgit.FreeBSD.org/doc/commit/?id=b5882a38ef4fefccf1f74b77947a35d276d25958 commit b5882a38ef4fefccf1f74b77947a35d276d25958 Author: Allan Jude AuthorDate: 2020-12-22 03:23:36 +0000 Commit: Allan Jude CommitDate: 2020-12-30 18:47:57 +0000 Add a section about Zstandard compression to the ZFS handbook Reviewed by: emaste, ygy, bcr, debdrup, pauamma@gundo.com Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D27715 --- en_US.ISO8859-1/books/handbook/zfs/chapter.xml | 66 ++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/en_US.ISO8859-1/books/handbook/zfs/chapter.xml b/en_US.ISO8859-1/books/handbook/zfs/chapter.xml index 4bc0031fa0..547cff76e6 100644 --- a/en_US.ISO8859-1/books/handbook/zfs/chapter.xml +++ b/en_US.ISO8859-1/books/handbook/zfs/chapter.xml @@ -2926,6 +2926,72 @@ mypool/compressed_dataset logicalused 496G - However since quotas do not consider compression, more data may be written than would fit with uncompressed backups. + + + Zstandard Compression + + In OpenZFS 2.0, a new compression + algorithm was added. Zstandard (Zstd) + offers higher compression ratios than the default + LZ4 while offering much greater speeds + than the alternative, gzip. + OpenZFS 2.0 is available starting with + &os; 12.1-RELEASE via + sysutils/openzfs and has been the + default in &os; 13-CURRENT since September 2020, and + will by in &os; 13.0-RELEASE. + + Zstd provides a large selection of + compression levels, providing fine-grained control over + performance versus compression ratio. One of the main + advantages of Zstd is that the + decompression speed is independent of the compression + level. For data that is written once but read many times, + Zstd allows the use of the highest + compression levels without a read performance + penalty. + + Even when data is updated frequently, there are often + performance gains that come from enabling compression. One + of the biggest advantages comes from the compressed ARC + feature. ZFS's Adaptive Replacement + Cache (ARC) caches the compressed version + of the data in RAM, decompressing it each + time it is needed. This allows the same amount of + RAM to store more data and metadata, + increasing the cache hit ratio. + + ZFS offers 19 levels of + Zstd compression, each offering + incrementally more space savings in exchange for slower + compression. The default level is + zstd-3 and offers greater compression + than LZ4 without being significantly + slower. Levels above 10 require significant amounts of + memory to compress each block, so they are discouraged on + systems with less than 16 GB of RAM. + ZFS also implements a selection of the + Zstd fast levels, + which get correspondingly faster but offer lower + compression ratios. ZFS supports + zstd-fast-1 through + zstd-fast-10, + zstd-fast-20 through + zstd-fast-100 in increments of 10, and + finally zstd-fast-500 and + zstd-fast-1000 which provide minimal + compression, but offer very high performance. + + If ZFS is not able to allocate the required memory to + compress a block with Zstd, it will fall + back to storing the block uncompressed. This is unlikely + to happen outside of the highest levels of + Zstd on systems that are memory + constrained. The sysctl + kstat.zfs.misc.zstd.compress_alloc_fail + counts how many times this has occurred since the + ZFS module was loaded. +