From nobody Mon Jul 21 14:58:17 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bm3Rs4wJJz62lGt; Mon, 21 Jul 2025 14:58:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bm3Rs3B3Wz3P7g; Mon, 21 Jul 2025 14:58:17 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1753109897; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vx1uODUfsGgwhjf2Qjhi5Rbl+AGBsjFH4L0BnJICKwM=; b=gaUpCyZCOEEaSavK9in5X+cNVYbE0c3cvgttm48HkK2ripsIZm8jtqYBuu0dYUNS6RVdJY 8mewm+7CYB9DUtaEFi9w0SH0U2bzxLLJtSthWrOGiZ08+NBNDnEPPTlHsuAtZCRL9vSVrw 0oTH5LWtp0+a9zhwZL4uZU24X6Wwmw+KiiCmcaoIF85i0VOvVXLSbrx1OSb2qb1oWzZ7sm zHno0OcJ4rqqlqFjYewNnATE8H84qoP+sEltUsBZ3FJ0Xj8C6F67hjMHTxanXzfe0LARYz Nc6NA1v+rOAUkQJsEl8dGTAN1cVasPNT1PUeKALZQ5yCDrb5513AD9taNHdJOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1753109897; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vx1uODUfsGgwhjf2Qjhi5Rbl+AGBsjFH4L0BnJICKwM=; b=xiJN8D+BAgK9EV/bm/COewCndtLjxGdFsUiFy0XhwWPx4BGGosXvrK4wuPpf9BExBENVpW vfi+lyMsVa6oN5+YjVn6awFx/tvCQV/32po9tvdGZUS/sQr6hjz5DyP2al/bHeIkzq5Paj Y7LHnc3brKPf+yRBjkRRfq07VJO3k8mY0msOfzPTOlY9pwgIN/BsdFwQ43vLRb5GewH1yT Xx1GnId2KSM92d0sYzQw7ArzonV13sE9I5TwHwBSd61hTA+TSdLurKVuf5HmIJBPUi4M5E PSCYlIRn3h24bY1WRPrlWgwM/vEUeq4CJj5D+prpzod7ITi6UpwfiegUmvkCHw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1753109897; a=rsa-sha256; cv=none; b=XO0TbUU7ik6VHSFD4PHxJLBXfgjudAkGbULurEtO9zF8YJexNPRRhkVLXUf+ZpytMnVtc0 3/XZYrXWD6HIoGQ4b++V9+wqK/S2bz1iKSKS1WKhFHIDZNPS0mr+nwsikjGW1JUb0XfoYo UrzaXycP5wrX3g4Wr3YSDOnuNZKu6G9B75VEqVNTqzye1ogM4uV4i0uGlI5QQkMzelxo82 y5rgYu/qWIgxhV+VthaGUHUTbBaNoyY3akd5i/GNRpq5gg95HGXepMWnNRqfP6w9K186QM aZ0L5Vwj9NKtiWIs1wo7efntyfzqZQx/nyTWmR4dCPVi5iB4uCkg/FJZ4YZygQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4bm3Rs2HR9z11pt; Mon, 21 Jul 2025 14:58:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 56LEwHA7053893; Mon, 21 Jul 2025 14:58:17 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 56LEwHTU053890; Mon, 21 Jul 2025 14:58:17 GMT (envelope-from git) Date: Mon, 21 Jul 2025 14:58:17 GMT Message-Id: <202507211458.56LEwHTU053890@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: d8f3e8781b2c - main - makefs/zfs: Make it possible to set a compression property List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: d8f3e8781b2ce316120e9a5a42bb1174e9a53187 Auto-Submitted: auto-generated The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=d8f3e8781b2ce316120e9a5a42bb1174e9a53187 commit d8f3e8781b2ce316120e9a5a42bb1174e9a53187 Author: Mark Johnston AuthorDate: 2025-07-21 13:35:12 +0000 Commit: Mark Johnston CommitDate: 2025-07-21 14:57:58 +0000 makefs/zfs: Make it possible to set a compression property makefs currently does not implement compression for ZFS datasets, as doing so seems somewhat fraught with compatibility issues. As a result, the root dataset has compression disabled, and all others inherit from that. However, it may be useful to enable compression for new files once the generated pool is actually imported. Thus, implement a per-data compression option. By default, compression is inherited from the parent dataset and disabled on the root dataset. Add a regression test. PR: 288241 MFC after: 1 month --- usr.sbin/makefs/makefs.8 | 10 +++- usr.sbin/makefs/tests/makefs_zfs_tests.sh | 90 +++++++++++++++++++++++++++++++ usr.sbin/makefs/zfs/dsl.c | 44 +++++++++++++-- 3 files changed, 137 insertions(+), 7 deletions(-) diff --git a/usr.sbin/makefs/makefs.8 b/usr.sbin/makefs/makefs.8 index a11eaf8206e9..c46f283f90a8 100644 --- a/usr.sbin/makefs/makefs.8 +++ b/usr.sbin/makefs/makefs.8 @@ -33,8 +33,7 @@ .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE .\" POSSIBILITY OF SUCH DAMAGE. .\" -.Dd January 19, 2024 - +.Dd July 19, 2025 .Dt MAKEFS 8 .Os .Sh NAME @@ -596,10 +595,17 @@ The following properties may be set for a dataset: .Bl -tag -compact -offset indent .It atime .It canmount +.It compression .It exec .It mountpoint .It setuid .El +Note that +.Nm +does not implement compression of files included in the image, +regardless of the value of the +.Dv compression +property. .El .Sh SEE ALSO .Xr mtree 5 , diff --git a/usr.sbin/makefs/tests/makefs_zfs_tests.sh b/usr.sbin/makefs/tests/makefs_zfs_tests.sh index 520d1f211ac3..d73da083a8c2 100644 --- a/usr.sbin/makefs/tests/makefs_zfs_tests.sh +++ b/usr.sbin/makefs/tests/makefs_zfs_tests.sh @@ -124,6 +124,95 @@ basic_cleanup() common_cleanup } +# +# Try configuring various compression algorithms. +# +atf_test_case compression cleanup +compression_body() +{ + create_test_inputs + + cd $TEST_INPUTS_DIR + mkdir dir + mkdir dir2 + cd - + + for alg in off on lzjb gzip gzip-1 gzip-2 gzip-3 gzip-4 \ + gzip-5 gzip-6 gzip-7 gzip-8 gzip-9 zle lz4 zstd; do + atf_check $MAKEFS -s 1g -o rootpath=/ \ + -o poolname=$ZFS_POOL_NAME \ + -o fs=${ZFS_POOL_NAME}\;compression=$alg \ + -o fs=${ZFS_POOL_NAME}/dir \ + -o fs=${ZFS_POOL_NAME}/dir2\;compression=off \ + $TEST_IMAGE $TEST_INPUTS_DIR + + import_image + + check_image_contents + + if [ $alg = gzip-6 ]; then + # ZFS reports gzip-6 as just gzip since it uses + # a default compression level of 6. + alg=gzip + fi + # The "dir" dataset's compression algorithm should be + # inherited from the root dataset. + atf_check -o inline:$alg\\n -e empty -s exit:0 \ + zfs get -H -o value compression ${ZFS_POOL_NAME} + atf_check -o inline:$alg\\n -e empty -s exit:0 \ + zfs get -H -o value compression ${ZFS_POOL_NAME}/dir + atf_check -o inline:off\\n -e empty -s exit:0 \ + zfs get -H -o value compression ${ZFS_POOL_NAME}/dir2 + + atf_check -e ignore dd if=/dev/random \ + of=${TEST_MOUNT_DIR}/dir/random bs=1M count=10 + atf_check -e ignore dd if=/dev/zero \ + of=${TEST_MOUNT_DIR}/dir/zero bs=1M count=10 + atf_check -e ignore dd if=/dev/zero \ + of=${TEST_MOUNT_DIR}/dir2/zero bs=1M count=10 + + # Export and reimport to ensure that everything is + # flushed to disk. + atf_check zpool export ${ZFS_POOL_NAME} + atf_check -o ignore -e empty -s exit:0 \ + zdb -e -p /dev/$(cat $TEST_MD_DEVICE_FILE) -mmm -ddddd \ + $ZFS_POOL_NAME + atf_check zpool import -R $TEST_MOUNT_DIR $ZFS_POOL_NAME + + if [ $alg = off ]; then + # If compression is off, the files should be the + # same size as the input. + atf_check -o match:"^11[[:space:]]+${TEST_MOUNT_DIR}/dir/random" \ + du -m ${TEST_MOUNT_DIR}/dir/random + atf_check -o match:"^11[[:space:]]+${TEST_MOUNT_DIR}/dir/zero" \ + du -m ${TEST_MOUNT_DIR}/dir/zero + atf_check -o match:"^11[[:space:]]+${TEST_MOUNT_DIR}/dir2/zero" \ + du -m ${TEST_MOUNT_DIR}/dir2/zero + else + # If compression is on, the dir/zero file ought + # to be smaller. + atf_check -o match:"^1[[:space:]]+${TEST_MOUNT_DIR}/dir/zero" \ + du -m ${TEST_MOUNT_DIR}/dir/zero + atf_check -o match:"^11[[:space:]]+${TEST_MOUNT_DIR}/dir/random" \ + du -m ${TEST_MOUNT_DIR}/dir/random + atf_check -o match:"^11[[:space:]]+${TEST_MOUNT_DIR}/dir2/zero" \ + du -m ${TEST_MOUNT_DIR}/dir2/zero + fi + + atf_check zpool destroy ${ZFS_POOL_NAME} + atf_check rm -f ${TEST_ZFS_POOL_NAME} + atf_check mdconfig -d -u $(cat ${TEST_MD_DEVICE_FILE}) + atf_check rm -f ${TEST_MD_DEVICE_FILE} + done +} +compression_cleanup() +{ + common_cleanup +} + +# +# Try destroying a dataset that was created by makefs. +# atf_test_case dataset_removal cleanup dataset_removal_body() { @@ -939,6 +1028,7 @@ atf_init_test_cases() { atf_add_test_case autoexpand atf_add_test_case basic + atf_add_test_case compression atf_add_test_case dataset_removal atf_add_test_case devfs atf_add_test_case empty_dir diff --git a/usr.sbin/makefs/zfs/dsl.c b/usr.sbin/makefs/zfs/dsl.c index f7264b9d2ca7..8a8cee7c82b2 100644 --- a/usr.sbin/makefs/zfs/dsl.c +++ b/usr.sbin/makefs/zfs/dsl.c @@ -193,6 +193,39 @@ dsl_dir_set_prop(zfs_opt_t *zfs, zfs_dsl_dir_t *dir, const char *key, nvlist_add_uint64(nvl, key, 0); else errx(1, "invalid value `%s' for %s", val, key); + } else if (strcmp(key, "compression") == 0) { + size_t i; + + const struct zfs_compression_algorithm { + const char *name; + enum zio_compress alg; + } compression_algorithms[] = { + { "off", ZIO_COMPRESS_OFF }, + { "on", ZIO_COMPRESS_ON }, + { "lzjb", ZIO_COMPRESS_LZJB }, + { "gzip", ZIO_COMPRESS_GZIP_6 }, + { "gzip-1", ZIO_COMPRESS_GZIP_1 }, + { "gzip-2", ZIO_COMPRESS_GZIP_2 }, + { "gzip-3", ZIO_COMPRESS_GZIP_3 }, + { "gzip-4", ZIO_COMPRESS_GZIP_4 }, + { "gzip-5", ZIO_COMPRESS_GZIP_5 }, + { "gzip-6", ZIO_COMPRESS_GZIP_6 }, + { "gzip-7", ZIO_COMPRESS_GZIP_7 }, + { "gzip-8", ZIO_COMPRESS_GZIP_8 }, + { "gzip-9", ZIO_COMPRESS_GZIP_9 }, + { "zle", ZIO_COMPRESS_ZLE }, + { "lz4", ZIO_COMPRESS_LZ4 }, + { "zstd", ZIO_COMPRESS_ZSTD }, + }; + for (i = 0; i < nitems(compression_algorithms); i++) { + if (strcmp(val, compression_algorithms[i].name) == 0) { + nvlist_add_uint64(nvl, key, + compression_algorithms[i].alg); + break; + } + } + if (i == nitems(compression_algorithms)) + errx(1, "invalid compression algorithm `%s'", val); } else { errx(1, "unknown property `%s'", key); } @@ -236,9 +269,6 @@ dsl_init(zfs_opt_t *zfs) zfs->rootdsldir = dsl_dir_alloc(zfs, NULL); - nvlist_add_uint64(zfs->rootdsldir->propsnv, "compression", - ZIO_COMPRESS_OFF); - zfs->rootds = dsl_dataset_alloc(zfs, zfs->rootdsldir); zfs->rootdsldir->headds = zfs->rootds; @@ -288,9 +318,13 @@ dsl_init(zfs_opt_t *zfs) } /* - * Set the root dataset's mount point if the user didn't override the - * default. + * Set the root dataset's mount point and compression strategy if the + * user didn't override the defaults. */ + if (nvpair_find(zfs->rootdsldir->propsnv, "compression") == NULL) { + nvlist_add_uint64(zfs->rootdsldir->propsnv, "compression", + ZIO_COMPRESS_OFF); + } if (nvpair_find(zfs->rootdsldir->propsnv, "mountpoint") == NULL) { nvlist_add_string(zfs->rootdsldir->propsnv, "mountpoint", zfs->rootpath);