From nobody Mon Mar 25 13:59:46 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V3F1K5Wd6z5F0ql for ; Mon, 25 Mar 2024 13:59:49 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V3F1K53tTz4DyG; Mon, 25 Mar 2024 13:59:49 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711375189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6i9lSmjo0POIXEEYEgl9rQwR+PHcG2UbIXjyCKd3JFY=; b=W+k6ipKPgTYg/ojAGnKNmYvda914qJbRZcZfg26S0lWo3Fht0mxDoF8Qw1qfnOB4TPrbeJ VVK/A7dD2VtWYUsqWw52TKlGtWrieNnH2eGSaB+Tv6Bu0GZWmN5TeYjwvY41nOXKWD2MlC BQXIBAYU7ozFcDSuhKlKg4DCdW/yGIXSIp3WvOLGMo+wAYCPHMwN/HoO3jK9d1TJ+rsfAk 3JH+WadJOjTaucSdG9y19u8kLZ10bdHHKSaMOgnnuvCkLupJU6qnmfhkFkm0tiY0cicC4X tS7H/+KG3hv/FQDxvUksZbLA5InwFTbbK14vTN0KkBEZwdqfLr+NJVPyyCdi8A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1711375189; a=rsa-sha256; cv=none; b=mxn4TOyOBoZRKnOZlrpJcg8lqKzKUuun6hHBfqP7whOXIFYvWUb88XJSJUg5jSlz4DjB3V X/OJG3FGYRZKXpdavNRP2lP1Q3C3TPT+r9P4hp8Xwjuo06hXag8h8DO0W4KSEOHKOrdXN0 V868sQ56Gs3X/Exuo+CrvQj9kxAtD/ZXrII0o/D3Iys2t7G8o+hWF5/A/dTUF6qiQvoqoJ tKXjGzn7TkaHLoOF2xlzJ19X38inOsasct+CzRF9HEDLqCVNGTR57eQmZUF2EDRVrlwX8G /SQI4lFgqNrbJu1p0NOmvxYNPhocpnozyTOns7kRHbY//YYgFA97d+In7iyXAA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711375189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6i9lSmjo0POIXEEYEgl9rQwR+PHcG2UbIXjyCKd3JFY=; b=hjPbsGZb3hpeoxBF9wHscPGaMEPp9MxXPYEwkUOGJtq1ezEpFuyrOyLZJkPGaJ0GU1G4vb K/NCdYDPppRx6t85jiMEL/fgfwqoLU2PGPWjTlAAxsJfypcqTcNkJ6j6B3swLKTRAq8Spz aQL7OSeE0AxGvpsEevO3DuRU/d4HnESfpJ7e4ZVqhdjyYe9/H09LnjBN1gxBwS/NmLvY+c c2lp7hjbfqb2wZsvUGNASDyfZJYbpruWkYtLatIiPLA4f37yvOWuvqkmNLlY7NcZrZ7Veg yXxAhXKkyU1w7xJIIOePaQvWdxyAg82GhJo5OCwD1h14PJAqacZVbR5gCRGtPg== Received: from ltc.des.dev (163.23.65.37.rev.sfr.net [37.65.23.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4V3F1K3qz2zV6B; Mon, 25 Mar 2024 13:59:49 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id D592E781B9; Mon, 25 Mar 2024 14:59:46 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mark Saad Cc: FreeBSD Hackers Subject: Re: TarFS In-Reply-To: (Mark Saad's message of "Fri, 22 Mar 2024 11:48:21 -0400") References: User-Agent: Gnus/5.13 (Gnus v5.13) Date: Mon, 25 Mar 2024 14:59:46 +0100 Message-ID: <86edbyh0od.fsf@ltc.des.dev> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mark Saad writes: > I was wondering if anyone has started to play with tarfs in FreeBSD 14? I wrote it, does that count? > It appears to puke with larger tarballs. On the contrary, one of the use cases I tested was buildworld from a tarball of the source tree, which it handled just fine. > For example, if I fetch a copy of ports or pkgsrc and decompress it to > just a posix tar archive I can't get it to mount. > > root@mono:/home/nonesuch # file ports.tar > ports.tar: POSIX tar archive > root@mono:/home/nonesuch # mount -t tarfs ./ports.tar /tarfs/ports > mount: ./ports.tar: Inappropriate file type or format % fetch ftp://ftp.freebsd.org/pub/FreeBSD/ports/ports/ports.tar.zst ports.tar.zst 47 MB 5431 kBps = 09s % sudo mount -rt tarfs $PWD/ports.tar.zst /mnt mount: /home/des/ports.tar.zst: Inappropriate file type or format % dmesg | grep tarfs=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 tarfs_alloc_one: unsupported global extended header at 0 % zcat ports.tar.zst | hexdump -C | head -3 00000000 70 61 78 5f 67 6c 6f 62 61 6c 5f 68 65 61 64 65 |pax_global_hea= de| 00000010 72 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |r.............= ..| 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |..............= ..| This is a non-standard extension header emitted by `git archive` which contains metadata about the git tree from which the archive was created. I believe you can safely strip it off (zcat ports.tar.zst | dd bs=3D1k skip=3D1 >ports.tar), but you'll run into issues with very long paths in devel/electron*. I will take a closer look when I find the time. Note that tarfs can mount tarballs compressed with zstd, but performance will be poor unless you create a multi-frame archive (try using --zstd --options=3Dzstd:frame-per-file,zstd:min-frame-size=3D65536). DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org