Date: Tue, 3 Oct 2023 11:56:11 -0600 From: Warner Losh <imp@bsdimp.com> To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= <des@freebsd.org> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: something magic about the size of a ports tree Message-ID: <CANCZdfrR1tHRdiNiQ1kk0%2BQMLt4gegkO0Xocp=1B5a-WR6cCCg@mail.gmail.com> In-Reply-To: <86y1gjhdx7.fsf@ltc.des.no> References: <ZRw8x58bxtp26A8e@c720-1400094.fritz.box> <86y1gjhdx7.fsf@ltc.des.no>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Tue, Oct 3, 2023, 10:24 AM Dag-Erling Smørgrav <des@freebsd.org> wrote: > Matthias Apitz <guru@unixarea.de> writes: > > I have on my poudriere build host a ports tree and wanted to move it to > > the host where the resulting packages are installed: > > > > root@jet:/usr/local/poudriere/ports # du -sh ports20230806 > > 397M ports20230806 > > root@jet:/usr/local/poudriere/ports # tar cf p.tar ports20230806 > > root@jet:/usr/local/poudriere/ports # ls -lh p.tar > > -rw-r--r-- 1 root wheel 672M Oct 3 18:00 p.tar > > > > already the size of the tar file is somewhat magic; but if you un-tar it > > on the other host I will get: > > > > [guru@c720-1400094 ~]$ ls -lh p.tar > > -rw-r--r-- 1 guru wheel 672M 3 oct. 18:00 p.tar > > [guru@c720-1400094 ~]$ tar xf p.tar > > [guru@c720-1400094 ~]$ du -sh ports20230806 > > 1,2G ports20230806 > > > > How this is possible? > > Most files in the ports tree are very small. On disk, each file gets > rounded up to the nearest multiple of the filesystem block size, which > could be as small as 512 bytes or as large as 8 kB (or even more in > pathological cases). In a tarball, they get rounded up to the nearest > multiple of 512 bytes plus an additional 512 bytes per file for > metadata. > > For instance, your average distinfo file (of which there are 30k in the > ports tree) is only 200-250 bytes long, but it occupies 512 bytes on an > FFS filesystem, 1 kB in a tarball, and 4 kB on a typical ZFS filesystem. > > Note that if the target system is FreeBSD 14 or newer, you can simply > mount the tarball (`sudo mount -rt tarfs p.tar /usr/ports`). > Do we support any compression on top of that? Has support for poudriere been added for it? Aldo I want a pony.... I'm mostly curious... I have no immediate plans here (though aligning with the boot loader and supporting this on a block device to support rootfs would be cool). Maybe some or all of these wishes would make good GSOC projects? Warner DES > -- > Dag-Erling Smørgrav - des@FreeBSD.org > > [-- Attachment #2 --] <div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 3, 2023, 10:24 AM Dag-Erling Smørgrav <<a href="mailto:des@freebsd.org">des@freebsd.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Matthias Apitz <<a href="mailto:guru@unixarea.de" target="_blank" rel="noreferrer">guru@unixarea.de</a>> writes:<br> > I have on my poudriere build host a ports tree and wanted to move it to<br> > the host where the resulting packages are installed:<br> ><br> > root@jet:/usr/local/poudriere/ports # du -sh ports20230806<br> > 397M ports20230806<br> > root@jet:/usr/local/poudriere/ports # tar cf p.tar ports20230806<br> > root@jet:/usr/local/poudriere/ports # ls -lh p.tar<br> > -rw-r--r-- 1 root wheel 672M Oct 3 18:00 p.tar<br> ><br> > already the size of the tar file is somewhat magic; but if you un-tar it<br> > on the other host I will get:<br> ><br> > [guru@c720-1400094 ~]$ ls -lh p.tar<br> > -rw-r--r-- 1 guru wheel 672M 3 oct. 18:00 p.tar<br> > [guru@c720-1400094 ~]$ tar xf p.tar<br> > [guru@c720-1400094 ~]$ du -sh ports20230806<br> > 1,2G ports20230806<br> ><br> > How this is possible?<br> <br> Most files in the ports tree are very small. On disk, each file gets<br> rounded up to the nearest multiple of the filesystem block size, which<br> could be as small as 512 bytes or as large as 8 kB (or even more in<br> pathological cases). In a tarball, they get rounded up to the nearest<br> multiple of 512 bytes plus an additional 512 bytes per file for<br> metadata.<br> <br> For instance, your average distinfo file (of which there are 30k in the<br> ports tree) is only 200-250 bytes long, but it occupies 512 bytes on an<br> FFS filesystem, 1 kB in a tarball, and 4 kB on a typical ZFS filesystem.<br> <br> Note that if the target system is FreeBSD 14 or newer, you can simply<br> mount the tarball (`sudo mount -rt tarfs p.tar /usr/ports`).<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">Do we support any compression on top of that? Has support for poudriere been added for it?</div><div dir="auto"><br></div><div dir="auto">Aldo I want a pony.... I'm mostly curious... I have no immediate plans here (though aligning with the boot loader and supporting this on a block device to support rootfs would be cool). Maybe some or all of these wishes would make good GSOC projects?</div><div dir="auto"><br></div><div dir="auto">Warner</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> DES<br> -- <br> Dag-Erling Smørgrav - des@FreeBSD.org<br> <br> </blockquote></div></div></div>help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrR1tHRdiNiQ1kk0%2BQMLt4gegkO0Xocp=1B5a-WR6cCCg>