Date: Thu, 09 Jul 2026 18:56:12 +0000 From: bugzilla-noreply@freebsd.org To: pkg@FreeBSD.org Subject: [Bug 296241] pkgbase: "make update-packages" causes too much disk churn Message-ID: <bug-296241-32340-4rWXLTa4mY@https.bugs.freebsd.org/bugzilla/> In-Reply-To: <bug-296241-32340@https.bugs.freebsd.org/bugzilla/>
index | next in thread | previous in thread | raw e-mail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296241 Alan Somers <asomers@FreeBSD.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Open --- Comment #2 from Alan Somers <asomers@FreeBSD.org> --- Lexi, I don't think you need to worry. Even though the new packages contain different mtimes in the +MANIFEST, the package checksum, as reported by 'pkg query -F ${PKG} "%X"', is unchanged. I guess pkg is smart enough to skip the mtime when it computes checksums. Or maybe it skips +MANIFEST entirely. At any rate, that's probably why we don't have to download hundreds of packages to update 15.1 to 15.1p1. As for the overall problem, I have a better understanding now: 1) First, create-world-packages creates new packages for everything in pkgbase and puts them into the final location in REPODIR. 2) Second, real-update-packages runs. If PKG_VERSION == PKG_VERSION_FROM, which will be the case when running "make update-packages" twice in a row on a release branch, then real-update-packages basically does nothing. All of the newly packages will have different embedded mtime values than the previous packages, which are by now deleted. BUT, on a CURRENT or STABLE branch, PKG_VERSION will contain an encoded timestamp, and thereby differ from PKG_VERSION_FROM. In that case, real-update-packages will compare the packages' checksums. If they are unchanged, then it will copy the old package into the new output directory. That's possible because the old and new directories are different. To fix this problem, I propose creating a PKG_OUTPUT_TMPDIR. It will located right next to PKG_OUTPUT_DIR, so they'll share a file system. Makefile targets like create-world-packages will write to PKG_OUTPUT_TMPDIR. Then real-update-packages will populate PKG_OUTPUT_DIR like this: * For stable branches, either copy the old package to the new output dir and delete the new tmp package if it's unchanged, or move the new package from the tmp location to the new output dir if it's changed. * For release branches, either delete the tmp package if the package is unchanged, or move the new one from the tmp location if it's changed. * When bootstrapping, move the entire tmp location to the new location. This should eliminate unnecessary churn in REPODIR, at least for release branches. For stable branches, "zfs dedup" will still be helpful. This proposal won't do anything to help runtimes, because we'll still be writing out a full set of packages every time. Fixing that would be a more difficult problem. -- You are receiving this mail because: You are on the CC list for the bug.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-296241-32340-4rWXLTa4mY>