Date: Mon, 21 Aug 2023 14:33:24 +0100 From: Doug Rabson <dfr@rabson.org> To: freebsd-pkgbase@freebsd.org Subject: Repeatable builds using pkgbase Message-ID: <CACA0VUgd0Az-=vj2qwirY081YEQ%2BVPutWhjU596qj05r6m%2BZyA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] While working on build scripts for FreeBSD container images, I wanted to get to the point where my builds are repeatable, i.e. if I create two images with the same set of packages installed in the same order, they should be identical. The main stumbling block is timestamps. I can force all the file timestamps to a fixed value with buildah using the '--timestamp' argument to either 'buildah commit' or 'buildah build' but even then, the two images have different hashes. Looking deeper, the difference is in /var/db/pkg/local.sqlite. If I compare SQL dumps of the databases from each image, I can see a timestamp embedded in the sqlite file: diff dump1 dump2 4c4 < INSERT INTO packages VALUES(1,'base','FreeBSD-zoneinfo','13.2p2','zoneinfo package','zoneinfo package',NULL,NULL,'FreeBSD:13:amd64','re@FreeBSD.org',' https://www.FreeBSD.org ','/',731014,0,0,1,1692446701,'2$2$c9w95oqai9bwhny1k4pcg8mji77xgk43zjxxb69j1duzq5jao18wak4deer85epmfpc8ngyysyt9wu74pg7sczkqc3ekyawkfgwzi8d',NULL,NULL,0); --- > INSERT INTO packages VALUES(1,'base','FreeBSD-zoneinfo','13.2p2','zoneinfo package','zoneinfo package',NULL,NULL,'FreeBSD:13:amd64','re@FreeBSD.org',' https://www.FreeBSD.org ','/',731014,0,0,1,1692622924,'2$2$c9w95oqai9bwhny1k4pcg8mji77xgk43zjxxb69j1duzq5jao18wak4deer85epmfpc8ngyysyt9wu74pg7sczkqc3ekyawkfgwzi8d',NULL,NULL,0); Looking at the pkg source, I can see that the prepared statement for inserting into the packages table explicitly uses NOW() for this column. Would it be reasonable to allow changing this, e.g. by adding a command line argument to pkg to override the default? I haven't tried this to see if that makes the two databases identical - if not, I guess I'll just remove pkg metadata altogether. [-- Attachment #2 --] <div dir="ltr">While working on build scripts for FreeBSD container images, I wanted to get to the point where my builds are repeatable, i.e. if I create two images with the same set of packages installed in the same order, they should be identical.<div><br></div><div>The main stumbling block is timestamps. I can force all the file timestamps to a fixed value with buildah using the '--timestamp' argument to either 'buildah commit' or 'buildah build' but even then, the two images have different hashes. Looking deeper, the difference is in /var/db/pkg/local.sqlite. If I compare SQL dumps of the databases from each image, I can see a timestamp embedded in the sqlite file:</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>diff dump1 dump2 <br>4c4<br>< INSERT INTO packages VALUES(1,'base','FreeBSD-zoneinfo','13.2p2','zoneinfo package','zoneinfo package',NULL,NULL,'FreeBSD:13:amd64','re@FreeBSD.org','<a href="https://www.FreeBSD.org">https://www.FreeBSD.org</a>','/',731014,0,0,1,1692446701,'2$2$c9w95oqai9bwhny1k4pcg8mji77xgk43zjxxb69j1duzq5jao18wak4deer85epmfpc8ngyysyt9wu74pg7sczkqc3ekyawkfgwzi8d',NULL,NULL,0);<br>---<br>> INSERT INTO packages VALUES(1,'base','FreeBSD-zoneinfo','13.2p2','zoneinfo package','zoneinfo package',NULL,NULL,'FreeBSD:13:amd64','re@FreeBSD.org','<a href="https://www.FreeBSD.org">https://www.FreeBSD.org</a>','/',731014,0,0,1,1692622924,'2$2$c9w95oqai9bwhny1k4pcg8mji77xgk43zjxxb69j1duzq5jao18wak4deer85epmfpc8ngyysyt9wu74pg7sczkqc3ekyawkfgwzi8d',NULL,NULL,0);<br></div></blockquote><div><br></div>Looking at the pkg source, I can see that the prepared statement for inserting into the packages table explicitly uses NOW() for this column. Would it be reasonable to allow changing this, e.g. by adding a command line argument to pkg to override the default? I haven't tried this to see if that makes the two databases identical - if not, I guess I'll just remove pkg metadata altogether.<div><br><br><div><br></div></div></div>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACA0VUgd0Az-=vj2qwirY081YEQ%2BVPutWhjU596qj05r6m%2BZyA>