Date: Sat, 18 Jul 2015 16:09:45 +0200 From: Holger Levsen <holger@layer-acht.org> To: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>, reproducible-builds@lists.alioth.debian.org Subject: Re: reproducible builds of FreeBSD in a chroot on Linux Message-ID: <201507181609.49815.holger@layer-acht.org> In-Reply-To: <CAPyFy2DExDdGf8hN2DNJCSgnP2dj_cLm_TXf1Y8tNJ%2BygvqRzg@mail.gmail.com> References: <201505071122.36037.holger@layer-acht.org> <201506162350.11646.holger@layer-acht.org> <CAPyFy2DExDdGf8hN2DNJCSgnP2dj_cLm_TXf1Y8tNJ%2BygvqRzg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--nextPart4827853.JqjbN4J8qL Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, so I made some progress on this: a.) there is a build host running freebsd= =20 10.1 (called freebsd-jenkins.debian.net) now, on which the jenkins user fro= m=20 jenkins.debian.net can login via ssh as jenkins user b.) besides the base=20 system it has "screen git vim sudo denyhosts" installed and c.) the=20 directories /srv/workspace/chroots/ and /srv/reproducible-results have been= =20 created (and are owned by the jenkins user) and d.) /usr/obj/srv is a link = to=20 /srv. With this,=20 http://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/bin/reproduci= ble_freebsd.sh=20 gets as far as=20 https://jenkins.debian.net/view/reproducible/job/reproducible_freebsd/7/con= sole=20 where "stage 2.1: cleaning up the object tree" fails on "make buildworld",= =20 because /srv/workspace/chroots/freebsd- XXXXXXXX.v1adN6Qo/freebsd/lib/libc/tests does not exist. And at this point I'm stuck as to why this happens. Any hint much welcome! (Please note that reproducible_freebsd.sh is just a work-in-progress now an= d=20 there are still some bits from it's source, reproducible_netbsd.sh visible.= =20 This need to be cleaned up, but shouldn't be too confusing know that this i= s=20 clear.) On Mittwoch, 17. Juni 2015, Ed Maste wrote: > > https://wiki.freebsd.org/ReproducibleBuilds claims there are 3 known > > issues (for "make world" AIUI) for HEAD, I would like to build twice and > > verify myself. > I'm interested in fixing the remaining kernel / world issues, with the > kernel being my higher priority. cool! =20 > For the kernel we have the username, hostname, and build timestamp. > The path is included too, but I don't anticipate trying to address it > at first; release builds are done in a consistent location anyhow > (/usr/src). /me nods - that's what we are doing in (reproducible builds for) Debian too= ,=20 the path has to be the same on rebuilds (as it is included in too many buil= d=20 artifacts to deeply.) > These are used only as user-facing strings for the kern.version sysctl > and reported by uname. An example kern.version string: > FreeBSD 10.1-STABLE #28 r280427+86df2de(stable-10): Thu Mar 26 16:07:47 E= DT > 2015 > emaste@feynman:/tank/emaste/obj/tank/emaste/src/git-stable-10/sys/GENERIC >=20 > From a technical perspective they're trivially eliminated. There may > be some 3rd party ports expect the precise format, but probably not > very many (and they should be fixed, anyhow). There's a much larger > social issue in convincing the FreeBSD developer community to accept > their removal, though :-) If any build (of the same sources) results in the exact same bits, the buil= d=20 time becomes meaningless and thus a.) can be dropped or b.) replaced with t= he=20 date of the last modification of the sources - which is meaningful informat= ion=20 again! While this is/was a new thought for most everyone (me included...) in my=20 experience it also has been convincing logic for most everyone. The technic= al=20 details to achieve this are sometimes a bit harder to achieve, but not=20 impossible. (eg they differ whether git, svn or tarballs are the means to g= et=20 access to sources.) In Debian we want 100% bit identical packages (=3D.deb files) as this allow= s us=20 to only require a checksum comparison to see whether two builds created=20 reproducible results. > > https://wiki.freebsd.org/PortsReproducibleBuilds says "Of the 23599 > > packages which were built in both runs, 15164 have the same checksum > > when using the previously mentioned patch, giving 64.25% reproducible > > packages." - I'm also curious to re-confirm this - and set up a test > > bed, which can be triggered regularily and easily. Our jenkins set up > > allows this and I'm interested to do this. >=20 > I'm pleasantly surprised by the ports results -- 64.25% seems quite > good for such a straightforward change. The test there is on the same > host though, and so avoids any non-reproducibility from host/user/path > leaks. ah > > My interest is to help FreeBSD with reproducible builds as I want to see > > reproducible builds become the norm in the free software world and as I > > believe FreeBSD is an important part of this world. And also because I'm > > curious. :) >=20 > Great! Hopefully we can help lend some weight in convincing upstream > projects to accept reproducibility patches (once we get further along > in our ports effort). I'm looking forward to see this happen! ;-) cheers, Holger --nextPart4827853.JqjbN4J8qL Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIVAwUAVapeLQkauFYGmqocAQoKbA/+L//t613SZ3MeYETSLn9fz9jRllyaFLQq USIkeRfahmsGLeRbTqwwosvnAPhUv4xcKAeeC4ofLzo4pb8ySNhpl+JyZ90vMCGj +tiyZCaPwBp9EHiHIup78rsFJZOaBP1CfhnrWOIn1uwQJ7puopQ2pnYfXdydy+uQ grWxY3YzkXTDFLr+biDeaLyg2Qi0MTexvf5udShoEpZdI8AS2+AIVpzwwaCa+2t9 V8VS6x23o4uB65TP+DqKKb+u7Lg43d0/lc+ZdAEHxMFPWSXO2BAVTXfIPW2Nnw1B PbAJJF3jEsZyFVFLrkgKTkQyVH0yK1wFfiQqq8TqWZRca/tL3gi5gcqFMuiD+x9N Cmste0dcUnNv7a4YrcWQB7wGIzlVhQZehkCDoNBdtcJSPOOtCOMBj5Mh2GoSiaMQ dUgWxqy0scWv4tSTCIO8R/J5wa+2hURS2iDc+hMajXSjWgYLlWixMC/uNCDubghA OWVHGoWV1yDAdBkyKMSe2/yysPUP4xmKqCf97fQcyjXHNDbsrsHLabEH30YEn1ML S7mtFBNeSP2Ia6suvgzt9Ugp+7UkwPSYpiVIrRw4Jf+QzZ073BVoto9aq9wc5f8E tt63Yd1jRasmTuB+ZyT0IUO92Sexm9vo7SKl+NAQRWuiUuVxhr99RFxe0vDiudso Mfn7W96ZgOs= =mucW -----END PGP SIGNATURE----- --nextPart4827853.JqjbN4J8qL--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201507181609.49815.holger>