From owner-svn-ports-all@freebsd.org Sat May 21 12:41:55 2016 Return-Path: Delivered-To: svn-ports-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0206AB44F9C; Sat, 21 May 2016 12:41:55 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 75D9C17E7; Sat, 21 May 2016 12:41:54 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by mail-lb0-x22a.google.com with SMTP id h1so42958506lbj.3; Sat, 21 May 2016 05:41:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=THZ/XLzaLHpDdRQiz8PfWIn+keuHfHxvidOVtMoagk0=; b=mlq/z6AlfEM5sP1LEIakhmK/xuBCg4zh/wSM9gY8UhrjD5Eo4Byt7ZZRMB4TrlYrqW 3SiYu+q3/XGVacYYQRcDKrWFAhb/1AuqmHf6Ps2MGx9WCK/TvO7hg0SBs6OhqP1A49tx J+2S/WJRQVN3+GMo0VRMH6ygnRXcL3AD9DcnHX7QQHOLFveqVEDDtyDWhg0LmsqPSk6Y dao92hVUbEEQGUItwpbD9Sz5g07P17fg9PdYFWFmHnVrLRmZqDfWnonnJIo1aNQGHRkt Y2nmPMVi++yaf3oab64qbOhof1QyWHa7d1YAFnqYFw69iC9fwBNaewz1gG0jeqzNtzd9 F6Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=THZ/XLzaLHpDdRQiz8PfWIn+keuHfHxvidOVtMoagk0=; b=YxXM9oecXmg/o0pdy00Lo3wUymYa8bJ82lwZ9CJpX2MsVO0BmFCjT1KPPRp3fE+L6z GMLOwEqQMZN4tDJfBoIxElDXZVU+Qs1x0WSdwdShq0x1IfMh2IjBia9C/yadpDhsQpl1 MjXeCmnQ+BexJx9hRL4XxgIu/X23jnAJWOE9S8429gQnyzjefmu/ODOxSd2FnhFixfj7 T/SRuWQhj7rAe7cEDBUSXnwi7oi/fTrL6qYy5nvapuRVnHJd3DoleiENCsfINtFd6UCU D07mV+1+MCTZGp21Bcc4dWKumWyE0v3atUhBFs8K945XyKzIc2EKTj8g3JiS0ADkdPVx jyhw== X-Gm-Message-State: AOPr4FUmL9QVL358H+EuPyihhFifEkcdybFFylFXxRWPlFDzUybp0Js8F9Hqjv21i9I57Q== X-Received: by 10.112.170.106 with SMTP id al10mr2800378lbc.12.1463834512434; Sat, 21 May 2016 05:41:52 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by smtp.gmail.com with ESMTPSA id oq7sm4122219lbb.47.2016.05.21.05.41.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 May 2016 05:41:51 -0700 (PDT) Sender: Baptiste Daroussin Date: Sat, 21 May 2016 14:41:48 +0200 From: Baptiste Daroussin To: marino@freebsd.org Cc: Alexey Dokuchaev , Ed Maste , ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: Re: svn commit: r415078 - in head: . Mk Message-ID: <20160521124148.GK21899@ivaldir.etoilebsd.net> References: <20160513160151.GA30219@FreeBSD.org> <20160513182837.GF49383@ivaldir.etoilebsd.net> <20160513201919.GA48945@FreeBSD.org> <20160519122306.GA24015@FreeBSD.org> <20160521112728.GA624@FreeBSD.org> <364d3d9f-63ff-18c8-c730-a11c57dc0673@marino.st> <20160521114358.GC624@FreeBSD.org> <20160521122522.GJ21899@ivaldir.etoilebsd.net> <70938d6b-0fab-91b9-28b0-9dd05302a503@marino.st> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="x+RZeZVNR8VILNfK" Content-Disposition: inline In-Reply-To: <70938d6b-0fab-91b9-28b0-9dd05302a503@marino.st> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: svn-ports-all@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the ports tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 May 2016 12:41:55 -0000 --x+RZeZVNR8VILNfK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, May 21, 2016 at 02:33:09PM +0200, John Marino wrote: > On 5/21/2016 2:25 PM, Baptiste Daroussin wrote: > > On Sat, May 21, 2016 at 11:43:58AM +0000, Alexey Dokuchaev wrote: > >> On Sat, May 21, 2016 at 01:33:36PM +0200, John Marino wrote: > >>> On 5/21/2016 1:27 PM, Alexey Dokuchaev wrote: > >>>> On Thu, May 19, 2016 at 12:23:06PM +0000, Alexey Dokuchaev wrote: > >>>>> ... > >>>>> I'm still not convinced though, sorry. Ports tree can be obtained = by > >>>>> a number of means, but this new ugly TIMESTAMP thingy is added for a > >>>>> very specific usecase, and there should be no problem to require th= at > >>>>> for that particular usecase, exported ports tree must have its file= s' > >>>>> mtimes correctly set. (If svn/git/hg are not setting right mtimes = on > >>>>> export, they should be fixed.) It looks more like quick'n'dirty ha= ck > >>>>> rather than thoroughly thought-out solution. > >>>> > >>>> Given lack of replies, I guess I'd have to elaborate a bit on proble= ms with > >>>> TIMESTAMP and why I'm against it. > >>>> > >>>> 1. It does not line up with distinfo format... > >>>> 2. It is not needed even if ports repo is obtained as tarball... > >>> > >>> Maybe it could/should be implemented as a makefile variable instead? > >>> > >>> e.g. REP_TIMESTAMP=3D > >>> > >>> Just a suggestion. I don't disagree with you. > >> > >> While still hackish, it's a *lot* less ugly and bogus as tainting dist= info. > >> (New variable in Makefile is OK because that's what Makefiles are made= of: > >> variables, targets, and recipes. Adding TIMESTAMP to distinfo is NOT = OK > >> because distinfo describes port's distfiles in a form of FOO(), BAR(),= ... > >> values per each distfile.) > >> > > Implementing it as a Makefile variable would make it not automatically = updated. > > Meaning more work for maintainers and more chances for mistakes to happ= en. > > > > As stated before using the mtime of the files will end up updating this > > timestamp too often and then defeating the goal of the reproducible bui= ld. > > > > If you have watched the differents talks about reproducible builds and = the > > different links provided earlier you would understand the huge benefit = of > > reproducible builds for Q/A for exp-run for example where you can quick= ly figure > > out the real inpact of a change in base or in ports by getting quickly = the list > > of packages that have changed and analyse them, instead of relying on b= uild > > failures and discovering later that there are runtime problems like fai= lures or > > unexpected changes. That would also allow to discover which ports needs= to be > > bumped because they do link statically (unoticed until now) to a lib th= at has > > received a security fix. Not stating the benefits for users using binary > > packages, the speedup on publishing packages etc. > > > > Now if you disagree with the TIMESTAMP in distinfo they please make sur= e to > > provide a reliable in all case solution for getting this information. > > > > The choice of using distinfo this way has been decided after actually t= esting > > getting informations from other places like mtime from the Makefile its= elf, from > > the distinfo files, from the distfiles etc. We may have missed an obvio= us better > > solution, if that is the case then please provide a solution and please= a tested > > one, because this change does not come out of nowhere, it has been sitt= ing for a > > wile after many many tests (I do work on reproducible builds for years)= =2E the > > ports is a very complicated place for that because upstream builds syst= ems are > > full of hidden dragons. > > > > There are many changes that would be added to the ports tree later, but= we need > > that TIMESTAMP source of information first, before getting further. > > > > As a conclusion this is the less worse solution we found it works, if o= ne has > > a better one, please provide it, but after understanding and taking in = account > > the requirements. > > >=20 > It's not clear to me if 26,000 ports need a timestamp or if it's only=20 > 26. Plus, it's possible that a timestamp could be automatically updated= =20 > even if it's a makefile variable. Finally, "make makesum" could also=20 > detect if timestamp needs updating. >=20 > I'm hearing that besides the "what are the real requirements here?" [1]= =20 > question that it's the location of the information (distfile) that=20 > people are pushing back on. >=20 > John >=20 > [1] e.g. which ports (if not all) really need it, what is impact of not= =20 > having it, what is impact of having a wrong timestamp, etc. I also=20 > agree with those that say this was not introduced in any way, at all. All needs it, I have provided links in previous mails that explain reproduc= ible builds and in particular the issue with timestamps A quick hint: this timestamp will be used as a timestamp for file inside ea= ch packages, (but not only) in order to be sure that the tar files itself is t= he has the same checksum if packaging the same files rebuilt laters. the timestamps are more tricky that they looks like because of how things l= ike Makefile.pl, bytecodes for python, emacs etc works and are regenerated. I really don't care about the location of the information, I care about the= fact that it is updated often enough so it does not break building and not too o= ften so we can benefit from reproducible build. Another benefit from reproducible builds is to be able to provide in the fu= tur reliable delta packages for example Once again I provided links to resources about it in previous mails Bapt --x+RZeZVNR8VILNfK Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJXQFeMAAoJEGOJi9zxtz5a4fQQAKHUCiNzybgX6F07UfExzT4S k8iDrmtnbXZAL8c0SEKyimanlunQ8CgSYCoaxQNwW1/W49mWATn+aTF6nwebqgXv Q6xL0tKd9LeE5+7XnPxyOcB4m6LpcW7bgT+Ly5oBy3DOddFxruBozpiVLAPyNvoT UdhbWXxErFblLD/2DOeX6HaQgCWWuxP+gLtAzMU/TjeDGPGeCv8KjCrQk4lzyQJ9 5+hzKhL/ozWt+dZwBhq+5LrjWQlgeA3oItnXZK06caA8JVcxx+JTH5y6aIImdDVg To5B8LghQZxdLwG7SuILa6c61ixQfba5pp7wryVDWxFX9lrOThr53Bw0ZmkNKXkd jRCbXvi8RcbZp3AXHbcTUajdNxO26UxQht7ql0DTSumJE3SEkYX5CChgmjo+NMkJ JUxGYnp7FPN3x0Y4lzSVkC2OD4G3f753Cqy2gE4Py7FQucbuEEsV/EY+/k9NiCxs 5K7Y8IbvJ1YttRgyNI3p3ZHAILLqtMztguch0CZxZaVKhhoXkIirzI4HLKxn2zR9 7bUTBdjvzd+mrQuDQFtnVdWCxbfkHmkIt5zjicsZHj8k8vchjFPVgzG0fQbcEnN2 ruDFE/2JZvyze2i8k+/g4M2/AubINOGjAh/Cj301hc/3XMcLTcb9XR6wD6km+qgm lmdBKzzs3soOgqrjCozA =lbIO -----END PGP SIGNATURE----- --x+RZeZVNR8VILNfK--