From owner-svn-src-all@freebsd.org Wed Aug 7 20:41:16 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DBF43B73B1; Wed, 7 Aug 2019 20:41:16 +0000 (UTC) (envelope-from freebsd@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 463k1R2q6lz4Jr1; Wed, 7 Aug 2019 20:41:14 +0000 (UTC) (envelope-from freebsd@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x77KfDIK089237; Wed, 7 Aug 2019 13:41:13 -0700 (PDT) (envelope-from freebsd@gndrsh.dnsmgr.net) Received: (from freebsd@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x77KfDv1089236; Wed, 7 Aug 2019 13:41:13 -0700 (PDT) (envelope-from freebsd) From: "Rodney W. Grimes" Message-Id: <201908072041.x77KfDv1089236@gndrsh.dnsmgr.net> Subject: Re: svn commit: r350550 - head/share/mk In-Reply-To: To: Pedro Giffuni Date: Wed, 7 Aug 2019 13:41:13 -0700 (PDT) CC: rgrimes@freebsd.org, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org Reply-To: rgrimes@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 463k1R2q6lz4Jr1 X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd@gndrsh.dnsmgr.net X-Spamd-Result: default: False [1.72 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[rgrimes@freebsd.org]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; NEURAL_SPAM_MEDIUM(0.73)[0.727,0]; IP_SCORE(0.05)[ip: (0.15), ipnet: 69.59.192.0/19(0.08), asn: 13868(0.05), country: US(-0.05)]; NEURAL_HAM_SHORT(-0.63)[-0.630,0]; NEURAL_SPAM_LONG(0.68)[0.679,0]; RCPT_COUNT_SEVEN(0.00)[7]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Aug 2019 20:41:16 -0000 > On 07/08/2019 15:12, Rodney W. Grimes wrote: > >> On 07/08/2019 11:00, John Baldwin wrote: > >>> On 8/6/19 9:56 AM, Glen Barber wrote: > >>>> On Sat, Aug 03, 2019 at 01:06:18AM +0000, John Baldwin wrote: > >>>>> Author: jhb > >>>>> Date: Sat Aug 3 01:06:17 2019 > >>>>> New Revision: 350550 > >>>>> URL: https://svnweb.freebsd.org/changeset/base/350550 > >>>>> > >>>>> Log: > >>>>> Flip REPRODUCIBLE_BUILD back to off by default in head. > >>>>> > >>>>> Having the full uname output can be useful on head even with > >>>>> unmodified trees or trees that newvers.sh fails to recognize as > >>>>> modified. > >>>>> > >>>>> Reviewed by: emaste > >>>>> Differential Revision: https://reviews.freebsd.org/D20895 > >>>>> > >>>> I would like to request this commit be reverted. While the original > >>>> commit message to enable this knob stated the commit would be reverted > >>>> after stable/12 branched, I have seen no public complaints about > >>>> enabling REPRODUCIBLE_BUILD by default (and quite honestly, do not see > >>>> the benefit of disabling it by default -- why wouldn't we want > >>>> reproducibility?). > >>>> > >>>> To me, this feels like a step backwards, with no tangible benefit. > >>>> Note, newvers.sh does properly detect a modified tree if it can find > >>>> the VCS metadata directory (i.e., .git, .svn) -- I know this because > >>>> I personally helped with it. > >>>> > >>>> In my opinion, those that want the non-reproducible metadata included in > >>>> output from 'uname -a' should set WITHOUT_REPRODUCIBLE_BUILDS in their > >>>> src.conf. Turning off a sane default for the benefit of what I suspect > >>>> is likely a short list of use cases feels like a step in the wrong > >>>> direction. > >>> My arguments for flipping this in head (and head only) are that the data > >>> provided in uname -a when this is disabled is useful for development, and > >>> that in head we do tailor settings towards development (e.g. GENERIC in > >>> head vs GENERIC in stable). > >>> > >>> The logic to handle modified trees has an inherent assumption that I think > >>> is false, at least for my workflow and I suspect many others. I do builds > >>> and tests of kernels on separate machines (VMs or bare metal) from where I > >>> use VCS to manage sources so that a kernel crash doesn't toast my source > >>> tree. The trees are then shared to the build/test machines via NFS. As > >>> a result, the build/test machines are not always able to detect that the > >>> tree is modified either because a subset of the checkout is exported via > >>> NFS, or the VCS tool isn't installed on the build/test machines because > >>> they are generally barebones systems with only a base installed. This > >>> does mean that flipping the knob off doesn't provide all of the same info, > >>> but it does provide the path, and the path matters because 'kgdb -n last' > >>> uses it, and because if you use separate directories for separate projects > >>> (e.g. git worktrees), then the path tells you which test kernel you booted. > >>> (It is not uncommon for me to have several test projects in flight on a > >>> single test machine for different branches.) > >>> > >>> In the original discussion on arch, we collectively recognized that > >>> developer builds vs release builds were different and needed different > >>> defaults. The compromise reached at that time was to depend on the VCS > >>> to detect developer builds to choose the policy. What I have found is that > >>> in practice for at least my workflow that doesn't actually work. I posit > >>> that the majority of kernels built from head are developer builds, not > >>> releases, and that the default should cater to that. You could also always > >>> patch release.sh to set WITH_REPRODUCIBLE_BUILD in the environment which I > >>> think would give a more accurate sense of when builds are releases or not. > >>> > >>> However, I will yield to whatever the consensus is. > >> +1 keeping metadata in head. > > I am conflicted on this one, and I think there is a reasonable argument > > on both sides, but from what I have read here this appears to be mostly > > the kernel that is at issue, loss of the meta data from newvers.sh in > > the kernel is infact a PITA, even on stable or production release > > systems. > > > > I propose a compromise, add 2 knobs: > > WITHOUT_REPRODUCIBLE_KERNEL (aka get your metadata in uname) > > WITH_REPRODUCIBLE_USERLAND (aka reproducible userland) > > > > WITH{,OUT}_REPRODUCIBLE_BUILD overrides both, for backwards compat, > > and neither should be defined by default. > Too complex IMHO. Either the system is reproducible or it isn't. > > Somehow set WITH_REPRODUCIBLE_KERNEL for builds of GENERIC > > for releases/snapshots, but do not ship the system with it > > set (I can here a growl from Glen on this) Thus we build > > a reproducible kernel and ship it with the system but if > > the user builds a kernel it gets meta data to indicate it > > is no longer a stock kernel. > > FYI, upon finding I could not figure out what kernel I was running > > after installing 12.0 release I turnd off REPRODUCIBLE on my kernel > > build VM for 12.0. I do leave it on if I am building userland. > > > > Thoughts? > > Among other things, reproducible builds implies that pkg upgrades are Do you mean freebsd-update? > smaller. I see it makes sense to make releases, and in fact -stable, > completely reproducible. For -current I am fine with it not being > reproducible, > > All just IMHO. Let me try to add a case for it on ^head, weekly snapshots are built, if ^head was running reproducible it would be possible to diff these snapshots in a meaniful way. It would also mean one could get pretty creative with ZFS, zfs-snapshots and the built snapshots to actually have on line almost all binary versions of ^head in a fairly compact form. > Pedro. -- Rod Grimes rgrimes@freebsd.org