From owner-svn-src-all@freebsd.org Wed Aug 7 20:27:50 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 738B9B6BA6; Wed, 7 Aug 2019 20:27:50 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 463jjy2YpKz4HkD; Wed, 7 Aug 2019 20:27:50 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from [192.168.0.5] (unknown [181.52.72.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: pfg) by smtp.freebsd.org (Postfix) with ESMTPSA id 59B6A17353; Wed, 7 Aug 2019 20:27:49 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Subject: Re: svn commit: r350550 - head/share/mk To: rgrimes@freebsd.org Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org References: <201908072012.x77KCObt089132@gndrsh.dnsmgr.net> From: Pedro Giffuni Organization: FreeBSD Message-ID: Date: Wed, 7 Aug 2019 15:27:48 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <201908072012.x77KCObt089132@gndrsh.dnsmgr.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Aug 2019 20:27:50 -0000 On 07/08/2019 15:12, Rodney W. Grimes wrote: >> On 07/08/2019 11:00, John Baldwin wrote: >>> On 8/6/19 9:56 AM, Glen Barber wrote: >>>> On Sat, Aug 03, 2019 at 01:06:18AM +0000, John Baldwin wrote: >>>>> Author: jhb >>>>> Date: Sat Aug 3 01:06:17 2019 >>>>> New Revision: 350550 >>>>> URL: https://svnweb.freebsd.org/changeset/base/350550 >>>>> >>>>> Log: >>>>> Flip REPRODUCIBLE_BUILD back to off by default in head. >>>>> >>>>> Having the full uname output can be useful on head even with >>>>> unmodified trees or trees that newvers.sh fails to recognize as >>>>> modified. >>>>> >>>>> Reviewed by: emaste >>>>> Differential Revision: https://reviews.freebsd.org/D20895 >>>>> >>>> I would like to request this commit be reverted. While the original >>>> commit message to enable this knob stated the commit would be reverted >>>> after stable/12 branched, I have seen no public complaints about >>>> enabling REPRODUCIBLE_BUILD by default (and quite honestly, do not see >>>> the benefit of disabling it by default -- why wouldn't we want >>>> reproducibility?). >>>> >>>> To me, this feels like a step backwards, with no tangible benefit. >>>> Note, newvers.sh does properly detect a modified tree if it can find >>>> the VCS metadata directory (i.e., .git, .svn) -- I know this because >>>> I personally helped with it. >>>> >>>> In my opinion, those that want the non-reproducible metadata included in >>>> output from 'uname -a' should set WITHOUT_REPRODUCIBLE_BUILDS in their >>>> src.conf. Turning off a sane default for the benefit of what I suspect >>>> is likely a short list of use cases feels like a step in the wrong >>>> direction. >>> My arguments for flipping this in head (and head only) are that the data >>> provided in uname -a when this is disabled is useful for development, and >>> that in head we do tailor settings towards development (e.g. GENERIC in >>> head vs GENERIC in stable). >>> >>> The logic to handle modified trees has an inherent assumption that I think >>> is false, at least for my workflow and I suspect many others. I do builds >>> and tests of kernels on separate machines (VMs or bare metal) from where I >>> use VCS to manage sources so that a kernel crash doesn't toast my source >>> tree. The trees are then shared to the build/test machines via NFS. As >>> a result, the build/test machines are not always able to detect that the >>> tree is modified either because a subset of the checkout is exported via >>> NFS, or the VCS tool isn't installed on the build/test machines because >>> they are generally barebones systems with only a base installed. This >>> does mean that flipping the knob off doesn't provide all of the same info, >>> but it does provide the path, and the path matters because 'kgdb -n last' >>> uses it, and because if you use separate directories for separate projects >>> (e.g. git worktrees), then the path tells you which test kernel you booted. >>> (It is not uncommon for me to have several test projects in flight on a >>> single test machine for different branches.) >>> >>> In the original discussion on arch, we collectively recognized that >>> developer builds vs release builds were different and needed different >>> defaults. The compromise reached at that time was to depend on the VCS >>> to detect developer builds to choose the policy. What I have found is that >>> in practice for at least my workflow that doesn't actually work. I posit >>> that the majority of kernels built from head are developer builds, not >>> releases, and that the default should cater to that. You could also always >>> patch release.sh to set WITH_REPRODUCIBLE_BUILD in the environment which I >>> think would give a more accurate sense of when builds are releases or not. >>> >>> However, I will yield to whatever the consensus is. >> +1 keeping metadata in head. > I am conflicted on this one, and I think there is a reasonable argument > on both sides, but from what I have read here this appears to be mostly > the kernel that is at issue, loss of the meta data from newvers.sh in > the kernel is infact a PITA, even on stable or production release > systems. > > I propose a compromise, add 2 knobs: > WITHOUT_REPRODUCIBLE_KERNEL (aka get your metadata in uname) > WITH_REPRODUCIBLE_USERLAND (aka reproducible userland) > > WITH{,OUT}_REPRODUCIBLE_BUILD overrides both, for backwards compat, > and neither should be defined by default. Too complex IMHO. Either the system is reproducible or it isn't. > Somehow set WITH_REPRODUCIBLE_KERNEL for builds of GENERIC > for releases/snapshots, but do not ship the system with it > set (I can here a growl from Glen on this) Thus we build > a reproducible kernel and ship it with the system but if > the user builds a kernel it gets meta data to indicate it > is no longer a stock kernel. > FYI, upon finding I could not figure out what kernel I was running > after installing 12.0 release I turnd off REPRODUCIBLE on my kernel > build VM for 12.0. I do leave it on if I am building userland. > > Thoughts? Among other things, reproducible builds implies that pkg upgrades are smaller. I see it makes sense to make releases, and in fact -stable, completely reproducible. For -current I am fine with it not being reproducible, All just IMHO. Pedro.