From owner-freebsd-toolchain@freebsd.org  Sat Nov  4 17:23:20 2017
Return-Path: <owner-freebsd-toolchain@freebsd.org>
Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F148E50D83;
 Sat,  4 Nov 2017 17:23:20 +0000 (UTC)
 (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 3BD42767C3;
 Sat,  4 Nov 2017 17:23:19 +0000 (UTC)
 (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
 by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id vA4HNABk001400;
 Sat, 4 Nov 2017 10:23:10 -0700 (PDT)
 (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
 by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id vA4HNANJ001399;
 Sat, 4 Nov 2017 10:23:10 -0700 (PDT) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201711041723.vA4HNANJ001399@pdx.rh.CN85.dnsmgr.net>
Subject: Re: Head build unsafe for /etc today
In-Reply-To: <ec4f16fc-eeb3-d778-94f6-6b564b0e6987@FreeBSD.org>
To: Bryan Drewery <bdrewery@freebsd.org>
Date: Sat, 4 Nov 2017 10:23:10 -0700 (PDT)
CC: Warner Losh <imp@bsdimp.com>,
 Steve Kargl <sgk@troutmask.apl.washington.edu>,
 freebsd-hackers <freebsd-hackers@freebsd.org>,
 FreeBSD Toolchain <freebsd-toolchain@freebsd.org>,
 FreeBSD Current <freebsd-current@freebsd.org>
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-toolchain@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Maintenance of FreeBSD's integrated toolchain
 <freebsd-toolchain.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-toolchain>, 
 <mailto:freebsd-toolchain-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-toolchain/>
List-Post: <mailto:freebsd-toolchain@freebsd.org>
List-Help: <mailto:freebsd-toolchain-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain>, 
 <mailto:freebsd-toolchain-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 04 Nov 2017 17:23:20 -0000

> On 11/2/2017 8:58 PM, Warner Losh wrote:
> > FreeBSD has grown too big to test every possible thing before you commit.

I would say that it has always been too big to test every possible
thing before a commit.  Breakage is just going to happen, we make
great efforts to minimize it, but like all risks sooner or later
your gona have a failure.

> 
> The build itself is massive.  I usually forget about release/ and the
> new 'make packages', external toolchain, "old style" kernel builds, etc.
Good starting list for "make build-regresion" ?

> 
> Steve's concerns have validity.   I do think it's time we have an
> automated suite to test most build cases for things like bmake upgrades
> or other high risk changes like META_MODE.
> 
> 
> I'll think about this and add to my list of things to implement.

I would even go so far as to say this may be what we should be
running in (a) Jenkins.  Or perhaps a deeper exp-run?

-- 
Rod Grimes                                                 rgrimes@freebsd.org