From owner-freebsd-current@FreeBSD.ORG Sat Jun 7 01:29:02 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D1B048D6; Sat, 7 Jun 2014 01:29:02 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A03632623; Sat, 7 Jun 2014 01:29:02 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id z10so3090248pdj.12 for ; Fri, 06 Jun 2014 18:29:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:reply-to:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=vQwMogn6NUfStlpOeGsi0d5+adZ3NtLjv50Aavqtvq0=; b=02gFJMe5Asmq3jICNND6VKge50MNRvON0MnrCP0KWQQZMPV/gRz6GjDOrHdZc6oB4W AqKT1ckBR3rmdo+ykWFYlEBX+5HN5OgXEiynjbrNV+Hb/b1Sa6uCFT3McMOTnPugX7dV sETp8l6nQYTj5UfUyIYq+ryU3/xiKBIBjejOiuCfnX4AlusyiomRBF3c8mjgzuhu8JZ0 /nxY1UYOFxX6lg0gtAyj6pO2TPcQbq5twK5XGHs3RFovUHaZd3tuJH63sib5TGHXvubh yxFAxGhzym7BiHKclwe5ZFJfPlqunRNOiasL9Utyo09gZIbxulI491W7wLFinX0iX1C7 mbmQ== X-Received: by 10.68.94.66 with SMTP id da2mr6545671pbb.106.1402104542193; Fri, 06 Jun 2014 18:29:02 -0700 (PDT) Received: from [192.168.1.7] (ppp59-167-128-11.static.internode.on.net. [59.167.128.11]) by mx.google.com with ESMTPSA id fv2sm41622685pbd.11.2014.06.06.18.29.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Jun 2014 18:29:01 -0700 (PDT) Sender: Kubilay Kocak Message-ID: <53926ACD.2080305@FreeBSD.org> Date: Sat, 07 Jun 2014 11:28:45 +1000 From: Kubilay Kocak Reply-To: koobs@FreeBSD.org User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Thunderbird/30.0 MIME-Version: 1.0 To: Julio Merino , freebsd-current@freebsd.org Subject: Re: Turning TESTS on by default References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 01:29:03 -0000 On 7/06/2014 5:14 AM, Julio Merino wrote: > Hello all, > > > TL;DR > ----- > > I plan to turn the TESTS src.conf knob ON by default on Tuesday once I > have been able to perform enough sanity-checks of the build and all of > them pass. > > The impact of this is that the FreeBSD Test Suite (see tests(7)) will > be built and installed by default under /usr/tests/ along with the > private atf libraries and binaries. There should be no other changes > and this should be transparent to everyone. > > If this happens to break the world in any way, we can trivially roll > the change back to fix the fallout. > > > Some details > ------------ > > TESTS was never intended to be disabled by default. However, the > original patches that were committed months ago related to this > feature broke the build and the easiest way out (instead of reverting > the commits) was to set the knob to disabled. Unfortunately, it stayed > that way even after the discovered problems were fixed. > > I am confident enough now that we have ironed out all major issues > that this might introduce, so it is about time to enable TESTS by > default again in HEAD. > > The benefits of this are that 1) we allow end users (especially > consumers of binary releases!) to run the tests out of the box, as it > has always been intended; and 2) we will be able to run the official > release builds through testing via Jenkins, instead of having to issue > custom builds. > > Actual change: https://phabric.freebsd.org/D188 > > > I will update this thread when the change is committed and/or with any updates. > > Please let me know if I'm missing anything. > > Cheers > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > Julio, Awesome! I look forward to automated review lint checks that test for addition of tests, coverage++ and issue ID's in a changeset :] On a related note, how challenging might it be to generate and expose coverage metrics? This is not to say they are a perfect measure of anything in particular, but ought to provide us a good high level idea of important candidate areas that would benefit from coverage and don't currently. -- Koobs