From owner-freebsd-hackers@freebsd.org Fri Aug 30 01:35:50 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E25DE5D91; Fri, 30 Aug 2019 01:35:50 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46KMW846DWz4VHg; Fri, 30 Aug 2019 01:35:48 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id 3VpHi2s32sAGk3VpIitl9V; Thu, 29 Aug 2019 19:35:46 -0600 X-Authority-Analysis: v=2.3 cv=WeVylHpX c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=8nJEP1OIZ-IA:10 a=FmdZ9Uzk2mMA:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=duxrOwH0z9cosxiZrWsA:9 a=wPNLvfGTeEIA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id 98DC4725; Thu, 29 Aug 2019 18:35:42 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x7U1Zg6M067244; Thu, 29 Aug 2019 18:35:42 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x7U1Zfh7067241; Thu, 29 Aug 2019 18:35:41 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201908300135.x7U1Zfh7067241@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: "Kristof Provost" cc: araujo@freebsd.org, Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , Ian Lepore , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy In-reply-to: References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> Comments: In-reply-to "Kristof Provost" message dated "Thu, 29 Aug 2019 17:09:32 +0200." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Thu, 29 Aug 2019 18:35:41 -0700 X-CMAE-Envelope: MS4wfCbf9EUat0hnmBN/VxddWxCz2VbiMs9RNbe9T2Rpl2FPyFF9yIjEFS3WPs2NbFDiNbEojxO8rUhOpBG8drXgSquMESIPMpGSN5Ofhq2zAo9nYNFmh6Le K6V3XJS434syjkd5snUoXy90o3NGoEfEgGlEtUyDhAZ570GTu82U0t1EwyEp69EDXkM1J0zD6+qaP+772Sxu1KI4OFfyZ+7+A1iQLp4ipWecHEHmuZsJ4B5X MWPI73lCwlqC3H7dKfzeMUa/BFZsW1/QrMlvOXr0RUdlCDGq+ZYkLglYuhoQPVrtPn2HHCgxd25MdU6MHnvfTw== X-Rspamd-Queue-Id: 46KMW846DWz4VHg X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.134.9) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-4.95 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.95)[-0.952,0]; RCVD_IN_DNSWL_NONE(0.00)[9.134.59.64.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[7]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-2.40)[ip: (-6.35), ipnet: 64.59.128.0/20(-3.13), asn: 6327(-2.43), country: CA(-0.09)]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 01:35:50 -0000 In message , "Kristof Provost " writes: > On 29 Aug 2019, at 17:01, Marcelo Araujo wrote: > > Em qui, 29 de ago de 2019 às 22:54, Kristof Provost > > escreveu: > > > >> On 29 Aug 2019, at 16:42, Ian Lepore wrote: > >>> (And I don't think breaking a test counts as > >>> breaking the build.) > >>> > >> I fundamentally disagree on this point. A test failure is, just like > >> a > >> compiler warning, a precious gift that should not be ignored. > >> The more distance (both in terms of time, and in terms of the people > >> involved) there is between a bug being introduced and it being > >> detected > >> the harder it is to fix it. Test accelerate detection of bugs. If we > >> do > >> not take test failures seriously (i.e. as an indication something is > >> wrong and should be fixed) the tests will inevitable become useless > >> in > >> one of two ways: we’ll either disable failing tests (which is what > >> we > >> tend to do now) reducing test coverage or we’ll have a test suite > >> with > >> many failures in it, which makes it useless as well. (As with > >> compiler > >> warnings, the best way to keep them under control is to consider them > >> to > >> be fatal errors.) > >> > > > > Could you elaborate where is the "fundamentally" you disagree? Where > > is the > > fundament? You guys are introducing something new, yes everybody knows > > about test, it is year 2019, but nobody can come with new rules tha in > > hours we gonna revert if you "dare to don't fix it". Sorry, this is > > not how > > people test software and fix it. > > > I do think that breaking a test breaks the build. Something used to work > and now it doesn’t. That’s breakage, even if it’s not as total as > it not compiling any more. I agree. A broken test is an indication of a regression. If a test is not a regression, then it is not a valid test. Having said that, 48 hours is a little draconian. Maybe 48 hours if no one steps up to the plate but if it's actively being worked on with end in sight I'd give the person working on it a little space. What about weekends? I notice commits tend to slow down during weekends. I suspect people who work on FreeBSD at $COMPANY for a living treat it as a job, which it is. However this should be considered. OTOH (arguing against myself to point out another consideration), it is foolhardy to commit a new feature just before a weekend and go camping (or be unreachable for whatever reason). A committer should be reachable N hours after a significant commit. Maybe 48 hours is justifiable then. (At $JOB we don't allow significant changes prior to vacation or even a flex day unless there is someone to cover for the person having done the work.) My point is, we pick a number like 48. What's the rationale? What is the desired result? Can the desired result be accomplished without a 48 hour auto-revert rule? Should it be 48 hours during the week and 72 over weekends? -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few.