From owner-freebsd-hackers@freebsd.org Thu Aug 29 15:09:34 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E8166D8C5E; Thu, 29 Aug 2019 15:09:34 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K5cZ5Yt8z3Jpg; Thu, 29 Aug 2019 15:09:34 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 76A2D1BFEA; Thu, 29 Aug 2019 15:09:34 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.193] (ptr-8rh08k12jlyqauo4swr.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240e:402:c59e:f85c:1ed8:db5b]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id EEAC138603; Thu, 29 Aug 2019 17:09:32 +0200 (CEST) From: "Kristof Provost" To: araujo@freebsd.org Cc: "Ian Lepore" , "Konstantin Belousov" , "FreeBSD Hackers" , "Li-Wen Hsu" , fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Date: Thu, 29 Aug 2019 17:09:32 +0200 X-Mailer: MailMate (2.0BETAr6137) Message-ID: In-Reply-To: References: <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 15:09:35 -0000 On 29 Aug 2019, at 17:01, Marcelo Araujo wrote: > Em qui, 29 de ago de 2019 =C3=A0s 22:54, Kristof Provost > escreveu: > >> On 29 Aug 2019, at 16:42, Ian Lepore wrote: >>> (And I don't think breaking a test counts as >>> breaking the build.) >>> >> I fundamentally disagree on this point. A test failure is, just like = >> a >> compiler warning, a precious gift that should not be ignored. >> The more distance (both in terms of time, and in terms of the people >> involved) there is between a bug being introduced and it being = >> detected >> the harder it is to fix it. Test accelerate detection of bugs. If we = >> do >> not take test failures seriously (i.e. as an indication something is >> wrong and should be fixed) the tests will inevitable become useless = >> in >> one of two ways: we=E2=80=99ll either disable failing tests (which is = what = >> we >> tend to do now) reducing test coverage or we=E2=80=99ll have a test su= ite = >> with >> many failures in it, which makes it useless as well. (As with = >> compiler >> warnings, the best way to keep them under control is to consider them = >> to >> be fatal errors.) >> > > Could you elaborate where is the "fundamentally" you disagree? Where = > is the > fundament? You guys are introducing something new, yes everybody knows > about test, it is year 2019, but nobody can come with new rules tha in > hours we gonna revert if you "dare to don't fix it". Sorry, this is = > not how > people test software and fix it. > I do think that breaking a test breaks the build. Something used to work = and now it doesn=E2=80=99t. That=E2=80=99s breakage, even if it=E2=80=99s= not as total as = it not compiling any more. >> In either scenario we end up reducing test coverage, which means = >> we=E2=80=99re >> going to push more bugs towards users. >> >>> I totally agree. This is an overly-bureaucratic solution in search = >>> of >>> a problem. >>> >>> If this needs to be addressed at all (and I'm not sure it does), = >>> then >>> another sentence or two in bullet item 10 in section 18.1 [*] of the >>> committer's guide should be enough. And even then it needn't be >>> overly-formal and should just mention that if a commit does break = >>> the >>> build the committer is expected to be responsive to that problem and >>> the commit might get reverted if they're unresponsive. I don't = >>> think >>> we need schedules. >>> >> I do feel that=E2=80=99s a better argument. We=E2=80=99ve always had a= policy of >> reverting on request (AIUI), so this is more or less trying to be a >> strong restatement of that, more than a fundamental shift in policy. >> > > We don't have a policy to revert commit, actually revert commit is > something bad, it is kind of punishment, I have been there, nobody = > wants to > be there. Stop to push this non-sense argument. > That=E2=80=99s how I=E2=80=99ve interpreted =E2=80=9911. Developer Relati= ons=E2=80=99 in in = https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/art= icle.html#conventions = (Specifically, =E2=80=9C If a commit does results in controversy erupti= ng, = it may be advisable to consider backing the change out again until the = matter is settled.=E2=80=9D) I understand that it=E2=80=99s not fun to see changes reverted, and it=E2= =80=99s = certainly not the intention to make that the preferred solution. = That=E2=80=99s why the FCP discusses adds waiting periods and discussion = with = the committer. Best regards, Kristof