Date: Thu, 29 Aug 2019 16:54:26 +0200 From: "Kristof Provost" <kp@FreeBSD.org> To: "Ian Lepore" <ian@freebsd.org> Cc: "Konstantin Belousov" <kostikbel@gmail.com>, "Li-Wen Hsu" <lwhsu@freebsd.org>, "FreeBSD Hackers" <freebsd-hackers@freebsd.org>, fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <A837EF78-DC69-4B52-A7D9-0363302A48FA@FreeBSD.org> In-Reply-To: <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org> References: <CAKBkRUwKKPKwRvUs00ja0%2BG9vCBB1pKhv6zBS-F-hb=pqMzSxQ@mail.gmail.com> <20190829114057.GZ71821@kib.kiev.ua> <28934eb780342605090bf365ac3a2e0d522256f5.camel@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 29 Aug 2019, at 16:42, Ian Lepore wrote: > (And I don't think breaking a test counts as > breaking the build.) > I fundamentally disagree on this point. A test failure is, just like a compiler warning, a precious gift that should not be ignored. The more distance (both in terms of time, and in terms of the people involved) there is between a bug being introduced and it being detected the harder it is to fix it. Test accelerate detection of bugs. If we do not take test failures seriously (i.e. as an indication something is wrong and should be fixed) the tests will inevitable become useless in one of two ways: we’ll either disable failing tests (which is what we tend to do now) reducing test coverage or we’ll have a test suite with many failures in it, which makes it useless as well. (As with compiler warnings, the best way to keep them under control is to consider them to be fatal errors.) In either scenario we end up reducing test coverage, which means we’re going to push more bugs towards users. > I totally agree. This is an overly-bureaucratic solution in search of > a problem. > > If this needs to be addressed at all (and I'm not sure it does), then > another sentence or two in bullet item 10 in section 18.1 [*] of the > committer's guide should be enough. And even then it needn't be > overly-formal and should just mention that if a commit does break the > build the committer is expected to be responsive to that problem and > the commit might get reverted if they're unresponsive. I don't think > we need schedules. > I do feel that’s a better argument. We’ve always had a policy of reverting on request (AIUI), so this is more or less trying to be a strong restatement of that, more than a fundamental shift in policy. Best regards, Kristof
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A837EF78-DC69-4B52-A7D9-0363302A48FA>