Date: Thu, 29 Aug 2019 19:35:57 -0400 From: Ed Maste <emaste@freebsd.org> To: Konstantin Belousov <kostikbel@gmail.com> Cc: Li-Wen Hsu <lwhsu@freebsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org>, fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <CAPyFy2BNrFfZ1PHaLesW%2Bu7YmbhC7mtiZ%2BBsYrqYE0J7KdagaA@mail.gmail.com> In-Reply-To: <20190829114057.GZ71821@kib.kiev.ua> References: <CAKBkRUwKKPKwRvUs00ja0%2BG9vCBB1pKhv6zBS-F-hb=pqMzSxQ@mail.gmail.com> <20190829114057.GZ71821@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 29 Aug 2019 at 07:41, Konstantin Belousov <kostikbel@gmail.com> wrote: > > More, I know that tests are of very low quality, which means that > brokeness of the tests is not an indicator of anything until root cause > is identified. "Low quality" needs clarification here. I can think of many attributes of a test that might lead someone to claim tests are low quality: - The test result is not consistent (e.g., a "flaky test") - The test does not actually test what it claims to - The test does as it claims, but there is no value in the result - Test coverage overall is insufficient (i.e., not an issue with a specific test) - The test has excessive requirements (run time, memory usage, etc.) - The test is difficult to maintain
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPyFy2BNrFfZ1PHaLesW%2Bu7YmbhC7mtiZ%2BBsYrqYE0J7KdagaA>