Date: Thu, 29 Aug 2019 22:44:38 +0800 From: Marcelo Araujo <araujobsdport@gmail.com> To: Kristof Provost <kp@freebsd.org> Cc: Konstantin Belousov <kostikbel@gmail.com>, FreeBSD Hackers <freebsd-hackers@freebsd.org>, Li-Wen Hsu <lwhsu@freebsd.org>, fcp@freebsd.org Subject: Re: FCP 20190401-ci_policy: CI policy Message-ID: <CAOfEmZgc6n48cPN--5VA2F_JQy04xhH7V%2BJNe4LmmC9jK7_ohw@mail.gmail.com> In-Reply-To: <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> References: <CAKBkRUwKKPKwRvUs00ja0%2BG9vCBB1pKhv6zBS-F-hb=pqMzSxQ@mail.gmail.com> <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Em qui, 29 de ago de 2019 =C3=A0s 20:03, Kristof Provost <kp@freebsd.org> escreveu: > On 29 Aug 2019, at 13:40, Konstantin Belousov wrote: > > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: > >> It seems I was doing wrong that just changed the content of this FCP > >> to "feedback", but did not send to the right mailing lists. > >> > >> So I would like to make an announcement that the FCP > >> 20190401-ci_policy "CI policy": > >> > >> https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md > >> > >> is officially in "feedback" state to hopefully receive more comments > >> and suggestions, then we can move on for the next FCP state. > > > > What problem does the document tries to solve ? Or rather, do we > > really > > have the problem that it claims to solve ? > > > There are, somewhat regularly, commits which break functionality, or at > the very least tests. > The main objective of this policy proposal is to try to improve overall > code quality by encouraging and empowering all committers to investigate > and fix test failures. > Sure, but it doesn't sounds like you are empowering people to works in their spare time for the project, quite the opposite. Could you show something more feasible than "somewhat" regularly breaks functionality? Most of the time personally I'm running HEAD, built daily and I can't see this. > > >> From my experience, normal peer pressure is enough to get things > >> fixed > > quickly when it is possible to fix them quickly. If there is something > > more non-trivial, esp. in the tests and not the build, I am sure that > > a rule allowing anybody to do blind revert is much more harmful than > > having a test broken. > > > > More, I know that tests are of very low quality, which means that > > brokeness of the tests is not an indicator of anything until root > > cause > > is identified. > > > I=E2=80=99m not sure I agree with the characterisation that the tests are= of > low quality. My own experience with the pf tests is that they test a > large section of the network stack and firewall code. They=E2=80=99ve > identified several very really issues (both pre- and post commit on the > epoch-isatin of the network stack, for example, as well as a fairly > important issue with IPv6 reassembly). > It=E2=80=99s certainly true that the pf tests often reveal issues that ar= e not > in pf but in other code. I wouldn=E2=80=99t agree that this is a sign of = low > quality tests, but instead I consider it a sign that we don=E2=80=99t hav= e > enough tests for the network stack itself. > All the tests and CI are pretty new, or at least it is something new for most of everybody and people are getting used to that. I stop here, I would elaborate more, but after Ian's email, I think I don't need anymore. > > > Can we rely on the common sense of developers until there is indeed > > the > > visible problem ? > > > I don=E2=80=99t want to suggest that people simply don=E2=80=99t care abo= ut test > failures, because that=E2=80=99s clearly not true. > > On the other hand, I do think we can do better. There are at least two > open problem in the network stack that I currently can=E2=80=99t get anyo= ne to > look at, and where I personally do not have sufficient context (or time) > to fix them myself. (#239380, #238870). > > This proposal isn=E2=80=99t a silver bullet, I don=E2=80=99t think there = is such a > thing, but I do believe that elevating the visibility and importance of > test failures can help us improve overall quality. > > Best regards, > Kristof > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org <http://www.freebsd.org/>; \/ \ ^ Power To Server. .\. /_)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOfEmZgc6n48cPN--5VA2F_JQy04xhH7V%2BJNe4LmmC9jK7_ohw>