From owner-freebsd-fcp@freebsd.org Thu Aug 29 14:44:51 2019 Return-Path: Delivered-To: freebsd-fcp@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F315FD7BCE for ; Thu, 29 Aug 2019 14:44:51 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 46K5433xWQz3GZb for ; Thu, 29 Aug 2019 14:44:51 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 87177D7BC9; Thu, 29 Aug 2019 14:44:51 +0000 (UTC) Delivered-To: fcp@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 86BE9D7BC8; Thu, 29 Aug 2019 14:44:51 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-lf1-x143.google.com (mail-lf1-x143.google.com [IPv6:2a00:1450:4864:20::143]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46K54331jvz3GZZ; Thu, 29 Aug 2019 14:44:51 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: by mail-lf1-x143.google.com with SMTP id r5so2760170lfc.3; Thu, 29 Aug 2019 07:44:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=vi6RluLvB3RVtOqMSwTBcCPHr3wMB6PjNtXI4mFu7xg=; b=CrqzX0hxzJTSQsxZAHboaUU8u3dqR11K6LpJ9jwdq73F9oUhj4CyNeuTNuEnZPnqRK VNrNsNiO9Qm25nw62K00nG3f3R2+b+Aq32tAYNPiRKyQq4WtEIvB2yb509mNJp2gyY4L B+djE2yXtmqcHFZmghtEOGO7kkbIQmhLMoNhaPEXuhMIydqrXASMw5eB9W3jMqZdGIAi w/UrGMPZpteOqaSkfbmWSG+iHfgqaY9HycWcvgv1ryyyGhB1LPEb2j2+uycQD68pOvA0 hyZcZ5JAUaSK0q+YdTafFjgB41LPbHDR3J8jU369ygq1BVxmdKsk+rw1CymZLtKWykru KrYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=vi6RluLvB3RVtOqMSwTBcCPHr3wMB6PjNtXI4mFu7xg=; b=dAtz8ut1UbkjdaNDJROUY/QJezK9em42OwfEJSWYxW8k1iqYNU+shgfM9zkl3TWX9K n65hlFVIhevM0BvB6/b0D3BPFpBorIMSXp4hDg0/r8DhmmcxPewFj63xzO9+gaFq5rrN 60bcRhS2IETfrb9zTR9sHQprjjkQYYzxX688nE5Q3kWJB1CO4xeo/SGMCF7GH+OfNWwm NVUMq+PvywV1gHrbG29/N267IeA1k4xdHQrBAzK+YERaWjukgTDfuXL9Cx/eCBhjN5mB KVF9+mqUG8B4NRzimvqYzYnVcWIqkE5vJ2gRIjwHgz7d9m7ndvbtBORzb3cWoaW5nmB7 wBzQ== X-Gm-Message-State: APjAAAVHgt6kUJT5rxAtZ3mIe+GU145Df2W/VmLKyVynYR4fBGpEP4gn mJtFu/G0+gWVJk255DejNPVboxZ97yF8cugEkcdlFeJ7DtU= X-Google-Smtp-Source: APXvYqyms0ZjwLVbkRFOQGGIENLSitckxYGyqhD4idJWirjrbzZH5VtzzWPtEQ2ybke4wT7q0ovg6JMYc79rGdWApz0= X-Received: by 2002:a19:674d:: with SMTP id e13mr6463979lfj.176.1567089889352; Thu, 29 Aug 2019 07:44:49 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> In-Reply-To: <412537DD-D98F-4B92-85F5-CB93CF33F281@FreeBSD.org> Reply-To: araujo@freebsd.org From: Marcelo Araujo Date: Thu, 29 Aug 2019 22:44:38 +0800 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Kristof Provost Cc: Konstantin Belousov , FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46K54331jvz3GZZ X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.983,0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fcp@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FreeBSD Community Proposals List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 14:44:52 -0000 Em qui, 29 de ago de 2019 =C3=A0s 20:03, Kristof Provost escreveu: > On 29 Aug 2019, at 13:40, Konstantin Belousov wrote: > > On Wed, Aug 28, 2019 at 12:29:58PM +0800, Li-Wen Hsu wrote: > >> It seems I was doing wrong that just changed the content of this FCP > >> to "feedback", but did not send to the right mailing lists. > >> > >> So I would like to make an announcement that the FCP > >> 20190401-ci_policy "CI policy": > >> > >> https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md > >> > >> is officially in "feedback" state to hopefully receive more comments > >> and suggestions, then we can move on for the next FCP state. > > > > What problem does the document tries to solve ? Or rather, do we > > really > > have the problem that it claims to solve ? > > > There are, somewhat regularly, commits which break functionality, or at > the very least tests. > The main objective of this policy proposal is to try to improve overall > code quality by encouraging and empowering all committers to investigate > and fix test failures. > Sure, but it doesn't sounds like you are empowering people to works in their spare time for the project, quite the opposite. Could you show something more feasible than "somewhat" regularly breaks functionality? Most of the time personally I'm running HEAD, built daily and I can't see this. > > >> From my experience, normal peer pressure is enough to get things > >> fixed > > quickly when it is possible to fix them quickly. If there is something > > more non-trivial, esp. in the tests and not the build, I am sure that > > a rule allowing anybody to do blind revert is much more harmful than > > having a test broken. > > > > More, I know that tests are of very low quality, which means that > > brokeness of the tests is not an indicator of anything until root > > cause > > is identified. > > > I=E2=80=99m not sure I agree with the characterisation that the tests are= of > low quality. My own experience with the pf tests is that they test a > large section of the network stack and firewall code. They=E2=80=99ve > identified several very really issues (both pre- and post commit on the > epoch-isatin of the network stack, for example, as well as a fairly > important issue with IPv6 reassembly). > It=E2=80=99s certainly true that the pf tests often reveal issues that ar= e not > in pf but in other code. I wouldn=E2=80=99t agree that this is a sign of = low > quality tests, but instead I consider it a sign that we don=E2=80=99t hav= e > enough tests for the network stack itself. > All the tests and CI are pretty new, or at least it is something new for most of everybody and people are getting used to that. I stop here, I would elaborate more, but after Ian's email, I think I don't need anymore. > > > Can we rely on the common sense of developers until there is indeed > > the > > visible problem ? > > > I don=E2=80=99t want to suggest that people simply don=E2=80=99t care abo= ut test > failures, because that=E2=80=99s clearly not true. > > On the other hand, I do think we can do better. There are at least two > open problem in the network stack that I currently can=E2=80=99t get anyo= ne to > look at, and where I personally do not have sufficient context (or time) > to fix them myself. (#239380, #238870). > > This proposal isn=E2=80=99t a silver bullet, I don=E2=80=99t think there = is such a > thing, but I do believe that elevating the visibility and importance of > test failures can help us improve overall quality. > > Best regards, > Kristof > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > --=20 --=20 Marcelo Araujo (__)araujo@FreeBSD.org \\\'',)http://www.FreeBSD.org \/ \ ^ Power To Server. .\. /_)