From owner-freebsd-hackers@freebsd.org Sat Aug 31 19:09:53 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E481D95EE for ; Sat, 31 Aug 2019 19:09:53 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46LQrw2yybz3JJY for ; Sat, 31 Aug 2019 19:09:52 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72e.google.com with SMTP id u190so9170867qkh.5 for ; Sat, 31 Aug 2019 12:09:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wuqIQ6VjJLqAmwFu3UeQjakHZH7aJZq7NTKf+mOEtTM=; b=l61dqkLO8zVFZ10+f7CDeWliXqe0iQLbGAnw6OnVhZOaMnHG47YBV1UnQ2Rw5fRPbB OeJFbH0FUR3ePvwjQd3tka9K3rO1r3c1iyzaEmyjhnYRErBT6wv65koPii8UdG9lLQyx 5Z+ZunLunX+lkH1MXVIUET25dDqEUhgcNrn+dPUldyr2EEVzfM4fYuTCB0jqeZc8oifu m28so9IIWE3okj0Rvj+/x6QhrLIS4X3RpVEfRyI6fB7i4QS3GNmK0Ub4REnnUGDE3uRC FGCggOz1c/DpD0+E8aGu+Ohu0+trHP3SPmhr/nfQIjexTf9oS+krBPYWCktekPMNUbR1 a3yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wuqIQ6VjJLqAmwFu3UeQjakHZH7aJZq7NTKf+mOEtTM=; b=nVP7Uuhmc0F58nOJEOOCQXMOXgTPMzLREGA9AHhmxgPq54bnruCHPAP7Rw3i4Tvyx9 LMLhbCNzdHmqLGc4KTyivGo18JCbE3lIbMCsKHOl1bZCLxh8T8SZqDYYHDCYp5diKHqc 020Yuu/GYU+GEYKjvrABAhJRv8LvLPvZsYlICrMKMfBSlLdQu2uRuttzobTfrbetszP6 dXA3OTVJxWil47IkhUh17R9ZtYKtMWmohDZfwp0IbBciGzdfqYZD6huBDom6Y5CpKs2e ZGYVVtxRTIAgFbnLEptdqZFJeEJUFKHIlHxuzP0ZnWzjrIyZCRaxMfo4qKqVxqkgqEm3 WODg== X-Gm-Message-State: APjAAAXsj6Wq2SpY1lxtTPnKnjJ3fh6M2tzUKyLnEqLXt2cV12z6f7Y3 e9Ri3XnXU6H+R1IA74QIt4CylyeoTC6zk8Jt6sWdvg== X-Google-Smtp-Source: APXvYqy0i3b2EuSlq92DA4ISuvDNoUY1nkICcebV1jzGWzapPJyw3Sm6pVFG67D/oSb/ECBBCLiJRNBkDEDBfRVGtvI= X-Received: by 2002:a37:30f:: with SMTP id 15mr12747587qkd.240.1567278590149; Sat, 31 Aug 2019 12:09:50 -0700 (PDT) MIME-Version: 1.0 References: <20190829114057.GZ71821@kib.kiev.ua> <20190830065534.GC71821@kib.kiev.ua> <73111.1567263258@kaos.jnpr.net> In-Reply-To: From: Warner Losh Date: Sat, 31 Aug 2019 13:09:38 -0600 Message-ID: Subject: Re: FCP 20190401-ci_policy: CI policy To: Ed Maste Cc: FreeBSD Hackers , Li-Wen Hsu , fcp@freebsd.org X-Rspamd-Queue-Id: 46LQrw2yybz3JJY X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=l61dqkLO; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72e) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-5.90 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[e.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.91)[ip: (-9.34), ipnet: 2607:f8b0::/32(-2.83), asn: 15169(-2.31), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2019 19:09:53 -0000 On Sat, Aug 31, 2019, 12:55 PM Ed Maste wrote: > On Sat, 31 Aug 2019 at 13:01, Warner Losh wrote: > > > > (7) Breaking tests is a problem, but our tests need to evolve because we > > have too high a rate of false positives. > > I'm not sure there's consensus on false positives. In particular, > Li-Wen has spent an incredible amount of time over more than six > months tracking down test failures and submitting PRs. Flaky tests and > false positives have generally been addressed by now (by submitting a > PR and disabling or xfailing the test). > Fair enough. Do we have recent data that can distinguish between false positives or bad perception? Warner >