From owner-freebsd-testing@freebsd.org Thu Nov 21 05:06:31 2019 Return-Path: Delivered-To: freebsd-testing@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F5D51AE248; Thu, 21 Nov 2019 05:06:31 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: from mail-lf1-x144.google.com (mail-lf1-x144.google.com [IPv6:2a00:1450:4864:20::144]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47JSFz33DNz3Mjx; Thu, 21 Nov 2019 05:06:30 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: by mail-lf1-x144.google.com with SMTP id q28so1497861lfp.2; Wed, 20 Nov 2019 21:06:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6ixIL2H4YfQ2ybK7ko2Jn/YhFLHm9ShV3G2nDKxniJ0=; b=GqPfRzm1Pk2Gz9CSAT73heDfB0zHMWAcarRZg/SBpHV/rcB2A201qy2XBmHjEVNeNM GX1dMDN5r/3o63X4jpcwkc/Yxsiajt1mwmqViFT7K2cfeZ5HlBWDishrpdvxcgxCht4S PhxM2nvWYDVof48Gq6+Ys6/s4P+6JsAIHMHhy5qTmr6GXHJLrzeJMCi92CW1HCGyJh2C Z7FzQYLVg5hdEIpQIbjxetVASunnz2PqWgRN0G8dakZ8kfBh/ISacif0ofixZfdH4Eo4 awaLy4lJYobJmAs45JMg9vXafIMy/EAC86IfdaMo+KQLHmD4cx0HKoH0MvO/xAeUQJkw P9Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6ixIL2H4YfQ2ybK7ko2Jn/YhFLHm9ShV3G2nDKxniJ0=; b=X2QqcBWJaRMQk7Y/sgu8MJtH9z6x0FMBUfhbECqQXSbikdOLKIiVNS+d4Tofimv8OC uXHPLoYfXWsEtRIZgz4IO5pBCYQNq8z9+eqRIyPsT2BvnwwYthWmbvqrHBq4FYYWnbiK n/TbFoWHkhN9HhNiPMg3qH2eKj+/yGDvhyXr/JI1kP5lq4yzWhWzijd/ZW8hGpEP9zAf oWxfeeggDLYW0yCWdtzwV9QV6T2v2Om9oTKGcG7O7mq3oaW/K4MRcHbZOjBujeJErwmL m+60UblL1Y8e5FzHh3fC3sKzh3XTGdKOC6Hn+rP2uKiejzdfuWQv8kLSLiQsEuYzRnmD dvzQ== X-Gm-Message-State: APjAAAWQacK7X04eHtwGV0um4bjK4Jg11OETbgXBoqwGs5aYg1ImaxTE 2W9ZV9uXunj8Q/iYb9Ggly/bCHbtf7KIhLhdboM3ZA6a X-Google-Smtp-Source: APXvYqyyr922y5adeTp6yfIJ5T+8THawRgMcSFT4zPfNBiZjcvse6yyqXmT4gqrSiInJ5TVCatAYYxs1Jd9ZUfMWd14= X-Received: by 2002:ac2:5222:: with SMTP id i2mr5724606lfl.138.1574312789002; Wed, 20 Nov 2019 21:06:29 -0800 (PST) MIME-Version: 1.0 References: <1ef80fd8-d695-010f-9dc8-a6b9a2a80f8e@FreeBSD.org> In-Reply-To: From: Matthew Macy Date: Wed, 20 Nov 2019 21:06:17 -0800 Message-ID: Subject: Re: Revisiting the ZFS test suite To: Alan Somers Cc: Andriy Gapon , freebsd-testing , freebsd-fs Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 47JSFz33DNz3Mjx X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 05:06:31 -0000 On Wed, Nov 20, 2019 at 8:18 AM Alan Somers wrote: > > On Wed, Nov 20, 2019 at 1:17 AM Andriy Gapon wrote: > > > On 04/10/2019 19:23, Alan Somers wrote: > > > My employment status changed last week, and I find myself once again > > > working on ZFS. It's hard to have confidence in any changes without a > > > solid test suite, so my main task right now is to fix the problems in > > > FreeBSD's ZFS test suite. Our most recent run had 64 failed tests and 99 > > > skipped ones; let's get those down to 0 and something small, > > respectively. > > > > > > https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/3880/testReport/ > > > > > > Would anybody be interested to review my changes to the test suite? It's > > > not terrible if I have to commit without review; I can't possibly break > > > anything but the test suite itself. > > > > Alan, > > > > just curious, did you have any plans to merge new tests and changes to > > existing > > tests from illumos or ZoL? > > I've imported quite a few code changes, many times there were test changes > > or > > new test, but I never took time to merge them. > > > > > > -- > > Andriy Gapon > > > > We'll need to do it sooner or later, but it's going to take a large > effort. I can't justify that much effort right now. And it will require > changing ZoL's test suite, too, not just ours. We'll need to convince them > to use ATF, for example. I highly doubt that Brian cares one way or the other. They currently have a large group of ksh tests run by a bespoke python test driver. The issue is that someone would need to convert the test suite and CI to use ATF. The OpenZFS CI will be the first stop for any changes, so adding / fixing tests in the OpenZFS test suite would have the most long term impact.