From owner-freebsd-fs@freebsd.org Thu Nov 21 05:14:40 2019 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 80A8F1AE62C for ; Thu, 21 Nov 2019 05:14:40 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47JSRN2mVwz3N1d for ; Thu, 21 Nov 2019 05:14:40 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: mmacy) by smtp.freebsd.org (Postfix) with ESMTPSA id 07779446B for ; Thu, 21 Nov 2019 05:14:40 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mail-lf1-f54.google.com with SMTP id l28so1517689lfj.1 for ; Wed, 20 Nov 2019 21:14:39 -0800 (PST) X-Gm-Message-State: APjAAAUIXZX45tV5ZXm+FHpkKf/lzGHd43ctV2pT8zid8TIJQ6Kmg+f1 CTXpVVzGb7A6ErBRsB0jn531z8Q9joVWXhgv0tU= X-Google-Smtp-Source: APXvYqxxq8kpovY4YWaj5pfWodKhvZ5H7YaDVqFFq0BcIeBm6hJLhYUhf4QAYINR/S1fAXu2UNqEtCYpBp9OOpAwI/M= X-Received: by 2002:ac2:5222:: with SMTP id i2mr5750848lfl.138.1574313278550; Wed, 20 Nov 2019 21:14:38 -0800 (PST) MIME-Version: 1.0 References: <1ef80fd8-d695-010f-9dc8-a6b9a2a80f8e@FreeBSD.org> In-Reply-To: From: Matthew Macy Date: Wed, 20 Nov 2019 21:14:27 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Revisiting the ZFS test suite To: freebsd-fs Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 05:14:40 -0000 On Wed, Nov 20, 2019 at 8:18 AM Alan Somers wrote: > > On Wed, Nov 20, 2019 at 1:17 AM Andriy Gapon wrote: > > > On 04/10/2019 19:23, Alan Somers wrote: > > > My employment status changed last week, and I find myself once again > > > working on ZFS. It's hard to have confidence in any changes without a > > > solid test suite, so my main task right now is to fix the problems in > > > FreeBSD's ZFS test suite. Our most recent run had 64 failed tests and 99 > > > skipped ones; let's get those down to 0 and something small, > > respectively. > > > > > > https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/3880/testReport/ > > > > > > Would anybody be interested to review my changes to the test suite? It's > > > not terrible if I have to commit without review; I can't possibly break > > > anything but the test suite itself. > > > > Alan, > > > > just curious, did you have any plans to merge new tests and changes to > > existing > > tests from illumos or ZoL? > > I've imported quite a few code changes, many times there were test changes > > or > > new test, but I never took time to merge them. > > > > > > -- > > Andriy Gapon > > > > We'll need to do it sooner or later, but it's going to take a large > effort. I can't justify that much effort right now. And it will require > changing ZoL's test suite, too, not just ours. We'll need to convince them > to use ATF, for example. I highly doubt that Brian cares one way or the other. They currently have a large group of ksh tests run by a bespoke python test driver. The issue is that someone would need to convert the test suite and CI to use ATF. The OpenZFS CI will be the first stop for any changes, so adding / fixing tests in the OpenZFS test suite would have the most long term impact.