From owner-freebsd-testing@freebsd.org Sun Oct 6 22:29:42 2019 Return-Path: Delivered-To: freebsd-testing@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1CAC71369EA for ; Sun, 6 Oct 2019 22:29:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46mdZs71dVz3wj6; Sun, 6 Oct 2019 22:29:41 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f41.google.com with SMTP id s22so9436167otr.6; Sun, 06 Oct 2019 15:29:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=L9bIkVwf9lITtmGeTHV/yupIVRzmpTfQrzF9fk0pLFs=; b=ljWiUDq35eeN3BXeEK8sRZCgSJukHyBAVqG6zCbwDxG5eKfjL58OCml+3meyOP49mX 8KP0q2H3e1MepSpPOCC6+fZlTbxZqxkqICPpjuJVH5WysH7oJiDScpUTbEGhDM3Wd3js 045iaf71K4H7hL1cufdZNkoHNpHm8TdYHWs8QjbidPBgYg5Y0Khd1g1M9mpPd8gsrrI1 ObtegPmYICgypzHTk65t44IHxOXlEkZWxpcD/lcS1I47MAIUCn6G9O68mpbcMyB9jiu7 nvJtvDErDw1teg/Dyf+VIEtDAdZlgX7uB+gyGmEayEwRuLfOKN+skrQHoYYFTD8Ns8i9 F5KA== X-Gm-Message-State: APjAAAV8Kb+ZxwbZvMrXzKeLfYBNFE6n8aURAqpiJm+nCfUgCcfuM/Qp s9aYKzmQjtvIEKD7YcdVDEIwWa7n81kmz1ao1GAAOA== X-Google-Smtp-Source: APXvYqzrcXKIFpeHRVR7YrI2xU34z2F+Ok3AKeyhRGQDApB2FJ+zgnsCraa0rycAodhA71WLAVY1fj3k1x59NwxhOmg= X-Received: by 2002:a9d:378a:: with SMTP id x10mr19315254otb.222.1570400979391; Sun, 06 Oct 2019 15:29:39 -0700 (PDT) MIME-Version: 1.0 References: <0cd4df2a-dedd-5cb0-da71-37247fd7bfb4@freebsd.org> In-Reply-To: <0cd4df2a-dedd-5cb0-da71-37247fd7bfb4@freebsd.org> From: Alan Somers Date: Sun, 6 Oct 2019 16:29:27 -0600 Message-ID: Subject: Re: Revisiting the ZFS test suite To: Allan Jude Cc: "freebsd-testing@freebsd.org" X-Rspamd-Queue-Id: 46mdZs71dVz3wj6 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Oct 2019 22:29:42 -0000 On Sun, Oct 6, 2019 at 10:20 AM Allan Jude wrote: > On 2019-10-04 12:23, Alan Somers wrote: > > My employment status changed last week, and I find myself once again > > working on ZFS. It's hard to have confidence in any changes without a > > solid test suite, so my main task right now is to fix the problems in > > FreeBSD's ZFS test suite. Our most recent run had 64 failed tests and 99 > > skipped ones; let's get those down to 0 and something small, > respectively. > > > > https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/3880/testReport/ > > > > Would anybody be interested to review my changes to the test suite? It's > > not terrible if I have to commit without review; I can't possibly break > > anything but the test suite itself. > > > > -Alan > > _______________________________________________ > > freebsd-testing@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > > To unsubscribe, send any mail to " > freebsd-testing-unsubscribe@freebsd.org" > > > > Have you followed the work here: > > https://github.com/zfsonfreebsd/ZoF > > That has ported the improved ZFS test suite from ZFSonLinux back to > FreeBSD (as part of the effort to upstream FreeBSD support into that > newer version of ZFS) > > -- > Allan Jude > Oh, boy, I didn't know about that fork of the test suite. When I made FreeBSD's, I didn't try for portability. Merging the two will not be easy. There are > 60,000 lines of diffs between the two, plus > 1,000 files that don't exist on both sides. Worse, there are some major structural differences: 1) FreeBSD's test suite runs as root and drops privileges selectively, but ZoL's runs as an unprivileged user and uses sudo selectively to gain privileges 2) FreeBSD's test suite uses ATF, just like everything else, but ZoL's test suite uses a bespoke python program 3) FreeBSD's test suite runs setup and cleanup once for every test file, but ZoL does it once for every directory of test files. 4) FreeBSD's test suite uses FreeBSD make, but ZoL's uses autotools and gmake Of those item 3 is the most troubling. I wonder if ZoL could be convinced to switch to ATF and Kyua? They're pretty portable. -Alan