From owner-svn-src-all@freebsd.org Tue Jan 29 17:11:35 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 92BC114C44C9; Tue, 29 Jan 2019 17:11:35 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3812C8FF8A; Tue, 29 Jan 2019 17:11:35 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id DD0D710C7B; Tue, 29 Jan 2019 17:11:34 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-lf1-f50.google.com with SMTP id z13so15187836lfe.11; Tue, 29 Jan 2019 09:11:34 -0800 (PST) X-Gm-Message-State: AJcUukdx5OKD+gMmpDnxywRkO6eT5i3qgJMk31ZYz0W5h7PQ5wc0W9gu DIu781LhbErQ0UFY85c3/5Cqp6VMloxNmepF3Q8= X-Google-Smtp-Source: ALg8bN4K3zDtVUgUJ7xt59Mc32/gSFCsPuu2MqkYkDA29760BNe61Miok2AquSxkIhBGvK3BN4qf9JPXh29RCDmq+vY= X-Received: by 2002:a19:920a:: with SMTP id u10mr13650337lfd.122.1548781893401; Tue, 29 Jan 2019 09:11:33 -0800 (PST) MIME-Version: 1.0 References: <201901290408.x0T48oJQ080243@repo.freebsd.org> <674E01AB-49C7-4B40-B2FD-6EB4AFEAB1FD@gmail.com> <5687EE07-74FB-4233-B51D-C8948641E8CB@gmail.com> In-Reply-To: <5687EE07-74FB-4233-B51D-C8948641E8CB@gmail.com> From: Kyle Evans Date: Tue, 29 Jan 2019 11:10:55 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r343543 - head/sbin/bectl/tests To: Enji Cooper Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 3812C8FF8A X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.976,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jan 2019 17:11:35 -0000 On Tue, Jan 29, 2019 at 10:55 AM Enji Cooper wrote: > > > > On Jan 29, 2019, at 08:39, Kyle Evans wrote: > > > >> On Tue, Jan 29, 2019 at 10:32 AM Enji Cooper w= rote: > >> > >> > >>>> On Jan 28, 2019, at 20:10, Kyle Evans wrote: > >>>> > >>>> On Mon, Jan 28, 2019 at 10:09 PM Kyle Evans wro= te: > >>>> > >>>> Author: kevans > >>>> Date: Tue Jan 29 04:08:49 2019 > >>>> New Revision: 343543 > >>>> URL: https://svnweb.freebsd.org/changeset/base/343543 > >>>> > >>>> Log: > >>>> bectl(8) test: Force destroy the zpool in cleanup > >>>> > >>>> This is a wild guess as to why bectl tests failed once upon a time i= n CI, > >>>> given no apparent way to see a transcript of cleanup routines with K= yua. The > >>>> bectl tests construct a new, clean zpool for every test. The failure > >>>> indicated was because of a mount that was leftover from a previous t= est, but > >>>> the previous test had succeeded so it's not clear how the mount rema= ined > >>>> leftover unless the `zpool get health ${pool}` had somehow failed. > >>>> > >>> > >>> I left out: the tests are supposed to be constructed to clean up any > >>> mounts that were left over in the course of the test, hence the > >>> assumption that the failure lies in the cleanup. > >> > >> Hi Kyle, > >> > >> The tests use a deterministic zpool name defined locally (not globally= ), and will only destroy the zpool if =E2=80=9Czpool get health=E2=80=9D su= cceeds. > >> > >> The tests will work the first time (when the zpool doesn=E2=80=99t exi= st), but I believe they=E2=80=99re actually introducing nondeterminism by a= ccident. I will propose a fix for this. > >> > >> There=E2=80=99s a way to decipher why things failed from /var/log/mess= ages and kyua output. It=E2=80=99s just nontrivial to those who don=E2=80= =99t know what to look for. Can you please provide a failing ci run? > >> > > > > Hi, > > > > Thanks! Any help is appreciated -- my inquiry to -testing@ in response > > to the weekly report where the failing test [1] was mentioned has been > > met with silence. The failure is consistent in the i386-test job, but > > amd64-test sees nothing of the sort (and neither does my local > > testing). > > Thanks for the reminder to rejoin that list. > > > [1] https://ci.freebsd.org/job/FreeBSD-stable-12-i386-test/426/testRepo= rt/sbin.bectl/bectl_test/bectl_mount/ > > It=E2=80=99s pretty obvious from the above run what=E2=80=99s going on fr= om the output. A similarly named zpool (same prefix; suffixed with a 2) is = being matched by grep, even though the test sets up one with a suffix in th= e preceding steps. > Right- this is the part that makes sense to me. =3D) The 'default2' BE is clearly a leftover from a previous test, but it should've been unmounted in the process and the zpool cleaned up. What's not clear is how the cleanup routine is failing; the `zpool get health` shouldn't have any reason to fail (though I can't test it here, apparently =3D() and `zpool destroy -f` should've unmounted this BE even if that hadn't happened throughout the course of the test, but the failure remains (even if probably incorrectly fixed) following the above commit. Setting the zpool name globally with a suffix unique to this test would likely be a great idea -- I think there's something else going on as well, though. Thanks, Kyle Evans