From owner-freebsd-testing@freebsd.org Tue Aug 7 16:53:02 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A37F61064240 for ; Tue, 7 Aug 2018 16:53:02 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com [IPv6:2a00:1450:4864:20::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17C5677710; Tue, 7 Aug 2018 16:53:02 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-x234.google.com with SMTP id v9-v6so13990494ljk.4; Tue, 07 Aug 2018 09:53:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=xE5dnbHoi0Ukr2O/BBdgtBXuhwI3jLW/ttj++dby7S0=; b=ERS7OSKmsQY4M88Mh1lKbmJc6+4Et5aFUZJfkZfs+98jzH49+6sUVF6EqxxB7cmTF6 UbFFm1U0xIhOUM6dpMbyd0fN4Oq+LlYV5U9GsNdLfr4e4XnAC2iOFqmyDU5Md6n8pMp5 OYtcTtG1dMf3isCpB0xZJRveiA07/bLEDTBi+TyXi1do/J8N2BEa10X0gQw//rYAFzVq 4TRVYpk36Ml+4paTu5tWMCf9e682eu27T91/0qlpmABGT6yMHa7C7DKscvmb0DvyvXSM QsoFdfdrS7VlEUpK4o2aUGQlFskiZ+haorvhsphoIJrNXB7AWqKA03qpOTQjbrtXx1ya OpOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=xE5dnbHoi0Ukr2O/BBdgtBXuhwI3jLW/ttj++dby7S0=; b=rHvESW6rUjv2HPOcMPYZCVNgwhWopMh5jSmio6q+7FtI5ZTPfVbwXn8CZza9tqtXpU n4ya1gU5d4MjGFdgwLgZWiDKwve7ovfa4ySHb5yW8nBQbl/em7cW8XUIWUx0EUqdfXWW GFFSjBJFkZuFiTDVbbc14pO3E+EBcQ9dJtqoY3JCvQTYk126dSGOr+TNDBaxku9ryh8c KEZzRhGvtB+Zd5G0xqKhaF7mVecMaKsQx9zPTeNdmf8TCZkIoxjhi6naIhW7RE7k0SRt xmFahLKjNqWSdYAJdE45Yum9GBaKvbJXk+L//8vpTEScnZpxCKGsxNK0Idm8FPJiztqb Sw3Q== X-Gm-Message-State: AOUpUlFwQ6nCGcAz9T+zt+FVUVCtRUNHc3pnyuEUhvJv5elCeoRFx6y3 FM1Mdj5Rrwi0AP6T6aBbUsX7mIj4OdISw7NseYrnMQ== X-Google-Smtp-Source: AAOMgpdcLkGXtj7SyRgQEFoNBKaxVXevIapMAValNCFYqBIOobCj7R6cHFkxLTSPLKEJ07eUYqnXlyYgc7pcPzwHWgs= X-Received: by 2002:a2e:87da:: with SMTP id v26-v6mr15897506ljj.69.1533660780404; Tue, 07 Aug 2018 09:53:00 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:7851:0:0:0:0:0 with HTTP; Tue, 7 Aug 2018 09:52:59 -0700 (PDT) In-Reply-To: <20180807163405.GH77150@raichu> References: <20180806191406.GA77150@raichu> <20180807150349.GF77150@raichu> <20180807152402.GG77150@raichu> <20180807163405.GH77150@raichu> From: Alan Somers Date: Tue, 7 Aug 2018 10:52:59 -0600 X-Google-Sender-Auth: RNto7bU5hCosYjb0gYaQa_0xRP0 Message-ID: Subject: Re: Skipping tests that are unimplemented in 32-bit emulation To: Mark Johnston Cc: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 16:53:03 -0000 On Tue, Aug 7, 2018 at 10:34 AM, Mark Johnston wrote: > On Tue, Aug 07, 2018 at 10:11:45AM -0600, Alan Somers wrote: > > On Tue, Aug 7, 2018 at 9:24 AM, Mark Johnston wrote: > > > Oops, right. For 2), then, couldn't we define test_suite("FreeBSD32") > > > in the kyuafiles for the 32-bit test suite and set a > > > test_suite.FreeBSD32.compat32=true variable in kyua.conf? Then a test > > > which uses, say, setfib(1) can query that variable and skip if it's > set. > > > > > > > Ok, but what will set the test_suite.FreeBSD32.compat32 variable? It > can't > > be set as part of the image build, because then it will have the wrong > > value in jails and chroots. It needs to be set at runtime. > > For my suggestion to work, the 32-bit test suite must be built with > TESTSUITE=FreeBSD32 instead of the default in suite.test.mk. This could > be accomplished in a few ways: > 1) Modify the amd64 build to additionally build a 32-bit copy of the > test suite with TESTSUITE=FreeBSD32 and install it to /usr/tests32. > Users that wish to run it would have to copy it into their > chroot/jail. > 2) When building a 32-bit image for a chroot/jail, just remember to > specify TESTSUITE=FreeBSD32 in the make(1) invocation. (I didn't > actually verify that this does what I expect.) > 3) Always define the test suite name to be FreeBSD32 or FreeBSD64 > depending on the target architecture. Then, tests that need to be > skipped when run under COMPAT32 would need to check for the > "compat32" variable _and_ determine if the host kernel is 64-bit > (e.g., by looking at the hw.machine sysctl). > > If you do 2) and just set test_suite.FreeBSD32.compat32 statically in > /usr/local/etc/kyua.conf, then I don't see why you need any special > runtime logic. Any tests that aren't supported by COMPAT32 can just be > skipped when the compat32 kyua variable is set. > Option 2) may avoid additional runtime logic, but at the expense of creating an image that won't work correctly when run natively. Plus, you would also have to copy all of test_suite.FreeBSD's Kyua variables to test_suite.FreeBSD32. For example, test_suites.FreeBSD.disks. I still think the simplest option is to build a single test suite, where the few tests that can't run under emulation will call a helper function that compares the build architecture to the current kernel and skip on a mismatch. It eliminates extra configuration, uses just a single image, and doesn't modify a large number of files. Are there any emulations besides amd64/i386 that we support? Like mips64/mips for example? -Alan