From owner-freebsd-testing@freebsd.org Tue Aug 7 16:34:11 2018 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDE7E10636FC for ; Tue, 7 Aug 2018 16:34:10 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-pl0-x235.google.com (mail-pl0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 525A376715; Tue, 7 Aug 2018 16:34:10 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-pl0-x235.google.com with SMTP id j8-v6so7323124pll.12; Tue, 07 Aug 2018 09:34:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=gO920L9XURYMFAWUYN9BuSb9TRwfns66ug16b0WEr1Q=; b=t16lgpxtdTM3xxtODBF9ePh2DxH+bweSec+8ymHzLsHRtO8NRQxrrA0uRW9/9EHN4O v8vV2fImvRahY8x74pc2+s1inU3pcRJUTb64n+Gfyl0wcwrpz+xiADMKcpEz/clkR/6R 2puegIplrnav/va0Va2+rnCSqV5oJ+e2XS5zKZt0kYjS/LLojTQATw0tlxLFU5Kz8+TS tRPGAnc4fKMIG2xqBPBE+LRaql2PAMcD3mqDcBu9amlHAFdZRNepH/AAxb4kdmDCvf9h CJCX1eDLivKYJK3zzjLYAJxBpvigyOQLREJD3T0Yn5dbrxf2khjzPT9gG/3oeumvc1SQ m4ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=gO920L9XURYMFAWUYN9BuSb9TRwfns66ug16b0WEr1Q=; b=hnod0zzUfI1kbjWuiPYj/Jd6wem7MvjPCOyyjHEXAMxc8K4Nfma7U6cL09oWEbZxuZ +6/3uAFZp0ZNWrxsWCULg1b+1Br546q+Q1VhGrocLI/NYcFaXPFFhLvz7YjgAmiFp27a X5gD8kcWLiOoRlSzU6KUkaE4jk90XxMFPvMYoy1cYeVJ/eRwrXg6RqRm6fDRWbxm/xLR PVHNSuBbW66IAkwkfpsG2yH+E+ijuHKhRAACX9bzt30Lqw9MinxW0JNwyJnJlXLlUtBc TUk8CA0kiLpTndmFL0ndZKG0JisutynHwJRvYUSqSIUyGHqHAE/hboc8Ny+zl7dD0GrU YhhQ== X-Gm-Message-State: AOUpUlGvM/BItAwOz3W0Fvv/yrZODOh8zlpxtqhoEHKSjkG6wMJNLFxn W2xCCCIzgoRmPrIXIlWxyOqK3BXQ X-Google-Smtp-Source: AAOMgpcTKq25vdYJcloX3ne/S9ZMXnKbO3xfVHCwOx3verGF2ltYf5EOyGZ4DKj4nsvB6en454lX2w== X-Received: by 2002:a17:902:123:: with SMTP id 32-v6mr18338832plb.181.1533659648947; Tue, 07 Aug 2018 09:34:08 -0700 (PDT) Received: from raichu (toroon0560w-lp130-09-70-52-224-239.dsl.bell.ca. [70.52.224.239]) by smtp.gmail.com with ESMTPSA id y20-v6sm1931614pgv.31.2018.08.07.09.34.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Aug 2018 09:34:07 -0700 (PDT) Sender: Mark Johnston Date: Tue, 7 Aug 2018 12:34:05 -0400 From: Mark Johnston To: Alan Somers Cc: "freebsd-testing@freebsd.org" Subject: Re: Skipping tests that are unimplemented in 32-bit emulation Message-ID: <20180807163405.GH77150@raichu> References: <20180806191406.GA77150@raichu> <20180807150349.GF77150@raichu> <20180807152402.GG77150@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Aug 2018 16:34:11 -0000 On Tue, Aug 07, 2018 at 10:11:45AM -0600, Alan Somers wrote: > On Tue, Aug 7, 2018 at 9:24 AM, Mark Johnston wrote: > > Oops, right. For 2), then, couldn't we define test_suite("FreeBSD32") > > in the kyuafiles for the 32-bit test suite and set a > > test_suite.FreeBSD32.compat32=true variable in kyua.conf? Then a test > > which uses, say, setfib(1) can query that variable and skip if it's set. > > > > Ok, but what will set the test_suite.FreeBSD32.compat32 variable? It can't > be set as part of the image build, because then it will have the wrong > value in jails and chroots. It needs to be set at runtime. For my suggestion to work, the 32-bit test suite must be built with TESTSUITE=FreeBSD32 instead of the default in suite.test.mk. This could be accomplished in a few ways: 1) Modify the amd64 build to additionally build a 32-bit copy of the test suite with TESTSUITE=FreeBSD32 and install it to /usr/tests32. Users that wish to run it would have to copy it into their chroot/jail. 2) When building a 32-bit image for a chroot/jail, just remember to specify TESTSUITE=FreeBSD32 in the make(1) invocation. (I didn't actually verify that this does what I expect.) 3) Always define the test suite name to be FreeBSD32 or FreeBSD64 depending on the target architecture. Then, tests that need to be skipped when run under COMPAT32 would need to check for the "compat32" variable _and_ determine if the host kernel is 64-bit (e.g., by looking at the hw.machine sysctl). If you do 2) and just set test_suite.FreeBSD32.compat32 statically in /usr/local/etc/kyua.conf, then I don't see why you need any special runtime logic. Any tests that aren't supported by COMPAT32 can just be skipped when the compat32 kyua variable is set.