From nobody Fri Feb 23 15:08:41 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ThD1c54R3z59v07 for ; Fri, 23 Feb 2024 15:09:08 +0000 (UTC) (envelope-from igor.ostapenko@pm.me) Received: from mail-40134.protonmail.ch (mail-40134.protonmail.ch [185.70.40.134]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ThD1c09Jqz4GM7 for ; Fri, 23 Feb 2024 15:09:07 +0000 (UTC) (envelope-from igor.ostapenko@pm.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail3; t=1708700944; x=1708960144; bh=EFJ7vncwfzcPVqIqVkex5Gi1pKyxGPQRx2mNqSH23bw=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=d0mGKJ6S+wJw3n1tO+mfYxMQ37B6GqLv2faOyUnxQyn6kllwbt0bkHQW1KsSlV+9b GWsYWCCynf7QarWrw2+avNDdop+fSL/vaj6I10wWJewcNNQLli0iFBCd3lwpmgJLWj /8NxuAirBSSzCdKhJ3k5cuCy53eV7zpcctSCi3NsBKIOWDYyFDw10d/ZfP129SE9nP VPQ+goU+DivaPlCyfyqFDitFlx5mcn+EtwlzPnr9wGWiK+eW1aDJ2A6IkWCtAyRpfp Q15R/cwwjNdOYhUWiy20L+a3FyhA+FNPQwnGP6atxGq/TUKJY8PFXC3Dqf1LKgQKpj 4YBHQ6FW0dE3A== Date: Fri, 23 Feb 2024 15:08:41 +0000 To: Paul Floyd From: Igor Ostapenko Cc: freebsd-hackers@freebsd.org Subject: Re: Add jail execution environment support to the FreeBSD test suite Message-ID: <5XGsuxCWojiK7f1wBuoC73PWMtjVB3oCvsy1vHQtNr5UHC_2LN8zNGczKwpQEs4Pcc8g1G3Rir4uIrQ9iUxKbWmonsEu5f_mHzldy4vuYk4=@pm.me> In-Reply-To: <188b3537-11b1-4169-aae2-070d349b6016@gmail.com> References: <2bjQNp1msrv-_AqyamMun6kY-SCqbgPm3Q7DqVQHAYlqvFkiE1i85svfIT-QQdUG1cg3cKippyTyv8Z-5nbLu4WaMutgZQ7KT-YYo_5Pbro=@pm.me> <188b3537-11b1-4169-aae2-070d349b6016@gmail.com> Feedback-ID: 8300135:user:proton List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] X-Rspamd-Queue-Id: 4ThD1c09Jqz4GM7 On Friday, February 23rd, 2024 at 10:30 AM, Paul Floyd wrote: >=20 > Will there be an option to turn this off globally? >=20 > I sometimes run test suits with Valgrind. I'm not aware of any option to > do that cleanly so I just do it with brute force: run kyua and all > children under Valgrind. >=20 > valgrind --trace-children=3Dyes -q kyua test -k Kyuafile >=20 > Adding jails into the mix is likely to give worse results. I did try > using an x86 jail as an alternative to VirtualBox but I gave up as I got > too many test failures. >=20 Thanks for your attention and mention of your use case. I've added this point to my TODO. I guess it could look as the following: =09valgrind --trace-children=3Dyes -q kyua -v execenv=3Dhost test -k Kyuafi= le It would skip tests designed for other execution environments and run only "host" ones. I think execenv Kyua config variable could be a list actually. By default, = if jail feature is present it could be set to "host jail", i.e. to run tests designed for all environments. As long as tests which opted-in to use execenv=3D"jail" seem to be out of y= our interest in your specific case it looks to be as one of the options. I was thinking about possiblity that a test could be ready to run within "host" a= nd "jail" env, I actually wrote such tests and they work, but it needs more effort from a test author and can be error prone. Anyway, Kyua itself canno= t help with duality of a test. Some tests can be freely run as "host" or "jai= l" without any change, others may need quite an effort to prepare for both cas= es. Does it look like something that would cover your production needs? Best regards, Igor.