From nobody Thu Oct 17 13:49:48 2024 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XTq2h6yJ3z5Z2Kh; Thu, 17 Oct 2024 13:49:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XTq2h6F3Cz4lSQ; Thu, 17 Oct 2024 13:49:48 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1729172988; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=6c7xQmoNgD0qStjnzYRA0U0jLzVnaahi11/gXd0Uldg=; b=ZzCzS+Ly+mmvrv7H5azZ08FLy5h8ZqAs0Rah8s7GxOGhYEnKbvHVMOGWdZnedKZSydhNL7 u8jBQWgIJjoX5mmxisvxHZBha6o7Lt4qkrkKEc5NgL4I5JieVRIbUR5eRDNWfm5NjfEBlb zBta7flvYYjrfRF8krelCYkzZWLXAuFTkNGDI7F2q9g8x1sLcxykaG+RlI++6I8QNWFhkp 9tqm837Xz60UyAsW33ur6FvS33UH3zr1jkFbQUQHFXozFjH1zehgQPx218T4TlZ2rPfw21 AU7EvrZrSkOUwvX8vK3JFe+dqlJSZU/eRI5eHadVQe5n3uGty46Nrip4N1Bu6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1729172988; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=6c7xQmoNgD0qStjnzYRA0U0jLzVnaahi11/gXd0Uldg=; b=q+q7ig08FvKLajK5ilqbxhTCQc4Ita4lu2WcULXZXdr3aM91vo3QjjCu/R4ZalwICFJcN2 dqTDLBdPmOopLEKNBzldDj0Aq5snk4Yz4qH3NvJP1HZjP7O5BRsDezWVbUJ7orDOQrKIw9 4rnmQ28XBt1vBjKiDSeqMQZwQ8nfGdHZYUUVhc8UwWhkd7DKAZsjZNmHXyni9YcRq3XJ2/ ELE2grDkpk0fpp6RJf8B6tL8sWerBFp/snEMy1y2ii2nk/5OdIykrzebOseKh21jCacPwd LO+J5hJMDXzJqnaauh2sHEwRnmXufYvd0NUlD7MugjNNr4pxv74fwM6AJTg02w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1729172988; a=rsa-sha256; cv=none; b=a+T3NPk6fQzjn+O5Hm0nbSFPWo+Gc2zGg+JHAG10l03v8QGuRh30QhAOWJ2hWczyr9MPlC QLVYrbMqApSpf31t+x+P75y2Ri5IcDopq5q7daGgAhebMrBQEMM2tO3u7sundnuOEC5D5T XrmVzcMzfRsPwaCDs/xCHjbqK9RrNKpy5XJu6DKQt/pYt83C2EpTMSUgiqrVdXtDsrfc3U QMXL1/RzeoE8CV9ao1i+XKxMaOzNv65l3+wFlWCDF8tGQLK6aFdJp+50vvA2xhqihex3KW xBnTYwUNCfhI0l2uHvAf3bGNq7t3kCK2Rk8EKocDNhVS6qPvF9kjAtmfTKgUGg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XTq2h5qzbzrbm; Thu, 17 Oct 2024 13:49:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 49HDnms7044601; Thu, 17 Oct 2024 13:49:48 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 49HDnmXv044598; Thu, 17 Oct 2024 13:49:48 GMT (envelope-from git) Date: Thu, 17 Oct 2024 13:49:48 GMT Message-Id: <202410171349.49HDnmXv044598@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Igor Ostapenko Subject: git: 26980dceaef3 - stable/14 - kyua: Add FreeBSD Jail execution environment support List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: igoro X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 26980dceaef31cb2fec887b3cd5d928438d8579a Auto-Submitted: auto-generated The branch stable/14 has been updated by igoro: URL: https://cgit.FreeBSD.org/src/commit/?id=26980dceaef31cb2fec887b3cd5d928438d8579a commit 26980dceaef31cb2fec887b3cd5d928438d8579a Author: Igor Ostapenko AuthorDate: 2024-07-16 18:41:12 +0000 Commit: Igor Ostapenko CommitDate: 2024-10-17 13:44:34 +0000 kyua: Add FreeBSD Jail execution environment support A new Kyua concept is added -- "execution environment". A test can be configured to be run within a specific environment. The test case lifecycle is extended respectively: - execenv init (creates a jail or does nothing for default execenv="host") - test exec - cleanup exec (optional) - execenv cleanup (removes a jail or does nothing for default execenv="host") The following new functionality is provided, from bottom to top: 1 ATF based tests - The new "execenv" metadata property can be set to explicitly ask for an execution environment: "host" or "jail". If it's not defined, as all existing tests do, then it implicitly means "host". - The new "execenv.jail.params" metadata property can be optionally defined to ask Kyua to use specific jail(8) parameters during creation of a temporary jail. An example is "vnet allow.raw_sockets". Kyua implicitly adds "children.max" to "execenv_jail_params" parameters with the maximum possible value. A test case can override it. 2 Kyuafile - The same new metadata properties can be defined on Kyuafile level: "execenv" and "execenv_jail_params". - Note that historically ATF uses dotted style of metadata naming, while Kyua uses underscore style. Hence "execenv.jail.params" vs. "execenv_jail_params". 3 kyua.conf, kyua CLI - The new "execenvs" engine configuration variable can be set to a list of execution environments to run only tests designed for. Tests of not listed environments are skipped. - By default, this variable lists all execution environments supported by a Kyua binary, e.g. execenvs="host jail". - This variable can be changed via "kyua.conf" or via kyua CLI's "-v" parameter. For example, "kyua -v execenvs=host test" will run only host-based tests and skip jail-based ones. - Current value of this variable can be examined with "kyua config". [markj] This feature has not landed upstream yet. See the discussion in https://github.com/freebsd/kyua/pull/224 . Having the ability to automatically jail tests allows many network tests to run in parallel, giving a drastic speedup. So, let's import the feature and start using it in main. Signed-off-by: Igor Ostapenko Reviewed by: markj, kp Tested by: markj, kp MFC after: 3 months Differential Revision: https://reviews.freebsd.org/D45865 (cherry picked from commit 257e70f1d5ee61037c8c59b116538d3b6b1427a2) Approved by: kp (mentor), markj (mentor) --- contrib/kyua/AUTHORS | 1 + contrib/kyua/cli/cmd_config_test.cpp | 12 +- contrib/kyua/doc/kyua.conf.5.in | 13 +- contrib/kyua/doc/kyuafile.5.in | 103 +++++++- contrib/kyua/drivers/report_junit_test.cpp | 8 + contrib/kyua/engine/atf.cpp | 11 +- contrib/kyua/engine/atf_list.cpp | 4 + contrib/kyua/engine/config.cpp | 18 ++ contrib/kyua/engine/execenv/execenv.cpp | 74 ++++++ contrib/kyua/engine/execenv/execenv.hpp | 149 +++++++++++ contrib/kyua/engine/execenv/execenv_host.cpp | 52 ++++ contrib/kyua/engine/execenv/execenv_host.hpp | 63 +++++ contrib/kyua/engine/plain.cpp | 7 +- contrib/kyua/engine/requirements.cpp | 33 +++ contrib/kyua/engine/scheduler.cpp | 274 ++++++++++++++++++- contrib/kyua/engine/scheduler.hpp | 1 + contrib/kyua/engine/tap.cpp | 7 +- contrib/kyua/examples/kyua.conf | 3 + contrib/kyua/integration/cmd_config_test.sh | 2 + contrib/kyua/integration/cmd_report_junit_test.sh | 8 + contrib/kyua/integration/cmd_report_test.sh | 2 + contrib/kyua/main.cpp | 3 + contrib/kyua/model/metadata.cpp | 76 ++++++ contrib/kyua/model/metadata.hpp | 5 + contrib/kyua/model/metadata_test.cpp | 8 +- contrib/kyua/model/test_case_test.cpp | 3 +- contrib/kyua/model/test_program_test.cpp | 13 +- contrib/kyua/os/freebsd/execenv_jail.cpp | 78 ++++++ contrib/kyua/os/freebsd/execenv_jail.hpp | 65 +++++ contrib/kyua/os/freebsd/execenv_jail_manager.cpp | 63 +++++ contrib/kyua/os/freebsd/execenv_jail_manager.hpp | 54 ++++ contrib/kyua/os/freebsd/execenv_jail_stub.cpp | 75 ++++++ contrib/kyua/os/freebsd/main.cpp | 54 ++++ contrib/kyua/os/freebsd/main.hpp | 41 +++ contrib/kyua/os/freebsd/utils/jail.cpp | 306 ++++++++++++++++++++++ contrib/kyua/os/freebsd/utils/jail.hpp | 64 +++++ contrib/kyua/utils/config/nodes.ipp | 9 +- contrib/kyua/utils/process/executor.cpp | 42 +++ contrib/kyua/utils/process/executor.hpp | 1 + usr.bin/kyua/Makefile | 15 +- 40 files changed, 1792 insertions(+), 28 deletions(-) diff --git a/contrib/kyua/AUTHORS b/contrib/kyua/AUTHORS index ac0998fb937c..26bfcd3471d3 100644 --- a/contrib/kyua/AUTHORS +++ b/contrib/kyua/AUTHORS @@ -9,3 +9,4 @@ # * Organization * Google Inc. +* Igor Ostapenko diff --git a/contrib/kyua/cli/cmd_config_test.cpp b/contrib/kyua/cli/cmd_config_test.cpp index f084f99bb90a..a5f6930ba027 100644 --- a/contrib/kyua/cli/cmd_config_test.cpp +++ b/contrib/kyua/cli/cmd_config_test.cpp @@ -61,6 +61,7 @@ fake_config(void) { config::tree user_config = engine::default_config(); user_config.set_string("architecture", "the-architecture"); + user_config.set_string("execenvs", "the-env"); user_config.set_string("parallelism", "128"); user_config.set_string("platform", "the-platform"); //user_config.set_string("unprivileged_user", ""); @@ -83,12 +84,13 @@ ATF_TEST_CASE_BODY(all) cmdline::ui_mock ui; ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, fake_config())); - ATF_REQUIRE_EQ(5, ui.out_log().size()); + ATF_REQUIRE_EQ(6, ui.out_log().size()); ATF_REQUIRE_EQ("architecture = the-architecture", ui.out_log()[0]); - ATF_REQUIRE_EQ("parallelism = 128", ui.out_log()[1]); - ATF_REQUIRE_EQ("platform = the-platform", ui.out_log()[2]); - ATF_REQUIRE_EQ("test_suites.foo.bar = first", ui.out_log()[3]); - ATF_REQUIRE_EQ("test_suites.foo.baz = second", ui.out_log()[4]); + ATF_REQUIRE_EQ("execenvs = the-env", ui.out_log()[1]); + ATF_REQUIRE_EQ("parallelism = 128", ui.out_log()[2]); + ATF_REQUIRE_EQ("platform = the-platform", ui.out_log()[3]); + ATF_REQUIRE_EQ("test_suites.foo.bar = first", ui.out_log()[4]); + ATF_REQUIRE_EQ("test_suites.foo.baz = second", ui.out_log()[5]); ATF_REQUIRE(ui.err_log().empty()); } diff --git a/contrib/kyua/doc/kyua.conf.5.in b/contrib/kyua/doc/kyua.conf.5.in index 05a9499b48c4..7188bb8888c3 100644 --- a/contrib/kyua/doc/kyua.conf.5.in +++ b/contrib/kyua/doc/kyua.conf.5.in @@ -1,4 +1,4 @@ -.\" Copyright 2012 The Kyua Authors. +.\" Copyright 2012-2024 The Kyua Authors. .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,7 +25,7 @@ .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -.Dd February 20, 2015 +.Dd March 22, 2024 .Dt KYUA.CONF 5 .Os .Sh NAME @@ -36,6 +36,7 @@ .Pp Variables: .Va architecture , +.Va execenvs , .Va platform , .Va test_suites , .Va unprivileged_user . @@ -72,6 +73,14 @@ The following variables are internally recognized by .Bl -tag -width XX -offset indent .It Va architecture Name of the system architecture (aka processor type). +.It Va execenvs +Whitespace-separated list of execution environment names. +.Pp +Only tests which require one of the given execution environments will be run. +.Pp +See +.Xr kyuafile 5 +for the list of possible execution environments. .It Va parallelism Maximum number of test cases to execute concurrently. .It Va platform diff --git a/contrib/kyua/doc/kyuafile.5.in b/contrib/kyua/doc/kyuafile.5.in index 06cb2dbc42a8..a667f5dc2816 100644 --- a/contrib/kyua/doc/kyuafile.5.in +++ b/contrib/kyua/doc/kyuafile.5.in @@ -1,4 +1,4 @@ -.\" Copyright 2012 The Kyua Authors. +.\" Copyright 2012-2024 The Kyua Authors. .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,7 +25,7 @@ .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -.Dd July 3, 2015 +.Dd March 23, 2024 .Dt KYUAFILE 5 .Os .Sh NAME @@ -173,6 +173,75 @@ Refer to the section below for clarification. .It Va description Textual description of the test. +.It Va execenv +The name of the execution environment to be used for running the test. +If empty or not defined, the +.Sq host +execution environment is meant. +The possible values are: +.Bl -tag -width xUnnnnnnn +.It host +The default environment which runs the test as a usual child process. +.It jail +The +.Fx +.Xr jail 8 +environment. +It creates a temporary jail to run the test and its optional cleanup logic +within. +.Pp +This feature requires +.Xr kyua 1 +to be running with superuser privileges. +.Pp +The difference between +.Va security.jail.children.max +and +.Va security.jail.children.cur +sysctl of the jail +.Xr kyua 1 +is running within must have a value high enough for the jail based tests +planned to be run. +For instance, the value 1 should be enough for a sequential run of simple +tests. +Otherwise, such aspects as parallel test execution and sub-jails spawned +by specific test cases should be considered. +.Pp +The formula of a temporary jail name is +.Sq kyua ++ +.Va test program path ++ +.Sq _ ++ +.Va test case name . +All non-alphanumeric characters are replaced with +.Sq _ . +.Sq kyua_usr_tests_sys_netpfil_pf_pass_block_v4 +is an example for /usr/tests/sys/netpfil/pf/pass_block:v4 test case. +.El +.It Va execenv_jail_params +Additional test-specific whitespace-separated parameters of +.Fx +.Xr jail 8 +to create a temporary jail within which the test is run. +It makes sense only if execenv is set to +.Sq jail . +.sp +.Xr kyua 1 +implicitly passes +.Sq children.max +parameter to +.Xr jail 8 +for a temporary jail with the maximum possible value according to +the jail +.Xr kyua 1 +itself is running within. +It allows tests to easily spawn their own sub-jails without additional +configuration. +It can be overridden via +.Va execenv_jail_params +if needed. .It Va is_exclusive If true, indicates that this test program cannot be executed along any other programs at the same time. @@ -360,6 +429,36 @@ test_suite('FreeBSD') plain_test_program{name='the_test', ['custom.FreeBSD-Bug-Id']='category/12345'} .Ed +.Ss FreeBSD jail execution environment +The following example configures the test to be run within a temporary jail +with +.Xr vnet 9 +support and the permission to create raw sockets: +.Bd -literal -offset indent +syntax(2) + +test_suite('FreeBSD') + +atf_test_program{name='network_test', + execenv='jail', + execenv_jail_params='vnet allow.raw_sockets', + required_user='root'} +.Ed +.Pp +A test case itself may have no requirements in superuser privileges, +but required_user='root' metadata property reminds that the jail execution +environment requires +.Xr kyua 1 +being running with root privileges, and the test is skipped otherwise with +the respective message. The combination of +.Va execenv +set to +.Sq jail +and +.Va required_user +set to +.Sq unprivileged +does not work respectively. .Ss Connecting disjoint test suites Now suppose you had various test suites on your file system and you would like to connect them together so that they could be executed and treated as diff --git a/contrib/kyua/drivers/report_junit_test.cpp b/contrib/kyua/drivers/report_junit_test.cpp index 462dca72f9be..0f009c6befd3 100644 --- a/contrib/kyua/drivers/report_junit_test.cpp +++ b/contrib/kyua/drivers/report_junit_test.cpp @@ -63,6 +63,8 @@ static const char* const default_metadata = "allowed_architectures is empty\n" "allowed_platforms is empty\n" "description is empty\n" + "execenv is empty\n" + "execenv_jail_params is empty\n" "has_cleanup = false\n" "is_exclusive = false\n" "required_configs is empty\n" @@ -80,6 +82,8 @@ static const char* const overriden_metadata = "allowed_architectures is empty\n" "allowed_platforms is empty\n" "description = Textual description\n" + "execenv is empty\n" + "execenv_jail_params is empty\n" "has_cleanup = false\n" "is_exclusive = false\n" "required_configs is empty\n" @@ -199,6 +203,8 @@ ATF_TEST_CASE_BODY(junit_metadata__overrides) .add_allowed_architecture("arch1") .add_allowed_platform("platform1") .set_description("This is a test") + .set_execenv("jail") + .set_execenv_jail_params("vnet") .set_has_cleanup(true) .set_is_exclusive(true) .add_required_config("config1") @@ -215,6 +221,8 @@ ATF_TEST_CASE_BODY(junit_metadata__overrides) + "allowed_architectures = arch1\n" + "allowed_platforms = platform1\n" + "description = This is a test\n" + + "execenv = jail\n" + + "execenv_jail_params = vnet\n" + "has_cleanup = true\n" + "is_exclusive = true\n" + "required_configs = config1\n" diff --git a/contrib/kyua/engine/atf.cpp b/contrib/kyua/engine/atf.cpp index eb63be20b0e7..f6746dd2f29f 100644 --- a/contrib/kyua/engine/atf.cpp +++ b/contrib/kyua/engine/atf.cpp @@ -39,6 +39,7 @@ extern "C" { #include "engine/atf_list.hpp" #include "engine/atf_result.hpp" #include "engine/exceptions.hpp" +#include "engine/execenv/execenv.hpp" #include "model/test_case.hpp" #include "model/test_program.hpp" #include "model/test_result.hpp" @@ -54,6 +55,7 @@ extern "C" { #include "utils/stream.hpp" namespace config = utils::config; +namespace execenv = engine::execenv; namespace fs = utils::fs; namespace process = utils::process; @@ -190,7 +192,10 @@ engine::atf_interface::exec_test(const model::test_program& test_program, args.push_back(F("-r%s") % (control_directory / result_name)); args.push_back(test_case_name); - process::exec(test_program.absolute_path(), args); + + auto e = execenv::get(test_program, test_case_name); + e->init(); + e->exec(args); } @@ -219,7 +224,9 @@ engine::atf_interface::exec_cleanup( } args.push_back(F("%s:cleanup") % test_case_name); - process::exec(test_program.absolute_path(), args); + + auto e = execenv::get(test_program, test_case_name); + e->exec(args); } diff --git a/contrib/kyua/engine/atf_list.cpp b/contrib/kyua/engine/atf_list.cpp index a16b889c74f0..c9c2fed70175 100644 --- a/contrib/kyua/engine/atf_list.cpp +++ b/contrib/kyua/engine/atf_list.cpp @@ -121,6 +121,10 @@ engine::parse_atf_metadata(const model::properties_map& props) mdbuilder.set_string("has_cleanup", value); } else if (name == "require.arch") { mdbuilder.set_string("allowed_architectures", value); + } else if (name == "execenv") { + mdbuilder.set_string("execenv", value); + } else if (name == "execenv.jail.params") { + mdbuilder.set_string("execenv_jail_params", value); } else if (name == "require.config") { mdbuilder.set_string("required_configs", value); } else if (name == "require.files") { diff --git a/contrib/kyua/engine/config.cpp b/contrib/kyua/engine/config.cpp index 3f162a94fbb5..a7c418e3164c 100644 --- a/contrib/kyua/engine/config.cpp +++ b/contrib/kyua/engine/config.cpp @@ -35,6 +35,7 @@ #include #include "engine/exceptions.hpp" +#include "engine/execenv/execenv.hpp" #include "utils/config/exceptions.hpp" #include "utils/config/parser.hpp" #include "utils/config/tree.ipp" @@ -43,6 +44,7 @@ #include "utils/text/operations.ipp" namespace config = utils::config; +namespace execenv = engine::execenv; namespace fs = utils::fs; namespace passwd = utils::passwd; namespace text = utils::text; @@ -59,6 +61,7 @@ static void init_tree(config::tree& tree) { tree.define< config::string_node >("architecture"); + tree.define< config::strings_set_node >("execenvs"); tree.define< config::positive_int_node >("parallelism"); tree.define< config::string_node >("platform"); tree.define< engine::user_node >("unprivileged_user"); @@ -74,6 +77,14 @@ static void set_defaults(config::tree& tree) { tree.set< config::string_node >("architecture", KYUA_ARCHITECTURE); + + std::set< std::string > supported; + for (auto em : execenv::execenvs()) + if (em->is_supported()) + supported.insert(em->name()); + supported.insert(execenv::default_execenv_name); + tree.set< config::strings_set_node >("execenvs", supported); + // TODO(jmmv): Automatically derive this from the number of CPUs in the // machine and forcibly set to a value greater than 1. Still testing // the new parallel implementation as of 2015-02-27 though. @@ -229,6 +240,13 @@ engine::empty_config(void) { config::tree tree(false); init_tree(tree); + + // Tests of Kyua itself tend to use an empty config, i.e. default + // execution environment is used. Let's allow it. + std::set< std::string > supported; + supported.insert(engine::execenv::default_execenv_name); + tree.set< config::strings_set_node >("execenvs", supported); + return tree; } diff --git a/contrib/kyua/engine/execenv/execenv.cpp b/contrib/kyua/engine/execenv/execenv.cpp new file mode 100644 index 000000000000..b043bcda52cb --- /dev/null +++ b/contrib/kyua/engine/execenv/execenv.cpp @@ -0,0 +1,74 @@ +// Copyright 2023 The Kyua Authors. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of Google Inc. nor the names of its contributors +// may be used to endorse or promote products derived from this software +// without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "engine/execenv/execenv.hpp" + +#include "engine/execenv/execenv_host.hpp" + +namespace execenv = engine::execenv; + +using utils::none; + + +const char* execenv::default_execenv_name = "host"; + + +/// List of registered execution environments, except default host one. +/// +/// Use register_execenv() to add an entry to this global list. +static std::vector< std::shared_ptr< execenv::manager > > + execenv_managers; + + +void +execenv::register_execenv(const std::shared_ptr< execenv::manager > manager) +{ + execenv_managers.push_back(manager); +} + + +const std::vector< std::shared_ptr< execenv::manager> > +execenv::execenvs() +{ + return execenv_managers; +} + + +std::unique_ptr< execenv::interface > +execenv::get(const model::test_program& test_program, + const std::string& test_case_name) +{ + for (auto m : execenv_managers) { + auto e = m->probe(test_program, test_case_name); + if (e != nullptr) + return e; + } + + return std::unique_ptr< execenv::interface >( + new execenv::execenv_host(test_program, test_case_name)); +} diff --git a/contrib/kyua/engine/execenv/execenv.hpp b/contrib/kyua/engine/execenv/execenv.hpp new file mode 100644 index 000000000000..e667ff205d85 --- /dev/null +++ b/contrib/kyua/engine/execenv/execenv.hpp @@ -0,0 +1,149 @@ +// Copyright 2023 The Kyua Authors. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of Google Inc. nor the names of its contributors +// may be used to endorse or promote products derived from this software +// without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +/// \file engine/execenv/execenv.hpp +/// Execution environment subsystem interface. + +#if !defined(ENGINE_EXECENV_EXECENV_HPP) +#define ENGINE_EXECENV_EXECENV_HPP + +#include "model/test_program.hpp" +#include "utils/optional.ipp" +#include "utils/process/operations_fwd.hpp" + +using utils::process::args_vector; +using utils::optional; + +namespace engine { +namespace execenv { + + +extern const char* default_execenv_name; + + +/// Abstract interface of an execution environment. +class interface { +protected: + const model::test_program& _test_program; + const std::string& _test_case_name; + +public: + /// Constructor. + /// + /// \param program The test program. + /// \param test_case_name Name of the test case. + interface(const model::test_program& test_program, + const std::string& test_case_name) : + _test_program(test_program), + _test_case_name(test_case_name) + {} + + /// Destructor. + virtual ~interface() {} + + /// Initializes execution environment. + /// + /// It's expected to be called inside a fork which runs + /// scheduler::interface::exec_test(), so we can fail a test fast if its + /// execution environment setup fails, and test execution could use the + /// configured proc environment, if expected. + virtual void init() const = 0; + + /// Cleanups or removes execution environment. + /// + /// It's expected to be called inside a fork for execenv cleanup. + virtual void cleanup() const = 0; + + /// Executes a test within the execution environment. + /// + /// It's expected to be called inside a fork which runs + /// scheduler::interface::exec_test() or exec_cleanup(). + /// + /// \param args The arguments to pass to the binary. + virtual void exec(const args_vector& args) const UTILS_NORETURN = 0; +}; + + +/// Abstract interface of an execution environment manager. +class manager { +public: + /// Destructor. + virtual ~manager() {} + + /// Returns name of an execution environment. + virtual const std::string& name() const = 0; + + /// Returns whether this execution environment is actually supported. + /// + /// It can be compile time and/or runtime check. + virtual bool is_supported() const = 0; + + /// Returns execution environment for a test. + /// + /// It checks if the given test is designed for this execution environment. + /// + /// \param program The test program. + /// \param test_case_name Name of the test case. + /// + /// \return An execenv object if the test conforms, or none. + virtual std::unique_ptr< interface > probe( + const model::test_program& test_program, + const std::string& test_case_name) const = 0; + + // TODO: execenv related extra metadata could be provided by a manager + // not to know how exactly and where it should be added to the kyua +}; + + +/// Registers an execution environment. +/// +/// \param manager Execution environment manager. +void register_execenv(const std::shared_ptr< manager > manager); + + +/// Returns list of registered execenv managers, except default host one. +/// +/// \return A vector of pointers to execenv managers. +const std::vector< std::shared_ptr< manager> > execenvs(); + + +/// Returns execution environment for a test case. +/// +/// \param program The test program. +/// \param test_case_name Name of the test case. +/// +/// \return An execution environment of a test. +std::unique_ptr< execenv::interface > get( + const model::test_program& test_program, + const std::string& test_case_name); + + +} // namespace execenv +} // namespace engine + +#endif // !defined(ENGINE_EXECENV_EXECENV_HPP) diff --git a/contrib/kyua/engine/execenv/execenv_host.cpp b/contrib/kyua/engine/execenv/execenv_host.cpp new file mode 100644 index 000000000000..4e37fca3e7d3 --- /dev/null +++ b/contrib/kyua/engine/execenv/execenv_host.cpp @@ -0,0 +1,52 @@ +// Copyright 2024 The Kyua Authors. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of Google Inc. nor the names of its contributors +// may be used to endorse or promote products derived from this software +// without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "engine/execenv/execenv_host.hpp" + +#include "utils/fs/path.hpp" +#include "utils/process/operations.hpp" + +void +execenv::execenv_host::init() const +{ + // nothing to do +} + + +void +execenv::execenv_host::cleanup() const +{ + // nothing to do +} + + +void +execenv::execenv_host::exec(const args_vector& args) const +{ + utils::process::exec(_test_program.absolute_path(), args); +} diff --git a/contrib/kyua/engine/execenv/execenv_host.hpp b/contrib/kyua/engine/execenv/execenv_host.hpp new file mode 100644 index 000000000000..2742366cfd6f --- /dev/null +++ b/contrib/kyua/engine/execenv/execenv_host.hpp @@ -0,0 +1,63 @@ +// Copyright 2024 The Kyua Authors. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of Google Inc. nor the names of its contributors +// may be used to endorse or promote products derived from this software +// without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +/// \file engine/execenv/execenv_host.hpp +/// Default execution environment. + +#if !defined(ENGINE_EXECENV_EXECENV_HOST_HPP) +#define ENGINE_EXECENV_EXECENV_HOST_HPP + +#include "engine/execenv/execenv.hpp" + +#include "utils/process/operations_fwd.hpp" + +namespace execenv = engine::execenv; + +using utils::process::args_vector; + +namespace engine { +namespace execenv { + + +class execenv_host : public execenv::interface { +public: + execenv_host(const model::test_program& test_program, + const std::string& test_case_name) : + execenv::interface(test_program, test_case_name) + {} + + void init() const; + void cleanup() const; + void exec(const args_vector& args) const UTILS_NORETURN; +}; + + +} // namespace execenv +} // namespace engine + +#endif // !defined(ENGINE_EXECENV_EXECENV_HOST_HPP) diff --git a/contrib/kyua/engine/plain.cpp b/contrib/kyua/engine/plain.cpp index 8346e50bbecf..9a2c63f8b663 100644 --- a/contrib/kyua/engine/plain.cpp +++ b/contrib/kyua/engine/plain.cpp @@ -34,6 +34,7 @@ extern "C" { #include +#include "engine/execenv/execenv.hpp" #include "model/test_case.hpp" #include "model/test_program.hpp" #include "model/test_result.hpp" @@ -47,6 +48,7 @@ extern "C" { #include "utils/sanity.hpp" namespace config = utils::config; +namespace execenv = engine::execenv; namespace fs = utils::fs; namespace process = utils::process; @@ -104,7 +106,10 @@ engine::plain_interface::exec_test( } process::args_vector args; - process::exec(test_program.absolute_path(), args); + + auto e = execenv::get(test_program, test_case_name); + e->init(); + e->exec(args); } diff --git a/contrib/kyua/engine/requirements.cpp b/contrib/kyua/engine/requirements.cpp index a7b0a90d97db..a6a4cae7511c 100644 --- a/contrib/kyua/engine/requirements.cpp +++ b/contrib/kyua/engine/requirements.cpp @@ -28,6 +28,7 @@ #include "engine/requirements.hpp" +#include "engine/execenv/execenv.hpp" #include "model/metadata.hpp" #include "model/types.hpp" #include "utils/config/nodes.ipp" @@ -100,6 +101,34 @@ check_allowed_architectures(const model::strings_set& allowed_architectures, } +/// Checks if test's execenv matches the user configuration. +/// +/// \param execenv Execution environment name a test is designed for. +/// \param user_config Runtime user configuration. +/// +/// \return Empty if the execenv is in the list or an error message otherwise. +static std::string +check_execenv(const std::string& execenv, const config::tree& user_config) +{ + std::string name = execenv; + if (name.empty()) + name = engine::execenv::default_execenv_name; // if test claims nothing + + std::set< std::string > execenvs; + try { + execenvs = user_config.lookup< config::strings_set_node >("execenvs"); + } catch (const config::unknown_key_error&) { + // okay, user config does not define it, empty set then + } + + if (execenvs.find(name) == execenvs.end()) + return F("'%s' execenv is not supported or not allowed by " + "the runtime user configuration") % name; + + return ""; +} + + /// Checks if the allowed platforms match the current architecture. /// /// \param allowed_platforms Set of allowed platforms. @@ -263,6 +292,10 @@ engine::check_reqs(const model::metadata& md, const config::tree& cfg, if (!reason.empty()) return reason; + reason = check_execenv(md.execenv(), cfg); + if (!reason.empty()) + return reason; + reason = check_allowed_platforms(md.allowed_platforms(), cfg); if (!reason.empty()) return reason; diff --git a/contrib/kyua/engine/scheduler.cpp b/contrib/kyua/engine/scheduler.cpp index e7b51d23acca..e75091a40e38 100644 --- a/contrib/kyua/engine/scheduler.cpp +++ b/contrib/kyua/engine/scheduler.cpp @@ -40,6 +40,7 @@ extern "C" { #include "engine/config.hpp" #include "engine/exceptions.hpp" +#include "engine/execenv/execenv.hpp" #include "engine/requirements.hpp" #include "model/context.hpp" #include "model/metadata.hpp" @@ -68,6 +69,7 @@ extern "C" { namespace config = utils::config; namespace datetime = utils::datetime; +namespace execenv = engine::execenv; namespace executor = utils::process::executor; namespace fs = utils::fs; namespace logging = utils::logging; @@ -87,6 +89,10 @@ using utils::optional; datetime::delta scheduler::cleanup_timeout(60, 0); +/// Timeout for the test case execenv cleanup operation. +datetime::delta scheduler::execenv_cleanup_timeout(60, 0); + + /// Timeout for the test case listing operation. /// /// TODO(jmmv): This is here only for testing purposes. Maybe we should expose @@ -206,6 +212,18 @@ struct test_exec_data : public exec_data { /// denote that no further attempts shall be made at cleaning this up. bool needs_cleanup; + /// Whether this test case still needs to have its execenv cleanup executed. + /// + /// This is set externally when the cleanup routine is actually invoked to + /// denote that no further attempts shall be made at cleaning this up. + bool needs_execenv_cleanup; + + /// Original PID of the test case subprocess. + /// + /// This is used for the cleanup upon termination by a signal, to reap the + /// leftovers and form missing exit_handle. *** 1724 LINES SKIPPED ***