Date: Thu, 18 Jul 2024 13:19:39 GMT From: Mark Johnston <markj@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: git: 257e70f1d5ee - main - kyua: Add FreeBSD Jail execution environment support Message-ID: <202407181319.46IDJdBj055008@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=257e70f1d5ee61037c8c59b116538d3b6b1427a2 commit 257e70f1d5ee61037c8c59b116538d3b6b1427a2 Author: Igor Ostapenko <pm@igoro.pro> AuthorDate: 2024-07-16 18:41:12 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2024-07-18 13:18:28 +0000 kyua: Add FreeBSD Jail execution environment support A new Kyua concept is added -- "execution environment". A test can be configured to be run within a specific environment. The test case lifecycle is extended respectively: - execenv init (creates a jail or does nothing for default execenv="host") - test exec - cleanup exec (optional) - execenv cleanup (removes a jail or does nothing for default execenv="host") The following new functionality is provided, from bottom to top: 1 ATF based tests - The new "execenv" metadata property can be set to explicitly ask for an execution environment: "host" or "jail". If it's not defined, as all existing tests do, then it implicitly means "host". - The new "execenv.jail.params" metadata property can be optionally defined to ask Kyua to use specific jail(8) parameters during creation of a temporary jail. An example is "vnet allow.raw_sockets". Kyua implicitly adds "children.max" to "execenv_jail_params" parameters with the maximum possible value. A test case can override it. 2 Kyuafile - The same new metadata properties can be defined on Kyuafile level: "execenv" and "execenv_jail_params". - Note that historically ATF uses dotted style of metadata naming, while Kyua uses underscore style. Hence "execenv.jail.params" vs. "execenv_jail_params". 3 kyua.conf, kyua CLI - The new "execenvs" engine configuration variable can be set to a list of execution environments to run only tests designed for. Tests of not listed environments are skipped. - By default, this variable lists all execution environments supported by a Kyua binary, e.g. execenvs="host jail". - This variable can be changed via "kyua.conf" or via kyua CLI's "-v" parameter. For example, "kyua -v execenvs=host test" will run only host-based tests and skip jail-based ones. - Current value of this variable can be examined with "kyua config". [markj] This feature has not landed upstream yet. See the discussion in https://github.com/freebsd/kyua/pull/224 . Having the ability to automatically jail tests allows many network tests to run in parallel, giving a drastic speedup. So, let's import the feature and start using it in main. Signed-off-by: Igor Ostapenko <pm@igoro.pro> Reviewed by: markj, kp Tested by: markj, kp MFC after: 3 months Differential Revision: https://reviews.freebsd.org/D45865 --- contrib/kyua/AUTHORS | 1 + contrib/kyua/cli/cmd_config_test.cpp | 12 +- contrib/kyua/doc/kyua.conf.5.in | 13 +- contrib/kyua/doc/kyuafile.5.in | 103 +++++++- contrib/kyua/drivers/report_junit_test.cpp | 8 + contrib/kyua/engine/atf.cpp | 11 +- contrib/kyua/engine/atf_list.cpp | 4 + contrib/kyua/engine/config.cpp | 18 ++ contrib/kyua/engine/execenv/execenv.cpp | 74 ++++++ contrib/kyua/engine/execenv/execenv.hpp | 149 +++++++++++ contrib/kyua/engine/execenv/execenv_host.cpp | 52 ++++ contrib/kyua/engine/execenv/execenv_host.hpp | 63 +++++ contrib/kyua/engine/plain.cpp | 7 +- contrib/kyua/engine/requirements.cpp | 33 +++ contrib/kyua/engine/scheduler.cpp | 274 ++++++++++++++++++- contrib/kyua/engine/scheduler.hpp | 1 + contrib/kyua/engine/tap.cpp | 7 +- contrib/kyua/examples/kyua.conf | 3 + contrib/kyua/integration/cmd_config_test.sh | 2 + contrib/kyua/integration/cmd_report_junit_test.sh | 8 + contrib/kyua/integration/cmd_report_test.sh | 2 + contrib/kyua/main.cpp | 3 + contrib/kyua/model/metadata.cpp | 76 ++++++ contrib/kyua/model/metadata.hpp | 5 + contrib/kyua/model/metadata_test.cpp | 8 +- contrib/kyua/model/test_case_test.cpp | 3 +- contrib/kyua/model/test_program_test.cpp | 13 +- contrib/kyua/os/freebsd/execenv_jail.cpp | 78 ++++++ contrib/kyua/os/freebsd/execenv_jail.hpp | 65 +++++ contrib/kyua/os/freebsd/execenv_jail_manager.cpp | 63 +++++ contrib/kyua/os/freebsd/execenv_jail_manager.hpp | 54 ++++ contrib/kyua/os/freebsd/execenv_jail_stub.cpp | 75 ++++++ contrib/kyua/os/freebsd/main.cpp | 54 ++++ contrib/kyua/os/freebsd/main.hpp | 41 +++ contrib/kyua/os/freebsd/utils/jail.cpp | 306 ++++++++++++++++++++++ contrib/kyua/os/freebsd/utils/jail.hpp | 64 +++++ contrib/kyua/utils/config/nodes.ipp | 9 +- contrib/kyua/utils/process/executor.cpp | 42 +++ contrib/kyua/utils/process/executor.hpp | 1 + usr.bin/kyua/Makefile | 15 +- 40 files changed, 1792 insertions(+), 28 deletions(-) diff --git a/contrib/kyua/AUTHORS b/contrib/kyua/AUTHORS index c7bd72ce776b..4cf65083015a 100644 --- a/contrib/kyua/AUTHORS +++ b/contrib/kyua/AUTHORS @@ -10,3 +10,4 @@ * The FreeBSD Foundation * Google Inc. +* Igor Ostapenko <pm@igoro.pro> diff --git a/contrib/kyua/cli/cmd_config_test.cpp b/contrib/kyua/cli/cmd_config_test.cpp index f084f99bb90a..a5f6930ba027 100644 --- a/contrib/kyua/cli/cmd_config_test.cpp +++ b/contrib/kyua/cli/cmd_config_test.cpp @@ -61,6 +61,7 @@ fake_config(void) { config::tree user_config = engine::default_config(); user_config.set_string("architecture", "the-architecture"); + user_config.set_string("execenvs", "the-env"); user_config.set_string("parallelism", "128"); user_config.set_string("platform", "the-platform"); //user_config.set_string("unprivileged_user", ""); @@ -83,12 +84,13 @@ ATF_TEST_CASE_BODY(all) cmdline::ui_mock ui; ATF_REQUIRE_EQ(EXIT_SUCCESS, cmd.main(&ui, args, fake_config())); - ATF_REQUIRE_EQ(5, ui.out_log().size()); + ATF_REQUIRE_EQ(6, ui.out_log().size()); ATF_REQUIRE_EQ("architecture = the-architecture", ui.out_log()[0]); - ATF_REQUIRE_EQ("parallelism = 128", ui.out_log()[1]); - ATF_REQUIRE_EQ("platform = the-platform", ui.out_log()[2]); - ATF_REQUIRE_EQ("test_suites.foo.bar = first", ui.out_log()[3]); - ATF_REQUIRE_EQ("test_suites.foo.baz = second", ui.out_log()[4]); + ATF_REQUIRE_EQ("execenvs = the-env", ui.out_log()[1]); + ATF_REQUIRE_EQ("parallelism = 128", ui.out_log()[2]); + ATF_REQUIRE_EQ("platform = the-platform", ui.out_log()[3]); + ATF_REQUIRE_EQ("test_suites.foo.bar = first", ui.out_log()[4]); + ATF_REQUIRE_EQ("test_suites.foo.baz = second", ui.out_log()[5]); ATF_REQUIRE(ui.err_log().empty()); } diff --git a/contrib/kyua/doc/kyua.conf.5.in b/contrib/kyua/doc/kyua.conf.5.in index 05a9499b48c4..7188bb8888c3 100644 --- a/contrib/kyua/doc/kyua.conf.5.in +++ b/contrib/kyua/doc/kyua.conf.5.in @@ -1,4 +1,4 @@ -.\" Copyright 2012 The Kyua Authors. +.\" Copyright 2012-2024 The Kyua Authors. .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,7 +25,7 @@ .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -.Dd February 20, 2015 +.Dd March 22, 2024 .Dt KYUA.CONF 5 .Os .Sh NAME @@ -36,6 +36,7 @@ .Pp Variables: .Va architecture , +.Va execenvs , .Va platform , .Va test_suites , .Va unprivileged_user . @@ -72,6 +73,14 @@ The following variables are internally recognized by .Bl -tag -width XX -offset indent .It Va architecture Name of the system architecture (aka processor type). +.It Va execenvs +Whitespace-separated list of execution environment names. +.Pp +Only tests which require one of the given execution environments will be run. +.Pp +See +.Xr kyuafile 5 +for the list of possible execution environments. .It Va parallelism Maximum number of test cases to execute concurrently. .It Va platform diff --git a/contrib/kyua/doc/kyuafile.5.in b/contrib/kyua/doc/kyuafile.5.in index 06cb2dbc42a8..a667f5dc2816 100644 --- a/contrib/kyua/doc/kyuafile.5.in +++ b/contrib/kyua/doc/kyuafile.5.in @@ -1,4 +1,4 @@ -.\" Copyright 2012 The Kyua Authors. +.\" Copyright 2012-2024 The Kyua Authors. .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,7 +25,7 @@ .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -.Dd July 3, 2015 +.Dd March 23, 2024 .Dt KYUAFILE 5 .Os .Sh NAME @@ -173,6 +173,75 @@ Refer to the section below for clarification. .It Va description Textual description of the test. +.It Va execenv +The name of the execution environment to be used for running the test. +If empty or not defined, the +.Sq host +execution environment is meant. +The possible values are: +.Bl -tag -width xUnnnnnnn +.It host +The default environment which runs the test as a usual child process. +.It jail +The +.Fx +.Xr jail 8 +environment. +It creates a temporary jail to run the test and its optional cleanup logic +within. +.Pp +This feature requires +.Xr kyua 1 +to be running with superuser privileges. +.Pp +The difference between +.Va security.jail.children.max +and +.Va security.jail.children.cur +sysctl of the jail +.Xr kyua 1 +is running within must have a value high enough for the jail based tests +planned to be run. +For instance, the value 1 should be enough for a sequential run of simple +tests. +Otherwise, such aspects as parallel test execution and sub-jails spawned +by specific test cases should be considered. +.Pp +The formula of a temporary jail name is +.Sq kyua ++ +.Va test program path ++ +.Sq _ ++ +.Va test case name . +All non-alphanumeric characters are replaced with +.Sq _ . +.Sq kyua_usr_tests_sys_netpfil_pf_pass_block_v4 +is an example for /usr/tests/sys/netpfil/pf/pass_block:v4 test case. +.El +.It Va execenv_jail_params +Additional test-specific whitespace-separated parameters of +.Fx +.Xr jail 8 +to create a temporary jail within which the test is run. +It makes sense only if execenv is set to +.Sq jail . +.sp +.Xr kyua 1 +implicitly passes +.Sq children.max +parameter to +.Xr jail 8 +for a temporary jail with the maximum possible value according to +the jail +.Xr kyua 1 +itself is running within. +It allows tests to easily spawn their own sub-jails without additional +configuration. +It can be overridden via +.Va execenv_jail_params +if needed. .It Va is_exclusive If true, indicates that this test program cannot be executed along any other programs at the same time. @@ -360,6 +429,36 @@ test_suite('FreeBSD') plain_test_program{name='the_test', ['custom.FreeBSD-Bug-Id']='category/12345'} .Ed +.Ss FreeBSD jail execution environment +The following example configures the test to be run within a temporary jail +with +.Xr vnet 9 +support and the permission to create raw sockets: +.Bd -literal -offset indent +syntax(2) + +test_suite('FreeBSD') + +atf_test_program{name='network_test', + execenv='jail', + execenv_jail_params='vnet allow.raw_sockets', + required_user='root'} +.Ed +.Pp +A test case itself may have no requirements in superuser privileges, +but required_user='root' metadata property reminds that the jail execution +environment requires +.Xr kyua 1 +being running with root privileges, and the test is skipped otherwise with +the respective message. The combination of +.Va execenv +set to +.Sq jail +and +.Va required_user +set to +.Sq unprivileged +does not work respectively. .Ss Connecting disjoint test suites Now suppose you had various test suites on your file system and you would like to connect them together so that they could be executed and treated as diff --git a/contrib/kyua/drivers/report_junit_test.cpp b/contrib/kyua/drivers/report_junit_test.cpp index 462dca72f9be..0f009c6befd3 100644 --- a/contrib/kyua/drivers/report_junit_test.cpp +++ b/contrib/kyua/drivers/report_junit_test.cpp @@ -63,6 +63,8 @@ static const char* const default_metadata = "allowed_architectures is empty\n" "allowed_platforms is empty\n" "description is empty\n" + "execenv is empty\n" + "execenv_jail_params is empty\n" "has_cleanup = false\n" "is_exclusive = false\n" "required_configs is empty\n" @@ -80,6 +82,8 @@ static const char* const overriden_metadata = "allowed_architectures is empty\n" "allowed_platforms is empty\n" "description = Textual description\n" + "execenv is empty\n" + "execenv_jail_params is empty\n" "has_cleanup = false\n" "is_exclusive = false\n" "required_configs is empty\n" @@ -199,6 +203,8 @@ ATF_TEST_CASE_BODY(junit_metadata__overrides) .add_allowed_architecture("arch1") .add_allowed_platform("platform1") .set_description("This is a test") + .set_execenv("jail") + .set_execenv_jail_params("vnet") .set_has_cleanup(true) .set_is_exclusive(true) .add_required_config("config1") @@ -215,6 +221,8 @@ ATF_TEST_CASE_BODY(junit_metadata__overrides) + "allowed_architectures = arch1\n" + "allowed_platforms = platform1\n" + "description = This is a test\n" + + "execenv = jail\n" + + "execenv_jail_params = vnet\n" + "has_cleanup = true\n" + "is_exclusive = true\n" + "required_configs = config1\n" diff --git a/contrib/kyua/engine/atf.cpp b/contrib/kyua/engine/atf.cpp index eb63be20b0e7..f6746dd2f29f 100644 --- a/contrib/kyua/engine/atf.cpp +++ b/contrib/kyua/engine/atf.cpp @@ -39,6 +39,7 @@ extern "C" { #include "engine/atf_list.hpp" #include "engine/atf_result.hpp" #include "engine/exceptions.hpp" +#include "engine/execenv/execenv.hpp" #include "model/test_case.hpp" #include "model/test_program.hpp" #include "model/test_result.hpp" @@ -54,6 +55,7 @@ extern "C" { #include "utils/stream.hpp" namespace config = utils::config; +namespace execenv = engine::execenv; namespace fs = utils::fs; namespace process = utils::process; @@ -190,7 +192,10 @@ engine::atf_interface::exec_test(const model::test_program& test_program, args.push_back(F("-r%s") % (control_directory / result_name)); args.push_back(test_case_name); - process::exec(test_program.absolute_path(), args); + + auto e = execenv::get(test_program, test_case_name); + e->init(); + e->exec(args); } @@ -219,7 +224,9 @@ engine::atf_interface::exec_cleanup( } args.push_back(F("%s:cleanup") % test_case_name); - process::exec(test_program.absolute_path(), args); + + auto e = execenv::get(test_program, test_case_name); + e->exec(args); } diff --git a/contrib/kyua/engine/atf_list.cpp b/contrib/kyua/engine/atf_list.cpp index a16b889c74f0..c9c2fed70175 100644 --- a/contrib/kyua/engine/atf_list.cpp +++ b/contrib/kyua/engine/atf_list.cpp @@ -121,6 +121,10 @@ engine::parse_atf_metadata(const model::properties_map& props) mdbuilder.set_string("has_cleanup", value); } else if (name == "require.arch") { mdbuilder.set_string("allowed_architectures", value); + } else if (name == "execenv") { + mdbuilder.set_string("execenv", value); + } else if (name == "execenv.jail.params") { + mdbuilder.set_string("execenv_jail_params", value); } else if (name == "require.config") { mdbuilder.set_string("required_configs", value); } else if (name == "require.files") { diff --git a/contrib/kyua/engine/config.cpp b/contrib/kyua/engine/config.cpp index 3f162a94fbb5..a7c418e3164c 100644 --- a/contrib/kyua/engine/config.cpp +++ b/contrib/kyua/engine/config.cpp @@ -35,6 +35,7 @@ #include <stdexcept> #include "engine/exceptions.hpp" +#include "engine/execenv/execenv.hpp" #include "utils/config/exceptions.hpp" #include "utils/config/parser.hpp" #include "utils/config/tree.ipp" @@ -43,6 +44,7 @@ #include "utils/text/operations.ipp" namespace config = utils::config; +namespace execenv = engine::execenv; namespace fs = utils::fs; namespace passwd = utils::passwd; namespace text = utils::text; @@ -59,6 +61,7 @@ static void init_tree(config::tree& tree) { tree.define< config::string_node >("architecture"); + tree.define< config::strings_set_node >("execenvs"); tree.define< config::positive_int_node >("parallelism"); tree.define< config::string_node >("platform"); tree.define< engine::user_node >("unprivileged_user"); @@ -74,6 +77,14 @@ static void set_defaults(config::tree& tree) { tree.set< config::string_node >("architecture", KYUA_ARCHITECTURE); + + std::set< std::string > supported; + for (auto em : execenv::execenvs()) + if (em->is_supported()) + supported.insert(em->name()); + supported.insert(execenv::default_execenv_name); + tree.set< config::strings_set_node >("execenvs", supported); + // TODO(jmmv): Automatically derive this from the number of CPUs in the // machine and forcibly set to a value greater than 1. Still testing // the new parallel implementation as of 2015-02-27 though. @@ -229,6 +240,13 @@ engine::empty_config(void) { config::tree tree(false); init_tree(tree); + + // Tests of Kyua itself tend to use an empty config, i.e. default + // execution environment is used. Let's allow it. + std::set< std::string > supported; + supported.insert(engine::execenv::default_execenv_name); + tree.set< config::strings_set_node >("execenvs", supported); + return tree; } diff --git a/contrib/kyua/engine/execenv/execenv.cpp b/contrib/kyua/engine/execenv/execenv.cpp new file mode 100644 index 000000000000..b043bcda52cb --- /dev/null +++ b/contrib/kyua/engine/execenv/execenv.cpp @@ -0,0 +1,74 @@ +// Copyright 2023 The Kyua Authors. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of Google Inc. nor the names of its contributors +// may be used to endorse or promote products derived from this software +// without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "engine/execenv/execenv.hpp" + +#include "engine/execenv/execenv_host.hpp" + +namespace execenv = engine::execenv; + +using utils::none; + + +const char* execenv::default_execenv_name = "host"; + + +/// List of registered execution environments, except default host one. +/// +/// Use register_execenv() to add an entry to this global list. +static std::vector< std::shared_ptr< execenv::manager > > + execenv_managers; + + +void +execenv::register_execenv(const std::shared_ptr< execenv::manager > manager) +{ + execenv_managers.push_back(manager); +} + + +const std::vector< std::shared_ptr< execenv::manager> > +execenv::execenvs() +{ + return execenv_managers; +} + + +std::unique_ptr< execenv::interface > +execenv::get(const model::test_program& test_program, + const std::string& test_case_name) +{ + for (auto m : execenv_managers) { + auto e = m->probe(test_program, test_case_name); + if (e != nullptr) + return e; + } + + return std::unique_ptr< execenv::interface >( + new execenv::execenv_host(test_program, test_case_name)); +} diff --git a/contrib/kyua/engine/execenv/execenv.hpp b/contrib/kyua/engine/execenv/execenv.hpp new file mode 100644 index 000000000000..e667ff205d85 --- /dev/null +++ b/contrib/kyua/engine/execenv/execenv.hpp @@ -0,0 +1,149 @@ +// Copyright 2023 The Kyua Authors. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of Google Inc. nor the names of its contributors +// may be used to endorse or promote products derived from this software +// without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +/// \file engine/execenv/execenv.hpp +/// Execution environment subsystem interface. + +#if !defined(ENGINE_EXECENV_EXECENV_HPP) +#define ENGINE_EXECENV_EXECENV_HPP + +#include "model/test_program.hpp" +#include "utils/optional.ipp" +#include "utils/process/operations_fwd.hpp" + +using utils::process::args_vector; +using utils::optional; + +namespace engine { +namespace execenv { + + +extern const char* default_execenv_name; + + +/// Abstract interface of an execution environment. +class interface { +protected: + const model::test_program& _test_program; + const std::string& _test_case_name; + +public: + /// Constructor. + /// + /// \param program The test program. + /// \param test_case_name Name of the test case. + interface(const model::test_program& test_program, + const std::string& test_case_name) : + _test_program(test_program), + _test_case_name(test_case_name) + {} + + /// Destructor. + virtual ~interface() {} + + /// Initializes execution environment. + /// + /// It's expected to be called inside a fork which runs + /// scheduler::interface::exec_test(), so we can fail a test fast if its + /// execution environment setup fails, and test execution could use the + /// configured proc environment, if expected. + virtual void init() const = 0; + + /// Cleanups or removes execution environment. + /// + /// It's expected to be called inside a fork for execenv cleanup. + virtual void cleanup() const = 0; + + /// Executes a test within the execution environment. + /// + /// It's expected to be called inside a fork which runs + /// scheduler::interface::exec_test() or exec_cleanup(). + /// + /// \param args The arguments to pass to the binary. + virtual void exec(const args_vector& args) const UTILS_NORETURN = 0; +}; + + +/// Abstract interface of an execution environment manager. +class manager { +public: + /// Destructor. + virtual ~manager() {} + + /// Returns name of an execution environment. + virtual const std::string& name() const = 0; + + /// Returns whether this execution environment is actually supported. + /// + /// It can be compile time and/or runtime check. + virtual bool is_supported() const = 0; + + /// Returns execution environment for a test. + /// + /// It checks if the given test is designed for this execution environment. + /// + /// \param program The test program. + /// \param test_case_name Name of the test case. + /// + /// \return An execenv object if the test conforms, or none. + virtual std::unique_ptr< interface > probe( + const model::test_program& test_program, + const std::string& test_case_name) const = 0; + + // TODO: execenv related extra metadata could be provided by a manager + // not to know how exactly and where it should be added to the kyua +}; + + +/// Registers an execution environment. +/// +/// \param manager Execution environment manager. +void register_execenv(const std::shared_ptr< manager > manager); + + +/// Returns list of registered execenv managers, except default host one. +/// +/// \return A vector of pointers to execenv managers. +const std::vector< std::shared_ptr< manager> > execenvs(); + + +/// Returns execution environment for a test case. +/// +/// \param program The test program. +/// \param test_case_name Name of the test case. +/// +/// \return An execution environment of a test. +std::unique_ptr< execenv::interface > get( + const model::test_program& test_program, + const std::string& test_case_name); + + +} // namespace execenv +} // namespace engine + +#endif // !defined(ENGINE_EXECENV_EXECENV_HPP) diff --git a/contrib/kyua/engine/execenv/execenv_host.cpp b/contrib/kyua/engine/execenv/execenv_host.cpp new file mode 100644 index 000000000000..4e37fca3e7d3 --- /dev/null +++ b/contrib/kyua/engine/execenv/execenv_host.cpp @@ -0,0 +1,52 @@ +// Copyright 2024 The Kyua Authors. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of Google Inc. nor the names of its contributors +// may be used to endorse or promote products derived from this software +// without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "engine/execenv/execenv_host.hpp" + +#include "utils/fs/path.hpp" +#include "utils/process/operations.hpp" + +void +execenv::execenv_host::init() const +{ + // nothing to do +} + + +void +execenv::execenv_host::cleanup() const +{ + // nothing to do +} + + +void +execenv::execenv_host::exec(const args_vector& args) const +{ + utils::process::exec(_test_program.absolute_path(), args); +} diff --git a/contrib/kyua/engine/execenv/execenv_host.hpp b/contrib/kyua/engine/execenv/execenv_host.hpp new file mode 100644 index 000000000000..2742366cfd6f --- /dev/null +++ b/contrib/kyua/engine/execenv/execenv_host.hpp @@ -0,0 +1,63 @@ +// Copyright 2024 The Kyua Authors. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of Google Inc. nor the names of its contributors +// may be used to endorse or promote products derived from this software +// without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +/// \file engine/execenv/execenv_host.hpp +/// Default execution environment. + +#if !defined(ENGINE_EXECENV_EXECENV_HOST_HPP) +#define ENGINE_EXECENV_EXECENV_HOST_HPP + +#include "engine/execenv/execenv.hpp" + +#include "utils/process/operations_fwd.hpp" + +namespace execenv = engine::execenv; + +using utils::process::args_vector; + +namespace engine { +namespace execenv { + + +class execenv_host : public execenv::interface { +public: + execenv_host(const model::test_program& test_program, + const std::string& test_case_name) : + execenv::interface(test_program, test_case_name) + {} + + void init() const; + void cleanup() const; + void exec(const args_vector& args) const UTILS_NORETURN; +}; + + +} // namespace execenv +} // namespace engine + +#endif // !defined(ENGINE_EXECENV_EXECENV_HOST_HPP) diff --git a/contrib/kyua/engine/plain.cpp b/contrib/kyua/engine/plain.cpp index 8346e50bbecf..9a2c63f8b663 100644 --- a/contrib/kyua/engine/plain.cpp +++ b/contrib/kyua/engine/plain.cpp @@ -34,6 +34,7 @@ extern "C" { #include <cstdlib> +#include "engine/execenv/execenv.hpp" #include "model/test_case.hpp" #include "model/test_program.hpp" #include "model/test_result.hpp" @@ -47,6 +48,7 @@ extern "C" { #include "utils/sanity.hpp" namespace config = utils::config; +namespace execenv = engine::execenv; namespace fs = utils::fs; namespace process = utils::process; @@ -104,7 +106,10 @@ engine::plain_interface::exec_test( } process::args_vector args; - process::exec(test_program.absolute_path(), args); + + auto e = execenv::get(test_program, test_case_name); + e->init(); + e->exec(args); } diff --git a/contrib/kyua/engine/requirements.cpp b/contrib/kyua/engine/requirements.cpp index a7b0a90d97db..a6a4cae7511c 100644 --- a/contrib/kyua/engine/requirements.cpp +++ b/contrib/kyua/engine/requirements.cpp @@ -28,6 +28,7 @@ #include "engine/requirements.hpp" +#include "engine/execenv/execenv.hpp" #include "model/metadata.hpp" #include "model/types.hpp" #include "utils/config/nodes.ipp" @@ -100,6 +101,34 @@ check_allowed_architectures(const model::strings_set& allowed_architectures, } +/// Checks if test's execenv matches the user configuration. +/// +/// \param execenv Execution environment name a test is designed for. +/// \param user_config Runtime user configuration. +/// +/// \return Empty if the execenv is in the list or an error message otherwise. +static std::string +check_execenv(const std::string& execenv, const config::tree& user_config) +{ + std::string name = execenv; + if (name.empty()) + name = engine::execenv::default_execenv_name; // if test claims nothing + + std::set< std::string > execenvs; + try { + execenvs = user_config.lookup< config::strings_set_node >("execenvs"); + } catch (const config::unknown_key_error&) { + // okay, user config does not define it, empty set then + } + + if (execenvs.find(name) == execenvs.end()) + return F("'%s' execenv is not supported or not allowed by " + "the runtime user configuration") % name; + + return ""; +} + + /// Checks if the allowed platforms match the current architecture. /// /// \param allowed_platforms Set of allowed platforms. @@ -263,6 +292,10 @@ engine::check_reqs(const model::metadata& md, const config::tree& cfg, if (!reason.empty()) return reason; + reason = check_execenv(md.execenv(), cfg); + if (!reason.empty()) + return reason; + reason = check_allowed_platforms(md.allowed_platforms(), cfg); if (!reason.empty()) return reason; diff --git a/contrib/kyua/engine/scheduler.cpp b/contrib/kyua/engine/scheduler.cpp index e7b51d23acca..e75091a40e38 100644 --- a/contrib/kyua/engine/scheduler.cpp +++ b/contrib/kyua/engine/scheduler.cpp @@ -40,6 +40,7 @@ extern "C" { #include "engine/config.hpp" #include "engine/exceptions.hpp" +#include "engine/execenv/execenv.hpp" #include "engine/requirements.hpp" #include "model/context.hpp" #include "model/metadata.hpp" @@ -68,6 +69,7 @@ extern "C" { namespace config = utils::config; namespace datetime = utils::datetime; +namespace execenv = engine::execenv; namespace executor = utils::process::executor; namespace fs = utils::fs; namespace logging = utils::logging; @@ -87,6 +89,10 @@ using utils::optional; datetime::delta scheduler::cleanup_timeout(60, 0); +/// Timeout for the test case execenv cleanup operation. +datetime::delta scheduler::execenv_cleanup_timeout(60, 0); + + /// Timeout for the test case listing operation. /// /// TODO(jmmv): This is here only for testing purposes. Maybe we should expose @@ -206,6 +212,18 @@ struct test_exec_data : public exec_data { /// denote that no further attempts shall be made at cleaning this up. bool needs_cleanup; + /// Whether this test case still needs to have its execenv cleanup executed. + /// + /// This is set externally when the cleanup routine is actually invoked to + /// denote that no further attempts shall be made at cleaning this up. + bool needs_execenv_cleanup; + + /// Original PID of the test case subprocess. + /// + /// This is used for the cleanup upon termination by a signal, to reap the + /// leftovers and form missing exit_handle. + pid_t pid; + /// The exit_handle for this test once it has completed. /// *** 1720 LINES SKIPPED ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202407181319.46IDJdBj055008>