Date: Wed, 06 Nov 2024 21:27:05 +0100 From: Kristof Provost <kp@FreeBSD.org> To: =?utf-8?q?Olivier_Cochard-Labb=C3=A9?= <olivier@freebsd.org> Cc: Igor Ostapenko <igoro@freebsd.org>, freebsd-testing@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: RFC: Add required_klds metadata to Kyua Message-ID: <9975A26D-6AB4-433F-B4B2-515F956BF366@FreeBSD.org> In-Reply-To: <CA%2Bq%2BTcpLeVqHhtL7VHvWsijQO9sxewY5opeUQ895G43EC24qgg@mail.gmail.com> References: <9fc6fc72-4c2c-4b09-bdb5-122a49b45295@FreeBSD.org> <CA%2Bq%2BTcpLeVqHhtL7VHvWsijQO9sxewY5opeUQ895G43EC24qgg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On 6 Nov 2024, at 21:13, Olivier Cochard-Labbé wrote: > On Wed, Nov 6, 2024 at 7:12 PM Igor Ostapenko <igoro@freebsd.org> > wrote: > >> >> >> The problem to solve here is a well-known one: typically we invoke >> "kyua >> test" and get a lot of "Skipped" due to many kernel module >> requirements. >> And >> the pain is to know which ones need loading, especially if we target >> a >> sub-tree of /usr/tests. Also, the FreeBSD CI needs periodic sync if a >> new >> test is added with new module requirements. And other related issues. >> >> >> Hello Igor, > > Thanks again for working on this topic > And there are even more challenging cases: what about tests that > assess the > behavior of different modules and need to unload them as well? > For example, sys/netipsec/tunnel/* or sys/opencrypto/, which involve > loading and unloading modules between tests." > That’s a good point. I ran into that a few days ago when I looked at using the new execenv=jail option to parallelise the netipsec tests, and discovered that that wasn’t straightforward. I wonder if an `forbidden_klds` or something is the answer there. At first to skip a test if the forbidden module is present, later on perhaps to enforce unloading of the module so the test can run. That may not be straightforward through, because module state is global, and if we only support loading required modules that’s easy (gather the list and load, or load when starting a test that requires a new module), but if we support unloading them it’ll affect test scheduling. (i.e. we can’t load modules while a `forbidden_klds` test is running, and can’t unload while a `require_klds` test is running). That said, perhaps we don’t need to worry about that just yet. It’s a minority of tests that forbid modules, so those can just keep doing what they’re doing already (i.e. run exclusively, do the checks in the test script itself). It shouldn’t stop us from making the majority of tests better. Best regards, Kristof [-- Attachment #2 --] <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/xhtml; charset=utf-8"> </head> <body><div style="font-family: sans-serif;"><div class="markdown" style="white-space: normal;"> <p dir="auto">On 6 Nov 2024, at 21:13, Olivier Cochard-Labbé wrote:</p> </div><div class="plaintext" style="white-space: normal;"><blockquote style="margin: 0 0 5px; padding-left: 5px; border-left: 2px solid #136BCE; color: #136BCE;"><p dir="auto">On Wed, Nov 6, 2024 at 7:12 PM Igor Ostapenko <igoro@freebsd.org> wrote:</p> <blockquote style="margin: 0 0 5px; padding-left: 5px; border-left: 2px solid #136BCE; border-left-color: #4B89CF; color: #4B89CF;"><p dir="auto">The problem to solve here is a well-known one: typically we invoke "kyua <br> test" and get a lot of "Skipped" due to many kernel module requirements. <br> And <br> the pain is to know which ones need loading, especially if we target a <br> sub-tree of /usr/tests. Also, the FreeBSD CI needs periodic sync if a new <br> test is added with new module requirements. And other related issues.</p> <p dir="auto">Hello Igor,</p> </blockquote><p dir="auto">Thanks again for working on this topic <br> And there are even more challenging cases: what about tests that assess the <br> behavior of different modules and need to unload them as well? <br> For example, sys/netipsec/tunnel/* or sys/opencrypto/, which involve <br> loading and unloading modules between tests."</p> <br></blockquote></div> <div class="markdown" style="white-space: normal;"> <p dir="auto">That’s a good point. I ran into that a few days ago when I looked at using the new execenv=jail option to parallelise the netipsec tests, and discovered that that wasn’t straightforward.</p> <p dir="auto">I wonder if an <code style="padding: 0 0.25em; background-color: #E4E4E4;">forbidden_klds</code> or something is the answer there. At first to skip a test if the forbidden module is present, later on perhaps to enforce unloading of the module so the test can run.</p> <p dir="auto">That may not be straightforward through, because module state is global, and if we only support loading required modules that’s easy (gather the list and load, or load when starting a test that requires a new module), but if we support unloading them it’ll affect test scheduling. (i.e. we can’t load modules while a <code style="padding: 0 0.25em; background-color: #E4E4E4;">forbidden_klds</code> test is running, and can’t unload while a <code style="padding: 0 0.25em; background-color: #E4E4E4;">require_klds</code> test is running).</p> <p dir="auto">That said, perhaps we don’t need to worry about that just yet. It’s a minority of tests that forbid modules, so those can just keep doing what they’re doing already (i.e. run exclusively, do the checks in the test script itself). It shouldn’t stop us from making the majority of tests better.</p> <p dir="auto">Best regards,<br> Kristof</p> </div> </div> </body> </html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9975A26D-6AB4-433F-B4B2-515F956BF366>