Date: Thu, 7 Nov 2024 08:16:06 -0700 From: Alan Somers <asomers@freebsd.org> To: Kristof Provost <kp@freebsd.org> Cc: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= <olivier@freebsd.org>, Igor Ostapenko <igoro@freebsd.org>, freebsd-testing@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: RFC: Add required_klds metadata to Kyua Message-ID: <CAOtMX2j%2BXFL_9CNi0iKUg9aCcH_0dNqzszC=M5wPPy6yhs57Rg@mail.gmail.com> In-Reply-To: <9975A26D-6AB4-433F-B4B2-515F956BF366@FreeBSD.org> References: <9fc6fc72-4c2c-4b09-bdb5-122a49b45295@FreeBSD.org> <CA%2Bq%2BTcpLeVqHhtL7VHvWsijQO9sxewY5opeUQ895G43EC24qgg@mail.gmail.com> <9975A26D-6AB4-433F-B4B2-515F956BF366@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Nov 6, 2024 at 1:27=E2=80=AFPM Kristof Provost <kp@freebsd.org> wro= te: > > On 6 Nov 2024, at 21:13, Olivier Cochard-Labb=C3=A9 wrote: > > On Wed, Nov 6, 2024 at 7:12=E2=80=AFPM Igor Ostapenko <igoro@freebsd.org>= wrote: > > The problem to solve here is a well-known one: typically we invoke "kyua > test" and get a lot of "Skipped" due to many kernel module requirements. > And > the pain is to know which ones need loading, especially if we target a > sub-tree of /usr/tests. Also, the FreeBSD CI needs periodic sync if a new > test is added with new module requirements. And other related issues. > > Hello Igor, > > Thanks again for working on this topic > And there are even more challenging cases: what about tests that assess t= he > behavior of different modules and need to unload them as well? > For example, sys/netipsec/tunnel/* or sys/opencrypto/, which involve > loading and unloading modules between tests." > > > That=E2=80=99s a good point. I ran into that a few days ago when I looked= at using the new execenv=3Djail option to parallelise the netipsec tests, = and discovered that that wasn=E2=80=99t straightforward. > > I wonder if an forbidden_klds or something is the answer there. At first = to skip a test if the forbidden module is present, later on perhaps to enfo= rce unloading of the module so the test can run. > > That may not be straightforward through, because module state is global, = and if we only support loading required modules that=E2=80=99s easy (gather= the list and load, or load when starting a test that requires a new module= ), but if we support unloading them it=E2=80=99ll affect test scheduling. (= i.e. we can=E2=80=99t load modules while a forbidden_klds test is running, = and can=E2=80=99t unload while a require_klds test is running). > > That said, perhaps we don=E2=80=99t need to worry about that just yet. It= =E2=80=99s a minority of tests that forbid modules, so those can just keep = doing what they=E2=80=99re doing already (i.e. run exclusively, do the chec= ks in the test script itself). It shouldn=E2=80=99t stop us from making the= majority of tests better. > > Best regards, > Kristof I too like the idea of Project A with required_klds or required_kmods. But how would you handle situations where a user customizes their kernel config to build some feature that's usually a module directly into the kernel? I would think that would break any test using required_klds. But project B is going to be a lot harder. "forbidden_klds", as suggested by kp, might be the way to go. In addition to the abovementioned suites, the fusefs tests are incompatible with the mac_bsdextended kld. I suggest focusing on Project A for now. That would be great even by itself. -Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2j%2BXFL_9CNi0iKUg9aCcH_0dNqzszC=M5wPPy6yhs57Rg>