Date: Mon, 26 Feb 2024 16:32:24 +0000 From: Igor Ostapenko <igor.ostapenko@pm.me> To: =?utf-8?Q?Olivier_Cochard-Labb=C3=A9?= <olivier@freebsd.org> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: Add jail execution environment support to the FreeBSD test suite Message-ID: <srdDbPKj-bU3DzlBDM23UMTrW9s9EwTzFZVvQrULqDLySbYuBxmWO6BMjAua_kFWbP-dKJKViUR-usMp51wHx8atAk5kaf_afhyh_cZcihs=@pm.me> In-Reply-To: <CA%2Bq%2BTcp5rLVucgsAmBQMRtb-tPzxs0p2Ojz4U6nPyTaVqxjfhg@mail.gmail.com> References: <2bjQNp1msrv-_AqyamMun6kY-SCqbgPm3Q7DqVQHAYlqvFkiE1i85svfIT-QQdUG1cg3cKippyTyv8Z-5nbLu4WaMutgZQ7KT-YYo_5Pbro=@pm.me> <CA%2Bq%2BTcp5rLVucgsAmBQMRtb-tPzxs0p2Ojz4U6nPyTaVqxjfhg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday, February 23rd, 2024 at 11:41 AM, Olivier Cochard-Labb=C3=A9 <oli= vier@freebsd.org> wrote: >=20 > On Thu, Feb 22, 2024 at 9:58=E2=80=AFPM <igor.ostapenko@pm.me> wrote: >=20 > >=20 > > There is a proposal to improve the FreeBSD test suite. > >=20 >=20 >=20 > Thanks a lot for working on improving the parallel tests! >=20 > At work, we are using: > 1. a Nanobsd based, so a lot of WITHOUT_ : WITHOUT_JAIL, WITHOUT_PF, no V= NET (a nightmare to debug network stack with it), etc.; > 2. The standard full test suite (about 8200 tests) is reduced to about 74= 00 tests with this lighter nanobsd; > 3. We have been using kyua parallelism=3D8 for years, and I had to skip o= nly about 10-20 tests that were not compliant with parallel mode. >=20 > It toke about 23 minutes to run them (in a bhyve VM): >=20 > Test cases: 7429 total, 905 skipped, 29 expected failures, 1 broken, 4 fa= iled > Start time: 2024-02-21T00:15:42.527291Z > End time: 2024-02-21T00:38:57.036211Z >=20 > Because we were using WITHOUT_JAIL, to improve the time spent, one of my = ideas was to divide the tests into groups of 4 or 8 and run as many bhyve V= Ms in parallel. > I assumed that by running regression tests in a jail, I was testing the j= ail system as much as the test object (and for our use case, I don=E2=80= =99t care about the jail because we are not using it). > So using the bhyve VM I had the feeling the tests were more confined. >=20 > Now if WITH_JAIL becomes mandatory to be able to run the test, I will jus= t have to modify my build system (like by adding a WITHOUT_JAIL just at ins= tallworld time and not during buildworld). >=20 It would be great if you can test the patch. The general test plan instructions are here: https://reviews.freebsd.org/D4= 2350. It includes instructions for your case WITHOUT_JAIL. It's expected that you need no changes in your build system, the Kyua should be built without jail support and your usual test runs should work as usually. And any execenv=3D= jail based test on your way should be automatically skipped by such instance of = Kyua. Best regards, Igor.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?srdDbPKj-bU3DzlBDM23UMTrW9s9EwTzFZVvQrULqDLySbYuBxmWO6BMjAua_kFWbP-dKJKViUR-usMp51wHx8atAk5kaf_afhyh_cZcihs=>