Date: Mon, 14 Sep 2020 18:51:59 +0000 From: Teran McKinney <sega01@go-beyond.org> To: Lawrence Greening <lg49152@outlook.com> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: Developers' Handbook: Chapter on testing Message-ID: <20200914185159.GA25758@daemon> In-Reply-To: <DM6PR04MB54847A47689C03D86619C3E5B2230@DM6PR04MB5484.namprd04.prod.outlook.com> References: <DM6PR04MB54847A47689C03D86619C3E5B2230@DM6PR04MB5484.namprd04.prod.outlook.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-09-14 08-25-21 , Lawrence Greening wrote: > Hi. > ??? > A variation of the following was posted to freebsd-testing. I would generally avoid cross-posting but I'm reposting here because of recent interest in testing on this list... > > I am writing a chapter on testing for the FreeBSD Developers' Handbook. > > The following topics are expected to be covered: > > * FreeBSD Test Suite > * Kyua > * Automated Testing Framework (ATF) > * Organization of current FreeBSD tests. > * Using automation and virtualization to reduce the overhead in setting up the test infrastructure. > * Examples of different types of testing. > * Reproducibility in building and testing. > > This list of topics is just a starting point. The scope and organization will evolve as the chapter is written. > > After reading the chapter, the reader should be able to do at least the following: > > * Install and configure the necessary testing tools, test suites, etc. for the purposes of testing FreeBSD. > * Run tests. > * Write tests. > * Submit tests. > > The chapter will include walkthroughs and guided or cumulative examples. > > At this preliminary stage, I am interested in the following feedback: > > (1) What material do you want to see covered? > (2) What material do you want to see prioritized? > (3) What general approach(es) do you want to see taken towards the subject matter? > (4) Are there good example tests or areas of the code base that deserve focus, either because of their importance, usefulness as examples, history of regressions, etc.? > (5) Anything else? > > Cheers, > Lawrence > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" Hi Lawrence, I personally like this idea a lot. I guess it might be a bit too late for 12.2-RELEASE but maybe could land in time for 13.0-RELEASE? I also wonder if there could be a single burn-in style command to run all of the tests (after making sure the kyua package is installed) once a box is installed. Or maybe an installer option to run the tests on first boot. I think though that some are not compatible with all of the security options, so may be some conflicts there. One angle I'm curious about is vulnerability-based testing, where a vulnerability comes out and a version of the exploit becomes a test case that can be ran. Not sure how much that is used with FreeBSD though or how relevant it is, just thought I'd mention it. Thank you, Teran
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20200914185159.GA25758>