Date: Sun, 18 Jan 2009 23:29:08 +0100 From: Peter Holm <peter@holm.cc> To: Alfred Perlstein <alfred@freebsd.org> Cc: Dag-Erling Sm??rgrav <des@des.no>, freebsd-arch@freebsd.org Subject: Re: stress2 is now in projects Message-ID: <20090118222908.GA42845@x2.osted.lan> In-Reply-To: <20090118203134.GF60686@elvis.mu.org> References: <20090118082145.GA18067@x2.osted.lan> <86iqocstjm.fsf@ds4.des.no> <49733419.5000407@FreeBSD.org> <20090118203134.GF60686@elvis.mu.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Jan 18, 2009 at 12:31:34PM -0800, Alfred Perlstein wrote: > * Kris Kennaway <kris@FreeBSD.org> [090118 05:52] wrote: > > Dag-Erling Sm??rgrav wrote: > > >Peter Holm <pho@freebsd.org> writes: > > >>The key functionality of this test suite is that it runs a random > > >>number of test programs for a random period, in random incarnations > > >>and in random sequence. > > > > > >In other words, it's non-deterministic and non-reproducable. > > > > > >You should at the very least allow the user to specify the random seed. > > > > > >DES > > > > I doubt this will help at all since the test suite is (by design) > > massively parallel, so you're at the mercy of small timing changes. > > If the start and stop times of the scripts were recorded one could > synch with the original potentially between runs, at least on the > same hardware it ran. > > Basically, replay the suite based on time instead of random. > During the more than 10 years I have used this test suite with FreeBSD I have always prioritized the ability to panic the kernel. I have never looked much into a method of re-creating a test stream. As I see it, it is a *slight* inconvenience that a panic or deadlock is not 100% reproducible in time. A fix for any problem still needs a thorough test (measured in days), IMHO. Several methods exists, as I see it, to create a test stream. One could for example generate the random work list first and then execute the tests after that. After a panic you would the have the work list that created the problem. But would re-running the work list reproduce the panic? I seriously doubt that. But there is only one way to find out. It should not be that hard to create deterministic execution of the the tests. - Peter > -Alfred
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090118222908.GA42845>