Date: Thu, 6 Mar 2014 07:51:46 -0800 From: Garrett Cooper <yanegomi@gmail.com> To: Peter Holm <peter@holm.cc> Cc: freebsd-testing@freebsd.org Subject: Re: A shell script test case Message-ID: <92FD8D48-FF95-41E6-A08D-F123E80AD8C3@gmail.com> In-Reply-To: <20140305123307.GA77355@x2.osted.lan> References: <20140305123307.GA77355@x2.osted.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mar 5, 2014, at 4:33 AM, Peter Holm <peter@holm.cc> wrote:
> I tried my hand with a test scenario for a current FreeBSD VM problem:
> At first I did not understand what I did wrong when I got:
>
> $ kyua test vm_pageout_test
> vm_pageout_test:__test_cases_list__ -> broken: Tester did not exit
> cleanly: kyua-atf-tester: Test program list did not return success
> [0.000s]
>
> 0/1 passed (1 failed)
> Committed action 15
> $
>
> but I figured it out.
>
> Now to the real question: Are regression tests like this one really
> ATF material? I would think that the long runtime could be prohibitive?
You should set the timeout variable to something other than 300 if it will take longer to execute (from atf-test-case(4)):
timeout Type: integral. Optional; defaults to `300'.
Specifies the maximum amount of time the test case can
run. This is particularly useful because some tests
can stall either because they are incorrectly coded or
because they trigger an anomalous behavior of the pro-
gram. It is not acceptable for these tests to stall
the whole execution of the test program.
Can optionally be set to zero, in which case the test
case has no run-time limit. This is discouraged.
> $ kyua debug vm_pageout_test:basic
> FAIL
> UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND
> 0 662 1 0 52 0 10268 0 pause IWs - 0:00.00 nfsuserd: master (nfsuserd)
> 25 845 1 0 20 0 12796 0 pause IWs - 0:00.00 sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue (sendmail)
> 0 850 1 0 20 0 10024 0 nanslp SWs - 0:00.00 /usr/sbin/cron -s
> 1001 4069 4068 0 20 0 11808 0 wait IWs 0 0:00.00 -bash (bash)
> 0 4115 4069 0 20 0 10496 0 wait IW 0 0:00.00 su
> 0 4116 4115 0 23 0 10772 0 pause IW 0 0:00.00 _su (csh)
> 1001 4354 4353 0 20 0 11808 0 wait IWs 1 0:00.00 -bash (bash)
> 0 4416 4354 0 20 0 10496 0 wait IW 1 0:00.00 su
> 0 4417 4416 0 23 0 10772 0 pause IW 1 0:00.00 _su (csh)
> 0 4419 4417 0 20 0 11808 0 wait IW 1 0:00.00 bash
> 0 6838 4419 0 52 0 13412 0 wait IW+ 1 0:00.00 kyua debug vm_pageout_test:basic
> 0 6857 6838 0 52 0 9900 0 wait IW 1 0:00.00 /usr/local/libexec/kyua-atf-tester -t1200 test -vhas.cleanup=false -vunprivileged-user=tests /usr/tests/sys/vm/vm_pageout_test basi
> vm_pageout_test:basic -> failed: Failed
> $ kyua test vm_pageout_test
> vm_pageout_test:basic -> failed: Failed [367.941s]
Why this took 67 seconds (367 total) to fail beyond the standard timeout (300) is a bit puzzling/concerning. I expect there to be an additional 1-5 seconds of needed cleanup, but 67 seems a bit high...
Thanks!
-Garrett
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?92FD8D48-FF95-41E6-A08D-F123E80AD8C3>