From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 15:51:49 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3E18870 for ; Thu, 6 Mar 2014 15:51:48 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BA77765C for ; Thu, 6 Mar 2014 15:51:48 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id ld10so2815648pab.12 for ; Thu, 06 Mar 2014 07:51:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Y+w/J2m/yR2ScYEooKfxhnBNZ59hPkt7BrojqmONEVc=; b=D/ZrLWTx7n6Ujlq6AFizcSp52ZM5VFq4w33l01L3mC95rnXMkVkNrZ9sfmLOepIb+/ hbdsJlVObr/IEW9ta55KYzt8BL5qIEiVarGPC97/29dAA+VlEWVsM4zy2zq3FBr/ZT18 c6OAG5BpIJta69VpSGTrkAZYg93wfSPBOu5qDiEI2eorKh/1MFpEsLwIoqvcHxTYt792 sEm8XB5J4val5I992YbjHJUzE3XUh/tWlBOThsiTE3Kf7RHN7v6vuaLCaayu6rQxgtLN upjeX2z30DfhaiFc6Av83F5Ifs1Roe0w7L6ET1Uj+/HzQrPkvsyQIH+xdk+X75YDyBRC N5ow== X-Received: by 10.68.189.33 with SMTP id gf1mr15080523pbc.111.1394121108461; Thu, 06 Mar 2014 07:51:48 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:1dcf:f87a:25ca:10b2? ([2601:8:ab80:7d6:1dcf:f87a:25ca:10b2]) by mx.google.com with ESMTPSA id n6sm21162761pbj.22.2014.03.06.07.51.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 06 Mar 2014 07:51:46 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: A shell script test case From: Garrett Cooper In-Reply-To: <20140305123307.GA77355@x2.osted.lan> Date: Thu, 6 Mar 2014 07:51:46 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <92FD8D48-FF95-41E6-A08D-F123E80AD8C3@gmail.com> References: <20140305123307.GA77355@x2.osted.lan> To: Peter Holm X-Mailer: Apple Mail (2.1874) Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 15:51:49 -0000 On Mar 5, 2014, at 4:33 AM, Peter Holm wrote: > I tried my hand with a test scenario for a current FreeBSD VM problem: > At first I did not understand what I did wrong when I got: >=20 > $ kyua test vm_pageout_test > vm_pageout_test:__test_cases_list__ -> broken: Tester did not exit > cleanly: kyua-atf-tester: Test program list did not return success > [0.000s] >=20 > 0/1 passed (1 failed) > Committed action 15 > $=20 >=20 > but I figured it out. >=20 > Now to the real question: Are regression tests like this one really > ATF material? I would think that the long runtime could be = prohibitive? You should set the timeout variable to something other than 300 if it = will take longer to execute (from atf-test-case(4)): timeout Type: integral. Optional; defaults to `300'. Specifies the maximum amount of time the test = case can run. This is particularly useful because some = tests can stall either because they are incorrectly = coded or because they trigger an anomalous behavior of = the pro- gram. It is not acceptable for these tests to = stall the whole execution of the test program. Can optionally be set to zero, in which case the = test case has no run-time limit. This is = discouraged. > $ kyua debug vm_pageout_test:basic > FAIL > UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND > 0 662 1 0 52 0 10268 0 pause IWs - 0:00.00 nfsuserd: = master (nfsuserd) > 25 845 1 0 20 0 12796 0 pause IWs - 0:00.00 sendmail: = Queue runner@00:30:00 for /var/spool/clientmqueue (sendmail) > 0 850 1 0 20 0 10024 0 nanslp SWs - 0:00.00 = /usr/sbin/cron -s > 1001 4069 4068 0 20 0 11808 0 wait IWs 0 0:00.00 -bash = (bash) > 0 4115 4069 0 20 0 10496 0 wait IW 0 0:00.00 su > 0 4116 4115 0 23 0 10772 0 pause IW 0 0:00.00 _su (csh) > 1001 4354 4353 0 20 0 11808 0 wait IWs 1 0:00.00 -bash = (bash) > 0 4416 4354 0 20 0 10496 0 wait IW 1 0:00.00 su > 0 4417 4416 0 23 0 10772 0 pause IW 1 0:00.00 _su (csh) > 0 4419 4417 0 20 0 11808 0 wait IW 1 0:00.00 bash > 0 6838 4419 0 52 0 13412 0 wait IW+ 1 0:00.00 kyua debug = vm_pageout_test:basic > 0 6857 6838 0 52 0 9900 0 wait IW 1 0:00.00 = /usr/local/libexec/kyua-atf-tester -t1200 test -vhas.cleanup=3Dfalse = -vunprivileged-user=3Dtests /usr/tests/sys/vm/vm_pageout_test basi > vm_pageout_test:basic -> failed: Failed > $ kyua test vm_pageout_test =20 > vm_pageout_test:basic -> failed: Failed [367.941s] Why this took 67 seconds (367 total) to fail beyond the standard timeout = (300) is a bit puzzling/concerning. I expect there to be an additional = 1-5 seconds of needed cleanup, but 67 seems a bit high... Thanks! -Garrett=