From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 16:37:25 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17F532A0 for ; Thu, 6 Mar 2014 16:37:25 +0000 (UTC) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.freebsd.org (Postfix) with SMTP id ABBB3B1F for ; Thu, 6 Mar 2014 16:37:24 +0000 (UTC) Received: (qmail 53311 invoked from network); 6 Mar 2014 16:37:17 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay01.pair.com with SMTP; 6 Mar 2014 16:37:17 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s26GbGjW026243; Thu, 6 Mar 2014 17:37:16 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s26GbGTu026242; Thu, 6 Mar 2014 17:37:16 +0100 (CET) (envelope-from pho) Date: Thu, 6 Mar 2014 17:37:16 +0100 From: Peter Holm To: Garrett Cooper Subject: Re: A shell script test case Message-ID: <20140306163716.GC25614@x2.osted.lan> References: <20140305123307.GA77355@x2.osted.lan> <92FD8D48-FF95-41E6-A08D-F123E80AD8C3@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <92FD8D48-FF95-41E6-A08D-F123E80AD8C3@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 16:37:25 -0000 On Thu, Mar 06, 2014 at 07:51:46AM -0800, Garrett Cooper wrote: > On Mar 5, 2014, at 4:33 AM, Peter Holm wrote: > > > I tried my hand with a test scenario for a current FreeBSD VM problem: > > At first I did not understand what I did wrong when I got: > > > > $ kyua test vm_pageout_test > > vm_pageout_test:__test_cases_list__ -> broken: Tester did not exit > > cleanly: kyua-atf-tester: Test program list did not return success > > [0.000s] > > > > 0/1 passed (1 failed) > > Committed action 15 > > $ > > > > but I figured it out. > > > > Now to the real question: Are regression tests like this one really > > ATF material? I would think that the long runtime could be prohibitive? > > You should set the timeout variable to something other than 300 if it will take longer to execute (from atf-test-case(4)): > I should have included the Makefile. I used: "TEST_METADATA.vm_pageout_test+= timeout="1200" > timeout Type: integral. Optional; defaults to `300'. > > Specifies the maximum amount of time the test case can > run. This is particularly useful because some tests > can stall either because they are incorrectly coded or > because they trigger an anomalous behavior of the pro- > gram. It is not acceptable for these tests to stall > the whole execution of the test program. > > Can optionally be set to zero, in which case the test > case has no run-time limit. This is discouraged. > > > $ kyua debug vm_pageout_test:basic > > FAIL > > UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND > > 0 662 1 0 52 0 10268 0 pause IWs - 0:00.00 nfsuserd: master (nfsuserd) > > 25 845 1 0 20 0 12796 0 pause IWs - 0:00.00 sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue (sendmail) > > 0 850 1 0 20 0 10024 0 nanslp SWs - 0:00.00 /usr/sbin/cron -s > > 1001 4069 4068 0 20 0 11808 0 wait IWs 0 0:00.00 -bash (bash) > > 0 4115 4069 0 20 0 10496 0 wait IW 0 0:00.00 su > > 0 4116 4115 0 23 0 10772 0 pause IW 0 0:00.00 _su (csh) > > 1001 4354 4353 0 20 0 11808 0 wait IWs 1 0:00.00 -bash (bash) > > 0 4416 4354 0 20 0 10496 0 wait IW 1 0:00.00 su > > 0 4417 4416 0 23 0 10772 0 pause IW 1 0:00.00 _su (csh) > > 0 4419 4417 0 20 0 11808 0 wait IW 1 0:00.00 bash > > 0 6838 4419 0 52 0 13412 0 wait IW+ 1 0:00.00 kyua debug vm_pageout_test:basic > > 0 6857 6838 0 52 0 9900 0 wait IW 1 0:00.00 /usr/local/libexec/kyua-atf-tester -t1200 test -vhas.cleanup=false -vunprivileged-user=tests /usr/tests/sys/vm/vm_pageout_test basi > > vm_pageout_test:basic -> failed: Failed > > $ kyua test vm_pageout_test > > vm_pageout_test:basic -> failed: Failed [367.941s] > > Why this took 67 seconds (367 total) to fail beyond the standard timeout (300) is a bit puzzling/concerning. I expect there to be an additional 1-5 seconds of needed cleanup, but 67 seems a bit high... > Sorry about the confusion. It's the "sort /dev/zero" that take up most of the time; Just a simple way of generate some VM usage. Anyway I think I have come to the conclusion that these kind of resource hogging tests probably do not belong in ATF. But as "my first shell script test" it was a success :) > Thanks! > -Garrett -- Peter