From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 18:42:53 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE4AA7B4 for ; Tue, 25 Feb 2014 18:42:53 +0000 (UTC) Received: from relay03.pair.com (relay03.pair.com [209.68.5.17]) by mx1.freebsd.org (Postfix) with SMTP id 912CD1ABA for ; Tue, 25 Feb 2014 18:42:53 +0000 (UTC) Received: (qmail 59729 invoked from network); 25 Feb 2014 18:36:12 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay03.pair.com with SMTP; 25 Feb 2014 18:36:12 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s1PIaBYu070871; Tue, 25 Feb 2014 19:36:11 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s1PIaB0Q070870; Tue, 25 Feb 2014 19:36:11 +0100 (CET) (envelope-from pho) Date: Tue, 25 Feb 2014 19:36:11 +0100 From: Peter Holm To: Mehmet Erol Sanliturk Subject: Re: My first ATF test Message-ID: <20140225183611.GA70774@x2.osted.lan> References: <20140225161129.GA59741@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 18:42:53 -0000 On Tue, Feb 25, 2014 at 12:12:38PM -0500, Mehmet Erol Sanliturk wrote: > On Tue, Feb 25, 2014 at 11:47 AM, Alan Somers <[1]asomers@freebsd.org> > wrote: > > On Tue, Feb 25, 2014 at 9:11 AM, Peter Holm wrote: > > In order to understand how ATF works I wrote a small test so I had > > something to work with: > > [2]http://people.freebsd.org/~pho/kern_descrip_test.diff > > Did I get it right? > ATF-wise, it looks good. Â However, it's a bad idea to use random > numbers in test code, except in stress tests. Â Random numbers > result > in irreproducible tests. Â How about replacing the body of > dup2_r234131 > with something like this? > Â int fd1, fd2, ret; > Â fd1 = Â open("/etc/passwd", O_RDONLY); > Â fd2 = INT_MAX; > Â ret = dup2(fd1, fd2); > Â ATF_CHECK_EQ(-1, ret); > Â ATF_CHECK_EQ(EBADF, errno); > On a side note, perhaps WARNS should be set in [3]atf.test.mk, so we > won't have to set it in every other Makefile. > -Alan > _______________________________________________ > > When random numbers are used , it is possible to make the runs > reproducible in the following way : > Generate a specified number of random numbers and store them into a > file . > During usage , for random numbers , traverse that file . > This may be repeated any number of times for different other parameters > . > All of the runs will use the same random numbers . > Then the results ( which they are generated from the same distribution > ) may be compared with suitable statistical tests . > Thank you very much . > Mehmet Erol Sanliturk > I guess I'm too used to using random values in tests :) Using random(3) with an initial seed of "1" could have been an alternative, but ... - Peter