From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 17:12:39 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EF4EF224; Tue, 25 Feb 2014 17:12:39 +0000 (UTC) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9408D10D9; Tue, 25 Feb 2014 17:12:39 +0000 (UTC) Received: by mail-vc0-f172.google.com with SMTP id lf12so7633918vcb.17 for ; Tue, 25 Feb 2014 09:12:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pkcj1ysaewpfHxLWw83e5kavEZ3b7dr3s69VFMNLP4o=; b=QGD4cgdHUVU90YlJEijfY0r6w5mALsuW+YW/ighFn3bbuMBZbPpvj914p4cOGRzc7y rFWs5l7pJQoMbmCnkIQymKVDHC5FNU63kOnfE2edJH5IvTV5CyVpv1CzOniL5htNeIBg T+QhHykaOOzws9WD3fncoJ546OZW+bKuuMKO9KnWZyMhn3QevFrpWpAJpCo53Db6gRgP 0VcMVV6UfH/bmi0OdkyIaENTR56cotOGJ4/Ve7NelOlGA41enoT7ketvf+61fwjY5IF2 vUN1stB0XlAba+vQLfEHsl16hmcx/BNhegMklNmriuS2r8zU4pKYeGvZXYaLZfCTokZ2 Xs7A== MIME-Version: 1.0 X-Received: by 10.58.170.69 with SMTP id ak5mr1890783vec.28.1393348358682; Tue, 25 Feb 2014 09:12:38 -0800 (PST) Received: by 10.58.235.166 with HTTP; Tue, 25 Feb 2014 09:12:38 -0800 (PST) In-Reply-To: References: <20140225161129.GA59741@x2.osted.lan> Date: Tue, 25 Feb 2014 12:12:38 -0500 Message-ID: Subject: Re: My first ATF test From: Mehmet Erol Sanliturk To: Alan Somers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 17:12:40 -0000 On Tue, Feb 25, 2014 at 11:47 AM, Alan Somers wrote: > On Tue, Feb 25, 2014 at 9:11 AM, Peter Holm wrote: > > In order to understand how ATF works I wrote a small test so I had > > something to work with: > > http://people.freebsd.org/~pho/kern_descrip_test.diff > > Did I get it right? > > ATF-wise, it looks good. However, it's a bad idea to use random > numbers in test code, except in stress tests. Random numbers result > in irreproducible tests. How about replacing the body of dup2_r234131 > with something like this? > > int fd1, fd2, ret; > fd1 = open("/etc/passwd", O_RDONLY); > fd2 = INT_MAX; > ret = dup2(fd1, fd2); > ATF_CHECK_EQ(-1, ret); > ATF_CHECK_EQ(EBADF, errno); > > On a side note, perhaps WARNS should be set in atf.test.mk, so we > won't have to set it in every other Makefile. > > -Alan > _______________________________________________ > > When random numbers are used , it is possible to make the runs reproducible in the following way : Generate a specified number of random numbers and store them into a file . During usage , for random numbers , traverse that file . This may be repeated any number of times for different other parameters . All of the runs will use the same random numbers . Then the results ( which they are generated from the same distribution ) may be compared with suitable statistical tests . Thank you very much . Mehmet Erol Sanliturk