From owner-freebsd-testing@FreeBSD.ORG Wed Mar 26 22:54:40 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A09E34EE for ; Wed, 26 Mar 2014 22:54:40 +0000 (UTC) Received: from mail-ve0-x22a.google.com (mail-ve0-x22a.google.com [IPv6:2607:f8b0:400c:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5ECF3927 for ; Wed, 26 Mar 2014 22:54:40 +0000 (UTC) Received: by mail-ve0-f170.google.com with SMTP id pa12so3209495veb.29 for ; Wed, 26 Mar 2014 15:54:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=8LOffpPiMqoeHQaBpAJQudzfKO9yKUkV16mpLOOij/k=; b=TqJ+3fgGG3rZLyGWOrk/o5mDFW8BbnZjVlolliTmc7V+B7+gKr0Hbe+R/fekKmrI3H q7xfubmRxgQbXp83LRdyTQWzINUrH3JD6pLhI3hP+xCDC1hProsnwHt2MUkQyWVBboss e5IIQCiZFD5rHGapauahRswHQHgYKkpkSab7qxnlZDU2PjJnVWzZos41GR16BfCGZNRP YAoKtYHVIEl/BJVqPWdDT5Eqp3lohBb04i+5lKLVsJC4ezCaLGyXeYUNaLaAEk81avHj zJSfqmzQpLwgKi1A7YnusOZhWD5HUg/pU5ExnLuNlBhzp+NUnPF5wmIZX9vEDKx7Yopr 9aiw== MIME-Version: 1.0 X-Received: by 10.58.57.67 with SMTP id g3mr61649531veq.3.1395874478409; Wed, 26 Mar 2014 15:54:38 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.58.207.169 with HTTP; Wed, 26 Mar 2014 15:54:38 -0700 (PDT) In-Reply-To: References: Date: Wed, 26 Mar 2014 16:54:38 -0600 X-Google-Sender-Auth: uPRCnph6yCpL1GuaZu1qtP0hcQs Message-ID: Subject: Re: ATF Test Cases From: Alan Somers To: "Kilner, Peter" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 22:54:40 -0000 On Wed, Mar 26, 2014 at 4:26 PM, Kilner, Peter wrote= : > Hello, > > I hope I am posting this question to the correct list. > > Is there a way to have a single ATF test case report multiple pass/fail r= esults? Currently I have seen that one can include a number of test condit= ions in a single test case (for example many aft_checks). However the test= case will only report one pass/fail result for that case. Nope. > > I would like to build a test case that will iterate though many configura= tions and would like to see a pass/fail for each config. However this is d= ifficult to implement with multiply test cases because of the for loops tha= t I am using. > > Any suggestions would be appreciated. Is this in C/C++ or sh? With atf-c-api(3), you can call atf_expect() instead of atf_require(). That way the test will continue even after one of the expectations failed. If multiple expectations fail, then I think kyua will print an error message for each failed expectation. I could be wrong however. Also, I don't think that equivalent functionality exists in atf-sh-api(3). Apart from that, you pretty much have to write multiple test cases. Note that separate test cases run in separate processes. If your for loop is very long and this is a C test program, using multiple test cases will be much slower than using a single test case with multiple expectations. -Alan > > Thank you, > > Peter Kilner > Software Engineer > EMC, Isilon Seattle > peter.kilner@isilon.com > 206-777-7859 > > > > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org= "