From owner-freebsd-testing@FreeBSD.ORG Wed Nov 27 04:21:25 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3E1F206 for ; Wed, 27 Nov 2013 04:21:25 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 80FCA252F for ; Wed, 27 Nov 2013 04:21:25 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id hi5so6181382wib.14 for ; Tue, 26 Nov 2013 20:21:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=QSxzmoDO+VuCxbJJdjOOhobp5JUNu6xW92DfHMnimBw=; b=ocl20E1KfvrA649VhRJ4+u4393kIWBXNaZgD0yYA/mjX/nhbWWMMhNe9qHQt9PLPHa USLGDCA6wJaaSU4zgEgwRMi+OFDuWYAVirUNK6qMTgfRkWrCT2rCjwDQenMWW13KAwIf aj3yKXWDa5HhyZC7wiexas4bUNw1nr/MMju7R1iU1LQWjAGs1xoo1FivEFo37r7fSivK ag8oVzp912QGvpj7nsGl1LxBZ3ZOS0aFJUzZidSAn5x8jp3+EzuaETqwE4uwfHCzLQqS jdqRivJ6RSVlHGm2+Y2TU4lhCt9kP4F2WmX/g6IQbbMxTLUPhXZ9dCj5xmL3D/OImapx dFCA== MIME-Version: 1.0 X-Received: by 10.194.232.133 with SMTP id to5mr9802770wjc.41.1385526084002; Tue, 26 Nov 2013 20:21:24 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.171.35 with HTTP; Tue, 26 Nov 2013 20:21:23 -0800 (PST) In-Reply-To: References: <82E5F5E2-EA2C-46D2-9068-C419A834686C@gmail.com> <751B8D9F-9A1B-46B5-80F8-CB9E44F98BFC@gmail.com> Date: Tue, 26 Nov 2013 21:21:23 -0700 X-Google-Sender-Auth: kqaB_iUE08v07DejG6cs-MmZbtQ Message-ID: Subject: Re: Adding new "test interfaces" to work with Kyua? From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Nov 2013 04:21:26 -0000 It looks like you're interfacing to TAP through its command-line interface. Unfortunately, as your man page notes, that means that "kyua list" can't list bundled tests, and thus it will treat them as a single unit. Within FreeBSD, most if not all of the TAP-based tests use Perl's Prove module. Have you considered writing a Prove backend for Kyua instead of a TAP backend? If you write it in Perl, it could have much more intimate access to the tests. Instead of interfacing through the CLI, it could dynamically load the tests and interrogate them directly. That's the approach that I'm taking with the Python backend that I've been working on. I'll post more details about that in a separate thread. -Alan On Tue, Nov 26, 2013 at 9:29 AM, Julio Merino wrote: > On Mon, Nov 25, 2013 at 12:13 PM, Julio Merino wrote: >> Bleh. Turns out that prove-based test programs do not return 1 on >> failure. Time to write a prove tester given that I have some spare >> time this week :-P > > Alright! I've committed a preliminary implementation of a tester for > TAP. See https://code.google.com/p/kyua/issues/detail?id=74 for the > relevant commits. > > Testing welcome. > > -- > Julio Merino / @jmmv > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org"