From owner-freebsd-testing@FreeBSD.ORG Wed Nov 27 13:37:31 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F531392 for ; Wed, 27 Nov 2013 13:37:31 +0000 (UTC) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1B28D20C7 for ; Wed, 27 Nov 2013 13:37:30 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id z5so5455825lbh.3 for ; Wed, 27 Nov 2013 05:37:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=a5Mng/yu6wkl7f+FczU6p8YDFhZgqodaAZRRreJBwM8=; b=jVwV+d4x1KUquESuxhy/NIrMHf3/I/FV8X7278sUof86ttZtn/bAmmlafVCQPDW3n7 SH3fiNnkMBTVFILmPjygdnJu7my7ocmCGkPYaL2/Ap+6FJT26cPIRRqPz/z4PfCyurxm RoziQ1Dj07aFTY/DQUk1a7g1DzJ8tsOb8WKqNjQ3wWmyboGhOvzd8DzjXlWvN+ONqc7Q kbJ8ycJFa3K+wudzGjQ+M6xumrE17+qj1DW6n9u4An49oXLJvCaCoxEARD8AHPOOE8JO kzG+vuF2n1FBqrVVY3M1WBFWld9H//+ul+MiQRcZ5ypDmXK5zibj1jbbfdNGz+90qGeD g1bQ== X-Gm-Message-State: ALoCoQkPm6JOOwsqttAUw5vksTbeFKV16nESo7Qr2gQmkBh6VODy33xulHc3a6M99WRk1MQpVg/Y X-Received: by 10.152.225.161 with SMTP id rl1mr29526728lac.5.1385559442967; Wed, 27 Nov 2013 05:37:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.133.74 with HTTP; Wed, 27 Nov 2013 05:37:02 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: References: <82E5F5E2-EA2C-46D2-9068-C419A834686C@gmail.com> <751B8D9F-9A1B-46B5-80F8-CB9E44F98BFC@gmail.com> From: Julio Merino Date: Wed, 27 Nov 2013 08:37:02 -0500 Message-ID: Subject: Re: Adding new "test interfaces" to work with Kyua? To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Nov 2013 13:37:31 -0000 On Tue, Nov 26, 2013 at 11:21 PM, Alan Somers wrote: > It looks like you're interfacing to TAP through its command-line > interface. Unfortunately, as your man page notes, that means that > "kyua list" can't list bundled tests, and thus it will treat them as a > single unit. > > Within FreeBSD, most if not all of the TAP-based tests use Perl's > Prove module. Have you considered writing a Prove backend for Kyua > instead of a TAP backend? If you write it in Perl, it could have much > more intimate access to the tests. Instead of interfacing through the > CLI, it could dynamically load the tests and interrogate them > directly. I hadn't considered that, but it is a good idea. However, having separate TAP and Prove backends are not mutually exclusive goals. That said, I don't find the Prove backend a pressing need nor I think it's as useful as a TAP backend, at least for the time being: what I'd like is a simple way to transition existing tests to the new layout, and the TAP backend works for that -- with the added benefit that it works also for non-Perl tests! Later on, if people want to keep the Perl-based tests and make them more versatile within Kyua, we can do that pretty easily. (And of course, if anyone wants to start writing a more versatile backend explicitly for Perl tests anyway, please do!) -- Julio Merino / @jmmv