From owner-freebsd-testing@FreeBSD.ORG Sun Nov 24 22:30:02 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B9B645E for ; Sun, 24 Nov 2013 22:30:02 +0000 (UTC) Received: from mail-lb0-f178.google.com (mail-lb0-f178.google.com [209.85.217.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BE524295A for ; Sun, 24 Nov 2013 22:30:01 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id c11so2558982lbj.23 for ; Sun, 24 Nov 2013 14:29:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=Frb5gadeB98qv1dhrWEGLKxg+zAv39ftm/l4ydRTEX4=; b=f7NSdZLcvGa9EfFR1YkxicnDMNqlTVrIlxNgFqb+QexWVSr+XeW66EuVJjcOzvUoBq 5DeNn9KnduzlrMgMFShwfhWtrKslXQ49dwPjVIGpVFwOWhq2zPKZbpEvbr0be/wqQMjx NdhpX2+K0T2BuvKPR0uR2nWRx3O6va8ZJ6fF5KG76sw7POQH5dW5ZOLeaqlYbzygSXUK Yonl0zo/LE/2mrCjOLcqhXcHgGqjRnYXt6d2Yl+jJ9tDXa9GLCelN2zfhzhjTjpbi+hc d4DVjSO4uxUp2vXq2DtHEL9qektNt3MDOPS99uu9/EltN78SBBdqr/Du2aWQO+ME3tw8 d1dw== X-Gm-Message-State: ALoCoQkLipYWY4fgwaSt3gctGp9+ivECCfPYxMeZYAMXgHCclkAvawSW0rkveplqSuW46OC+2QNT X-Received: by 10.152.116.46 with SMTP id jt14mr3654lab.31.1385332193563; Sun, 24 Nov 2013 14:29:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.168.65 with HTTP; Sun, 24 Nov 2013 14:29:33 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: <82E5F5E2-EA2C-46D2-9068-C419A834686C@gmail.com> References: <82E5F5E2-EA2C-46D2-9068-C419A834686C@gmail.com> From: Julio Merino Date: Sun, 24 Nov 2013 17:29:33 -0500 Message-ID: Subject: Re: Adding new "test interfaces" to work with Kyua? To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2013 22:30:02 -0000 On Sun, Nov 24, 2013 at 5:09 PM, Garrett Cooper wro= te: > Trying to reconcile my diffs and get things sane again, I realize= that due to refactoring (and especially after I installed devel/kyua) that= prove.test.mk no longer functions for me off my branch. Are there directio= ns for how I can program in a new test interface so I can integrate prove t= esting into Kyua? What kind of interface do prove-based test programs have? Are they binaries/scripts that you can just run and assume that a 0 exit code means passed and 1 means failed? If yes, you'd just funnel them through the 'plain' interface and they should just work as a first cut. (I believe this approach works for most test programs out there.) But there are some details here: https://code.google.com/p/kyua/wiki/TestersDesign and more in the kyua-tester manual page and its references. That said... I'm not sure now if I have detailed the "protocol" in detail anywhere... and if not, that's something for me to start doing :P --=20 Julio Merino / @jmmv