From owner-freebsd-testing@FreeBSD.ORG Thu Dec 18 21:13:39 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D1417D2 for ; Thu, 18 Dec 2014 21:13:39 +0000 (UTC) Received: from mail-qc0-f174.google.com (mail-qc0-f174.google.com [209.85.216.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1340C264C for ; Thu, 18 Dec 2014 21:13:39 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id c9so1536781qcz.33 for ; Thu, 18 Dec 2014 13:13:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=86biJhAbCQHsnLTrneyLYPcHOzzlmKW3wj9mMJY3r0E=; b=fF1CX+bjhyAjFoAbIgKNP2Ms8NXAnICMwRWrHllGgeVPxEeNBCU/iATjUZ75WT1rMH CphUZZQNcEIolgaKyq1BxM9LWhV8m0eN2lGSt/5gAFuAFZtwk9hJQx1vuymbORcJpSU2 za+2wiWjBjQYj+iPj5tbmfsOIp05ha3jcSLLibNwVMkQDezn5fou3l//6W/CO/MWZcq4 QUDkh7V+X10zYwKrqTeqw6kLEgWXDuB15gxfg5j2TSd67gEatqhu7QdTACTVW/Zn3j4m LhlgPJi3RhcW/Pkx2WK+4tRPOxM2bmTrCGKDoLf/T5i83MvjV5XvSoEKb1L3LeBySvJk 1y0Q== X-Gm-Message-State: ALoCoQnkHGV+B1o/YyK3WibCoQnbKjHojuWCr3Ne2+lYUJHDqDXrQSaNsFfZpRL7bVCbJgaLD8ev X-Received: by 10.224.95.67 with SMTP id c3mr7688550qan.3.1418937218183; Thu, 18 Dec 2014 13:13:38 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.154.4 with HTTP; Thu, 18 Dec 2014 13:13:17 -0800 (PST) X-Originating-IP: [2620:0:1003:1007:51a6:ae44:98bd:55f] In-Reply-To: References: <20141016001543.GD1852@funkthat.com> <69C71414-AE05-4AC2-B07D-23F5569C9FCA@gmail.com> <20141016003822.GE1852@funkthat.com> From: Julio Merino Date: Thu, 18 Dec 2014 16:13:17 -0500 X-Google-Sender-Auth: lyEFBhPTYvoGZP3hE6LFjnaY5Dg Message-ID: Subject: Re: help on testing for FreeBSD... To: NGie Cooper Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , John-Mark Gurney X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Dec 2014 21:13:39 -0000 On Wed, Oct 15, 2014 at 8:46 PM, NGie Cooper wrote: > On Wed, Oct 15, 2014 at 5:38 PM, John-Mark Gurney wrote: >> Garrett Cooper wrote this message on Wed, Oct 15, 2014 at 17:25 -0700: >>> - Why are they written in python? >> >> Because I'm too lazy to write code in C... I had already written >> framework code in Python, and C doesn't have a standard function to >> decode hex.. :) And error handling it much easier in python... > > I ask because we don't have a means for doing unittest level execution > right now and not being written in something that's in base or > installed automatically (perl for instance) would mean that your test > code wouldn't be run by default on jenkins runs. There are ways around this. pyUnit test programs are simple executables that return 0 on success and 1 on failure, so they can work with the "plain" interface of Kyua. We can then mark the test program as requiring python in the Kyuafile, just as we already do for some in-tree perl-based test programs. And then, we can tell whichever driver we are using to set up the test VMs to "pkg install python" right before running the test suite. We do this already, at least from autotest, to install perl and other required utilities. I'm sure we'd do the same for the Jenkins case.