From owner-freebsd-testing@FreeBSD.ORG Fri Jun 27 00:00:11 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2DC21ED; Fri, 27 Jun 2014 00:00:11 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E43E26EE; Fri, 27 Jun 2014 00:00:11 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id tr6so3822273ieb.1 for ; Thu, 26 Jun 2014 17:00:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8yQTqWxq9Snlayu15hPV9sFZvabPJMZWIGu6EQsMKFI=; b=YSSvZ37DbrG1e7Fo7k/R9hURk2pY7XesJ5S1pF8frWhintX0yMAsFk+vkq/S2FB7JQ A8FzAIXuTPF6KqvemWS0JYqhOwjnrhEvCgXXFiA1KeAXK1Ee1G3WJlCFa/2hNUPO91X0 tIceL4U8dVGjUULFH6L2LcT2NUp5zR7mi0KHmEGEzK/TDCn/aQ6fPEKJoPVD1njK9SE5 6Vds169kUPJipXxF9fdYDMt1jy/yPZopU8fxaUFNOJYUKTYBLCoUsOFRha3G59m9BvG7 yqobJbym5c19m06MDNB3Kyd6HtKDrdYYg1SvramT2lTCstRQjhODzXa8LtyOSdqoN+cX TBpw== MIME-Version: 1.0 X-Received: by 10.50.138.105 with SMTP id qp9mr8468032igb.2.1403827210513; Thu, 26 Jun 2014 17:00:10 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Thu, 26 Jun 2014 17:00:10 -0700 (PDT) In-Reply-To: References: Date: Thu, 26 Jun 2014 17:00:10 -0700 Message-ID: Subject: Re: How do I extend Kyua requirements checking for FreeBSD? From: Garrett Cooper To: Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 00:00:11 -0000 On Thu, Jun 26, 2014 at 12:54 PM, Garrett Cooper wrote: > On Thu, Jun 26, 2014 at 12:39 PM, Alan Somers wrote: >> On Thu, Jun 26, 2014 at 1:05 PM, Garrett Cooper wrote: >>> Hello Julio! >>> I'm looking at extending Kyua to create special FreeBSD-specific >>> functionality, and I'm wondering how I should do it. In particular >>> there are a couple items that could be checked as prerequisite >>> requirements before running tests to avoid running tests: >>> 1. Is a service running? >>> 2. Is a driver loaded? >>> 3. Is a driver loaded as a kernel module or is it statically >>> compiled in to the kernel? >>> 4. Am I running on a particular filesystem? >>> Etc. >>> If I was to add these "requirements functions" to kyua, how would >>> I query the values in an interface agnostic way? I would usually set >>> environment variables, but this makes things potentially messier >>> (especially in C/C++ unless I create a standalone library to handle >>> this)/harder to grep for than having a command which fetches this >>> information, like atf-config does for the ATF tester interface. >>> Thank you! >>> -Garrett >> >> I did it just be creating a library of shell functions that I call >> from test bodies. For example, something like this >> function require_module() { >> kldstat $1 || atf_skip "Module $1 is not loaded" >> } > > This is sort of what I'm doing, but this only works for the ATF test > interface :/. I used kldstat -m $1 by the way for detecting statically > compiled kernel modules, like aio: > > # kldstat -q -m ada && echo ada is loaded > ada is loaded > # kldstat -q -m foobar && echo foobar is loaded > # > > It's significantly trickier with drivers that don't have moduledata_t > which matches the kernel module that's loaded, e.g. if_em: > > # kldstat -m pci/em > Id Refs Name > 117 1 pci/em > # ifconfig em0 > em0: flags=8843 metric 0 mtu 1500 > options=9b > ether 00:0c:29:a3:e9:9c > inet 10.7.216.233 netmask 0xfffffe00 broadcast 10.7.217.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > # kldload if_em > kldload: can't load if_em: module already loaded or in kernel > >> But that's not useful for C and C++ tests. And it would be way cooler >> if it were more tightly integrated into Kyua. Would it be possible to >> load site-specific Lua code that would handle this sort of >> functionality, so the test could simply say "atf_set require.modules >> aio"? > > Exactly! > > I'm going to write the C/shell libraries for ATF at least, add some > testcases, then we can look at making them generic via Kyuafile/the > testcase header/etc. I'm started work on it on github: https://github.com/yaneurabeya/freebsd_atf_kyua_test_extensions . Any review would be much appreciated! Testcases coming soon of course. Thanks! -Garrett