From owner-freebsd-testing@FreeBSD.ORG Mon Jul 28 00:01:08 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1C91700 for ; Mon, 28 Jul 2014 00:01:08 +0000 (UTC) Received: from mail-qg0-f49.google.com (mail-qg0-f49.google.com [209.85.192.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6486F275D for ; Mon, 28 Jul 2014 00:01:07 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id j107so7621846qga.22 for ; Sun, 27 Jul 2014 17:01:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=JGSF2KrvI1m0Jr+UGEgoMMYkU29eGyk7CLVI1BCrgFs=; b=VfGiS0F6eet965gytMZY2ALQS2HWJQQZCCBcPv2eU/HOhUQrkTBQv17zqN2WnTQt9r 6wz3w8WKjPgipK/zuGqVFjrMEI2TKNsp2JLkUB7hZSn5BKTNwBKZ0IUhg4KEb+91te5f Qimix0XjQ5EpdiLZAplGoXnByruIJfRhA0aRLoO/oxq3cAsmjtrYC+o2sUgXTYH1q/G+ g8fgqXxX7kQO87PB7B1MJ0qeRZHMY28xnkHXkzrYNDOBk9SuzNGXLCAnxp3pysk+ernV ga6d2obZteLZsZBxtwzsY4VHUr6qNKzkZXm8IUb0H5o993qLGKT9ykIxkhlH7Ppd6y+a sLCA== X-Gm-Message-State: ALoCoQksqOCWaxOsrlS1yzYcfRPR6vm7ClqEXIx5WsYyjmNUmWpc1ZgE6NSDoLAyfSXRKQKvOW6D X-Received: by 10.224.42.196 with SMTP id t4mr52567304qae.48.1406505661645; Sun, 27 Jul 2014 17:01:01 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Sun, 27 Jul 2014 17:00:41 -0700 (PDT) X-Originating-IP: [50.202.45.106] In-Reply-To: <41d8a2fb.75b47028@fabiankeil.de> References: <41d8a2fb.75b47028@fabiankeil.de> From: Julio Merino Date: Sun, 27 Jul 2014 17:00:41 -0700 X-Google-Sender-Auth: lQX-5yW9PqBTTMYwtLLkg2hN9Ac Message-ID: Subject: Re: atf_check() equivalent to test individual shell functions? To: Fabian Keil Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 00:01:08 -0000 On Sat, Jul 26, 2014 at 4:30 AM, Fabian Keil wrote: > Here's another zogftw-related problem. What's zogftw? > Putting something like this in a test case works as expected: > > | atf_check -s exit:0 -o empty -e empty $ZOGFTW cmd zogftw_string_ends_with "abc" "c" What is "$ZOGFTW cmd ..." supposed to be doing? > It would be preferably to source zogftw once at the beginning > and call the shell function zogftw_string_ends_with() directly > afterwards to reduce the overhead, though. > > The following sort of does this, but the stdout and stderr > output isn't verified, it's tedious to write and if nothing > fails, the "kyua debug" output isn't particular useful: > > | . $ZOGFTW source || atf_fail "Failed to source zogftw" > | > | if ! zogftw_string_ends_with "abc" "c"; then > | atf_fail "Failed to find 'c' at the end of 'abc'" > | fi > > The following also sort of works, but in case of errors the output > isn't helpful (because atf_check_equal() doesn't see the actual > function) and the stderr output isn't verified either: > > | . $ZOGFTW source || atf_fail "Failed to source zogftw" > | > | atf_check_equal $(zogftw_string_ends_with "abc" "c"; echo $?) "0" Or just: zogftw_string_ends_with "abc" "c" || atf_fail "abc doesn't end with c" There is no reason to compare the exit code literally with atf_check_equal. > What I'm looking for is something like this: > > | atf_check_shell_function -s return:0 -o empty -e empty zogftw_string_ends_with "abc" "c" That could be interesting. However, note that atf_check currently is just a thin wrapper over the atf-check binary and that's the reason why you cannot feed shell functions into it. Implementing an atf_check_shell_function would involve rewriting atf-check's functionality in libatf-sh.subr. > atf-sh-api(3) doesn't seem to mention anything like it, > so I'm wondering how other people are testing individual > shell functions? Take a look at the test programs in here for other ideas: https://github.com/jmmv/shtk The background is that atf-sh has been mostly used for integration testing until now so the support for unit-testing shell functions hasn't been necessary. I agree that building some would be good.