From owner-svn-src-head@freebsd.org Wed Jun 28 08:22:06 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BA8FD9BB62; Wed, 28 Jun 2017 08:22:06 +0000 (UTC) (envelope-from ngie@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD0DD6F729; Wed, 28 Jun 2017 08:22:05 +0000 (UTC) (envelope-from ngie@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v5S8M5Bn095883; Wed, 28 Jun 2017 08:22:05 GMT (envelope-from ngie@FreeBSD.org) Received: (from ngie@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v5S8M4P4095877; Wed, 28 Jun 2017 08:22:04 GMT (envelope-from ngie@FreeBSD.org) Message-Id: <201706280822.v5S8M4P4095877@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: ngie set sender to ngie@FreeBSD.org using -f From: Ngie Cooper Date: Wed, 28 Jun 2017 08:22:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r320443 - in head/share/examples/tests/tests: . tap X-SVN-Group: head X-SVN-Commit-Author: ngie X-SVN-Commit-Paths: in head/share/examples/tests/tests: . tap X-SVN-Commit-Revision: 320443 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jun 2017 08:22:06 -0000 Author: ngie Date: Wed Jun 28 08:22:04 2017 New Revision: 320443 URL: https://svnweb.freebsd.org/changeset/base/320443 Log: Add kyua TAP test integration examples The examples are patterned loosely after the ATF examples, similar to the plain test examples. MFC after: 1 month Added: head/share/examples/tests/tests/tap/ head/share/examples/tests/tests/tap/Kyuafile - copied, changed from r320415, head/share/examples/tests/tests/plain/Kyuafile head/share/examples/tests/tests/tap/Makefile - copied, changed from r320415, head/share/examples/tests/tests/plain/Makefile head/share/examples/tests/tests/tap/Makefile.depend - copied unchanged from r320415, head/share/examples/tests/tests/plain/Makefile.depend head/share/examples/tests/tests/tap/cp_test.sh - copied, changed from r320415, head/share/examples/tests/tests/plain/cp_test.sh head/share/examples/tests/tests/tap/printf_test.c - copied, changed from r320415, head/share/examples/tests/tests/plain/printf_test.c Modified: head/share/examples/tests/tests/Makefile Modified: head/share/examples/tests/tests/Makefile ============================================================================== --- head/share/examples/tests/tests/Makefile Wed Jun 28 08:20:51 2017 (r320442) +++ head/share/examples/tests/tests/Makefile Wed Jun 28 08:22:04 2017 (r320443) @@ -19,6 +19,7 @@ # the auto-generated Kyuafile to recurse into these directories. TESTS_SUBDIRS+= atf TESTS_SUBDIRS+= plain +TESTS_SUBDIRS+= tap # We leave KYUAFILE unset so that bsd.test.mk auto-generates a Kyuafile # for us based on the contents of the TESTS_SUBDIRS line above. The Copied and modified: head/share/examples/tests/tests/tap/Kyuafile (from r320415, head/share/examples/tests/tests/plain/Kyuafile) ============================================================================== --- head/share/examples/tests/tests/plain/Kyuafile Tue Jun 27 17:23:20 2017 (r320415, copy source) +++ head/share/examples/tests/tests/tap/Kyuafile Wed Jun 28 08:22:04 2017 (r320443) @@ -43,5 +43,5 @@ test_suite('FreeBSD') -- any metadata properties in here. These have the exact same meaning as -- their ATF counterparts. These properties are often useful to define -- prerequisites for the execution of the tests. -plain_test_program{name='cp_test', required_programs='/bin/cp'} -plain_test_program{name='printf_test'} +tap_test_program{name='cp_test', required_programs='/bin/cp'} +tap_test_program{name='printf_test'} Copied and modified: head/share/examples/tests/tests/tap/Makefile (from r320415, head/share/examples/tests/tests/plain/Makefile) ============================================================================== --- head/share/examples/tests/tests/plain/Makefile Tue Jun 27 17:23:20 2017 (r320415, copy source) +++ head/share/examples/tests/tests/tap/Makefile Wed Jun 28 08:22:04 2017 (r320443) @@ -1,7 +1,5 @@ # $FreeBSD$ -.include - # The release package to use for the tests contained within the directory # # This applies to components which rely on ^/projects/release-pkg support @@ -17,12 +15,12 @@ PACKAGE= tests # # For example: if this Makefile were in src/bin/cp/tests/, its TESTSDIR # would point at ${TESTSBASE}/bin/cp/. -TESTSDIR= ${TESTSBASE}/share/examples/tests/plain +TESTSDIR= ${TESTSBASE}/share/examples/tests/tap # List of test programs to build. Note that we can build more than one # test from a single directory, and this is expected. -PLAIN_TESTS_C= printf_test -PLAIN_TESTS_SH= cp_test +TAP_TESTS_C= printf_test +TAP_TESTS_SH= cp_test # Tell bsd.test.mk that we are providing a hand-crafted Kyuafile in this # directory. We do so because the file in this directory exists for Copied: head/share/examples/tests/tests/tap/Makefile.depend (from r320415, head/share/examples/tests/tests/plain/Makefile.depend) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/share/examples/tests/tests/tap/Makefile.depend Wed Jun 28 08:22:04 2017 (r320443, copy of r320415, head/share/examples/tests/tests/plain/Makefile.depend) @@ -0,0 +1,18 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + include/xlocale \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libcompiler_rt \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Copied and modified: head/share/examples/tests/tests/tap/cp_test.sh (from r320415, head/share/examples/tests/tests/plain/cp_test.sh) ============================================================================== --- head/share/examples/tests/tests/plain/cp_test.sh Tue Jun 27 17:23:20 2017 (r320415, copy source) +++ head/share/examples/tests/tests/tap/cp_test.sh Wed Jun 28 08:22:04 2017 (r320443) @@ -1,84 +1,99 @@ -#! /bin/sh -# $FreeBSD$ +#!/bin/sh # -# Copyright 2013 Google Inc. +# Copyright (c) 2017 Ngie Cooper # All rights reserved. # # Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are -# met: +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. # -# * Redistributions of source code must retain the above copyright -# notice, this list of conditions and the following disclaimer. -# * Redistributions in binary form must reproduce the above copyright -# notice, this list of conditions and the following disclaimer in the -# documentation and/or other materials provided with the distribution. -# * Neither the name of Google Inc. nor the names of its contributors -# may be used to endorse or promote products derived from this software -# without specific prior written permission. +# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS +# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED +# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS +# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +# POSSIBILITY OF SUCH DAMAGE. # -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +# $FreeBSD$ +# # # INTRODUCTION # -# This plain test program mimics the structure and contents of its +# This TAP test program mimics the structure and contents of its # ATF-based counterpart. It attempts to represent various test cases # in different separate functions and just calls them all from main. # -# In reality, plain test programs can be much simpler. All they have -# to do is return 0 on success and non-0 otherwise. -# -set -e +test_num=1 +TEST_COUNT=4 -# Prints an error message and exits. -err() { - echo "${@}" 1>&2 - exit 1 +result() +{ + local result=$1; shift + local result_string + + result_string="$result $test_num" + if [ $# -gt 0 ]; then + result_string="$result_string - $@" + fi + echo "$result_string" + : $(( test_num += 1 )) } # Auxiliary function to compare two files for equality. verify_copy() { - if ! cmp -s "${1}" "${2}"; then + if cmp -s "${1}" "${2}"; then + result "ok" + else + result "not ok" "${1} and ${2} differ, but they should be equal" diff -u "${1}" "${2}" - err "${1} and ${2} differ, but they should be equal" fi } simple_test() { cp "$(dirname "${0}")/file1" . - cp file1 file2 || err "cp failed" - verify_copy file1 file2 + if cp file1 file2; then + result "ok" + verify_copy file1 file2 + else + result "not ok" "cp failed" + result "not ok" "# SKIP" + fi } force_test() { echo 'File 3' >file3 chmod 400 file3 - cp -f file1 file3 || err "cp failed" - verify_copy file1 file3 + if cp -f file1 file3; then + result "ok" + verify_copy file1 file3 + else + result "not ok" "cp -f failed" + result "not ok" "# SKIP" + fi } # If you have read the cp_test.sh counterpart in the atf/ directory, you # may think that the sequencing of tests below and the exposed behavior # to the user is very similar. But you'd be wrong. # -# There are two major differences with this and the ATF version. The -# first is that the code below has no provisions to detect failures in -# one test and continue running the other tests: the first failure -# causes the whole test program to exit. The second is that this -# particular "main" has no arguments: without ATF, all test programs may -# expose a different command-line interface, and this is an issue for -# consistency purposes. +# There are two major differences with this and the ATF version. First off, +# the TAP test doesn't isolate simple_test from force_test, whereas the ATF +# version does. Secondly, the test script accepts arbitrary command line +# inputs. +echo "1..$TEST_COUNT" + simple_test force_test +exit 0 Copied and modified: head/share/examples/tests/tests/tap/printf_test.c (from r320415, head/share/examples/tests/tests/plain/printf_test.c) ============================================================================== --- head/share/examples/tests/tests/plain/printf_test.c Tue Jun 27 17:23:20 2017 (r320415, copy source) +++ head/share/examples/tests/tests/tap/printf_test.c Wed Jun 28 08:22:04 2017 (r320443) @@ -40,21 +40,69 @@ */ #include +#include #include #include #include +static int failed; +static int test_num = 1; + +#define TEST_COUNT 7 + static void +fail(const char *fmt, ...) +{ + char *msg; + va_list ap; + + failed = 1; + + va_start(ap, fmt); + if (vasprintf(&msg, fmt, ap) == -1) + err(1, NULL); + va_end(ap); + printf("not ok %d - %s\n", test_num, msg); + free(msg); + + test_num++; +} + +static void +pass(void) +{ + + printf("ok %d\n", test_num); + test_num++; +} + +static void +skip(int skip_num) +{ + int i; + + for (i = 0; i < skip_num; i++) { + printf("not ok %d # SKIP\n", test_num); + test_num++; + } +} + +static void snprintf__two_formatters(void) { char buffer[128]; if (snprintf(buffer, sizeof(buffer), "%s, %s!", "Hello", - "tests") <= 0) - errx(EXIT_FAILURE, "snprintf with two formatters failed"); - - if (strcmp(buffer, "Hello, tests!") != 0) - errx(EXIT_FAILURE, "Bad formatting: got %s", buffer); + "tests") <= 0) { + fail("snprintf with two formatters failed"); + skip(1); + } else { + pass(); + if (strcmp(buffer, "Hello, tests!") != 0) + fail("Bad formatting: got %s", buffer); + else + pass(); + } } static void @@ -62,12 +110,18 @@ snprintf__overflow(void) { char buffer[10]; - if (snprintf(buffer, sizeof(buffer), "0123456789abcdef") != 16) - errx(EXIT_FAILURE, "snprintf did not return the expected " + if (snprintf(buffer, sizeof(buffer), "0123456789abcdef") != 16) { + fail("snprintf did not return the expected " "number of characters"); + skip(1); + return; + } + pass(); if (strcmp(buffer, "012345678") != 0) - errx(EXIT_FAILURE, "Bad formatting: got %s", buffer); + fail("Bad formatting: got %s", buffer); + else + pass(); } static void @@ -79,17 +133,27 @@ fprintf__simple_string(void) const char *contents = "This is a message\n"; file = fopen("test.txt", "w+"); - if (fprintf(file, "%s", contents) <= 0) - err(EXIT_FAILURE, "fprintf failed to write to file"); + if (fprintf(file, "%s", contents) <= 0) { + fail("fprintf failed to write to file"); + skip(2); + return; + } + pass(); rewind(file); length = fread(buffer, 1, sizeof(buffer) - 1, file); - if (length != strlen(contents)) - err(EXIT_FAILURE, "fread failed"); + if (length != strlen(contents)) { + fail("fread failed"); + skip(1); + return; + } + pass(); buffer[length] = '\0'; fclose(file); if (strcmp(buffer, contents) != 0) - errx(EXIT_FAILURE, "Written and read data differ"); + fail("Written and read data differ"); + else + pass(); /* Of special note here is that we are NOT deleting the temporary * files we created in this test. Kyua takes care of this cleanup @@ -111,9 +175,11 @@ main(void) * is that this particular main() has no arguments: without ATF, * all test programs may expose a different command-line interface, * and this is an issue for consistency purposes. */ + printf("1..%d\n", TEST_COUNT); + snprintf__two_formatters(); snprintf__overflow(); fprintf__simple_string(); - return EXIT_SUCCESS; + return (failed); }