From owner-svn-src-all@FreeBSD.ORG Wed Aug 13 04:14:54 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3D19FC0; Wed, 13 Aug 2014 04:14:53 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD68425E7; Wed, 13 Aug 2014 04:14:53 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id s7D4ErH3069393; Wed, 13 Aug 2014 04:14:53 GMT (envelope-from ngie@FreeBSD.org) Received: (from ngie@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id s7D4Eo98069371; Wed, 13 Aug 2014 04:14:50 GMT (envelope-from ngie@FreeBSD.org) Message-Id: <201408130414.s7D4Eo98069371@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: ngie set sender to ngie@FreeBSD.org using -f From: Garrett Cooper Date: Wed, 13 Aug 2014 04:14:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r269902 - in head: bin/sh/tests bin/sh/tests/builtins bin/sh/tests/errors bin/sh/tests/execution bin/sh/tests/expansion bin/sh/tests/parameters bin/sh/tests/parser bin/sh/tests/set-e to... X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Aug 2014 04:14:54 -0000 Author: ngie Date: Wed Aug 13 04:14:50 2014 New Revision: 269902 URL: http://svnweb.freebsd.org/changeset/base/269902 Log: Convert bin/sh/tests to ATF The new code uses a "test discovery mechanism" to determine what tests are available for execution The test shell can be specified via: kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh Sponsored by: EMC / Isilon Storage Division Approved by: jmmv (mentor) Reviewed by: jilles (maintainer) Added: head/bin/sh/tests/functional_test.sh (contents, props changed) Deleted: head/bin/sh/tests/legacy_test.sh Modified: head/bin/sh/tests/Makefile head/bin/sh/tests/builtins/Makefile head/bin/sh/tests/errors/Makefile head/bin/sh/tests/errors/bad-parm-exp2.2.stderr head/bin/sh/tests/errors/bad-parm-exp3.2.stderr head/bin/sh/tests/errors/bad-parm-exp4.2.stderr head/bin/sh/tests/errors/bad-parm-exp5.2.stderr head/bin/sh/tests/errors/bad-parm-exp6.2.stderr head/bin/sh/tests/execution/Makefile head/bin/sh/tests/expansion/Makefile head/bin/sh/tests/parameters/Makefile head/bin/sh/tests/parser/Makefile head/bin/sh/tests/set-e/Makefile head/tools/build/mk/OptionalObsoleteFiles.inc Modified: head/bin/sh/tests/Makefile ============================================================================== --- head/bin/sh/tests/Makefile Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/Makefile Wed Aug 13 04:14:50 2014 (r269902) @@ -4,15 +4,12 @@ TESTSDIR= ${TESTSBASE}/bin/sh -TAP_TESTS_SH= legacy_test -TAP_TESTS_SH_SED_legacy_test= -e 's,__SH__,/bin/sh,g' -# Some tests in here are silently not run when the tests are executed as -# root. Explicitly tell Kyua to drop privileges. -# -# TODO(jmmv): Kyua needs to do this by default, not only when explicitly -# requested. See https://code.google.com/p/kyua/issues/detail?id=6 -TEST_METADATA.legacy_test+= required_user="unprivileged" - -SUBDIR+= builtins errors execution expansion parameters parser set-e +TESTS_SUBDIRS+= builtins +TESTS_SUBDIRS+= errors +TESTS_SUBDIRS+= execution +TESTS_SUBDIRS+= expansion +TESTS_SUBDIRS+= parameters +TESTS_SUBDIRS+= parser +TESTS_SUBDIRS+= set-e .include Modified: head/bin/sh/tests/builtins/Makefile ============================================================================== --- head/bin/sh/tests/builtins/Makefile Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/builtins/Makefile Wed Aug 13 04:14:50 2014 (r269902) @@ -1,9 +1,13 @@ # $FreeBSD$ -.include +.include -FILESDIR= ${TESTSBASE}/bin/sh/builtins -KYUAFILE= no +TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T} + +.PATH: ${.CURDIR:H} +ATF_TESTS_SH= functional_test + +FILESDIR= ${TESTSDIR} FILES= alias.0 alias.0.stdout FILES+= alias.1 alias.1.stderr Modified: head/bin/sh/tests/errors/Makefile ============================================================================== --- head/bin/sh/tests/errors/Makefile Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/errors/Makefile Wed Aug 13 04:14:50 2014 (r269902) @@ -1,9 +1,11 @@ # $FreeBSD$ -.include +TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T} -FILESDIR= ${TESTSBASE}/bin/sh/errors -KYUAFILE= no +.PATH: ${.CURDIR:H} +ATF_TESTS_SH= functional_test + +FILESDIR= ${TESTSDIR} FILES= assignment-error1.0 FILES+= assignment-error2.0 Modified: head/bin/sh/tests/errors/bad-parm-exp2.2.stderr ============================================================================== --- head/bin/sh/tests/errors/bad-parm-exp2.2.stderr Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/errors/bad-parm-exp2.2.stderr Wed Aug 13 04:14:50 2014 (r269902) @@ -1 +1 @@ -./errors/bad-parm-exp2.2: ${}: Bad substitution +./bad-parm-exp2.2: ${}: Bad substitution Modified: head/bin/sh/tests/errors/bad-parm-exp3.2.stderr ============================================================================== --- head/bin/sh/tests/errors/bad-parm-exp3.2.stderr Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/errors/bad-parm-exp3.2.stderr Wed Aug 13 04:14:50 2014 (r269902) @@ -1 +1 @@ -./errors/bad-parm-exp3.2: ${foo/}: Bad substitution +./bad-parm-exp3.2: ${foo/}: Bad substitution Modified: head/bin/sh/tests/errors/bad-parm-exp4.2.stderr ============================================================================== --- head/bin/sh/tests/errors/bad-parm-exp4.2.stderr Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/errors/bad-parm-exp4.2.stderr Wed Aug 13 04:14:50 2014 (r269902) @@ -1 +1 @@ -./errors/bad-parm-exp4.2: ${foo:@...}: Bad substitution +./bad-parm-exp4.2: ${foo:@...}: Bad substitution Modified: head/bin/sh/tests/errors/bad-parm-exp5.2.stderr ============================================================================== --- head/bin/sh/tests/errors/bad-parm-exp5.2.stderr Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/errors/bad-parm-exp5.2.stderr Wed Aug 13 04:14:50 2014 (r269902) @@ -1 +1 @@ -./errors/bad-parm-exp5.2: ${/}: Bad substitution +./bad-parm-exp5.2: ${/}: Bad substitution Modified: head/bin/sh/tests/errors/bad-parm-exp6.2.stderr ============================================================================== --- head/bin/sh/tests/errors/bad-parm-exp6.2.stderr Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/errors/bad-parm-exp6.2.stderr Wed Aug 13 04:14:50 2014 (r269902) @@ -1 +1 @@ -./errors/bad-parm-exp6.2: ${foo...}: Bad substitution +./bad-parm-exp6.2: ${foo...}: Bad substitution Modified: head/bin/sh/tests/execution/Makefile ============================================================================== --- head/bin/sh/tests/execution/Makefile Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/execution/Makefile Wed Aug 13 04:14:50 2014 (r269902) @@ -1,9 +1,11 @@ # $FreeBSD$ -.include +TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T} -FILESDIR= ${TESTSBASE}/bin/sh/execution -KYUAFILE= no +.PATH: ${.CURDIR:H} +ATF_TESTS_SH= functional_test + +FILESDIR= ${TESTSDIR} FILES= bg1.0 FILES+= bg2.0 Modified: head/bin/sh/tests/expansion/Makefile ============================================================================== --- head/bin/sh/tests/expansion/Makefile Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/expansion/Makefile Wed Aug 13 04:14:50 2014 (r269902) @@ -1,9 +1,11 @@ # $FreeBSD$ -.include +TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T} -FILESDIR= ${TESTSBASE}/bin/sh/expansion -KYUAFILE= no +.PATH: ${.CURDIR:H} +ATF_TESTS_SH= functional_test + +FILESDIR= ${TESTSDIR} FILES= arith1.0 FILES+= arith2.0 Added: head/bin/sh/tests/functional_test.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/bin/sh/tests/functional_test.sh Wed Aug 13 04:14:50 2014 (r269902) @@ -0,0 +1,78 @@ +# +# Copyright 2014 EMC Corp. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +# +# $FreeBSD$ + +SRCDIR=$(atf_get_srcdir) + +check() +{ + local tc=${1}; shift + + export SH=$(atf_config_get bin.sh.test_shell /bin/sh) + + local err_file="${SRCDIR}/${tc}.stderr" + [ -f "${err_file}" ] && err_flag="-e file:${err_file}" + local out_file="${SRCDIR}/${tc}.stdout" + [ -f "${out_file}" ] && out_flag="-o file:${out_file}" + + # We need to copy the testcase scenario file because some of the + # testcases hardcode relative paths in the stderr/stdout. + # + # TODO: we might be able to generate this path at build time + cp ${SRCDIR}/${tc} . + + atf_check -s exit:${tc##*.} ${err_flag} ${out_flag} ${SH} "./${tc}" +} + +add_testcase() +{ + local tc=${1} + local tc_escaped word + + case "${tc%.*}" in + *-*) + local IFS="-" + for word in ${tc%.*}; do + tc_escaped="${tc_escaped:+${tc_escaped}_}${word}" + done + ;; + *) + tc_escaped=${tc%.*} + ;; + esac + + atf_test_case ${tc_escaped} + eval "${tc_escaped}_body() { check ${tc}; }" + atf_add_test_case ${tc_escaped} +} + +atf_init_test_cases() +{ + for path in $(find -Es "${SRCDIR}" -regex '.*\.[0-9]+$'); do + add_testcase ${path##*/} + done +} Modified: head/bin/sh/tests/parameters/Makefile ============================================================================== --- head/bin/sh/tests/parameters/Makefile Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/parameters/Makefile Wed Aug 13 04:14:50 2014 (r269902) @@ -1,9 +1,11 @@ # $FreeBSD$ -.include +TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T} -FILESDIR= ${TESTSBASE}/bin/sh/parameters -KYUAFILE= no +.PATH: ${.CURDIR:H} +ATF_TESTS_SH= functional_test + +FILESDIR= ${TESTSDIR} FILES= env1.0 FILES+= exitstatus1.0 Modified: head/bin/sh/tests/parser/Makefile ============================================================================== --- head/bin/sh/tests/parser/Makefile Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/parser/Makefile Wed Aug 13 04:14:50 2014 (r269902) @@ -1,9 +1,11 @@ # $FreeBSD$ -.include +TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T} -FILESDIR= ${TESTSBASE}/bin/sh/parser -KYUAFILE= no +.PATH: ${.CURDIR:H} +ATF_TESTS_SH= functional_test + +FILESDIR= ${TESTSDIR} FILES= alias1.0 FILES+= alias2.0 Modified: head/bin/sh/tests/set-e/Makefile ============================================================================== --- head/bin/sh/tests/set-e/Makefile Wed Aug 13 03:44:30 2014 (r269901) +++ head/bin/sh/tests/set-e/Makefile Wed Aug 13 04:14:50 2014 (r269902) @@ -1,9 +1,11 @@ # $FreeBSD$ -.include +TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T} -FILESDIR= ${TESTSBASE}/bin/sh/set-e -KYUAFILE= no +.PATH: ${.CURDIR:H} +ATF_TESTS_SH= functional_test + +FILESDIR= ${TESTSDIR} FILES= and1.0 FILES+= and2.1 Modified: head/tools/build/mk/OptionalObsoleteFiles.inc ============================================================================== --- head/tools/build/mk/OptionalObsoleteFiles.inc Wed Aug 13 03:44:30 2014 (r269901) +++ head/tools/build/mk/OptionalObsoleteFiles.inc Wed Aug 13 04:14:50 2014 (r269902) @@ -4168,6 +4168,7 @@ OLD_FILES+=usr/share/aclocal/atf-common. OLD_FILES+=usr/share/aclocal/atf-sh.m4 OLD_DIRS+=usr/share/aclocal OLD_FILES+=usr/tests/bin/chown/units_basics +OLD_FILES+=usr/tests/bin/sh/legacy_test OLD_FILES+=usr/tests/usr.bin/atf/Kyuafile OLD_FILES+=usr/tests/usr.bin/atf/atf-sh/Kyuafile OLD_FILES+=usr/tests/usr.bin/atf/atf-sh/atf_check_test