From owner-dev-commits-src-main@freebsd.org Tue Mar 2 18:28:41 2021 Return-Path: Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9357754B4BD; Tue, 2 Mar 2021 18:28:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dqly13mNPz4rZb; Tue, 2 Mar 2021 18:28:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 749AE1B7A9; Tue, 2 Mar 2021 18:28:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 122ISf8K092874; Tue, 2 Mar 2021 18:28:41 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 122ISf7D092873; Tue, 2 Mar 2021 18:28:41 GMT (envelope-from git) Date: Tue, 2 Mar 2021 18:28:41 GMT Message-Id: <202103021828.122ISf7D092873@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alex Richardson Subject: git: 53a535c1d80a - main - Simplify the capsicum-test wrapper script MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 53a535c1d80a2e5ea33a4e8807647e600402b1d8 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-main@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for the main branch of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Mar 2021 18:28:41 -0000 The branch main has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=53a535c1d80a2e5ea33a4e8807647e600402b1d8 commit 53a535c1d80a2e5ea33a4e8807647e600402b1d8 Author: Alex Richardson AuthorDate: 2021-03-02 18:27:34 +0000 Commit: Alex Richardson CommitDate: 2021-03-02 18:27:36 +0000 Simplify the capsicum-test wrapper script Instead of running tests one-by-one with the shell wrapper we now run the full gtest testsuite twice (once as root, once as non root). This significantly speeds up running tests despite running them twice. This change also passes the missing -u flag to capsicum-test that caused test failures (https://bugs.freebsd.org/250178) Previously, running the testsuite with the wrapper script took ~3s per test on aarch64 QEMU, i.e. a total of almost 5 minutes. Now it takes 6 seconds to run all tests twice. Before: root@freebsd-aarch64:/usr/tests/sys/capsicum # /usr/bin/time kyua test functional 94/96 passed (2 failed) 309.97 real 58.46 user 244.31 sys After: root@freebsd-aarch64:/usr/tests/sys/capsicum # /usr/bin/time kyua test functional functional:test_root -> passed [2.659s] functional:test_unprivileged -> passed [2.391s] 2/2 passed (0 failed) 5.48 real 1.06 user 2.52 sys This overhead is caused by kyua + atf-sh spawning lots of additional processes and can be avoided by just running the googletest test binary. syscall seconds calls errors fork 39.810229456 1275 0 sigprocmask 13.546928736 572 0 i.e. 1275 processes spawned to run a single test. Test Plan: All tests pass with D28907. PR: 250178 Reviewed By: lwhsu Differential Revision: https://reviews.freebsd.org/D29014 --- tests/sys/capsicum/functional.sh | 70 ++++++++++------------------------------ 1 file changed, 17 insertions(+), 53 deletions(-) diff --git a/tests/sys/capsicum/functional.sh b/tests/sys/capsicum/functional.sh index 6e11a9c4621a..0bd6b2d5e983 100755 --- a/tests/sys/capsicum/functional.sh +++ b/tests/sys/capsicum/functional.sh @@ -29,69 +29,33 @@ # # $FreeBSD$ -SRCDIR=$(atf_get_srcdir) CAPSICUM_TEST_BIN=capsicum-test -check() -{ - local tc=${1} +atf_test_case "test_root" +test_root_head() { - atf_check -s exit:0 -o match:PASSED -e ignore \ - ${SRCDIR}/${CAPSICUM_TEST_BIN} --gtest_filter=${tc} + atf_set descr 'Run capsicum-test as root' + atf_set require.user root } -skip() -{ - local reason=${1} - - atf_skip "${reason}" +test_root_body() { + atf_check -s exit:0 -o match:PASSED -e ignore \ + "$(atf_get_srcdir)/${CAPSICUM_TEST_BIN}" -u "$(id -u tests)" } -add_testcase() -{ - local tc=${1} - local tc_escaped word - - tc_escaped=$(echo ${tc} | sed -e 's/\./__/') - - atf_test_case ${tc_escaped} - - if [ "$(atf_config_get ci false)" = "true" ]; then - case "${tc_escaped}" in - ForkedOpenatTest_WithFlagInCapabilityMode___|OpenatTest__WithFlag) - eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/249960\"; }" - ;; - PipePdfork__WildcardWait) - eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/244165\"; }" - ;; - Capability__NoBypassDAC) - eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/250178\"; }" - ;; - Pdfork__OtherUserForked) - eval "${tc_escaped}_body() { skip \"https://bugs.freebsd.org/250179\"; }" - ;; - *) - eval "${tc_escaped}_body() { check ${tc}; }" - ;; - esac - else - eval "${tc_escaped}_body() { check ${tc}; }" - fi +atf_test_case "test_unprivileged" +test_unprivileged_head() { - atf_add_test_case ${tc_escaped} + atf_set descr 'Run capsicum-test as an unprivileged user' + atf_set require.user unprivileged } -list_tests() -{ - ${SRCDIR}/${CAPSICUM_TEST_BIN} --gtest_list_tests | awk ' - /^[^ ]/ { CAT=$0 } - /^[ ]/ { print CAT $1}' +test_unprivileged_body() { + atf_check -s exit:0 -o match:PASSED -e ignore \ + "$(atf_get_srcdir)/${CAPSICUM_TEST_BIN}" -u "$(id -u)" } -atf_init_test_cases() -{ - local t - for t in `list_tests`; do - add_testcase $t - done +atf_init_test_cases() { + atf_add_test_case test_root + atf_add_test_case test_unprivileged }