From nobody Wed Jun 4 19:58:40 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bCJL86nnDz5xXp2; Wed, 04 Jun 2025 19:58:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bCJL847Rhz3Xb5; Wed, 04 Jun 2025 19:58:40 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1749067120; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0cjv7w8CNqINpz+zr5run5qDUHH/WAXZoe/mJiJspvw=; b=RWnZgwrkpWKXbvDmsB0P07B1SWmKMfabCvWASrN1nVNm6CeXcmYXhfPLnxCb6Ej4jP+B+G aIPE/yC7+z5DZ15r3gFG99Rp3Vqi/s3o1l24LZ34TIJLZkhtZnTjxMur/a22CfMOCyWVq5 egVg32Mw6BzVyzo8Gtw2M+SQnHXxmz6zpAe11R0G6BfT5v03JMHiMLkI8R0uBzMFthookq k8Tm1zKsMCMxex152w+hZAPCZ2g67kxSA7p3bNX6fNZwEUTH5l6s8+LKOStreugMwFrbwq 46TBrOfUgUH7fXJwq74rVl1UoWQ2Yr7OFjHRGi5dvhoDYw3XCwZiW8gckSzQuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1749067120; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0cjv7w8CNqINpz+zr5run5qDUHH/WAXZoe/mJiJspvw=; b=ylei/haLIjP03H93DcnhBvNyblkTCbS86W5d1t1MTS+u4S5otavKf9cZ3bQMgLcBqls1Ps +CQBNV25WME0mEBHE5TroA17vseJoBeBVEe1FizCtS6ziHdIYSBpcoRMfRi8Ij3xpb+6j2 qeWSIFLX3ne/x4TzhTHBRVLfNmNN3v8NYcjl3mBGzl4s/gJpFunMHKVzN4w/4Y/j3WzCCd 8TP6F5p2tFfB5qg5A/pmqnn3U1N/1nWj5F19dKNIqMC4ni6fniHFXaiuXMru5YdxG8MH6M gjhYs9JbItiHbqgLxl+V82Uui3571ylVWiu58F/EscXCc3xsEpmGFoMEDGyJNQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1749067120; a=rsa-sha256; cv=none; b=MzXJsIKxH6dxxlw8tTPpZKyLHzS14t8w9kD9LgU7ZLIOGvMkFSce09c0v7Zwqh4L9GDXRp cY95Lhl1hcwMB/SBNhA/q6S5k2yPnv6pnhMP7snt1K7zzScs62xyeX9OEs/WWVsxrLUq9e A4ufQiD7o6cAzrhoWrFdJSjhXr7wctnotrElKW5XecPcgQEIWhWAoyaDNrEsF0O29/6Ypd rpn+TJluwSWvXxR6fRTU1CQ+RX3c1LQZemip0I27betMDGTAh2t+PwL7eLXGS6Og4zDR/U tDcv5/g+Kfj02C4O+m0DCTZ6TD/4D1wSkY/bw16Lg55SrbKyTsP91lF/MF2ZwA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4bCJL83NMgzdkY; Wed, 04 Jun 2025 19:58:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 554Jwe4p056713; Wed, 4 Jun 2025 19:58:40 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 554JweU5056710; Wed, 4 Jun 2025 19:58:40 GMT (envelope-from git) Date: Wed, 4 Jun 2025 19:58:40 GMT Message-Id: <202506041958.554JweU5056710@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: "Simon J. Gerraty" Subject: git: 8e1e989c77d4 - main - Add host.test.mk for DIRDEPS_BUILD List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: sjg X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 8e1e989c77d4e35ddc5ca3c143553f973c4a1764 Auto-Submitted: auto-generated The branch main has been updated by sjg: URL: https://cgit.FreeBSD.org/src/commit/?id=8e1e989c77d4e35ddc5ca3c143553f973c4a1764 commit 8e1e989c77d4e35ddc5ca3c143553f973c4a1764 Author: Simon J. Gerraty AuthorDate: 2025-06-04 19:56:44 +0000 Commit: Simon J. Gerraty CommitDate: 2025-06-04 19:56:44 +0000 Add host.test.mk for DIRDEPS_BUILD If we are doing DIRDEPS_BUILD and MK_RUN_TESTS is yes we can hook tests associated with dirs to be built into the graph such that we avoid circular dependencies but the build will fail should any test fail. Make MK_RUN_TESTS independent of MK_TESTS also for non-DIRDEPS_BUILD ensure we have RELDIR and check it against patterns in HOST_RUN_TESTS. Reviewed by: stevek Sponsored by: Juniper Networks, Inc. Differential Revision: https://reviews.freebsd.org/D50387 --- share/mk/Makefile | 1 + share/mk/bsd.opts.mk | 1 + share/mk/bsd.test.mk | 5 +++ share/mk/host.test.mk | 89 +++++++++++++++++++++++++++++++++++++++++++++++ share/mk/local.dirdeps.mk | 56 +++++++++++++++++++++++++++++ 5 files changed, 152 insertions(+) diff --git a/share/mk/Makefile b/share/mk/Makefile index 29de04f89670..837f7da68b4b 100644 --- a/share/mk/Makefile +++ b/share/mk/Makefile @@ -62,6 +62,7 @@ FILES= \ dirdeps-targets.mk \ gendirdeps.mk \ host-target.mk \ + host.test.mk \ install-new.mk \ jobs.mk \ meta.autodep.mk \ diff --git a/share/mk/bsd.opts.mk b/share/mk/bsd.opts.mk index a9400186f78a..85247d733a14 100644 --- a/share/mk/bsd.opts.mk +++ b/share/mk/bsd.opts.mk @@ -79,6 +79,7 @@ __DEFAULT_NO_OPTIONS = \ CTF \ INSTALL_AS_USER \ RETPOLINE \ + RUN_TESTS \ STALE_STAGED \ UBSAN \ UNDEFINED_VERSION \ diff --git a/share/mk/bsd.test.mk b/share/mk/bsd.test.mk index b324ac173335..a2e15c840e02 100644 --- a/share/mk/bsd.test.mk +++ b/share/mk/bsd.test.mk @@ -91,6 +91,11 @@ MAN= .include .endif +.if ${MK_RUN_TESTS} == "yes" +# Run tests as part of the build +.include +.endif + .if !target(realcheck) realcheck: .PHONY @echo "$@ not defined; skipping" diff --git a/share/mk/host.test.mk b/share/mk/host.test.mk new file mode 100644 index 000000000000..17bb00a0b28b --- /dev/null +++ b/share/mk/host.test.mk @@ -0,0 +1,89 @@ +# +# If MK_RUN_TESTS is "yes"; we want to run tests for "host" +# during the build - and fail the build if any test fails. +# +# HOST_RUN_TESTS can be used to filter which tests should be built and +# run. It should be a list of glob patterns to apply to RELDIR, +# so "*" would mean all. +# +# For the DIRDEPS_BUILD, HOST_RUN_TESTS defaults to "*" as the +# selection of tests to build and run is taken controlled at level 0 +# in local.dirdeps.mk +# + +.if !target(____) +.error ${.PARSEFILE} cannot be included directly. +.endif + +all: + +.if ${.MAKE.LEVEL} > 0 && !empty(_TESTS) && ${MACHINE:Nhost*} == "" + +# allow for customization +.-include + +.if ${MK_DIRDEPS_BUILD} == "yes" +# orchestration choices are already made +HOST_RUN_TESTS ?= * +.else +.if empty(RELDIR) +RELDIR:= ${.CURDIR:S,${SRCTOP}/,,:S,${OBJTOP}/,,} +.endif +.endif + +.if ${HOST_RUN_TESTS:Uno:@x@${RELDIR:M$x}@} != "" +all: run-tests +.endif + +KYUA?= kyua + +# we need to make sure kyua-report can find the results +KYUA_RESULTS?= ${.OBJDIR}/kyua.results +KYUA_ARGS?= --results-file=${KYUA_RESULTS} +KYUA_ENV?= HOME=${KYUA_HOME} TMPDIR=${.OBJDIR} +KYUA_FLAGS?= --config none --loglevel=${KYUA_LOGLEVEL:Uinfo} +KYUA_HOME?= ${OBJTOP} + +.if make(debug*) +KYUA_LOGLEVEL?= debug +.endif + +# some tests have files they need +.if ${${PACKAGE}FILES:U:NKyuafile} != "" +run-tests run-tests.log: link-test-files +link-test-files: ${${PACKAGE}FILES:NKyuafile} + @for f in ${.ALLSRC:N*Kyuafile:M*/*}; do \ + ln -sf $$f .; \ + done +.endif + +# we do not want to stage any of this +RUN_TESTS_LOG= run-tests.log +MK_STAGING= no +META_XTRAS+= ${RUN_TESTS_LOG} + +run-tests: ${RUN_TESTS_LOG} + +# This is the main event. +# Run kyua-test followed by kyua-report. +# If we have any test failues we want to run kyua-report --verbose +# Also on fail, we rename run-tests.log to run-tests.err so we save the +# output but the target will be out-of-date. +# We prepend ${.OBJDIR}:${.OBJDIR:H}: to PATH seen by kyua +# so tests for things like cat, cp, cmp etc can find the one we just built +# rather than the one from the host. +${RUN_TESTS_LOG}: ${_TESTS} Kyuafile + @( export PATH=${.OBJDIR}:${.OBJDIR:H}:${PATH}; \ + rm -f ${KYUA_RESULTS}; \ + ${KYUA_ENV} ${KYUA} ${KYUA_FLAGS} test ${KYUA_ARGS} -k ${.OBJDIR}/Kyuafile --build-root=${.OBJDIR} && \ + ${KYUA_ENV} ${KYUA} ${KYUA_FLAGS} report ${KYUA_ARGS} ) > ${.TARGET} || \ + { mv ${.TARGET} ${.TARGET:R}.err; \ + ${KYUA_ENV} ${KYUA} ${KYUA_FLAGS} report ${KYUA_ARGS} --verbose --results-filter broken,failed; echo See ${.TARGET:R:tA}.err; \ + exit 1; } + +# make kyua-debug KYUA_DEBUG_ARGS=app:test +kyua-debug: + @(export PATH=${.OBJDIR}:${.OBJDIR:H}:${PATH}; \ + ${KYUA_ENV} ${KYUA} ${KYUA_FLAGS} debug ${KYUA_DEBUG_ARGS}) || true + +.endif diff --git a/share/mk/local.dirdeps.mk b/share/mk/local.dirdeps.mk index f34a4542e534..2173ea0be446 100644 --- a/share/mk/local.dirdeps.mk +++ b/share/mk/local.dirdeps.mk @@ -250,3 +250,59 @@ BOOT_MACHINE_DIR:= ${BOOT_MACHINE_DIR.${DEP_MACHINE}} KERNEL_NAME:= ${KERNEL_NAME.${DEP_MACHINE}} .-include + +.if ${MK_RUN_TESTS} == "yes" +# some the tests below will not run correctly on the host +# as they require support files not present in .CURDIR +BROKEN_HOST_TESTS += \ + ${BROKEN_HOST_DEP_TESTS} \ + bin/cp/tests \ + lib/atf/libatf-c++/tests \ + lib/atf/libatf-c/tests \ + lib/libarchive/tests \ + lib/libnv/tests \ + libexec/rtld-elf/tests \ + sbin/devd/tests \ + usr.bin/comm/tests \ + usr.bin/du/tests \ + usr.bin/procstat/tests \ + usr.bin/sed/tests \ + usr.bin/tar/tests \ + usr.bin/tr/tests \ + usr.bin/xargs/tests \ + usr.bin/yacc/tests \ + usr.sbin/makefs/tests \ + usr.sbin/rpcbind/tests \ + +# these all have broken dependencies which we choose not to fix for host +BROKEN_HOST_DEP_TESTS += \ + lib/msun/tests \ + usr.bin/cut/tests \ + usr.bin/diff/tests \ + usr.bin/grep/tests \ + usr.bin/gzip/tests \ + usr.bin/printf/tests \ + +.endif + +TESTS_DIR_LIST += tests +# most of the tree only has Makefile.depend +# by default we only want to run host tests for Makefile.depend.host +TESTS_DEPENDFILE_PREFERENCE ?= Makefile.depend.host +EXCLUDE_TESTS_FILTER += . \ + lib/libcasper/services/cap_* \ + +.for t in ${TESTS_DIR_LIST:O:u} +.if !target(${DEP_RELDIR}/$t.test) +.if ${BROKEN_HOST_TESTS:U:M${DEP_RELDIR}/$t} == "" && \ + ${RUN_TESTS_FILTER:U*:@m@${DEP_RELDIR:M$m}@} != "" && \ + ${DEP_RELDIR:${EXCLUDE_TESTS_FILTER:Uno:${M_ListToSkip}}} != "" && \ + ${TESTS_DEPENDFILE_PREFERENCE:@m@${exists(${SRCTOP}/${DEP_RELDIR}/$t/$m):?$m:}@} != "" +# build but not via DIRDEPS to avoid circular dependencies +_build_xtra_dirs += ${SRCTOP}/${DEP_RELDIR}/$t.host +# mark it done +${DEP_RELDIR}/$t.test: +.endif +.endif +.endfor +.endif