From owner-freebsd-testing@FreeBSD.ORG Mon Jan 20 16:28:31 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B746E3B for ; Mon, 20 Jan 2014 16:28:31 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E8BC31086 for ; Mon, 20 Jan 2014 16:28:30 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id x12so6984860wgg.25 for ; Mon, 20 Jan 2014 08:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:cc:content-type; bh=G5RBzOJuIHJsKC7dfx5F/qRUnzXtNsrcuzK4o7yNhSA=; b=kyjEPRwlLwxjqp1e+gpkv0ywDFrVOr1yDr2woVOteA1d63tLA2sRYWkcS6mZDN07F+ DBB6AjgaV867KcmfwAIjRBpyUc4zwhCeIuUZh5A8MsNvnjRv2ODzazdlHm1Zz9OjAz3M qO+f78I5RawFU3VRj5/oVtM7ydxV4Vfvo7Pb2NFTZSczXRdU+jjYbFXhqqtOab9ggCUE OYFL723X5ugcAijGoLldGph5rDn37xd6+74h8yV0Evei/ezs4DuASU01d0iv/XAjIIUu L9QP9qK816LI+bYv3fNnpFDyT+S+vaRK1N5f1lYRdWyVz5E6Cs8I+ZBjrfrH/V73ScC+ 8Gug== MIME-Version: 1.0 X-Received: by 10.195.12.164 with SMTP id er4mr9174wjd.92.1390235309344; Mon, 20 Jan 2014 08:28:29 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Mon, 20 Jan 2014 08:28:29 -0800 (PST) In-Reply-To: References: Date: Mon, 20 Jan 2014 09:28:29 -0700 X-Google-Sender-Auth: 0DGRXBrZHSHpdHHsEEnzyPR1Uy8 Message-ID: Subject: Re: WITH_TESTS=1 make installworld fails to install ATF C tests From: Alan Somers Cc: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 16:28:31 -0000 On Fri, Jan 17, 2014 at 11:58 AM, Alan Somers wrote: > I'm unable to do a make installworld on my FreeBSD/head machine. When > I try, I get errors about "No such file or directory" for some ATF > tests. The odd thing is, if I do "make buildenv" and run the command > that failed, it works. How can "make buildworld" fail even though the > same commands work in a "make buildenv" shell? Has anyone seen this > before? > > For the record, this is AMD64, I just did a completely clean > buildworld, and my source tree is at rev 260786. > > > install -o root -g wheel -m 444 Kyuafile.auto > /usr/tests/lib/atf/libatf-c/detail/Kyuafile > (cd /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail && make > -f /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail/Makefile > _RECURSING_PROGS= SUBDIR= PROG=process_helpers install) > install -s -o root -g wheel -m 555 process_helpers > /usr/tests/lib/atf/libatf-c/detail/process_helpers > install: process_helpers: No such file or directory > *** Error code 71 > > Stop. > make[9]: stopped in /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail > *** Error code 1 > > Stop. > make[8]: stopped in /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail > *** Error code 1 > > Stop. > make[7]: stopped in /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests > *** Error code 1 > > Stop. > make[6]: stopped in /usr/home/alans/freebsd/head/lib/atf/libatf-c > *** Error code 1 > > Stop. > make[5]: stopped in /usr/home/alans/freebsd/head/lib/atf > *** Error code 1 > > Stop. > make[4]: stopped in /usr/home/alans/freebsd/head/lib > *** Error code 1 > > Stop. > make[3]: stopped in /usr/home/alans/freebsd/head > *** Error code 1 > > Stop. > make[2]: stopped in /usr/home/alans/freebsd/head > *** Error code 1 > > Stop. > make[1]: stopped in /usr/home/alans/freebsd/head > *** Error code 1 > > Stop. > make: stopped in /usr/home/alans/freebsd/head > [root@alans-fbsd-head /usr/home/alans/freebsd/head]# make buildenv > Entering world for amd64:amd64 > # (cd /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail && > make -f /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail/Makefile > _RECURSING_PROGS= SUBDIR= PROG=process_helpers install) > sh /usr/home/alans/freebsd/head/tools/install.sh -s -o root -g wheel > -m 555 process_helpers > /usr/tests/lib/atf/libatf-c/detail/process_helpers > sh /usr/home/alans/freebsd/head/tools/install.sh -o root -g wheel -m > 444 Kyuafile.auto /usr/tests/lib/atf/libatf-c/detail/Kyuafile I've just reproduced this using a freshly installed image from the latest CURRENT snapshot (ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/11.0-CURRENT/amd64/20140106/FreeBSD-11.0-CURRENT-amd64-20140106-r260369.vmdk.xz) and a freshly checked out source tree at /usr/src revision 260833. WITH_TESTS=1 is the only line in my /etc/src.conf. This is definitely a bug. So I'll rephrase the question. Has anybody successfully done "WITH_TESTS=1 make installworld" since late November? If so, what revision did you use? -Alan From owner-freebsd-testing@FreeBSD.ORG Mon Jan 20 16:30:33 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71251F7E; Mon, 20 Jan 2014 16:30:33 +0000 (UTC) Received: from mail-yh0-x232.google.com (mail-yh0-x232.google.com [IPv6:2607:f8b0:4002:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1F23910A8; Mon, 20 Jan 2014 16:30:33 +0000 (UTC) Received: by mail-yh0-f50.google.com with SMTP id f10so2404465yha.37 for ; Mon, 20 Jan 2014 08:30:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=OeTcKr/9NIfsPVZvekXrtI3/QmgPIKaiMi6zQYTF3L8=; b=vAG+cFT83KBSkcweu0+5RH9fF9gCdEyBTH1NisEvg5SN+JGpFgW79LP0/Jegqa2Jvl YWH2UxexspdaN+3/7Dyj/6b4ZnyR1MSgpTYHbdNzcCXwccRXh0gxK4ipVwPwMt2eabIn sAOQHRdRpMVPb6wshoa1A1wwsxQJl5KI4Q1YePk1zSerTqU1zyJfwcD7v7mw6pgiq55y LqgCjAdheX8GfQcDmH1IYNDrVumTK4SoWEZdZY5+OR+q06Ee3przKdmzELykxbgueFMM LX0+YUEq6tQwRa7d7n69ey3qc4Bfl7HIIZW+trWCgMxzpU6/xkcq4OXq9e0Gywo1F4CY 2a3g== X-Received: by 10.236.84.18 with SMTP id r18mr2914038yhe.99.1390235432219; Mon, 20 Jan 2014 08:30:32 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:6c59:6ab9:da13:72a9? ([2601:8:ab80:7d6:6c59:6ab9:da13:72a9]) by mx.google.com with ESMTPSA id o27sm3556655yhb.19.2014.01.20.08.30.30 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 20 Jan 2014 08:30:31 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: WITH_TESTS=1 make installworld fails to install ATF C tests From: Garrett Cooper In-Reply-To: Date: Mon, 20 Jan 2014 08:30:30 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <42175357-C8B8-46CE-993B-B4DAEE11020E@gmail.com> References: To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 16:30:33 -0000 On Jan 20, 2014, at 8:28 AM, Alan Somers wrote: > On Fri, Jan 17, 2014 at 11:58 AM, Alan Somers = wrote: >> I'm unable to do a make installworld on my FreeBSD/head machine. = When >> I try, I get errors about "No such file or directory" for some ATF >> tests. The odd thing is, if I do "make buildenv" and run the command >> that failed, it works. How can "make buildworld" fail even though = the >> same commands work in a "make buildenv" shell? Has anyone seen this >> before? >>=20 >> For the record, this is AMD64, I just did a completely clean >> buildworld, and my source tree is at rev 260786. >>=20 >>=20 >> install -o root -g wheel -m 444 Kyuafile.auto >> /usr/tests/lib/atf/libatf-c/detail/Kyuafile >> (cd /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail && = make >> -f = /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail/Makefile >> _RECURSING_PROGS=3D SUBDIR=3D PROG=3Dprocess_helpers install) >> install -s -o root -g wheel -m 555 process_helpers >> /usr/tests/lib/atf/libatf-c/detail/process_helpers >> install: process_helpers: No such file or directory >> *** Error code 71 >>=20 >> Stop. >> make[9]: stopped in = /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail >> *** Error code 1 >>=20 >> Stop. >> make[8]: stopped in = /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail >> *** Error code 1 >>=20 >> Stop. >> make[7]: stopped in = /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests >> *** Error code 1 >>=20 >> Stop. >> make[6]: stopped in /usr/home/alans/freebsd/head/lib/atf/libatf-c >> *** Error code 1 >>=20 >> Stop. >> make[5]: stopped in /usr/home/alans/freebsd/head/lib/atf >> *** Error code 1 >>=20 >> Stop. >> make[4]: stopped in /usr/home/alans/freebsd/head/lib >> *** Error code 1 >>=20 >> Stop. >> make[3]: stopped in /usr/home/alans/freebsd/head >> *** Error code 1 >>=20 >> Stop. >> make[2]: stopped in /usr/home/alans/freebsd/head >> *** Error code 1 >>=20 >> Stop. >> make[1]: stopped in /usr/home/alans/freebsd/head >> *** Error code 1 >>=20 >> Stop. >> make: stopped in /usr/home/alans/freebsd/head >> [root@alans-fbsd-head /usr/home/alans/freebsd/head]# make buildenv >> Entering world for amd64:amd64 >> # (cd /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail && >> make -f = /usr/home/alans/freebsd/head/lib/atf/libatf-c/tests/detail/Makefile >> _RECURSING_PROGS=3D SUBDIR=3D PROG=3Dprocess_helpers install) >> sh /usr/home/alans/freebsd/head/tools/install.sh -s -o root -g wheel >> -m 555 process_helpers >> /usr/tests/lib/atf/libatf-c/detail/process_helpers >> sh /usr/home/alans/freebsd/head/tools/install.sh -o root -g wheel -m >> 444 Kyuafile.auto /usr/tests/lib/atf/libatf-c/detail/Kyuafile >=20 > I've just reproduced this using a freshly installed image from the > latest CURRENT snapshot > = (ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/11.0-CURRENT/amd64/= 20140106/FreeBSD-11.0-CURRENT-amd64-20140106-r260369.vmdk.xz) > and a freshly checked out source tree at /usr/src revision 260833. > WITH_TESTS=3D1 is the only line in my /etc/src.conf. This is = definitely > a bug. >=20 > So I'll rephrase the question. Has anybody successfully done > "WITH_TESTS=3D1 make installworld" since late November? If so, what > revision did you use? I have been for some time, but as you know my tree is very = forked from svn. I=92ll reconcile my changes and send them out for = review this week.. Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Mon Jan 20 16:45:02 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8531064D for ; Mon, 20 Jan 2014 16:45:02 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 55A99125D for ; Mon, 20 Jan 2014 16:45:02 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id x10so5780982pdj.11 for ; Mon, 20 Jan 2014 08:45:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:subject:date:message-id:cc:to:mime-version; bh=YW8xJkFUfeIHrrLZsCh+hCsbnC37a54+k4pGhF1+WIU=; b=Zx9joEQxQ3rcNbmoiOaOSKs8nGUpd0IyOZZaakzt/RC0xj4n+9fmRpRMuwcASH6Hdy LSwFhixjx3V3YolKoSv5D7pNTpY3YnYlJezrDD6PBaRx2+92BzeNvMXiCrVjYmYjHyqb CKAe6b4Rp7hsxfzYuCIYR7StVn2u32buIRqn5wZKC+GMb5uZoTO41oodCDAzzqEhvfO2 syr/7hnMetIFgReQ29LXzzcBTMViShmEYMur0SsK2MR1ptIQUaeBunPlL4Zs8ypf4hUw uj8Au9t/qiCQOA/sjimd2t5Kz8frwlyKvw1FGp2Y62NpMr4xUNqLuAQB8a5jUUgF+mcl dV2A== X-Received: by 10.68.139.228 with SMTP id rb4mr19301819pbb.15.1390236301968; Mon, 20 Jan 2014 08:45:01 -0800 (PST) Received: from [192.168.20.5] (c-50-181-163-89.hsd1.wa.comcast.net. [50.181.163.89]) by mx.google.com with ESMTPSA id by1sm4533613pbd.25.2014.01.20.08.44.59 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 20 Jan 2014 08:45:00 -0800 (PST) From: Garrett Cooper Content-Type: multipart/mixed; boundary="Apple-Mail=_553187D7-FF9C-47AD-AB26-0E925B076CC3" Subject: [PATCH] convert /bin/sh tests over to ATF Date: Mon, 20 Jan 2014 08:44:58 -0800 Message-Id: To: Julio Merino Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@FreeBSD.org, Jilles Tjoelker X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 16:45:02 -0000 --Apple-Mail=_553187D7-FF9C-47AD-AB26-0E925B076CC3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 The attached patch converts the /bin/sh tests using the existing = framework bin/sh/tests. All of the tests pass except locale.0, and that=92s due to some = [currently] unknown issue with my environment =97 see misc/181151). I = could make some of the entries more permanent, but I was trying to make = the tests easier to backport by using shell globs. Also, I added some functionality so the tests expect failures on = certain versions of FreeBSD due to missing functionality or conformance = modifications (see the .valid_osreldate files). Thoughts? Comments? Thanks! -Garrett= --Apple-Mail=_553187D7-FF9C-47AD-AB26-0E925B076CC3 Content-Disposition: attachment; filename=convert-bin-sh-over-to-atf.patch Content-Type: application/octet-stream; name="convert-bin-sh-over-to-atf.patch" Content-Transfer-Encoding: 7bit diff --git a/bin/sh/tests/Makefile b/bin/sh/tests/Makefile index f6ddb8a..e1424f5 100644 --- a/bin/sh/tests/Makefile +++ b/bin/sh/tests/Makefile @@ -4,15 +4,8 @@ TESTSDIR= ${TESTSBASE}/bin/sh -TAP_TESTS_SH= legacy_test -TAP_TESTS_SH_SED_legacy_test= -e 's,__SH__,/bin/sh,g' -# Some tests in here are silently not run when the tests are executed as -# root. Explicitly tell Kyua to drop privileges. -# -# TODO(jmmv): Kyua needs to do this by default, not only when explicitly -# requested. See https://code.google.com/p/kyua/issues/detail?id=6 -TEST_METADATA.legacy_test+= required_user="unprivileged" +KYUAFILE= auto -SUBDIR+= builtins errors execution expansion parameters parser set-e +ATF_TESTS_SUBDIRS+= builtins errors execution expansion parameters parser set-e -.include +.include diff --git a/bin/sh/tests/Makefile.inc b/bin/sh/tests/Makefile.inc new file mode 100644 index 0000000..0a79fd8 --- /dev/null +++ b/bin/sh/tests/Makefile.inc @@ -0,0 +1,51 @@ +# $FreeBSD$ + +CLEANFILES= regress.sh + +KYUAFILE= auto + +SH_TESTS!= cd ${.CURDIR} && find -Es . -regex '.*\.[0-9]+' | sed -e 's,^\./,,g' + +SH_TESTS_FILES!= cd ${.CURDIR} && find . \( -type f -and \! -name Makefile\* \) | \ + sed -e 's,^\./,,g' + +TESTSDIR= ${TESTSBASE}/bin/sh/${.CURDIR:T} + +FILES+= ${SH_TESTS_FILES} +FILESDIR= ${TESTSDIR} + +regress.sh: ../Makefile.inc ${SH_TESTS_FILES} + @echo '' > ${.TARGET} + @echo 'OSRELDATE=$$(sysctl -n kern.osreldate)' >> ${.TARGET} + @echo ': $${SH=/bin/sh}' >> ${.TARGET} + @echo 'export SH' >> ${.TARGET} + @echo '' >> ${.TARGET} +.for test in ${SH_TESTS} + @echo 'atf_test_case ${test:S/\//_/g:S/./__/g:S/-/____/g}' >> ${.TARGET} + @echo '${test:S/\//_/g:S/./__/g:S/-/____/g}_body() {' >> ${.TARGET} + @echo ' cd $$(atf_get_srcdir)' >> ${.TARGET} +.if exists(${test}.valid_osreldate) + @echo -n ' [ '`cat ${.CURDIR}/${test}.valid_osreldate`' ] && ' \ + >> ${.TARGET} + @echo 'atf_expect_fail "expecting failure based on OS version"' \ + >> ${.TARGET} +.endif + @echo -n ' atf_check -s exit:${test:E} ' >> ${.TARGET} +.if exists(${test}.stderr) + @echo -n '-e "file:$$(atf_get_srcdir)/${test}.stderr" ' >> ${.TARGET} +.endif +.if exists(${test}.stdout) + @echo -n '-o "file:$$(atf_get_srcdir)/${test}.stdout" ' >> ${.TARGET} +.endif + @echo '$${SH} "./${test}"' >> ${.TARGET} + @echo '}' >> ${.TARGET} + @echo '' >> ${.TARGET} +.endfor + @echo '' >> ${.TARGET} + @echo 'atf_init_test_cases() {' >> ${.TARGET} +.for test in ${SH_TESTS} + @echo ' atf_add_test_case ${test:S/\//_/g:S/./__/g:S/-/____/g}' >> ${.TARGET} +.endfor + @echo '}' >> ${.TARGET} + +ATF_TESTS_SH+= regress diff --git a/bin/sh/tests/builtins/Makefile b/bin/sh/tests/builtins/Makefile index 3b1d24b..343219d 100644 --- a/bin/sh/tests/builtins/Makefile +++ b/bin/sh/tests/builtins/Makefile @@ -2,146 +2,4 @@ .include -FILESDIR= ${TESTSBASE}/bin/sh/builtins -KYUAFILE= no - -FILES= alias.0 alias.0.stdout -FILES+= alias.1 alias.1.stderr -FILES+= alias3.0 alias3.0.stdout -FILES+= alias4.0 -FILES+= break1.0 -FILES+= break2.0 break2.0.stdout -FILES+= break3.0 -FILES+= break4.4 -FILES+= break5.4 -FILES+= builtin1.0 -FILES+= case1.0 -FILES+= case2.0 -FILES+= case3.0 -FILES+= case4.0 -FILES+= case5.0 -FILES+= case6.0 -FILES+= case7.0 -FILES+= case8.0 -FILES+= case9.0 -FILES+= case10.0 -FILES+= case11.0 -FILES+= case12.0 -FILES+= case13.0 -FILES+= case14.0 -FILES+= case15.0 -FILES+= case16.0 -FILES+= case17.0 -FILES+= case18.0 -FILES+= case19.0 -FILES+= cd1.0 -FILES+= cd2.0 -FILES+= cd3.0 -FILES+= cd4.0 -FILES+= cd5.0 -FILES+= cd6.0 -FILES+= cd7.0 -FILES+= cd8.0 -FILES+= command1.0 -FILES+= command2.0 -FILES+= command3.0 -FILES+= command3.0.stdout -FILES+= command4.0 -FILES+= command5.0 -FILES+= command5.0.stdout -FILES+= command6.0 -FILES+= command6.0.stdout -FILES+= command7.0 -FILES+= command8.0 -FILES+= command9.0 -FILES+= command10.0 -FILES+= command11.0 -FILES+= command12.0 -FILES+= dot1.0 -FILES+= dot2.0 -FILES+= dot3.0 -FILES+= dot4.0 -FILES+= eval1.0 -FILES+= eval2.0 -FILES+= eval3.0 -FILES+= eval4.0 -FILES+= eval5.0 -FILES+= eval6.0 -FILES+= exec1.0 -FILES+= exec2.0 -FILES+= exit1.0 -FILES+= exit2.8 -FILES+= exit3.0 -FILES+= export1.0 -FILES+= fc1.0 -FILES+= fc2.0 -FILES+= for1.0 -FILES+= for2.0 -FILES+= for3.0 -FILES+= getopts1.0 getopts1.0.stdout -FILES+= getopts2.0 getopts2.0.stdout -FILES+= hash1.0 hash1.0.stdout -FILES+= hash2.0 hash2.0.stdout -FILES+= hash3.0 hash3.0.stdout -FILES+= hash4.0 -FILES+= jobid1.0 -FILES+= jobid2.0 -FILES+= lineno.0 lineno.0.stdout -FILES+= local1.0 -FILES+= local2.0 -FILES+= local3.0 -FILES+= local4.0 -FILES+= locale1.0 -FILES+= printf1.0 -FILES+= printf2.0 -FILES+= printf3.0 -FILES+= printf4.0 -FILES+= read1.0 read1.0.stdout -FILES+= read2.0 -FILES+= read3.0 read3.0.stdout -FILES+= read4.0 read4.0.stdout -FILES+= read5.0 -FILES+= read6.0 -FILES+= read7.0 -FILES+= return1.0 -FILES+= return2.1 -FILES+= return3.1 -FILES+= return4.0 -FILES+= return5.0 -FILES+= return6.4 -FILES+= return7.4 -FILES+= return8.0 -FILES+= set1.0 -FILES+= set2.0 -FILES+= trap1.0 -FILES+= trap10.0 -FILES+= trap11.0 -FILES+= trap12.0 -FILES+= trap13.0 -FILES+= trap14.0 -FILES+= trap2.0 -FILES+= trap3.0 -FILES+= trap4.0 -FILES+= trap5.0 -FILES+= trap6.0 -FILES+= trap7.0 -FILES+= trap8.0 -FILES+= trap9.0 -FILES+= type1.0 type1.0.stderr -FILES+= type2.0 -FILES+= type3.0 -FILES+= unalias.0 -FILES+= var-assign.0 -FILES+= var-assign2.0 -FILES+= wait1.0 -FILES+= wait2.0 -FILES+= wait3.0 -FILES+= wait4.0 -FILES+= wait5.0 -FILES+= wait6.0 -FILES+= wait7.0 -FILES+= wait8.0 -FILES+= wait9.127 -FILES+= wait10.0 - -.include +.include diff --git a/bin/sh/tests/builtins/alias.1.valid_osreldate b/bin/sh/tests/builtins/alias.1.valid_osreldate new file mode 100644 index 0000000..96bc086 --- /dev/null +++ b/bin/sh/tests/builtins/alias.1.valid_osreldate @@ -0,0 +1 @@ +\( ${OSRELDATE} -gt 802506 -a ${OSRELDATE} -lt 900000 \) diff --git a/bin/sh/tests/errors/Makefile b/bin/sh/tests/errors/Makefile index 9f8b0f2..f0ba343 100644 --- a/bin/sh/tests/errors/Makefile +++ b/bin/sh/tests/errors/Makefile @@ -2,9 +2,6 @@ .include -FILESDIR= ${TESTSBASE}/bin/sh/errors -KYUAFILE= no - FILES= assignment-error1.0 FILES+= assignment-error2.0 FILES+= backquote-error1.0 @@ -27,4 +24,4 @@ FILES+= redirection-error6.0 FILES+= redirection-error7.0 FILES+= write-error1.0 -.include +.include diff --git a/bin/sh/tests/errors/bad-parm-exp2.2.stderr b/bin/sh/tests/errors/bad-parm-exp2.2.stderr index d027a5a..1135af0 100644 --- a/bin/sh/tests/errors/bad-parm-exp2.2.stderr +++ b/bin/sh/tests/errors/bad-parm-exp2.2.stderr @@ -1 +1 @@ -./errors/bad-parm-exp2.2: ${}: Bad substitution +./bad-parm-exp2.2: ${}: Bad substitution diff --git a/bin/sh/tests/errors/bad-parm-exp3.2.stderr b/bin/sh/tests/errors/bad-parm-exp3.2.stderr index ef40251..b28fdff 100644 --- a/bin/sh/tests/errors/bad-parm-exp3.2.stderr +++ b/bin/sh/tests/errors/bad-parm-exp3.2.stderr @@ -1 +1 @@ -./errors/bad-parm-exp3.2: ${foo/}: Bad substitution +./bad-parm-exp3.2: ${foo/}: Bad substitution diff --git a/bin/sh/tests/errors/bad-parm-exp4.2.stderr b/bin/sh/tests/errors/bad-parm-exp4.2.stderr index 89bd80f..1487b61 100644 --- a/bin/sh/tests/errors/bad-parm-exp4.2.stderr +++ b/bin/sh/tests/errors/bad-parm-exp4.2.stderr @@ -1 +1 @@ -./errors/bad-parm-exp4.2: ${foo:@...}: Bad substitution +./bad-parm-exp4.2: ${foo:@...}: Bad substitution diff --git a/bin/sh/tests/errors/bad-parm-exp5.2.stderr b/bin/sh/tests/errors/bad-parm-exp5.2.stderr index 89b1997..2138c2e 100644 --- a/bin/sh/tests/errors/bad-parm-exp5.2.stderr +++ b/bin/sh/tests/errors/bad-parm-exp5.2.stderr @@ -1 +1 @@ -./errors/bad-parm-exp5.2: ${/}: Bad substitution +./bad-parm-exp5.2: ${/}: Bad substitution diff --git a/bin/sh/tests/errors/bad-parm-exp6.2.stderr b/bin/sh/tests/errors/bad-parm-exp6.2.stderr index dbf14b5..ba3b671 100644 --- a/bin/sh/tests/errors/bad-parm-exp6.2.stderr +++ b/bin/sh/tests/errors/bad-parm-exp6.2.stderr @@ -1 +1 @@ -./errors/bad-parm-exp6.2: ${foo...}: Bad substitution +./bad-parm-exp6.2: ${foo...}: Bad substitution diff --git a/bin/sh/tests/execution/Makefile b/bin/sh/tests/execution/Makefile index 302d0d8..343219d 100644 --- a/bin/sh/tests/execution/Makefile +++ b/bin/sh/tests/execution/Makefile @@ -2,52 +2,4 @@ .include -FILESDIR= ${TESTSBASE}/bin/sh/execution -KYUAFILE= no - -FILES= bg1.0 -FILES+= bg2.0 -FILES+= bg3.0 -FILES+= bg4.0 -FILES+= bg5.0 -FILES+= bg6.0 bg6.0.stdout -FILES+= bg7.0 -FILES+= bg8.0 -FILES+= bg9.0 -FILES+= bg10.0 bg10.0.stdout -FILES+= fork1.0 -FILES+= fork2.0 -FILES+= fork3.0 -FILES+= func1.0 -FILES+= func2.0 -FILES+= func3.0 -FILES+= hash1.0 -FILES+= int-cmd1.0 -FILES+= killed1.0 -FILES+= killed2.0 -FILES+= not1.0 -FILES+= not2.0 -FILES+= path1.0 -FILES+= redir1.0 -FILES+= redir2.0 -FILES+= redir3.0 -FILES+= redir4.0 -FILES+= redir5.0 -FILES+= redir6.0 -FILES+= redir7.0 -FILES+= set-n1.0 -FILES+= set-n2.0 -FILES+= set-n3.0 -FILES+= set-n4.0 -FILES+= set-x1.0 -FILES+= set-x2.0 -FILES+= set-x3.0 -FILES+= shellproc1.0 -FILES+= subshell1.0 subshell1.0.stdout -FILES+= subshell2.0 -FILES+= subshell3.0 -FILES+= subshell4.0 -FILES+= unknown1.0 -FILES+= var-assign1.0 - -.include +.include diff --git a/bin/sh/tests/expansion/Makefile b/bin/sh/tests/expansion/Makefile index bd24319..343219d 100644 --- a/bin/sh/tests/expansion/Makefile +++ b/bin/sh/tests/expansion/Makefile @@ -2,85 +2,4 @@ .include -FILESDIR= ${TESTSBASE}/bin/sh/expansion -KYUAFILE= no - -FILES= arith1.0 -FILES+= arith2.0 -FILES+= arith3.0 -FILES+= arith4.0 -FILES+= arith5.0 -FILES+= arith6.0 -FILES+= arith7.0 -FILES+= arith8.0 -FILES+= arith9.0 -FILES+= arith10.0 -FILES+= arith11.0 -FILES+= arith12.0 -FILES+= arith13.0 -FILES+= assign1.0 -FILES+= cmdsubst1.0 -FILES+= cmdsubst2.0 -FILES+= cmdsubst3.0 -FILES+= cmdsubst4.0 -FILES+= cmdsubst5.0 -FILES+= cmdsubst6.0 -FILES+= cmdsubst7.0 -FILES+= cmdsubst8.0 -FILES+= cmdsubst9.0 -FILES+= cmdsubst10.0 -FILES+= cmdsubst11.0 -FILES+= cmdsubst12.0 -FILES+= cmdsubst13.0 -FILES+= cmdsubst14.0 -FILES+= cmdsubst15.0 -FILES+= cmdsubst16.0 -FILES+= cmdsubst17.0 -FILES+= export1.0 -FILES+= export2.0 -FILES+= export3.0 -FILES+= heredoc1.0 -FILES+= heredoc2.0 -FILES+= ifs1.0 -FILES+= ifs2.0 -FILES+= ifs3.0 -FILES+= ifs4.0 -FILES+= length1.0 -FILES+= length2.0 -FILES+= length3.0 -FILES+= length4.0 -FILES+= length5.0 -FILES+= length6.0 -FILES+= length7.0 -FILES+= length8.0 -FILES+= local1.0 -FILES+= local2.0 -FILES+= pathname1.0 -FILES+= pathname2.0 -FILES+= pathname3.0 -FILES+= pathname4.0 -FILES+= plus-minus1.0 -FILES+= plus-minus2.0 -FILES+= plus-minus3.0 -FILES+= plus-minus4.0 -FILES+= plus-minus5.0 -FILES+= plus-minus6.0 -FILES+= plus-minus7.0 -FILES+= plus-minus8.0 -FILES+= question1.0 -FILES+= readonly1.0 -FILES+= set-u1.0 -FILES+= set-u2.0 -FILES+= set-u3.0 -FILES+= tilde1.0 -FILES+= tilde2.0 -FILES+= trim1.0 -FILES+= trim2.0 -FILES+= trim3.0 -FILES+= trim4.0 -FILES+= trim5.0 -FILES+= trim6.0 -FILES+= trim7.0 -FILES+= trim8.0 - -.include +.include diff --git a/bin/sh/tests/legacy_test.sh b/bin/sh/tests/legacy_test.sh deleted file mode 100644 index d43f5dd..0000000 --- a/bin/sh/tests/legacy_test.sh +++ /dev/null @@ -1,46 +0,0 @@ -# $FreeBSD$ - -: ${SH:="__SH__"} -export SH - -# TODO(jmmv): The Kyua TAP interface should be passing us the value of -# "srcdir" as an environment variable, just as it does with the ATF -# interface in the form of a configuration variable. For now, just try -# to guess this. -: ${TESTS_DATA:=$(dirname ${0})} - -COUNTER=1 - -do_test() { - c=${COUNTER} - COUNTER=$((COUNTER+1)) - ${SH} $1 > tmp.stdout 2> tmp.stderr - if [ $? -ne $2 ]; then - echo "not ok ${c} - ${1} # wrong exit status" - rm tmp.stdout tmp.stderr - return - fi - sed -I '' -e "s|^${TESTS_DATA}|.|" tmp.stderr - for i in stdout stderr; do - if [ -f ${1}.${i} ]; then - if ! cmp -s tmp.${i} ${1}.${i}; then - echo "not ok ${c} - ${1} # wrong output on ${i}" - rm tmp.stdout tmp.stderr - return - fi - elif [ -s tmp.${i} ]; then - echo "not ok ${c} - ${1} # wrong output on ${i}" - rm tmp.stdout tmp.stderr - return - fi - done - echo "ok ${c} - ${1}" - rm tmp.stdout tmp.stderr -} - -TESTS=$(find -Es ${TESTS_DATA} -regex ".*\.[0-9]+") -printf "1..%d\n" $(echo ${TESTS} | wc -w) - -for i in ${TESTS} ; do - do_test ${i} ${i##*.} -done diff --git a/bin/sh/tests/parameters/Makefile b/bin/sh/tests/parameters/Makefile index fafc059..343219d 100644 --- a/bin/sh/tests/parameters/Makefile +++ b/bin/sh/tests/parameters/Makefile @@ -2,18 +2,4 @@ .include -FILESDIR= ${TESTSBASE}/bin/sh/parameters -KYUAFILE= no - -FILES= env1.0 -FILES+= exitstatus1.0 -FILES+= mail1.0 -FILES+= mail2.0 -FILES+= optind1.0 -FILES+= optind2.0 -FILES+= positional1.0 -FILES+= positional2.0 -FILES+= pwd1.0 -FILES+= pwd2.0 - -.include +.include diff --git a/bin/sh/tests/parser/Makefile b/bin/sh/tests/parser/Makefile index 71bd370..343219d 100644 --- a/bin/sh/tests/parser/Makefile +++ b/bin/sh/tests/parser/Makefile @@ -2,58 +2,4 @@ .include -FILESDIR= ${TESTSBASE}/bin/sh/parser -KYUAFILE= no - -FILES= alias1.0 -FILES+= alias2.0 -FILES+= alias3.0 -FILES+= alias4.0 -FILES+= alias5.0 -FILES+= alias6.0 -FILES+= alias7.0 -FILES+= alias8.0 -FILES+= alias9.0 -FILES+= alias10.0 -FILES+= and-pipe-not.0 -FILES+= case1.0 -FILES+= case2.0 -FILES+= dollar-quote1.0 -FILES+= dollar-quote2.0 -FILES+= dollar-quote3.0 -FILES+= dollar-quote4.0 -FILES+= dollar-quote5.0 -FILES+= dollar-quote6.0 -FILES+= dollar-quote7.0 -FILES+= dollar-quote8.0 -FILES+= dollar-quote9.0 -FILES+= dollar-quote10.0 -FILES+= dollar-quote11.0 -FILES+= empty-braces1.0 -FILES+= empty-cmd1.0 -FILES+= for1.0 -FILES+= for2.0 -FILES+= func1.0 -FILES+= func2.0 -FILES+= func3.0 -FILES+= heredoc1.0 -FILES+= heredoc2.0 -FILES+= heredoc3.0 -FILES+= heredoc4.0 -FILES+= heredoc5.0 -FILES+= heredoc6.0 -FILES+= heredoc7.0 -FILES+= heredoc8.0 -FILES+= heredoc9.0 -FILES+= heredoc10.0 -FILES+= heredoc11.0 -FILES+= no-space1.0 -FILES+= no-space2.0 -FILES+= only-redir1.0 -FILES+= only-redir2.0 -FILES+= only-redir3.0 -FILES+= only-redir4.0 -FILES+= pipe-not1.0 -FILES+= var-assign1.0 - -.include +.include diff --git a/bin/sh/tests/set-e/Makefile b/bin/sh/tests/set-e/Makefile index 55d7917..343219d 100644 --- a/bin/sh/tests/set-e/Makefile +++ b/bin/sh/tests/set-e/Makefile @@ -2,43 +2,4 @@ .include -FILESDIR= ${TESTSBASE}/bin/sh/set-e -KYUAFILE= no - -FILES= and1.0 -FILES+= and2.1 -FILES+= and3.0 -FILES+= and4.0 -FILES+= background1.0 -FILES+= cmd1.0 -FILES+= cmd2.1 -FILES+= elif1.0 -FILES+= elif2.0 -FILES+= eval1.0 -FILES+= eval2.1 -FILES+= for1.0 -FILES+= func1.0 -FILES+= func2.1 -FILES+= if1.0 -FILES+= if2.0 -FILES+= if3.0 -FILES+= not1.0 -FILES+= not2.0 -FILES+= or1.0 -FILES+= or2.0 -FILES+= or3.1 -FILES+= pipe1.1 -FILES+= pipe2.0 -FILES+= return1.0 -FILES+= semi1.1 -FILES+= semi2.1 -FILES+= subshell1.0 -FILES+= subshell2.1 -FILES+= until1.0 -FILES+= until2.0 -FILES+= until3.0 -FILES+= while1.0 -FILES+= while2.0 -FILES+= while3.0 - -.include +.include --Apple-Mail=_553187D7-FF9C-47AD-AB26-0E925B076CC3-- From owner-freebsd-testing@FreeBSD.ORG Mon Jan 20 18:40:43 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AFC131B7; Mon, 20 Jan 2014 18:40:43 +0000 (UTC) Received: from mail-pd0-x229.google.com (mail-pd0-x229.google.com [IPv6:2607:f8b0:400e:c02::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 74B201DE0; Mon, 20 Jan 2014 18:40:43 +0000 (UTC) Received: by mail-pd0-f169.google.com with SMTP id v10so7224452pde.0 for ; Mon, 20 Jan 2014 10:40:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:subject:date:message-id:cc:to:mime-version; bh=t4XgPJf7H2W1x8oF2gGYTvXO4j/DVdaqweykQBA+jD4=; b=0Km50/rh0+5g08Bzp4A1CXJ/TkQCpJRjGEiMom3illKQknmB7NRPAxsx5Mo3DJA1P0 15betvnrBbw9z+446vusRdXGYY2zRrM9fKMHPX88x61bn9TOAqWwSqGZvYcGTY+s4aAy D9q0l2LoC1pi7JxTm6FBwbzjiQJsyXsPfqhnBGDGbktmew2JJCrFsYzpNd9r1AJ4jPUR deb9JcWX2Kwd6XLtuv2oiLuoLpT9gETpSBlCR43zp14u/FBOmoQqCG8sbipDVlOC9gux KXQkipGS2/ZOXkQwxUghM0Z1BhPcgr7heUncYC/PIwlTW+e1r6skYDMYtMERAs8TK/Nd 0+bQ== X-Received: by 10.67.23.135 with SMTP id ia7mr19874594pad.5.1390243242375; Mon, 20 Jan 2014 10:40:42 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id jn12sm5194397pbd.37.2014.01.20.10.40.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 20 Jan 2014 10:40:41 -0800 (PST) From: Garrett Cooper Content-Type: multipart/mixed; boundary="Apple-Mail=_AD906FCC-501F-4A0D-9822-A6E8655CA547" Subject: [PATCH] convert bin/date over to ATF Date: Mon, 20 Jan 2014 10:40:38 -0800 Message-Id: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> To: Julio Merino Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@FreeBSD.org, Giorgos Keramidas X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 18:40:43 -0000 --Apple-Mail=_AD906FCC-501F-4A0D-9822-A6E8655CA547 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii This is based on work done by Giorgos a couple years ago. Thanks! -Garrett # kyua test regress:A -> passed [0.059s] regress:a -> passed [0.144s] regress:B -> passed [0.036s] regress:b -> passed [0.037s] regress:C -> passed [0.045s] regress:c -> passed [0.035s] regress:D -> passed [0.061s] regress:d -> passed [0.034s] regress:e -> passed [0.040s] regress:F -> passed [0.036s] regress:G -> passed [0.039s] regress:g -> passed [0.042s] regress:H -> passed [0.034s] regress:h -> passed [0.036s] regress:I -> passed [0.043s] regress:j -> passed [0.034s] regress:k -> passed [0.035s] regress:l -> passed [0.047s] regress:M -> passed [0.034s] regress:m -> passed [0.039s] regress:p -> passed [0.036s] regress:R -> passed [0.037s] regress:r -> passed [0.035s] regress:S -> passed [0.045s] regress:s -> passed [0.035s] regress:U -> passed [0.035s] regress:u -> passed [0.034s] regress:V -> passed [0.034s] regress:v -> passed [0.035s] regress:W -> passed [0.051s] regress:w -> passed [0.038s] regress:X -> passed [0.043s] regress:x -> passed [0.041s] regress:Y -> passed [0.045s] regress:y -> passed [0.044s] regress:Z -> passed [0.036s] regress:z -> passed [0.035s] regress:pct -> passed [0.036s] regress:plus -> passed [0.036s] 39/39 passed (0 failed) Committed action 28 --Apple-Mail=_AD906FCC-501F-4A0D-9822-A6E8655CA547 Content-Disposition: attachment; filename=convert-bin-date-over-to-atf.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="convert-bin-date-over-to-atf.patch" Content-Transfer-Encoding: 7bit diff --git a/bin/date/tests/Makefile b/bin/date/tests/Makefile index 540008b..459d019 100644 --- a/bin/date/tests/Makefile +++ b/bin/date/tests/Makefile @@ -4,6 +4,6 @@ TESTSDIR= ${TESTSBASE}/bin/date -TAP_TESTS_SH= legacy_test +ATF_TESTS_SH= regress -.include +.include diff --git a/bin/date/tests/legacy_test.sh b/bin/date/tests/legacy_test.sh deleted file mode 100644 index 981bdd0..0000000 --- a/bin/date/tests/legacy_test.sh +++ /dev/null @@ -1,91 +0,0 @@ -#!/bin/sh - -# -# Regression tests for date(1) -# -# Submitted by Edwin Groothuis -# -# $FreeBSD$ -# - -# -# These two date/times have been chosen carefully, they -# create both the single digit and double/multidigit version of -# the values. -# -# To create a new one, make sure you are using the UTC timezone! -# - -TEST1=3222243 # 1970-02-07 07:04:03 -TEST2=1005600000 # 2001-11-12 21:11:12 - -export LC_ALL=C -export TZ=UTC -count=0 - -check() -{ - S=$1 - A1=$2 - A2=$3 - - count=`expr ${count} + 1` - - if [ -z "${A2}" ]; then A2=${A1}; fi - - R=`date -r ${TEST1} +%${S}` - if [ "${R}" = "${A1}" ]; then - echo "ok ${S}{t1}" - else - echo "no ok ${S}{t1} - (got ${R}, expected ${A1})" - fi - - R=`date -r ${TEST2} +%${S}` - if [ "${R}" = "${A2}" ]; then - echo "ok ${S}{t2}" - else - echo "no ok ${S}{t2} - (got ${R}, expected ${A2})" - fi -} - -echo "1..78" - -check A Saturday Monday -check a Sat Mon -check B February November -check b Feb Nov -check C 19 20 -check c "Sat Feb 7 07:04:03 1970" "Mon Nov 12 21:20:00 2001" -check D 02/07/70 11/12/01 -check d 07 12 -check e " 7" 12 -check F "1970-02-07" "2001-11-12" -check G 1970 2001 -check g 70 01 -check H 07 21 -check h Feb Nov -check I 07 09 -check j 038 316 -check k " 7" 21 -check l " 7" " 9" -check M 04 20 -check m 02 11 -check p AM PM -check R 07:04 21:20 -check r "07:04:03 AM" "09:20:00 PM" -check S 03 00 -check s ${TEST1} ${TEST2} -check U 05 45 -check u 6 1 -check V 06 46 -check v " 7-Feb-1970" "12-Nov-2001" -check W 05 46 -check w 6 1 -check X "07:04:03" "21:20:00" -check x "02/07/70" "11/12/01" -check Y 1970 2001 -check y 70 01 -check Z UTC UTC -check z +0000 +0000 -check % % % -check + "Sat Feb 7 07:04:03 UTC 1970" "Mon Nov 12 21:20:00 UTC 2001" diff --git a/bin/date/tests/regress.sh b/bin/date/tests/regress.sh new file mode 100644 index 0000000..f5f72a6 --- /dev/null +++ b/bin/date/tests/regress.sh @@ -0,0 +1,557 @@ +#!/bin/sh + +# +# Regression tests for date(1) +# +# Submitted by Edwin Groothuis +# +# $FreeBSD: src/tools/regression/bin/date/regress.sh,v 1.4 2011/01/09 22:05:09 keramida Exp $ +# + +# +# These two date/times have been chosen carefully, they +# create both the single digit and double/multidigit version of +# the values. +# +# To create a new one, make sure you are using the UTC timezone! +# + +TEST1=3222243 # 1970-02-07 07:04:03 +TEST2=1005600000 # 2001-11-12 21:11:12 + +export LC_ALL=C +export TZ=UTC + +check() +{ + S=$1 + A1=$2 + A2=$3 + + # If the second sample text for formatted output has not been + # passed, assume it should match exactly the first one. + if [ -z "${A2}" ]; then + A2=${A1} + fi + + R=`date -r ${TEST1} +%${S}` + atf_check test "${R}" = "${A1}" + + R=`date -r ${TEST2} +%${S}` + atf_check test "${R}" = "${A2}" +} + +# ---------------------------------------------------------------------- + +atf_test_case A +A_head() +{ + atf_set "descr" "Verifies that 'A' formatting spec works" +} +A_body() +{ + check A Saturday Monday +} + +# ---------------------------------------------------------------------- + +atf_test_case a +a_head() +{ + atf_set "descr" "Verifies that 'a' formatting spec works" +} +a_body() +{ + check a Sat Mon +} + +# ---------------------------------------------------------------------- + +atf_test_case B +B_head() +{ + atf_set "descr" "Verifies that 'B' formatting spec works" +} +B_body() +{ + check B February November +} + +# ---------------------------------------------------------------------- + +atf_test_case b +b_head() +{ + atf_set "descr" "Verifies that 'b' formatting spec works" +} +b_body() +{ + check b Feb Nov +} + +# ---------------------------------------------------------------------- + +atf_test_case C +C_head() +{ + atf_set "descr" "Verifies that 'C' formatting spec works" +} +C_body() +{ + check C 19 20 +} + +# ---------------------------------------------------------------------- + +atf_test_case c +c_head() +{ + atf_set "descr" "Verifies that 'c' formatting spec works" +} +c_body() +{ + check c "Sat Feb 7 07:04:03 1970" "Mon Nov 12 21:20:00 2001" +} + +# ---------------------------------------------------------------------- + +atf_test_case D +D_head() +{ + atf_set "descr" "Verifies that 'D' formatting spec works" +} +D_body() +{ + check D 02/07/70 11/12/01 +} + +# ---------------------------------------------------------------------- + +atf_test_case d +d_head() +{ + atf_set "descr" "Verifies that 'd' formatting spec works" +} +d_body() +{ + check d 07 12 +} + +# ---------------------------------------------------------------------- + +atf_test_case e +e_head() +{ + atf_set "descr" "Verifies that 'e' formatting spec works" +} +e_body() +{ + check e " 7" 12 +} + +# ---------------------------------------------------------------------- + +atf_test_case F +F_head() +{ + atf_set "descr" "Verifies that 'F' formatting spec works" +} +F_body() +{ + check F "1970-02-07" "2001-11-12" +} + +# ---------------------------------------------------------------------- + +atf_test_case G +G_head() +{ + atf_set "descr" "Verifies that 'G' formatting spec works" +} +G_body() +{ + check G 1970 2001 +} + +# ---------------------------------------------------------------------- + +atf_test_case g +g_head() +{ + atf_set "descr" "Verifies that 'g' formatting spec works" +} +g_body() +{ + check g 70 01 +} + +# ---------------------------------------------------------------------- + +atf_test_case H +H_head() +{ + atf_set "descr" "Verifies that 'H' formatting spec works" +} +H_body() +{ + check H 07 21 +} + +# ---------------------------------------------------------------------- + +atf_test_case h +h_head() +{ + atf_set "descr" "Verifies that 'h' formatting spec works" +} +h_body() +{ + check h Feb Nov +} + +# ---------------------------------------------------------------------- + +atf_test_case I +I_head() +{ + atf_set "descr" "Verifies that 'I' formatting spec works" +} +I_body() +{ + check I 07 09 +} + +# ---------------------------------------------------------------------- + +atf_test_case j +j_head() +{ + atf_set "descr" "Verifies that 'j' formatting spec works" +} +j_body() +{ + check j 038 316 +} + +# ---------------------------------------------------------------------- + +atf_test_case k +k_head() +{ + atf_set "descr" "Verifies that 'k' formatting spec works" +} +k_body() +{ + check k " 7" 21 +} + +# ---------------------------------------------------------------------- + +atf_test_case l +l_head() +{ + atf_set "descr" "Verifies that 'l' formatting spec works" +} +l_body() +{ + check l " 7" " 9" +} + +# ---------------------------------------------------------------------- + +atf_test_case M +M_head() +{ + atf_set "descr" "Verifies that 'M' formatting spec works" +} +M_body() +{ + check M 04 20 +} + +# ---------------------------------------------------------------------- + +atf_test_case m +m_head() +{ + atf_set "descr" "Verifies that 'm' formatting spec works" +} +m_body() +{ + check m 02 11 +} + +# ---------------------------------------------------------------------- + +atf_test_case p +p_head() +{ + atf_set "descr" "Verifies that 'p' formatting spec works" +} +p_body() +{ + check p AM PM +} + +# ---------------------------------------------------------------------- + +atf_test_case R +R_head() +{ + atf_set "descr" "Verifies that 'R' formatting spec works" +} +R_body() +{ + check R 07:04 21:20 +} + +# ---------------------------------------------------------------------- + +atf_test_case r +r_head() +{ + atf_set "descr" "Verifies that 'r' formatting spec works" +} +r_body() +{ + check r "07:04:03 AM" "09:20:00 PM" +} + +# ---------------------------------------------------------------------- + +atf_test_case S +S_head() +{ + atf_set "descr" "Verifies that 'S' formatting spec works" +} +S_body() +{ + check S 03 00 +} + +# ---------------------------------------------------------------------- + +atf_test_case s +s_head() +{ + atf_set "descr" "Verifies that 's' formatting spec works" +} +s_body() +{ + check s ${TEST1} ${TEST2} +} + +# ---------------------------------------------------------------------- + +atf_test_case U +U_head() +{ + atf_set "descr" "Verifies that 'U' formatting spec works" +} +U_body() +{ + check U 05 45 +} + +# ---------------------------------------------------------------------- + +atf_test_case u +u_head() +{ + atf_set "descr" "Verifies that 'u' formatting spec works" +} +u_body() +{ + check u 6 1 +} + +# ---------------------------------------------------------------------- + +atf_test_case V +V_head() +{ + atf_set "descr" "Verifies that 'V' formatting spec works" +} +V_body() +{ + check V 06 46 +} + +# ---------------------------------------------------------------------- + +atf_test_case v +v_head() +{ + atf_set "descr" "Verifies that 'v' formatting spec works" +} +v_body() +{ + check v " 7-Feb-1970" "12-Nov-2001" +} + +# ---------------------------------------------------------------------- + +atf_test_case W +W_head() +{ + atf_set "descr" "Verifies that 'W' formatting spec works" +} +W_body() +{ + check W 05 46 +} + +# ---------------------------------------------------------------------- + +atf_test_case w +w_head() +{ + atf_set "descr" "Verifies that 'w' formatting spec works" +} +w_body() +{ + check w 6 1 +} + +# ---------------------------------------------------------------------- + +atf_test_case X +X_head() +{ + atf_set "descr" "Verifies that 'X' formatting spec works" +} +X_body() +{ + check X "07:04:03" "21:20:00" +} + +# ---------------------------------------------------------------------- + +atf_test_case x +x_head() +{ + atf_set "descr" "Verifies that 'x' formatting spec works" +} +x_body() +{ + check x "02/07/70" "11/12/01" +} + +# ---------------------------------------------------------------------- + +atf_test_case Y +Y_head() +{ + atf_set "descr" "Verifies that 'Y' formatting spec works" +} +Y_body() +{ + check Y 1970 2001 +} + +# ---------------------------------------------------------------------- + +atf_test_case y +y_head() +{ + atf_set "descr" "Verifies that 'y' formatting spec works" +} +y_body() +{ + check y 70 01 +} + +# ---------------------------------------------------------------------- + +atf_test_case Z +Z_head() +{ + atf_set "descr" "Verifies that 'Z' formatting spec works" +} +Z_body() +{ + check Z UTC UTC +} + +# ---------------------------------------------------------------------- + +atf_test_case z +z_head() +{ + atf_set "descr" "Verifies that 'z' formatting spec works" +} +z_body() +{ + check z +0000 +0000 +} + +# ---------------------------------------------------------------------- + +atf_test_case pct +pct_head() +{ + atf_set "descr" "Verifies that '%' formatting spec works" +} +pct_body() +{ + check % % % +} + +# ---------------------------------------------------------------------- + +atf_test_case plus +plus_head() +{ + atf_set "descr" "Verifies that '+' formatting spec works" +} +plus_body() +{ + check + "Sat Feb 7 07:04:03 UTC 1970" "Mon Nov 12 21:20:00 UTC 2001" +} + +# ---------------------------------------------------------------------- +# Main test case driving script. +# ---------------------------------------------------------------------- + +atf_init_test_cases() +{ + atf_add_test_case A + atf_add_test_case a + atf_add_test_case B + atf_add_test_case b + atf_add_test_case C + atf_add_test_case c + atf_add_test_case D + atf_add_test_case d + atf_add_test_case e + atf_add_test_case F + atf_add_test_case G + atf_add_test_case g + atf_add_test_case H + atf_add_test_case h + atf_add_test_case I + atf_add_test_case j + atf_add_test_case k + atf_add_test_case l + atf_add_test_case M + atf_add_test_case m + atf_add_test_case p + atf_add_test_case R + atf_add_test_case r + atf_add_test_case S + atf_add_test_case s + atf_add_test_case U + atf_add_test_case u + atf_add_test_case V + atf_add_test_case v + atf_add_test_case W + atf_add_test_case w + atf_add_test_case X + atf_add_test_case x + atf_add_test_case Y + atf_add_test_case y + atf_add_test_case Z + atf_add_test_case z + atf_add_test_case pct + atf_add_test_case plus +} --Apple-Mail=_AD906FCC-501F-4A0D-9822-A6E8655CA547-- From owner-freebsd-testing@FreeBSD.ORG Mon Jan 20 20:57:42 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0596662; Mon, 20 Jan 2014 20:57:42 +0000 (UTC) Received: from mail-pb0-x22e.google.com (mail-pb0-x22e.google.com [IPv6:2607:f8b0:400e:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7EB2A1A09; Mon, 20 Jan 2014 20:57:42 +0000 (UTC) Received: by mail-pb0-f46.google.com with SMTP id um1so2428554pbc.19 for ; Mon, 20 Jan 2014 12:57:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=sTHjZmmwzCKgYkUco511h18n2DzDO95tOTe1EDJp8QA=; b=clkneM8QpzmEGIvMbLAwDLEBB0Rw13nm8A+X+j5A+BC4ip4xiCHEswF0iLV3MquPSF pBzmJNLtXx5ZoZ7N1btJMLurUkCUahvlnPlymgrBf55XYVuFTXYWXo3gS/yE9oDcD6wA wIQZTzhCv8tVZLr+0wu0v/lfa2pcipEG/NtaUHnnGerRu3o5SPTgN8wHLIZNVKVwFVx3 wOSeDhSF3iTM7/jLjB5m81xtm6duJYU+LHtusWuzx5Y3n9MPdoMVQFKK9iJiIwnLrcHo FJydXUfn29islTFjn/u5RHtXh6KJpjuYWv9xl1jgYBBTIDbYSfMEkU7ULyERPH1ZchWO lfdg== X-Received: by 10.68.213.41 with SMTP id np9mr20645924pbc.90.1390251462042; Mon, 20 Jan 2014 12:57:42 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id ug2sm11064042pac.21.2014.01.20.12.57.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 20 Jan 2014 12:57:41 -0800 (PST) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: [PATCH] convert bin/date over to ATF From: Garrett Cooper In-Reply-To: Date: Mon, 20 Jan 2014 12:57:38 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <4D9C5320-2A97-4CCF-A5FB-8AA6BC961207@gmail.com> References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> To: Steven Kreuzer X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@FreeBSD.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 20:57:42 -0000 On Jan 20, 2014, at 12:44 PM, Steven Kreuzer = wrote: > Hey Garrett- >=20 > Unless I am doing something stupid, I can't get this patch doesn't > apply cleaning against head r260913 Hi Steven! svn patch works better (in fact I never ever use raw patch(1) nowadays = when working with svn) and it seems to work properly with git produced = diffs: # svn patch /root/convert-bin-date-over-to-atf.patch=20 U bin/date/tests/Makefile D bin/date/tests/legacy_test.sh A bin/date/tests/regress.sh # svn --version | head -1 svn, version 1.8.5 (r1542147) Cheers, -Garrett= From owner-freebsd-testing@FreeBSD.ORG Mon Jan 20 21:03:50 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3D117FD for ; Mon, 20 Jan 2014 21:03:50 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A42E1A9B for ; Mon, 20 Jan 2014 21:03:50 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kp14so7469389pab.34 for ; Mon, 20 Jan 2014 13:03:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6ZAFDRFUYVOlFpCNXJ/0EVyYoabWjc0lWHJVuDZLKM4=; b=gExSqxXxdiHspTKj0Hs75Pc97j6Nof2yL5bXA5lWxhhFSo3v/YPftFIkVBM31obsfl yJioPA6a4+ltNdV9MGGvrPMUo7VqttlCh/NAHSeAUK2BjXy/HN7KoP09VTkYhAGNUhdA 21CECof8GHXR6Aq3802IQ2402xSVy+IMabLmulBNYgl7ybiD7MhUzSGLsLX8bUAb05gk 2JMGbnRctjpR8jN90nipjYyNnKL8HSJd/aAV71+ccViqVY7PbIbnol90Yjq1Rk+uzvzO BRxHmHOBUJELHngS8wJ8etDMgw32OZKagEOhlXrxgN0q6rtKWP8EFqAjjVUH2Hy/b+oR XiWw== MIME-Version: 1.0 X-Received: by 10.66.163.2 with SMTP id ye2mr20561015pab.110.1390251830231; Mon, 20 Jan 2014 13:03:50 -0800 (PST) Sender: sjk4015@gmail.com Received: by 10.68.114.101 with HTTP; Mon, 20 Jan 2014 13:03:50 -0800 (PST) In-Reply-To: <4D9C5320-2A97-4CCF-A5FB-8AA6BC961207@gmail.com> References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> <4D9C5320-2A97-4CCF-A5FB-8AA6BC961207@gmail.com> Date: Mon, 20 Jan 2014 16:03:50 -0500 X-Google-Sender-Auth: 4uutOV9cXCrAR2dpvWOZuPgJCpg Message-ID: Subject: Re: [PATCH] convert bin/date over to ATF From: Steven Kreuzer To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 21:03:50 -0000 On Mon, Jan 20, 2014 at 3:57 PM, Garrett Cooper wrote: > On Jan 20, 2014, at 12:44 PM, Steven Kreuzer wrote: > >> Hey Garrett- >> >> Unless I am doing something stupid, I can't get this patch doesn't >> apply cleaning against head r260913 > > Hi Steven! > > svn patch works better (in fact I never ever use raw patch(1) nowadays when working with svn) and it seems to work properly with git produced diffs: > > # svn patch /root/convert-bin-date-over-to-atf.patch > U bin/date/tests/Makefile > D bin/date/tests/legacy_test.sh > A bin/date/tests/regress.sh > # svn --version | head -1 > svn, version 1.8.5 (r1542147) heh... turns out I was doing something stupid :) I am able to get both of your patches to apply cleanly with svn patch. Thanks! From owner-freebsd-testing@FreeBSD.ORG Mon Jan 20 21:05:19 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEEB0822; Mon, 20 Jan 2014 21:05:19 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8BA9D1AA8; Mon, 20 Jan 2014 21:05:19 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id y10so3736787pdj.12 for ; Mon, 20 Jan 2014 13:05:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6wb6b/rFwHx7LYE2c3/cuQ71gwbI+YTZ5IDPyDV3vTk=; b=ANND53+4odQLKluBIgc6rD/ayDLVyO/2TiFQblqrcsGw8apsU2aPKXavDAbWchQ9Ye uxvJGnWI+wWV9QX1ON9sh/UNNljxrtdGA9bBCuuaGpJajukdc29kLjNotWSxD7kr1r1V 7/scTBdXD+JL0LeJwfsFUTSGRMHkEJh3IOKsqaDPPt8oOIB4AXPmnMeElJ8dFBkoaAY3 kmT56p0yEE5bPF5uCkSJeaWvvuqCBsN0XZuTaBwrVN3DqO7jkFD7a1pExzQX90gildC6 ckJWyGOIOSM/yRllCt8jvECGSQhHixdIAewKctbe4Ztykeq76lZeFynKvcCfRiV6ueid oQ/A== X-Received: by 10.66.51.137 with SMTP id k9mr21086403pao.11.1390251918977; Mon, 20 Jan 2014 13:05:18 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id un5sm11251027pab.3.2014.01.20.13.05.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 20 Jan 2014 13:05:17 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: [PATCH] convert bin/date over to ATF From: Garrett Cooper In-Reply-To: Date: Mon, 20 Jan 2014 13:05:15 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <3BA09C41-09FE-4F87-951E-D4BE19936EB6@gmail.com> References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> <4D9C5320-2A97-4CCF-A5FB-8AA6BC961207@gmail.com> To: Steven Kreuzer X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 21:05:19 -0000 On Jan 20, 2014, at 1:03 PM, Steven Kreuzer = wrote: > On Mon, Jan 20, 2014 at 3:57 PM, Garrett Cooper = wrote: >> On Jan 20, 2014, at 12:44 PM, Steven Kreuzer = wrote: >>=20 >>> Hey Garrett- >>>=20 >>> Unless I am doing something stupid, I can't get this patch doesn't >>> apply cleaning against head r260913 >>=20 >> Hi Steven! >>=20 >> svn patch works better (in fact I never ever use raw patch(1) = nowadays when working with svn) and it seems to work properly with git = produced diffs: >>=20 >> # svn patch /root/convert-bin-date-over-to-atf.patch >> U bin/date/tests/Makefile >> D bin/date/tests/legacy_test.sh >> A bin/date/tests/regress.sh >> # svn --version | head -1 >> svn, version 1.8.5 (r1542147) >=20 > heh... turns out I was doing something stupid :) Not really! It=92s just not a very well advertised tool (and people like = using patch, but then you have to svn add/rm things after the fact which = becomes a minor pain) :D. > I am able to get both of your patches to apply cleanly with svn patch. Awesome :). Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Tue Jan 21 18:00:20 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ACA097C5 for ; Tue, 21 Jan 2014 18:00:20 +0000 (UTC) Received: from mail-we0-x233.google.com (mail-we0-x233.google.com [IPv6:2a00:1450:400c:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 48DB91748 for ; Tue, 21 Jan 2014 18:00:20 +0000 (UTC) Received: by mail-we0-f179.google.com with SMTP id w62so8491605wes.10 for ; Tue, 21 Jan 2014 10:00:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=BJsrih2lEZ8yCIboNiALUNWMycp3x9WUiHlsMQElkoE=; b=PY4sz+KDfiD1iIW7ln/xRmm7yhgKG8pHdvJ6CaCqFK7nzwGKKYCkL1vcseQSyye9hN FcLiRBTGvQaafhAzNGzJyncOwg0fgaVKkmkqIOkgO5fv5Ev5+YilcLPD4oXd8X3maAJu lXKDO/08a1OygnwPRVn7cXMuuXvHtSnFJyHZVNuP6L2jxm5NjQcNrPKJzWDE7xr8o0aq u5xRdwqPkuv3ZgKN3DIHqsEHgMJQYHb/knepzY2+hpE97yc3gJXFUT32JTF7HIdtuLjt a77Bj8x4iRBFh4E/3Ia09oOqrd3dgaN4vSz0c2SW0NfRQGtVbigLmvbmiv62hZxslKbg 6LxQ== MIME-Version: 1.0 X-Received: by 10.180.211.39 with SMTP id mz7mr15527516wic.53.1390327218587; Tue, 21 Jan 2014 10:00:18 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Tue, 21 Jan 2014 10:00:18 -0800 (PST) Date: Tue, 21 Jan 2014 11:00:18 -0700 X-Google-Sender-Auth: mUUtiUmejcnnQA7Dis2WBP3gja8 Message-ID: Subject: PATCH: fix "WITH_TESTS=yes make installworld" on amd64 From: Alan Somers To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 18:00:20 -0000 I finally found the problem. Make was trying to install tests during the install32 phase. During this phase, it adds "lib32" to MAKEOBJDIRPREFIX, so it can't find the tests, which were never built with that prefix. The command worked fine when I ran it under "make buildenv", because that uses the normal MAKEOBJDIRPREFIX. I'm not sure what prevented the tests from being built during the lib32 build phase, but this patch will disable tests during both building and installing 32 bit libraries. At work, we never noticed this problem in our SpectraBSD branch because we set WITHOUT_LIB32=1 in /etc/src.conf. Does this patch look ok? Should I be disabling tests some other way? I looked for a way to disable recursing into the tests directories whenever LIBRARIES_ONLY was set, but didn't see an easy way. -Alan Index: Makefile.inc1 =================================================================== --- Makefile.inc1 (revision 260986) +++ Makefile.inc1 (working copy) @@ -417,7 +417,7 @@ -DNO_LINT LIB32WMAKE= ${LIB32WMAKEENV} ${MAKE} ${LIB32WMAKEFLAGS} \ - -DWITHOUT_MAN -DWITHOUT_INFO -DWITHOUT_HTML + -DWITHOUT_MAN -DWITHOUT_INFO -DWITHOUT_HTML -DNO_TESTS LIB32IMAKE= ${LIB32WMAKE:NINSTALL=*:NDESTDIR=*:N_LDSCRIPTROOT=*} -DNO_INCS \ ${IMAKE_INSTALL} .endif From owner-freebsd-testing@FreeBSD.ORG Tue Jan 21 20:03:16 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C72B0803; Tue, 21 Jan 2014 20:03:16 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 91D95137A; Tue, 21 Jan 2014 20:03:16 +0000 (UTC) Received: by mail-pd0-f173.google.com with SMTP id y10so5082921pdj.32 for ; Tue, 21 Jan 2014 12:03:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=yzAPJcaYMgNxZ8LlatUinNLcXq3zUnF4UQot0fw+ffA=; b=vMlHmhTv+LANpsPsqcM5RXcZPatwArtm8b07bZ/ensKgvVQ4fy+lBzl1+OCIQ8huDx L2sSwNBlPDdCFmiu1OFZLekgue9iGbnXjx8Y5YwEDw3sUg8FpxoM4y1LkVbtqWoIRA9J Tg1lsXkuHCMSKvi2O3rCIR6qE+YYqWa/w6AbdiW0AViHmKeTJxLqXphOy7gcYS6ffcX7 gIq6secF1QC2X4C9wtngx2N5X7Z/3dUaEu5af9OgsOjGxLd2EcOLuhI6Kwakcai1mNvD MISYLBdl7JZw6GzDiBB7bWvWr7eHwjcN161NaDmGhWvEIyzvGKUDHpWQR2iDtTGROJKx /TPg== X-Received: by 10.68.224.34 with SMTP id qz2mr27328125pbc.84.1390334596151; Tue, 21 Jan 2014 12:03:16 -0800 (PST) Received: from [10.0.2.128] ([64.14.143.130]) by mx.google.com with ESMTPSA id qw8sm14990144pbb.27.2014.01.21.12.03.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 21 Jan 2014 12:03:14 -0800 (PST) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <9EF7C79E-B9D0-4DDE-A425-8D6FFBC0FC1E@gmail.com> X-Mailer: iPhone Mail (11B554a) From: Garrett Cooper Subject: Re: PATCH: fix "WITH_TESTS=yes make installworld" on amd64 Date: Tue, 21 Jan 2014 12:03:08 -0800 To: Alan Somers Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 20:03:16 -0000 > On Jan 21, 2014, at 10:00, Alan Somers wrote: >=20 > I finally found the problem. Make was trying to install tests during > the install32 phase. During this phase, it adds "lib32" to > MAKEOBJDIRPREFIX, so it can't find the tests, which were never built > with that prefix. The command worked fine when I ran it under "make > buildenv", because that uses the normal MAKEOBJDIRPREFIX. >=20 > I'm not sure what prevented the tests from being built during the > lib32 build phase, but this patch will disable tests during both > building and installing 32 bit libraries. >=20 > At work, we never noticed this problem in our SpectraBSD branch > because we set WITHOUT_LIB32=3D1 in /etc/src.conf. >=20 >=20 > Does this patch look ok? Should I be disabling tests some other way? > I looked for a way to disable recursing into the tests directories > whenever LIBRARIES_ONLY was set, but didn't see an easy way. >=20 >=20 > -Alan >=20 >=20 > Index: Makefile.inc1 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- Makefile.inc1 (revision 260986) > +++ Makefile.inc1 (working copy) > @@ -417,7 +417,7 @@ > -DNO_LINT >=20 > LIB32WMAKE=3D ${LIB32WMAKEENV} ${MAKE} ${LIB32WMAKEFLAGS} \ > - -DWITHOUT_MAN -DWITHOUT_INFO -DWITHOUT_HTML > + -DWITHOUT_MAN -DWITHOUT_INFO -DWITHOUT_HTML -DNO_TESTS > LIB32IMAKE=3D ${LIB32WMAKE:NINSTALL=3D*:NDESTDIR=3D*:N_LDSCRIPTROOT=3D*= } -DNO_INCS \ > ${IMAKE_INSTALL} > .endif I have to check my git tree, but I'm pretty sure this was already accoun= ted for (along with some other changes). I'll get back to sometime today about this... Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Tue Jan 21 22:17:07 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 150CF7CD; Tue, 21 Jan 2014 22:17:07 +0000 (UTC) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [IPv6:2607:f8b0:400e:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D64BE1F2B; Tue, 21 Jan 2014 22:17:06 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id rd3so9037119pab.2 for ; Tue, 21 Jan 2014 14:17:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:content-transfer-encoding:subject:mime-version:from :message-id:date:cc:to; bh=OmxRazhF5cjYWKdoIP3W6KGm6ZABWyowrS8ETfxhF44=; b=ShzYIqUgTrDS3zwBY6g5LJXZsNsEG24ioTQ0Tsk4wMv4duIVMuVOi3iRkrq57C+I/b ARQk17XyjgtlUGMd6VI26N05DkqUns41fC9vQ+hwayNtjYeTUXncjGGmQjMaj/GcFxqW l9Dt7DmYkgu5R8Rk4Q1pm7gOxd+pEWJOPi9xElQ9PzKI6+vS++wLcyqEb/5ad7NbifLe WlCMGS/54CpkOWuz3ewGbImReZ6GpsKVEZ0WhRvep2+yUGeY08nPhivJk1GwOcWFJRVM m+nUCoJe9OuQCmMTMx471c/2nnqWR/d5OafSfreRsUF1GK7IlR6rWL6gLXaZ3FLq40We /OjQ== X-Received: by 10.68.164.4 with SMTP id ym4mr27763024pbb.53.1390342626449; Tue, 21 Jan 2014 14:17:06 -0800 (PST) Received: from [10.0.2.128] ([64.14.143.130]) by mx.google.com with ESMTPSA id gn5sm15576404pbc.29.2014.01.21.14.17.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 21 Jan 2014 14:17:05 -0800 (PST) Content-Type: multipart/mixed; boundary=Apple-Mail-5D1BF951-BCB4-4526-BB7B-9189041BF7B9 Content-Transfer-Encoding: 7bit Subject: Makefile.inc1.patch Mime-Version: 1.0 (1.0) From: Garrett Cooper Message-Id: Date: Tue, 21 Jan 2014 14:17:01 -0800 To: Alan Somers X-Mailer: iPhone Mail (11B554a) Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 22:17:07 -0000 --Apple-Mail-5D1BF951-BCB4-4526-BB7B-9189041BF7B9 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Hi all, The attached patch fixes/enhances the build so -DNO_TESTS is passed prop= erly through various sections of the build that don't require tests in order= to build/install. This should also fix Alan's recent reported issue with -D= WITH_TESTS failing on amd64. Cheers! -Garrett --Apple-Mail-5D1BF951-BCB4-4526-BB7B-9189041BF7B9 Content-Type: text/plain; name=Makefile.inc1.patch Content-Disposition: attachment; filename=Makefile.inc1.patch Content-Transfer-Encoding: 7bit diff --git a/Makefile.inc1 b/Makefile.inc1 index bf968a6..c5078f9 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -246,7 +247,7 @@ BMAKE= MAKEOBJDIRPREFIX=${WORLDTMP} \ SSP_CFLAGS= \ -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT -DWITHOUT_MAN \ -DNO_PIC -DNO_PROFILE -DNO_SHARED \ - -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD + -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD -DNO_TESTS # build-tools stage TMAKE= MAKEOBJDIRPREFIX=${OBJTREE} \ @@ -256,12 +257,12 @@ TMAKE= MAKEOBJDIRPREFIX=${OBJTREE} \ BOOTSTRAPPING=${OSRELDATE} \ SSP_CFLAGS= \ -DNO_LINT \ - -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD + -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD -DNO_TESTS # cross-tools stage XMAKE= TOOLS_PREFIX=${WORLDTMP} ${BMAKE} \ TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} \ - -DWITHOUT_GDB + -DWITHOUT_GDB -DNO_TESTS # kernel-tools stage KTMAKEENV= INSTALL="sh ${.CURDIR}/tools/install.sh" \ @@ -414,7 +415,8 @@ LIB32WMAKEFLAGS+= \ -DLIBRARIES_ONLY \ -DNO_CPU_CFLAGS \ -DNO_CTF \ - -DNO_LINT + -DNO_LINT \ + -DNO_TESTS LIB32WMAKE= ${LIB32WMAKEENV} ${MAKE} ${LIB32WMAKEFLAGS} \ -DWITHOUT_MAN -DWITHOUT_INFO -DWITHOUT_HTML @@ -569,7 +579,7 @@ _libraries: @echo "--------------------------------------------------------------" ${_+_}cd ${.CURDIR}; \ ${WMAKE} -DNO_FSCHG -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT \ - -DWITHOUT_MAN -DNO_PROFILE libraries + -DWITHOUT_MAN -DNO_PROFILE -DNO_TESTS libraries _depend: @echo @echo "--------------------------------------------------------------" @@ -1877,7 +1893,7 @@ XDEV_CPUTYPE?=${TARGET_CPUTYPE} NOFUN=-DNO_FSCHG -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT \ -DWITHOUT_MAN -DWITHOUT_NLS -DNO_PROFILE \ - -DWITHOUT_KERBEROS -DWITHOUT_RESCUE -DNO_WARNS \ + -DWITHOUT_KERBEROS -DWITHOUT_RESCUE -DNO_TESTS -DNO_WARNS \ TARGET=${XDEV} TARGET_ARCH=${XDEV_ARCH} \ CPUTYPE=${XDEV_CPUTYPE} --Apple-Mail-5D1BF951-BCB4-4526-BB7B-9189041BF7B9-- From owner-freebsd-testing@FreeBSD.ORG Tue Jan 21 22:45:57 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45DB5EC6 for ; Tue, 21 Jan 2014 22:45:57 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D64651175 for ; Tue, 21 Jan 2014 22:45:56 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id hr1so4975282wib.0 for ; Tue, 21 Jan 2014 14:45:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=URW61zoK1s72f6DfErMLd1aBU8elxQ1ceoJC2gZS7o4=; b=FWZT3pcyUX/pQHaMmIUhnfnWHTF8tttbD9qWil9o0WsNLy5wC9gVjbzCN7kpxAyqkI FcP5EvE46sN7dzs5Lg2aghzLa7FIHCLfLJ9JCxX+HmdxRfnyW/iR5jOLC5Igci/WyNKC EArUFx2kWQ2fmSr03PLgQHf+nhSueGb82YOq57W18vrpmZbTSrieRGodFwXeWYOnha0f k9bNhh5bqb1qWm6oG0T7rRmid9bf4Pg67M3UK6viidjY0sBHsRStGKX1HB5Qy+SzOXpP UTkbtNHMk79QbbVpTSullUffNI5MZ4jEGFLetonQYV0odSqSNwWPFHwfpozh+BGOFtAV ur9Q== MIME-Version: 1.0 X-Received: by 10.194.63.228 with SMTP id j4mr21743664wjs.34.1390344355160; Tue, 21 Jan 2014 14:45:55 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Tue, 21 Jan 2014 14:45:55 -0800 (PST) Date: Tue, 21 Jan 2014 15:45:55 -0700 X-Google-Sender-Auth: syPep2oTHs3NgXG2HDsTNKzVasA Message-ID: Subject: PATCH: add ATF tests in sys From: Alan Somers To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 22:45:57 -0000 I rewrote the unix seqpacket tests in ATF. The hard part is adding them to the build. The linked patch adds them in the directory sys/kern/tests and builds them as part of buildworld, not buildkernel. They get installed to /usr/tests/sys/kern, but the intermediate objects are stored in /usr/obj/sys/kern/tests. That means that you can't have different tests associated with different kernel configs. I think that this is desirable, because you wouldn't be able to install tests for different kernel configs anyway, given our chosen layout of /usr/tests. Please comment on the parts of this patch that deal with Makefiles. Is this the appropriate way to add sys tests to the build? Shouldn't I be building them in buildkernel instead of buildworld? I couldn't find a good way to do that. https://gist.github.com/asomers/5f98656d574e68314029 From owner-freebsd-testing@FreeBSD.ORG Tue Jan 21 23:56:13 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8EDEB237; Tue, 21 Jan 2014 23:56:13 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5329916E7; Tue, 21 Jan 2014 23:56:13 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kp14so9086986pab.34 for ; Tue, 21 Jan 2014 15:56:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=cxqtVQc40EyxsN9fr6zNu3t8uCgj2fPXfFqUoQk/UTM=; b=hk5YJLbG3+fi84tMKJfFZWpvOTrOPSnJRZVCC/B03zEpcPJx0oioC3UQEmpzCb7rJD GRNKqIiDZfxVkhAWnUjcyZgT2fMaxwQ4yxWoJc2VvSBVVcjcX2jocj8sthQEtBOSnFlH R7lshMVhpOJVKysqBtcPk+b0mUQqPbojcr0SSEnEODaVLKpN/dHI8v4q8dCuECZaOmFu AHrpuglfGkZKBMr59KCBteSWKyaesZ6VYOCSLTlTEygeuvILZvsbISQMVI2Spg7s4Him xmeDoONIXdLdDQjeqXvDOE1qV7g3VRFpMvcev9oiB2Jh4ZCVel6gcbhTKIY5gUdq0c9z ZccA== X-Received: by 10.69.1.104 with SMTP id bf8mr27672042pbd.75.1390348572865; Tue, 21 Jan 2014 15:56:12 -0800 (PST) Received: from [10.0.2.128] ([64.14.143.130]) by mx.google.com with ESMTPSA id yd4sm16041597pbc.13.2014.01.21.15.56.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 21 Jan 2014 15:56:11 -0800 (PST) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (11B554a) From: Garrett Cooper Subject: Re: PATCH: add ATF tests in sys Date: Tue, 21 Jan 2014 15:56:10 -0800 To: Alan Somers Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 23:56:13 -0000 > On Jan 21, 2014, at 14:45, Alan Somers wrote: >=20 > I rewrote the unix seqpacket tests in ATF. The hard part is adding > them to the build. The linked patch adds them in the directory > sys/kern/tests and builds them as part of buildworld, not buildkernel. > They get installed to /usr/tests/sys/kern, but the intermediate > objects are stored in /usr/obj/sys/kern/tests. That means that you > can't have different tests associated with different kernel configs. > I think that this is desirable, because you wouldn't be able to > install tests for different kernel configs anyway, given our chosen > layout of /usr/tests. >=20 > Please comment on the parts of this patch that deal with Makefiles. > Is this the appropriate way to add sys tests to the build? Shouldn't > I be building them in buildkernel instead of buildworld? I couldn't > find a good way to do that. I bypassed that for simplicity and placed the tests in tests/sys/kern/... et= c. Shoehorning things into our overly complex kernel build system just seems= like a really bad idea... Just make it load as a driver, loadable, and depe= ndent on kern.features to run..? Cheers! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Wed Jan 22 00:02:51 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EB943D3; Wed, 22 Jan 2014 00:02:51 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A4C5F178C; Wed, 22 Jan 2014 00:02:50 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id u57so8820619wes.14 for ; Tue, 21 Jan 2014 16:02:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=WQC4ZbpYdtP9CgIA+zCIXk57eNyhuEDlA80mSdbdgWA=; b=YbBzU16q9m+hpEI+dd/Pr0K+2S7X780mrDlf7B8A5crYllClQcyR1/f3AONZkkKaKX vkcDTQjfV+JTzbA/1Mwnfyfz3G908BonD5eJtNs9XWmz6gWMI2m6b16kdvY7PRlW+lCW B3kP+LyZDHM254OVwSgZn2SiuaA9ZDN1uMb/e6tw+QO5f1OdkBMKmhEied0Nv23G9NMC dxQrJifD13/2mB+b8QwKxVUQ7zxUYhcCWBQyO71iG57vxIDrvk7ghQvr4Eir1/yhSAKz dz0VczjedAYiYu96okVzAddsgf5JzAe0LbtXxD/STef3+kx/noa6AQj+IxWoJqHta+PD 7jOg== MIME-Version: 1.0 X-Received: by 10.194.78.141 with SMTP id b13mr21140236wjx.32.1390348969092; Tue, 21 Jan 2014 16:02:49 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Tue, 21 Jan 2014 16:02:49 -0800 (PST) In-Reply-To: References: Date: Tue, 21 Jan 2014 17:02:49 -0700 X-Google-Sender-Auth: Z7kbzWE0-JVRvdOC07Wpq9oL0o4 Message-ID: Subject: Re: PATCH: add ATF tests in sys From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 00:02:51 -0000 On Tue, Jan 21, 2014 at 4:56 PM, Garrett Cooper wro= te: > >> On Jan 21, 2014, at 14:45, Alan Somers wrote: >> >> I rewrote the unix seqpacket tests in ATF. The hard part is adding >> them to the build. The linked patch adds them in the directory >> sys/kern/tests and builds them as part of buildworld, not buildkernel. >> They get installed to /usr/tests/sys/kern, but the intermediate >> objects are stored in /usr/obj/sys/kern/tests. That means that you >> can't have different tests associated with different kernel configs. >> I think that this is desirable, because you wouldn't be able to >> install tests for different kernel configs anyway, given our chosen >> layout of /usr/tests. >> >> Please comment on the parts of this patch that deal with Makefiles. >> Is this the appropriate way to add sys tests to the build? Shouldn't >> I be building them in buildkernel instead of buildworld? I couldn't >> find a good way to do that. > > I bypassed that for simplicity and placed the tests in tests/sys/kern/...= etc. Shoehorning things into our overly complex kernel build system just s= eems like a really bad idea... Just make it load as a driver, loadable, and= dependent on kern.features to run..? > Cheers! > -Garrett I tried to place the tests as closely as possible to the source code that they're testing, according to https://wiki.freebsd.org/TestSuite/Structure#Makefiles_for_test_programs . I don't feel strongly about it either way, but I think Simon and jmmv do. FWIW, the tests run as a userland program, not in kernel mode. -Alan From owner-freebsd-testing@FreeBSD.ORG Wed Jan 22 00:23:32 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A24BA679; Wed, 22 Jan 2014 00:23:32 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6E0DE18FF; Wed, 22 Jan 2014 00:23:32 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id p10so4613821pdj.15 for ; Tue, 21 Jan 2014 16:23:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=SQChJ83zbhssvW3hfMbbXLSXuytpHLK06/5XVb0gdKQ=; b=klO1yksjfjbQ61HPesfznXyrGj0MDWqM7yGgHMn1iBBSefebTjzgpYnxVsz0pD+dZJ iQymx4dsH6LsWzggVnv/pUv/4nQtOQE0hsrcK3S5sAtl+HHPcaf11w/VvK9TgwnH/STc cGLHBGO2sd0Lvc1/m5IGS1K2qRX+wPVckuZhWQJbF6ieXf/lEAni9xYo7iPBTtPa1E4u yj2TnC9p+/+jAGI0SsGpp2TxvbALIlsMrfpL8BjD3Xy4qWEuwDj0tkx5/3Dvfv2rnwpm bprAD2eo4B1AfEe3YupwSSmXxLN1rTVU0DGHNfD8HnwUdXNPXXS9xyh9jq6SU3bBla1M On1g== X-Received: by 10.68.34.37 with SMTP id w5mr12212513pbi.159.1390350212141; Tue, 21 Jan 2014 16:23:32 -0800 (PST) Received: from [10.0.2.128] ([64.14.143.130]) by mx.google.com with ESMTPSA id tu3sm16129571pbc.40.2014.01.21.16.23.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 21 Jan 2014 16:23:31 -0800 (PST) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <7051E061-C054-47BF-A5B2-747B9D3E85A1@gmail.com> X-Mailer: iPhone Mail (11B554a) From: Garrett Cooper Subject: Re: PATCH: add ATF tests in sys Date: Tue, 21 Jan 2014 16:23:25 -0800 To: Alan Somers Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 00:23:32 -0000 > On Jan 21, 2014, at 16:02, Alan Somers wrote: >=20 >> On Tue, Jan 21, 2014 at 4:56 PM, Garrett Cooper w= rote: >>=20 >>> On Jan 21, 2014, at 14:45, Alan Somers wrote: >>>=20 >>> I rewrote the unix seqpacket tests in ATF. The hard part is adding >>> them to the build. The linked patch adds them in the directory >>> sys/kern/tests and builds them as part of buildworld, not buildkernel. >>> They get installed to /usr/tests/sys/kern, but the intermediate >>> objects are stored in /usr/obj/sys/kern/tests. That means that you >>> can't have different tests associated with different kernel configs. >>> I think that this is desirable, because you wouldn't be able to >>> install tests for different kernel configs anyway, given our chosen >>> layout of /usr/tests. >>>=20 >>> Please comment on the parts of this patch that deal with Makefiles. >>> Is this the appropriate way to add sys tests to the build? Shouldn't >>> I be building them in buildkernel instead of buildworld? I couldn't >>> find a good way to do that. >>=20 >> I bypassed that for simplicity and placed the tests in tests/sys/kern/...= etc. Shoehorning things into our overly complex kernel build system just se= ems like a really bad idea... Just make it load as a driver, loadable, and d= ependent on kern.features to run..? >> Cheers! >> -Garrett >=20 > I tried to place the tests as closely as possible to the source code > that they're testing, according to > https://wiki.freebsd.org/TestSuite/Structure#Makefiles_for_test_programs > . I don't feel strongly about it either way, but I think Simon and > jmmv do. >=20 > FWIW, the tests run as a userland program, not in kernel mode. Fwiw I spent months mulling over various details (this was just one point) w= ith gnn, marcel, and mfleming. The consensus was that that was a really bad i= dea because it wasn't terribly clear that tests covered a particular area, a= nd some sys/ tests could cover multiple areas (pjdfstest for instance). I mirrored the exact structure in the source tree, but avoided tainting thin= gs in sys/... Because integrating sys/ into buildworld and starting to mix k= ernel and world build logic together will irritate a large number of kernel d= evs and userland devs. This was the path of least resistance... HTH, -Garrett= From owner-freebsd-testing@FreeBSD.ORG Wed Jan 22 15:42:20 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 46ECAE6F for ; Wed, 22 Jan 2014 15:42:20 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D2FE31539 for ; Wed, 22 Jan 2014 15:42:19 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id hi8so727684wib.8 for ; Wed, 22 Jan 2014 07:42:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=1bT/sKlNxD9LFzdWOLPR5ldA74jl0jGbZgTLA9wL9yo=; b=z3U/lDwmGmx19Ugzs8WVbw8/CUojKdID7z9WeyEwheOWogQbtG8VNWSj86pQbhsR3D qEm2G/IfAUoiohNYJSOeHSvHwzGkE5M/KzgxfDkEGF4gBz0CSGcYuTFeeWewHILoG+G4 wTbMyjlobr10/Wv7dnlY45gaxjzUPeoV8UlHfP3CGujaPDA21RKUVFjVpMyHAbfOooo8 +CmZ98QvQsDn/y55x+7vqejjA3pu8AUbLWF12Q6C0g6X26bsafgRBCHHvnovVhLcZQCE MQgfYuEVLXFMVbQDZOd1f4DWP28q9t912R39W+St41ky2Xt8soIFPshd/ww2Thylfkg1 erGA== MIME-Version: 1.0 X-Received: by 10.194.58.180 with SMTP id s20mr2552374wjq.54.1390405338309; Wed, 22 Jan 2014 07:42:18 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Wed, 22 Jan 2014 07:42:18 -0800 (PST) In-Reply-To: References: Date: Wed, 22 Jan 2014 08:42:18 -0700 X-Google-Sender-Auth: 74N-Toq7zWiWgLiYm3FusxHFLDM Message-ID: Subject: Re: Makefile.inc1.patch From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 15:42:20 -0000 On Tue, Jan 21, 2014 at 3:17 PM, Garrett Cooper wro= te: > Hi all, > The attached patch fixes/enhances the build so -DNO_TESTS is passed p= roperly through various sections of the build that don't require tests in o= rder to build/install. This should also fix Alan's recent reported issue wi= th -DWITH_TESTS failing on amd64. > Cheers! > -Garrett > The patch doesn't work for me. I get the following error during buildworld during stage 4.4. --- atf-check --- --- games.all__D --- =3D=3D=3D> games/morse (all) --- libexec.all__D --- c++ -O2 -pipe -DHAVE_CONFIG_H -I/usr/home/alans/freebsd/head/contrib/atf -Qunused-arguments -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wpointer-arith -Wno-uninitialized -Wno-empty-body -Wno-string-plus-int -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-c++11-extensions -L/vmpool/obj/usr/home/alans/freebsd/head/libexec/atf/atf-check/../../../li= b/atf/libatf-c++ -L/vmpool/obj/usr/home/alans/freebsd/head/libexec/atf/atf-check/../../../li= b/atf/libatf-c -o atf-check atf-check.o -latf-c++ -latf-c --- lib.all__D --- --- unmount.po --- cc -DPROF -O2 -pipe -I/usr/home/alans/freebsd/head/lib/libc/include -I/usr/home/alans/freebsd/head/lib/libc/../../include -I/usr/home/alans/freebsd/head/lib/libc/amd64 -DNLS -D__DBINTERFACE_PRIVATE -I/usr/home/alans/freebsd/head/lib/libc/../../contrib/gdtoa -I/usr/home/alans/freebsd/head/lib/libc/../../contrib/libc-vis -DINET6 -I/vmpool/obj/usr/home/alans/freebsd/head/lib/libc -I/usr/home/alans/freebsd/head/lib/libc/resolv -D_ACL_PRIVATE -DPOSIX_MISTAKE -I/usr/home/alans/freebsd/head/lib/libc/../../contrib/jemalloc/include -I/usr/home/alans/freebsd/head/lib/libc/../../contrib/tzcode/stdtime -I/usr/home/alans/freebsd/head/lib/libc/stdtime -I/usr/home/alans/freebsd/head/lib/libc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/usr/home/alans/freebsd/head/lib/libc/rpc -DYP -DNS_CACHING -DSYMBOL_VERSIONING -std=3Dgnu99 -Qunused-arguments -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -c unmount.S -o unmount.po --- gnu.all__D --- --- checklist.po --- cc -pg -O2 -pipe -I/usr/home/alans/freebsd/head/gnu/lib/libdialog -I/usr/home/alans/freebsd/head/gnu/lib/libdialog/../../../contrib/dialog -D_XOPEN_SOURCE_EXTENDED -DGCC_UNUSED=3D__unused -std=3Dgnu99 -Qunused-arguments -fstack-protector -Wsystem-headers -Werror -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parentheses -c /usr/home/alans/freebsd/head/gnu/lib/libdialog/../../../contrib/dialog/chec= klist.c -o checklist.po --- libexec.all__D --- /vmpool/obj/usr/home/alans/freebsd/head/tmp/usr/bin/ld: cannot find -latf-c= ++ c++: error: linker command failed with exit code 1 (use -v to see invocatio= n) *** [atf-check] Error code 1 From owner-freebsd-testing@FreeBSD.ORG Wed Jan 22 16:34:30 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6EEDC2; Wed, 22 Jan 2014 16:34:30 +0000 (UTC) Received: from mail-gg0-x234.google.com (mail-gg0-x234.google.com [IPv6:2607:f8b0:4002:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 930EF19B5; Wed, 22 Jan 2014 16:34:30 +0000 (UTC) Received: by mail-gg0-f180.google.com with SMTP id q3so3110304gge.39 for ; Wed, 22 Jan 2014 08:34:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=LryzOxGCYY/cCNHbIL9eVc324RA9g34xMhSGlws0S0E=; b=P0ZiMjmEahB7+EUnyiRtFBjzfWh1JkQgvwXUoPm057fXKL4Pcb0W+lUyTq2pxc5r4K vQ1a/SIJESZJ5MxuxSNo3a+vGcCjXYo6MSI66lgcuB/pdy8zkSt5xatK8W2CL6+d9reI hPrpuOj6vVmwGvqgrzg3rYL0falf8dIDU6z08jC3pKXoCMHr4oEFJcx4oHip8O7vKrhm IDDCZvxSy5WYsBujWI8h4YpnRK+Ri5/QV9lZigcWeJLB2JYpdOY8+uPG5rClbpt3fB92 xj1ob9UqBX9C1T+1omRnIOMxdxNqxx6ct6cTjM8D+FzDbaUDUMuADK88Llti7ts+Ji0A rk5Q== X-Received: by 10.236.130.138 with SMTP id k10mr2504461yhi.31.1390408469725; Wed, 22 Jan 2014 08:34:29 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:b068:e4eb:186a:8dff? ([2601:8:ab80:7d6:b068:e4eb:186a:8dff]) by mx.google.com with ESMTPSA id 57sm25690115yhl.4.2014.01.22.08.34.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 Jan 2014 08:34:28 -0800 (PST) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> X-Mailer: iPhone Mail (11B554a) From: Garrett Cooper Subject: Re: Makefile.inc1.patch Date: Wed, 22 Jan 2014 08:34:27 -0800 To: Alan Somers Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 16:34:31 -0000 > On Jan 22, 2014, at 7:42, Alan Somers wrote: >=20 >> On Tue, Jan 21, 2014 at 3:17 PM, Garrett Cooper w= rote: >> Hi all, >> The attached patch fixes/enhances the build so -DNO_TESTS is passed pr= operly through various sections of the build that don't require tests in ord= er to build/install. This should also fix Alan's recent reported issue with -= DWITH_TESTS failing on amd64. >> Cheers! >> -Garrett >=20 > The patch doesn't work for me. I get the following error during > buildworld during stage 4.4. ... Oh yeah... It's that chicken and egg issue due to ATF now being controlled b= y MK_TESTS=3D=3Dyes... There's more to this patch I need to provide you in o= rder for things to work... Cheers, -Garrett= From owner-freebsd-testing@FreeBSD.ORG Wed Jan 22 23:16:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68EFFF8A for ; Wed, 22 Jan 2014 23:16:38 +0000 (UTC) Received: from mail-qc0-f171.google.com (mail-qc0-f171.google.com [209.85.216.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 218791EE9 for ; Wed, 22 Jan 2014 23:16:37 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id n7so1507057qcx.2 for ; Wed, 22 Jan 2014 15:16:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=WtjEAPDWeCo2Yqycs8VqKiQgY944/V3HFmUMhwdp6sc=; b=gg0GEhKsffhIISAoFxREqC6KeuZ7IpjCCwKwusE5h0+Y5G00CMOU8m9siAKWA43C7R NC2tdY+f5MR+NfqkJqx8pWCIZ2kGzNX9QV/IARTSqeGCczmmvgJXHRAA3ar7KxQEhYI4 v5eUpv/WuV6fw/+QMZIlL5wwYPoJey2KJutX5hNfnWAIgNx6jLzhvQV05ddUfJkCwtRQ DCM2qPVy1M1MaE65m8ElxxOGmK1/vgxjLxw7KbOOz89pz6ihEOfvVusjZwMT43jwws9E OH61/kusRXR5oPshOA8cbpiH6rqDaVvGIb5vQHqDCpsdABqhXkWX64aRpAuuJr0MHlur eIAA== X-Gm-Message-State: ALoCoQmALQonLCKOALjitd4/aRxq06znPa7I0FLKDVbBNBciF90fFkPepYjMgeR+XgxAa3OBSdOg X-Received: by 10.224.72.72 with SMTP id l8mr6782542qaj.51.1390432596915; Wed, 22 Jan 2014 15:16:36 -0800 (PST) Received: from barcelona.nyc.corp.google.com (barcelona.nyc.corp.google.com [172.26.105.74]) by mx.google.com with ESMTPSA id v92sm5901066qge.6.2014.01.22.15.16.33 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 Jan 2014 15:16:33 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: WITH_TESTS=1 make installworld fails to install ATF C tests From: Julio Merino In-Reply-To: Date: Wed, 22 Jan 2014 18:16:32 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <35ADDB01-8EF4-4944-8A82-04805E8701EA@meroh.net> References: To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 23:16:38 -0000 On Jan 20, 2014, at 11:28, Alan Somers wrote: > I've just reproduced this using a freshly installed image from the > latest CURRENT snapshot > = (ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/11.0-CURRENT/amd64/= 20140106/FreeBSD-11.0-CURRENT-amd64-20140106-r260369.vmdk.xz) > and a freshly checked out source tree at /usr/src revision 260833. > WITH_TESTS=3D1 is the only line in my /etc/src.conf. This is = definitely > a bug. >=20 > So I'll rephrase the question. Has anybody successfully done > "WITH_TESTS=3D1 make installworld" since late November? If so, what > revision did you use? Apologies for the delay in replying to this and other emails. Just came = back from vacations :-) I'm not sure I have seen this specific issue, but I know that the build = is broken with the default of WITH_LIB32 and haven't had a chance to fix = that yet. Can you try setting WITHOUT_LIB32 in src.conf and seeing if = that helps?= From owner-freebsd-testing@FreeBSD.ORG Wed Jan 22 23:36:53 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5C0E33FD for ; Wed, 22 Jan 2014 23:36:53 +0000 (UTC) Received: from mail-qc0-f177.google.com (mail-qc0-f177.google.com [209.85.216.177]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 12BB81034 for ; Wed, 22 Jan 2014 23:36:52 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id i8so1511582qcq.8 for ; Wed, 22 Jan 2014 15:36:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=dCmsMr4Yb+QSVJYQM0mDk3TvvUR78Yp+V4zaD77D8Xo=; b=b5ayY5zYAHpXOEwKRauWuQpkURgYwRXsTxLUjwZ2LKPYrSjhA7Zn0XrEoU1q/vUD4N Uoa+bbGgauQAFxOuFrTpE2MOfue7UpH/WxwDWtKeSp6AoraJ87YIH3BJLzKfAlscEXBa Ui66dnhgSFuU8rai1xdPH204jHXPKYP/gQ4wx8BM5lUPLCT3u34eDxi0Lg1LOO005bSs 6LilmW6sH8DajgE+tsiU2RFCh21SEVOe4u4j1iLdIAJ9jQVG8fdJv3CDAD/4OYhwtuW+ cD1Yv2LN36jlkDFoP5dWuMa+qzBKTwtej8o9F4I0BfHjLO1Xz6Kjh0zzFdazxjkCra0U bywg== X-Gm-Message-State: ALoCoQlQi8WWH4yuWzA+QhAcLE5uhr8JZt8+vUXOi/KrFVagLgqMmnxiEaa7jE2IUJxkpDIo++4P X-Received: by 10.224.147.75 with SMTP id k11mr6946320qav.22.1390433811762; Wed, 22 Jan 2014 15:36:51 -0800 (PST) Received: from barcelona.nyc.corp.google.com (barcelona.nyc.corp.google.com [172.26.105.74]) by mx.google.com with ESMTPSA id j65sm5965467qgj.18.2014.01.22.15.36.50 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 Jan 2014 15:36:50 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: [PATCH] convert /bin/sh tests over to ATF From: Julio Merino In-Reply-To: Date: Wed, 22 Jan 2014 18:36:48 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <2222060C-CA92-41E5-9B10-9EA075A2B96B@meroh.net> References: To: Garrett Cooper X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@FreeBSD.org, Jilles Tjoelker X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 23:36:53 -0000 On Jan 20, 2014, at 11:44, Garrett Cooper wrote: > The attached patch converts the /bin/sh tests using the existing = framework bin/sh/tests. > All of the tests pass except locale.0, and that=92s due to some = [currently] unknown issue with my environment =97 see misc/181151). I = could make some of the entries more permanent, but I was trying to make = the tests easier to backport by using shell globs. > Also, I added some functionality so the tests expect failures on = certain versions of FreeBSD due to missing functionality or conformance = modifications (see the .valid_osreldate files). > Thoughts? Comments? > Thanks! > -Garrett Pasted your diff inline for easier commenting but trimmed stuff. > diff --git a/bin/sh/tests/Makefile b/bin/sh/tests/Makefile > index f6ddb8a..e1424f5 100644 > --- a/bin/sh/tests/Makefile > +++ b/bin/sh/tests/Makefile > @@ -4,15 +4,8 @@ > =20 > TESTSDIR=3D ${TESTSBASE}/bin/sh > =20 > -TAP_TESTS_SH=3D legacy_test > -TAP_TESTS_SH_SED_legacy_test=3D -e 's,__SH__,/bin/sh,g' > -# Some tests in here are silently not run when the tests are executed = as > -# root. Explicitly tell Kyua to drop privileges. > -# > -# TODO(jmmv): Kyua needs to do this by default, not only when = explicitly > -# requested. See https://code.google.com/p/kyua/issues/detail?id=3D6 > -TEST_METADATA.legacy_test+=3D required_user=3D"unprivileged" > +KYUAFILE=3D auto KYUAFILE=3Dauto is the default; remove here and anywhere else. =20 > -SUBDIR+=3D builtins errors execution expansion parameters parser = set-e > +ATF_TESTS_SUBDIRS+=3D builtins errors execution expansion = parameters parser set-e > =20 > -.include > +.include > diff --git a/bin/sh/tests/Makefile.inc b/bin/sh/tests/Makefile.inc > new file mode 100644 > index 0000000..0a79fd8 > --- /dev/null > +++ b/bin/sh/tests/Makefile.inc > @@ -0,0 +1,51 @@ > +# $FreeBSD$ > + > +CLEANFILES=3D regress.sh > + > +KYUAFILE=3D auto > + > +SH_TESTS!=3D cd ${.CURDIR} && find -Es . -regex '.*\.[0-9]+' | sed -e = 's,^\./,,g' > + > +SH_TESTS_FILES!=3D cd ${.CURDIR} && find . \( -type f -and \! -name = Makefile\* \) | \ > + sed -e 's,^\./,,g' This won't work well if you don't use an OBJDIR, will it? > + > +TESTSDIR=3D ${TESTSBASE}/bin/sh/${.CURDIR:T} > + > +FILES+=3D ${SH_TESTS_FILES} > +FILESDIR=3D ${TESTSDIR} > + > +regress.sh: ../Makefile.inc ${SH_TESTS_FILES} > + @echo '' > ${.TARGET} Don't write directly to ${.TARGET} if you have more than one command = doing so. Otherwise, if the target fails half-way through (or is = interrupted), a second invocation of make won't properly regenerate the = file. You can either do: @{ \ echo ... \ echo ... \ ... \ } >${.TARGET} or create a .tmp file and move "atomically" to the target at the end. > + @echo 'OSRELDATE=3D$$(sysctl -n kern.osreldate)' >> ${.TARGET} > + @echo ': $${SH=3D/bin/sh}' >> ${.TARGET} > + @echo 'export SH' >> ${.TARGET} > + @echo '' >> ${.TARGET} > +.for test in ${SH_TESTS} > + @echo 'atf_test_case ${test:S/\//_/g:S/./__/g:S/-/____/g}' >> = ${.TARGET} > + @echo '${test:S/\//_/g:S/./__/g:S/-/____/g}_body() {' >> = ${.TARGET} > + @echo ' cd $$(atf_get_srcdir)' >> ${.TARGET} > +.if exists(${test}.valid_osreldate) > + @echo -n ' [ '`cat ${.CURDIR}/${test}.valid_osreldate`' ] = && ' \ > + >> ${.TARGET} > + @echo 'atf_expect_fail "expecting failure based on OS version"' = \ > + >> ${.TARGET} > +.endif > + @echo -n ' atf_check -s exit:${test:E} ' >> ${.TARGET} > +.if exists(${test}.stderr) > + @echo -n '-e "file:$$(atf_get_srcdir)/${test}.stderr" ' >> = ${.TARGET} > +.endif > +.if exists(${test}.stdout) > + @echo -n '-o "file:$$(atf_get_srcdir)/${test}.stdout" ' >> = ${.TARGET} > +.endif > + @echo '$${SH} "./${test}"' >> ${.TARGET} > + @echo '}' >> ${.TARGET} > + @echo '' >> ${.TARGET} > +.endfor > + @echo '' >> ${.TARGET} > + @echo 'atf_init_test_cases() {' >> ${.TARGET} > +.for test in ${SH_TESTS} > + @echo ' atf_add_test_case ${test:S/\//_/g:S/./__/g:S/-/____/g}' = >> ${.TARGET} > +.endfor > + @echo '}' >> ${.TARGET} All this logic in a Makefile is too complex and should really not be = there. Just create a simple shell script that defines test cases dynamically = based on the available files and install that instead. It's shell after = all, so you can define functions at run time with ease! The results = will be much easier to digest. ---- But now, on a more abstract level... This approach does not fit the "model" of atf-sh -- nor the model of = most unit testing libraries for that matter. An individual test program = for every test case is overkill and keeping all these tiny files around = is suboptimal. The following page has more details on this and the = corresponding rationales: https://wiki.freebsd.org/TestSuite/Structure I recommend to not do this in this manner. Instead, what I'd do is the = following: First, define one test program for every directory that currently = exists. This would leave you with builtins_test errors_test = execution_test expansion_test parameters_test parser_test set-e_test, = all in /usr/tests/bin/sh. There is no reason to put them in separate = subdirectories. Second, make each of these top-level test programs "auto-discover" the = various data files in their corresponding subdirectories, define an = individual test case for each, and run them "as is". For example, = builtins_test would inspect srcdir/builtins/*.0, dynamically define an = atf_test_case, and run the test code from within it. These two steps change the "glue code" to be atf and expose the = individual test cases to kyua with minimal changes to the existing code. And third, separately once the above works, consider merging the = contents of the individual files into the main test program. I'm not = sure keeping tons of small files is worth it, but that's a separate = discussion once you have got the basic atf structure in place. This is all very similar to what was done with the ipf tests in NetBSD. = You can take a look at them here: = http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/ipf/?only_with_tag=3DMAIN (Before this was done, something similar to what you did above was = attempted by using awk to generate test programs... it was a nightmare = to maintain.) Hope this helps and thanks for looking into this!= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 00:01:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 380A4D68 for ; Thu, 23 Jan 2014 00:01:09 +0000 (UTC) Received: from mail-qc0-f179.google.com (mail-qc0-f179.google.com [209.85.216.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E60FE1296 for ; Thu, 23 Jan 2014 00:01:08 +0000 (UTC) Received: by mail-qc0-f179.google.com with SMTP id e16so1545028qcx.24 for ; Wed, 22 Jan 2014 16:01:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=8UUwD6jLxBssxbN9kOJlOZ8OCZ5lEsh8dLxJYtHQoq8=; b=BO2WEqtZsNVHRdQiC+e1AMdeAmH2xch625wSrSI+2Ozw+YylylJWvZX5ePN96HZy5l zqCk00Vg0vNQrFHVN87gEX23hm/DNUL3zeHjRDffp2u/BgFMr/GTKjZWhwDDCE+Y8S/J KVaxWdAOGhwKPEWMP4125E4DQl+rdKFfv5qEsyMMicWOSwiuecpKyByIi+BdweL6zqvb /Vy8Z9qzSc4Gc+08SoBaTbNQrtOHOX3x/q2MqhcQItqdY9fqOZjNlvUH+uAc6qPMlbn5 buWX80/IwypIoTxeWcS1wgcMbOvObeae6gj4jWX2kpZrrc/IfYe+9UA48GHIS7m/ZcnT rOIQ== X-Gm-Message-State: ALoCoQm9LE2v8uHKPrUfyLAm/A11FxmeWbrAmfx02H+xJTBFoDLVw4aMfGqeHm/m6uxu9JzvD5HM X-Received: by 10.224.137.129 with SMTP id w1mr6957526qat.86.1390435262109; Wed, 22 Jan 2014 16:01:02 -0800 (PST) Received: from barcelona.nyc.corp.google.com (barcelona.nyc.corp.google.com [172.26.105.74]) by mx.google.com with ESMTPSA id nk1sm10588689qeb.8.2014.01.22.16.00.59 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 Jan 2014 16:01:00 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: PATCH: fix "WITH_TESTS=yes make installworld" on amd64 From: Julio Merino In-Reply-To: Date: Wed, 22 Jan 2014 19:00:58 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 00:01:09 -0000 On Jan 21, 2014, at 13:00, Alan Somers wrote: > I finally found the problem. Make was trying to install tests during > the install32 phase. During this phase, it adds "lib32" to > MAKEOBJDIRPREFIX, so it can't find the tests, which were never built > with that prefix. The command worked fine when I ran it under "make > buildenv", because that uses the normal MAKEOBJDIRPREFIX. >=20 > I'm not sure what prevented the tests from being built during the > lib32 build phase, but this patch will disable tests during both > building and installing 32 bit libraries. >=20 > At work, we never noticed this problem in our SpectraBSD branch > because we set WITHOUT_LIB32=3D1 in /etc/src.conf. >=20 >=20 > Does this patch look ok? Yes it does. Can you commit and merge into stable/10 please? (This is the issue I mentioned in your other email a little bit earlier, = before seeing this other email. Glad to see we reached the same = conclusion :-) You'll notice that the testing cluster was cheating on this area if you = look at line 17 below :-P = http://svnweb.freebsd.org/base/user/jmmv/autotest/node/configs/kyua1/head.= conf?revision=3D260453&view=3Dmarkup= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 00:01:46 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 263F3D8F; Thu, 23 Jan 2014 00:01:46 +0000 (UTC) Received: from mail-pd0-x236.google.com (mail-pd0-x236.google.com [IPv6:2607:f8b0:400e:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E7DEB129C; Thu, 23 Jan 2014 00:01:45 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id v10so1033529pde.41 for ; Wed, 22 Jan 2014 16:01:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=zW6CP3SmoP1L8SH+pE2Zr9uAzqUSNNNTiQ5wcC6bFSI=; b=CdTItgeJ2uZqJngF+4OejQpBNh5jL78PnWjNaBXm+pMJyHJ4SMO//oCSbajCjrG50T UfinaKDU333n6o6vCyLK5o2NQPm3GAuoAKeF76Q2IhfyZIUlKEw5xQi35NHvXiPwk7R2 5ezffuvrRQ/NGoHMRlhyKC9Lkkl097lWmOytzpujdVMFMpXp4Wbmsyq6KY2/L/SEi5AH iG2ep/zWMwS8fSg0mN3LtY3Y9Pc95AzKWE4DrqrI/NOaQaTBWOH3ktxqwmNec/z8PXN9 yRBdeBZYXxVUFLA8lmWOiWmzW2EPM8QvEBpRMrSxZApPmNJP5bkmu894hzXSYy8S1EyU hnjg== X-Received: by 10.66.43.135 with SMTP id w7mr4571977pal.97.1390435305505; Wed, 22 Jan 2014 16:01:45 -0800 (PST) Received: from [10.0.2.128] ([64.14.143.130]) by mx.google.com with ESMTPSA id dr1sm28438322pbc.18.2014.01.22.16.01.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 Jan 2014 16:01:44 -0800 (PST) References: <35ADDB01-8EF4-4944-8A82-04805E8701EA@meroh.net> Mime-Version: 1.0 (1.0) In-Reply-To: <35ADDB01-8EF4-4944-8A82-04805E8701EA@meroh.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (11B554a) From: Garrett Cooper Subject: Re: WITH_TESTS=1 make installworld fails to install ATF C tests Date: Wed, 22 Jan 2014 16:01:44 -0800 To: Julio Merino Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 00:01:46 -0000 > On Jan 22, 2014, at 15:16, Julio Merino wrote: >=20 >> On Jan 20, 2014, at 11:28, Alan Somers wrote: >>=20 >> I've just reproduced this using a freshly installed image from the >> latest CURRENT snapshot >> (ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/11.0-CURRENT/amd64= /20140106/FreeBSD-11.0-CURRENT-amd64-20140106-r260369.vmdk.xz) >> and a freshly checked out source tree at /usr/src revision 260833. >> WITH_TESTS=3D1 is the only line in my /etc/src.conf. This is definitely >> a bug. >>=20 >> So I'll rephrase the question. Has anybody successfully done >> "WITH_TESTS=3D1 make installworld" since late November? If so, what >> revision did you use? >=20 > Apologies for the delay in replying to this and other emails. Just came ba= ck from vacations :-) >=20 > I'm not sure I have seen this specific issue, but I know that the build is= broken with the default of WITH_LIB32 and haven't had a chance to fix that y= et. Can you try setting WITHOUT_LIB32 in src.conf and seeing if that helps? I'll update my patch and send it out sometime tonight. It will fix this issu= e (I forgot the nuance of a fix I made to Makefile.inc1). Unfortunately I done have access to my gmail account from !iphone at work. Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 00:06:56 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 757ACFA4 for ; Thu, 23 Jan 2014 00:06:56 +0000 (UTC) Received: from mail-qc0-f170.google.com (mail-qc0-f170.google.com [209.85.216.170]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2C84712DD for ; Thu, 23 Jan 2014 00:06:56 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id e9so1603419qcy.1 for ; Wed, 22 Jan 2014 16:06:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=WI+RPJIGYVX/cG0jKkxJPsakucvRbFa37K8Je0zR/Rw=; b=GwxX5b/XxZEipOoK6o41kwvrmcfx7gDoLbrxyJwYLiB/apmB2LUnSSrK36WGtgYIED cEncNlT94uufeeH8r6GAjeahZe5b51gLSUgng/fLuBTGyARVv8eZVm90ffXs00zWk+LS 8alS2WinPCNwEd0UCkKK7e9ztrXI455WwlKJ62RtHlS0VHWsvCbQQP3dGlehYJSXUI47 srJrRtVIFFC6PbMHn2F1Nda0+G9WJflADBoyg671T8oYOUeBd4kiQhzRVha3nlBixuEI DSXHV68tf3cKx42iCjfd2f2HcWCQ6dUrYy0GuPF891N0SqGI1PQcSYoq/IIRePl1m25C yuGA== X-Gm-Message-State: ALoCoQkPgMk0DJCYH8AlVA0K4eY5bKbduv3vCWHhqOS+VYg/bkil2FTbq4GttiS06EFP+ATwAcjM X-Received: by 10.229.71.69 with SMTP id g5mr7182291qcj.6.1390435615073; Wed, 22 Jan 2014 16:06:55 -0800 (PST) Received: from barcelona.nyc.corp.google.com (barcelona.nyc.corp.google.com [172.26.105.74]) by mx.google.com with ESMTPSA id b18sm16780686qaw.0.2014.01.22.16.06.51 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 Jan 2014 16:06:52 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: PATCH: add ATF tests in sys From: Julio Merino In-Reply-To: <7051E061-C054-47BF-A5B2-747B9D3E85A1@gmail.com> Date: Wed, 22 Jan 2014 19:06:49 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <7A53F7F4-349F-480D-802E-F61C9308D009@meroh.net> References: <7051E061-C054-47BF-A5B2-747B9D3E85A1@gmail.com> To: Garrett Cooper X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 00:06:56 -0000 On Jan 21, 2014, at 19:23, Garrett Cooper wrote: >>=20 >> On Jan 21, 2014, at 16:02, Alan Somers wrote: >>=20 >>> On Tue, Jan 21, 2014 at 4:56 PM, Garrett Cooper = wrote: >>>=20 >>>> On Jan 21, 2014, at 14:45, Alan Somers wrote: >>>>=20 >>>> I rewrote the unix seqpacket tests in ATF. The hard part is adding >>>> them to the build. The linked patch adds them in the directory >>>> sys/kern/tests and builds them as part of buildworld, not = buildkernel. >>>> They get installed to /usr/tests/sys/kern, but the intermediate >>>> objects are stored in /usr/obj/sys/kern/tests. That means that you >>>> can't have different tests associated with different kernel = configs. >>>> I think that this is desirable, because you wouldn't be able to >>>> install tests for different kernel configs anyway, given our chosen >>>> layout of /usr/tests. >>>>=20 >>>> Please comment on the parts of this patch that deal with Makefiles. >>>> Is this the appropriate way to add sys tests to the build? = Shouldn't >>>> I be building them in buildkernel instead of buildworld? I = couldn't >>>> find a good way to do that. >>>=20 >>> I bypassed that for simplicity and placed the tests in = tests/sys/kern/... etc. Shoehorning things into our overly complex = kernel build system just seems like a really bad idea... Just make it = load as a driver, loadable, and dependent on kern.features to run..? >>> Cheers! >>> -Garrett >>=20 >> I tried to place the tests as closely as possible to the source code >> that they're testing, according to >> = https://wiki.freebsd.org/TestSuite/Structure#Makefiles_for_test_programs >> . I don't feel strongly about it either way, but I think Simon and >> jmmv do. Yes in general, but... see below. >>=20 >> FWIW, the tests run as a userland program, not in kernel mode. >=20 > Fwiw I spent months mulling over various details (this was just one = point) with gnn, marcel, and mfleming. The consensus was that that was a = really bad idea because it wasn't terribly clear that tests covered a = particular area, and some sys/ tests could cover multiple areas = (pjdfstest for instance). >=20 > I mirrored the exact structure in the source tree, but avoided = tainting things in sys/... Because integrating sys/ into buildworld and = starting to mix kernel and world build logic together will irritate a = large number of kernel devs and userland devs. This was the path of = least resistance... I think Garrett has a good point here and, if the build within src/sys/ = is significantly different from the build of userspace tools and has = different restrictions, it is probably a good idea to treat the kernel = tests as "special". After all, these tests are userland programs as you well said and they = are a kind of "integration" tests because there is no way currently to = unit-test kernel code. Such programs will just do "stuff" hoping to = validate kernel code, but there are so many levels of indirection that = they cannot be though of as testing things in very fine-grain detail. (If we had rump-like features as NetBSD, things may be a bit different = though because in that case you really are writing unit tests for very = specific kernel code.) Using src/tests/sys/ and replicating the layout of src/sys/ as much as = possible within it may be a reasonable and easy compromise for now.= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 00:21:13 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CABA3748 for ; Thu, 23 Jan 2014 00:21:13 +0000 (UTC) Received: from mail-yh0-f54.google.com (mail-yh0-f54.google.com [209.85.213.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 87E591483 for ; Thu, 23 Jan 2014 00:21:13 +0000 (UTC) Received: by mail-yh0-f54.google.com with SMTP id z6so471327yhz.27 for ; Wed, 22 Jan 2014 16:21:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=2TldFqyv8KK9uLPR4DyEX/KXpF9S7eroD9CrxTZmucE=; b=HEHFq4mWrJsG11U/Q04vshXVFtEppNExc8t8TK0G2S3eKaBvHr2HNUUyJO6Smo2Ehx u6f3a/aGGk3EN0eNSq/+m8RXKGmP71k4TXILqhh4NN/iP4CMwFSSxi0C4+N9vydadsJ1 /O/JnS7zMLNMvqWCgqChisdfFa/Z1znmNRtBg7XDdiucBlcZNY6K7YyUGIWIJJYbQH9/ bzLDIp8esWq7cpHyKw26DSoI7/VyoY8aY6/nXYWhJId0257Am1GX/jlXJeosEvGAvBPn pcT6kFv3MQba+S9r76loCJHd7dhY3aUflrmZn1v4e+BaDEY3OZrw+xp8ZQbrs4A+FZ1L r6WQ== X-Gm-Message-State: ALoCoQlTCvDSXNdZWzOHYzI1l1C/wf2D5/2smYxefjlentHB8qusBPz+nsiy31VRJb6QGcRFr3ec X-Received: by 10.236.171.195 with SMTP id r43mr4727747yhl.56.1390435016100; Wed, 22 Jan 2014 15:56:56 -0800 (PST) Received: from ?IPv6:2620::1003:1007:6102:b049:b28e:1b56? ([2620:0:1003:1007:6102:b049:b28e:1b56]) by mx.google.com with ESMTPSA id e50sm29694671yhd.26.2014.01.22.15.56.51 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 Jan 2014 15:56:52 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: [PATCH] convert bin/date over to ATF From: Julio Merino In-Reply-To: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> Date: Wed, 22 Jan 2014 18:56:50 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> To: Garrett Cooper X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@FreeBSD.org, Giorgos Keramidas X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 00:21:13 -0000 On Jan 20, 2014, at 13:40, Garrett Cooper wrote: > This is based on work done by Giorgos a couple years ago. > Thanks! > -Garrett Pasting patch contents and commenting inline: > diff --git a/bin/date/tests/Makefile b/bin/date/tests/Makefile > index 540008b..459d019 100644 > --- a/bin/date/tests/Makefile > +++ b/bin/date/tests/Makefile > @@ -4,6 +4,6 @@ > =20 > TESTSDIR=3D ${TESTSBASE}/bin/date > =20 > -TAP_TESTS_SH=3D legacy_test > +ATF_TESTS_SH=3D regress Tests ought to end with _test per the description in = https://wiki.freebsd.org/TestSuite/Structure Also, "regress_test" is not a very indicative name. Will this only = contain test cases for bugs to prevent regressions? date_test or integration_test (due to the lack of unit tests for the = code) may be better choices. =20 > --- /dev/null > +++ b/bin/date/tests/regress.sh > @@ -0,0 +1,557 @@ > +#!/bin/sh Remove. This comes from atf.test.mk automatically. > + > +# > +# Regression tests for date(1) > +# > +# Submitted by Edwin Groothuis > +# > +# $FreeBSD: src/tools/regression/bin/date/regress.sh,v 1.4 2011/01/09 = 22:05:09 keramida Exp $ > +# > + > +# > +# These two date/times have been chosen carefully, they > +# create both the single digit and double/multidigit version of > +# the values. > +# > +# To create a new one, make sure you are using the UTC timezone! > +# > + > +TEST1=3D3222243 # 1970-02-07 07:04:03 > +TEST2=3D1005600000 # 2001-11-12 21:11:12 > + > +export LC_ALL=3DC > +export TZ=3DUTC Kyua does this already as part of the contract between atf and the = runtime engine. You should not be resetting these. > + > +check() > +{ > + S=3D$1 > + A1=3D$2 > + A2=3D$3 S, A1, A2? What do these mean? Also, make local. > + > + # If the second sample text for formatted output has not been > + # passed, assume it should match exactly the first one. > + if [ -z "${A2}" ]; then > + A2=3D${A1} > + fi > + > + R=3D`date -r ${TEST1} +%${S}` Prefer $() over ``. > + atf_check test "${R}" =3D "${A1}" > + > + R=3D`date -r ${TEST2} +%${S}` > + atf_check test "${R}" =3D "${A2}" > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case A > +A_head() > +{ > + atf_set "descr" "Verifies that 'A' formatting spec works" These test case names are truly non-descriptive. I'd recommend renaming = them to something like formatting_spec__A, etc. and omitting the = definition of head() altogether. Will result on a much shorter test = program, and being concise here for readability matters a lot. ... and then you can just create a "macro" to define test cases. Like = this, but untested: formatting_spec_test_case() { local subname=3D"${1}"; shift local name=3D"formatting_spec__${subname}" atf_test_case "${name}" eval "${name}_body() { check ${*} }" } formatting_spec_test_case a a Sat Mon formatting_spec_test_case B B February November ... formatting_spec_test_case plus + "Sat Feb 7 07:04:03 UTC 1970" "Mon Nov = 12 21:20:00 UTC 2001" > +} > +A_body() > +{ > + check A Saturday Monday > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case a > +a_head() > +{ > + atf_set "descr" "Verifies that 'a' formatting spec works" > +} > +a_body() > +{ > + check a Sat Mon > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case B > +B_head() > +{ > + atf_set "descr" "Verifies that 'B' formatting spec works" > +} > +B_body() > +{ > + check B February November > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case b > +b_head() > +{ > + atf_set "descr" "Verifies that 'b' formatting spec works" > +} > +b_body() > +{ > + check b Feb Nov > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case C > +C_head() > +{ > + atf_set "descr" "Verifies that 'C' formatting spec works" > +} > +C_body() > +{ > + check C 19 20 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case c > +c_head() > +{ > + atf_set "descr" "Verifies that 'c' formatting spec works" > +} > +c_body() > +{ > + check c "Sat Feb 7 07:04:03 1970" "Mon Nov 12 21:20:00 2001" > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case D > +D_head() > +{ > + atf_set "descr" "Verifies that 'D' formatting spec works" > +} > +D_body() > +{ > + check D 02/07/70 11/12/01 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case d > +d_head() > +{ > + atf_set "descr" "Verifies that 'd' formatting spec works" > +} > +d_body() > +{ > + check d 07 12 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case e > +e_head() > +{ > + atf_set "descr" "Verifies that 'e' formatting spec works" > +} > +e_body() > +{ > + check e " 7" 12 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case F > +F_head() > +{ > + atf_set "descr" "Verifies that 'F' formatting spec works" > +} > +F_body() > +{ > + check F "1970-02-07" "2001-11-12" > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case G > +G_head() > +{ > + atf_set "descr" "Verifies that 'G' formatting spec works" > +} > +G_body() > +{ > + check G 1970 2001 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case g > +g_head() > +{ > + atf_set "descr" "Verifies that 'g' formatting spec works" > +} > +g_body() > +{ > + check g 70 01 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case H > +H_head() > +{ > + atf_set "descr" "Verifies that 'H' formatting spec works" > +} > +H_body() > +{ > + check H 07 21 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case h > +h_head() > +{ > + atf_set "descr" "Verifies that 'h' formatting spec works" > +} > +h_body() > +{ > + check h Feb Nov > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case I > +I_head() > +{ > + atf_set "descr" "Verifies that 'I' formatting spec works" > +} > +I_body() > +{ > + check I 07 09 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case j > +j_head() > +{ > + atf_set "descr" "Verifies that 'j' formatting spec works" > +} > +j_body() > +{ > + check j 038 316 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case k > +k_head() > +{ > + atf_set "descr" "Verifies that 'k' formatting spec works" > +} > +k_body() > +{ > + check k " 7" 21 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case l > +l_head() > +{ > + atf_set "descr" "Verifies that 'l' formatting spec works" > +} > +l_body() > +{ > + check l " 7" " 9" > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case M > +M_head() > +{ > + atf_set "descr" "Verifies that 'M' formatting spec works" > +} > +M_body() > +{ > + check M 04 20 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case m > +m_head() > +{ > + atf_set "descr" "Verifies that 'm' formatting spec works" > +} > +m_body() > +{ > + check m 02 11 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case p > +p_head() > +{ > + atf_set "descr" "Verifies that 'p' formatting spec works" > +} > +p_body() > +{ > + check p AM PM > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case R > +R_head() > +{ > + atf_set "descr" "Verifies that 'R' formatting spec works" > +} > +R_body() > +{ > + check R 07:04 21:20 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case r > +r_head() > +{ > + atf_set "descr" "Verifies that 'r' formatting spec works" > +} > +r_body() > +{ > + check r "07:04:03 AM" "09:20:00 PM" > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case S > +S_head() > +{ > + atf_set "descr" "Verifies that 'S' formatting spec works" > +} > +S_body() > +{ > + check S 03 00 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case s > +s_head() > +{ > + atf_set "descr" "Verifies that 's' formatting spec works" > +} > +s_body() > +{ > + check s ${TEST1} ${TEST2} > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case U > +U_head() > +{ > + atf_set "descr" "Verifies that 'U' formatting spec works" > +} > +U_body() > +{ > + check U 05 45 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case u > +u_head() > +{ > + atf_set "descr" "Verifies that 'u' formatting spec works" > +} > +u_body() > +{ > + check u 6 1 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case V > +V_head() > +{ > + atf_set "descr" "Verifies that 'V' formatting spec works" > +} > +V_body() > +{ > + check V 06 46 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case v > +v_head() > +{ > + atf_set "descr" "Verifies that 'v' formatting spec works" > +} > +v_body() > +{ > + check v " 7-Feb-1970" "12-Nov-2001" > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case W > +W_head() > +{ > + atf_set "descr" "Verifies that 'W' formatting spec works" > +} > +W_body() > +{ > + check W 05 46 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case w > +w_head() > +{ > + atf_set "descr" "Verifies that 'w' formatting spec works" > +} > +w_body() > +{ > + check w 6 1 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case X > +X_head() > +{ > + atf_set "descr" "Verifies that 'X' formatting spec works" > +} > +X_body() > +{ > + check X "07:04:03" "21:20:00" > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case x > +x_head() > +{ > + atf_set "descr" "Verifies that 'x' formatting spec works" > +} > +x_body() > +{ > + check x "02/07/70" "11/12/01" > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case Y > +Y_head() > +{ > + atf_set "descr" "Verifies that 'Y' formatting spec works" > +} > +Y_body() > +{ > + check Y 1970 2001 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case y > +y_head() > +{ > + atf_set "descr" "Verifies that 'y' formatting spec works" > +} > +y_body() > +{ > + check y 70 01 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case Z > +Z_head() > +{ > + atf_set "descr" "Verifies that 'Z' formatting spec works" > +} > +Z_body() > +{ > + check Z UTC UTC > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case z > +z_head() > +{ > + atf_set "descr" "Verifies that 'z' formatting spec works" > +} > +z_body() > +{ > + check z +0000 +0000 > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case pct > +pct_head() > +{ > + atf_set "descr" "Verifies that '%' formatting spec works" > +} > +pct_body() > +{ > + check % % % > +} > + > +# = ---------------------------------------------------------------------- > + > +atf_test_case plus > +plus_head() > +{ > + atf_set "descr" "Verifies that '+' formatting spec works" > +} > +plus_body() > +{ > + check + "Sat Feb 7 07:04:03 UTC 1970" "Mon Nov 12 21:20:00 UTC = 2001" > +} > + > +# = ---------------------------------------------------------------------- > +# Main test case driving script. > +# = ---------------------------------------------------------------------- > + > +atf_init_test_cases() > +{ > + atf_add_test_case A > + atf_add_test_case a > + atf_add_test_case B > + atf_add_test_case b > + atf_add_test_case C > + atf_add_test_case c > + atf_add_test_case D > + atf_add_test_case d > + atf_add_test_case e > + atf_add_test_case F > + atf_add_test_case G > + atf_add_test_case g > + atf_add_test_case H > + atf_add_test_case h > + atf_add_test_case I > + atf_add_test_case j > + atf_add_test_case k > + atf_add_test_case l > + atf_add_test_case M > + atf_add_test_case m > + atf_add_test_case p > + atf_add_test_case R > + atf_add_test_case r > + atf_add_test_case S > + atf_add_test_case s > + atf_add_test_case U > + atf_add_test_case u > + atf_add_test_case V > + atf_add_test_case v > + atf_add_test_case W > + atf_add_test_case w > + atf_add_test_case X > + atf_add_test_case x > + atf_add_test_case Y > + atf_add_test_case y > + atf_add_test_case Z > + atf_add_test_case z > + atf_add_test_case pct > + atf_add_test_case plus > +} >=20 >=20 From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 20:53:37 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8FA61183; Thu, 23 Jan 2014 20:53:37 +0000 (UTC) Received: from mail-pb0-x22e.google.com (mail-pb0-x22e.google.com [IPv6:2607:f8b0:400e:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 524C719C3; Thu, 23 Jan 2014 20:53:37 +0000 (UTC) Received: by mail-pb0-f46.google.com with SMTP id um1so2310518pbc.5 for ; Thu, 23 Jan 2014 12:53:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=fiGw63QLMskZaBCYlYF85s/3VkI9XqoDSbBDxYa6cXY=; b=Ij1Jh1638aIXlwVVS5HYcf56ONVnlDSEWoAtfxMEDMXjmGRyl6Kn/TJ2BApO2e+jM6 PtRMm3aOvNVRpwSGYrRl5oODtVf8cebWnCWj9ybKRBMKwyIHP7+CJTloGxNxaeUPL9Hb xupAKjndugbHbUfxd8+VN6PpVmshHwG0m/x4JDf6T5VNM7ucGhOH/+89WZjTVaJc00Bc tQuupIt3y13/rvU4CjMev9es/6rqyfi37IHdzQJdmR7pIis+JbKKA9BOuUJSYQ3G5Lj6 abB1pU8EFkERL0ohGCjYM8At6pj9D2Qq5K1OIetrAhAH+fUNHf1lBsnG/70fqxbiy8fv NRJw== X-Received: by 10.68.134.98 with SMTP id pj2mr10273497pbb.110.1390510416886; Thu, 23 Jan 2014 12:53:36 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id un5sm65763032pab.3.2014.01.23.12.53.32 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 12:53:35 -0800 (PST) Content-Type: multipart/mixed; boundary="Apple-Mail=_BD061265-5FF4-416C-A559-2A84005BC8B5" Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> Date: Thu, 23 Jan 2014 12:53:31 -0800 Message-Id: <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 20:53:37 -0000 --Apple-Mail=_BD061265-5FF4-416C-A559-2A84005BC8B5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jan 22, 2014, at 8:34 AM, Garrett Cooper = wrote: >=20 >> On Jan 22, 2014, at 7:42, Alan Somers wrote: >>=20 >>> On Tue, Jan 21, 2014 at 3:17 PM, Garrett Cooper = wrote: >>> Hi all, >>> The attached patch fixes/enhances the build so -DNO_TESTS is = passed properly through various sections of the build that don't require = tests in order to build/install. This should also fix Alan's recent = reported issue with -DWITH_TESTS failing on amd64. >>> Cheers! >>> -Garrett >>=20 >> The patch doesn't work for me. I get the following error during >> buildworld during stage 4.4. >=20 > ... >=20 > Oh yeah... It's that chicken and egg issue due to ATF now being = controlled by MK_TESTS=3D=3Dyes... There's more to this patch I need to = provide you in order for things to work... Here=92s the working patch. The difference between this one and = the prior version is that you have to explicitly override -DNO_TESTS = when building lib/atf* (Simon: do you have any comments?): -.if ${MK_TESTS} !=3D "no" +.if defined(WITH_ATF) || ${MK_TESTS} !=3D "no" +# Make sure WITH_ATF overrules -DNO_TESTS +.if !defined(WITH_ATF) +MAKE+=3D -DWITH_ATF +.endif I unrolled most of the local changes to Makefile.inc1 on my = github fork so it=92ll be easier to spot if you diff it against my tree. Thanks! -Garrett --Apple-Mail=_BD061265-5FF4-416C-A559-2A84005BC8B5 Content-Disposition: attachment; filename=Makefile.inc1.patch Content-Type: application/octet-stream; name="Makefile.inc1.patch" Content-Transfer-Encoding: 7bit Index: Makefile.inc1 =================================================================== --- Makefile.inc1 (revision 260925) +++ Makefile.inc1 (working copy) @@ -246,7 +246,7 @@ SSP_CFLAGS= \ -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT -DWITHOUT_MAN \ -DNO_PIC -DNO_PROFILE -DNO_SHARED \ - -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD + -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD -DNO_TESTS # build-tools stage TMAKE= MAKEOBJDIRPREFIX=${OBJTREE} \ @@ -256,12 +256,12 @@ BOOTSTRAPPING=${OSRELDATE} \ SSP_CFLAGS= \ -DNO_LINT \ - -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD + -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD -DNO_TESTS # cross-tools stage XMAKE= TOOLS_PREFIX=${WORLDTMP} ${BMAKE} \ TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} \ - -DWITHOUT_GDB + -DWITHOUT_GDB -DNO_TESTS # kernel-tools stage KTMAKEENV= INSTALL="sh ${.CURDIR}/tools/install.sh" \ @@ -414,7 +414,8 @@ -DLIBRARIES_ONLY \ -DNO_CPU_CFLAGS \ -DNO_CTF \ - -DNO_LINT + -DNO_LINT \ + -DNO_TESTS LIB32WMAKE= ${LIB32WMAKEENV} ${MAKE} ${LIB32WMAKEFLAGS} \ -DWITHOUT_MAN -DWITHOUT_INFO -DWITHOUT_HTML @@ -569,7 +570,7 @@ @echo "--------------------------------------------------------------" ${_+_}cd ${.CURDIR}; \ ${WMAKE} -DNO_FSCHG -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT \ - -DWITHOUT_MAN -DNO_PROFILE libraries + -DWITHOUT_MAN -DNO_PROFILE -DNO_TESTS libraries _depend: @echo @echo "--------------------------------------------------------------" @@ -1501,7 +1502,11 @@ ${_secure_lib_libcrypto} ${_lib_libldns} \ ${_secure_lib_libssh} ${_secure_lib_libssl} -.if ${MK_TESTS} != "no" +.if defined(WITH_ATF) || ${MK_TESTS} != "no" +# Make sure WITH_ATF overrules -DNO_TESTS +.if !defined(WITH_ATF) +MAKE+= -DWITH_ATF +.endif _lib_atf= lib/atf .endif @@ -1877,7 +1882,7 @@ NOFUN=-DNO_FSCHG -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT \ -DWITHOUT_MAN -DWITHOUT_NLS -DNO_PROFILE \ - -DWITHOUT_KERBEROS -DWITHOUT_RESCUE -DNO_WARNS \ + -DWITHOUT_KERBEROS -DWITHOUT_RESCUE -DNO_TESTS -DNO_WARNS \ TARGET=${XDEV} TARGET_ARCH=${XDEV_ARCH} \ CPUTYPE=${XDEV_CPUTYPE} --Apple-Mail=_BD061265-5FF4-416C-A559-2A84005BC8B5-- From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 20:56:08 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB7A6376; Thu, 23 Jan 2014 20:56:08 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3D6AD19F2; Thu, 23 Jan 2014 20:56:08 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id u57so1841930wes.0 for ; Thu, 23 Jan 2014 12:56:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=vg58nvju7ridrrb1GwxYnMdvrq0EqTqC9LoMX0LmIPg=; b=AQ27qDhOZe2DrA9k6Z6ByOcOQGy1F4IJ+Q/PvfXJuoipugUk/+izaoyMehNiLSuC2/ IGaKK/a/fY1Xg3a0PTXJ9QvFkQd242SSL78a3899+wGt/X4B0VpX860Eu1XbupcVnWsp HAzo4OyjZQuyCF5kCPIte4VyoNUk1ZrVfnHqrCrkXrMzy9AnTHx+L8CsuJ/X+W96WDah 90Bwxd85swsBf0cbYXtlYt5m4bURGle/Rp7l9qSYDCLkkTVQleiW9l3UkswjjgIHMV4m XVrC/sTWCnvNivKko537NkT11wU8b2VQilUG7UHX5DxJ4ItAsP1n8CLdCOgj/gzR5ISN SZUA== MIME-Version: 1.0 X-Received: by 10.194.240.41 with SMTP id vx9mr45927wjc.70.1390510566697; Thu, 23 Jan 2014 12:56:06 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Thu, 23 Jan 2014 12:56:06 -0800 (PST) In-Reply-To: <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> Date: Thu, 23 Jan 2014 13:56:06 -0700 X-Google-Sender-Auth: 2165WS5vQUHSRVMt9Sg1F8xwDzc Message-ID: Subject: Re: Makefile.inc1.patch From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 20:56:08 -0000 On Thu, Jan 23, 2014 at 1:53 PM, Garrett Cooper wro= te: > On Jan 22, 2014, at 8:34 AM, Garrett Cooper wrote= : > >> >>> On Jan 22, 2014, at 7:42, Alan Somers wrote: >>> >>>> On Tue, Jan 21, 2014 at 3:17 PM, Garrett Cooper wrote: >>>> Hi all, >>>> The attached patch fixes/enhances the build so -DNO_TESTS is passed = properly through various sections of the build that don't require tests in = order to build/install. This should also fix Alan's recent reported issue w= ith -DWITH_TESTS failing on amd64. >>>> Cheers! >>>> -Garrett >>> >>> The patch doesn't work for me. I get the following error during >>> buildworld during stage 4.4. >> >> ... >> >> Oh yeah... It's that chicken and egg issue due to ATF now being controll= ed by MK_TESTS=3D=3Dyes... There's more to this patch I need to provide you= in order for things to work... > > Here=92s the working patch. The difference between this one and t= he prior version is that you have to explicitly override -DNO_TESTS when bu= ilding lib/atf* (Simon: do you have any comments?): > > -.if ${MK_TESTS} !=3D "no" > +.if defined(WITH_ATF) || ${MK_TESTS} !=3D "no" > +# Make sure WITH_ATF overrules -DNO_TESTS > +.if !defined(WITH_ATF) > +MAKE+=3D -DWITH_ATF > +.endif > > I unrolled most of the local changes to Makefile.inc1 on my githu= b fork so it=92ll be easier to spot if you diff it against my tree. > Thanks! > -Garrett Shouldn't that be "WITH_TESTS" instead of "WITH_ATF"? Those options got combined awhile back. -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:00:43 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 811024DE; Thu, 23 Jan 2014 21:00:43 +0000 (UTC) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 466A51A36; Thu, 23 Jan 2014 21:00:43 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id hz1so2345683pad.36 for ; Thu, 23 Jan 2014 13:00:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=tZ18/nyziLRY2MULWEgL3QbzW/kuIuMMMiunlRFsqng=; b=UZGVwJT9zCnmD5AIzI2yl0fDAvC2GHV8giYBa1GYQG+FQ1Khn91S4KgokB2/Mj6doI ygjTcdFLn4DaDJ49PlnYnbRd38AAhsF35DUK2tJ6l56ZaLMFL8Vl+6YO8i3YTKExM5nm wowaC9DLTbUJTM6/zz6XY89M2XNqi4VZf3DBAPZvvPlEkeXWF64vEmIxqy9gCeURrU1r EqrweQCBgNuobOJpKESURRtG7fj4Vx855Ta1WULFZEYPNdhYSDQ+K3U1UOdbkpTVpGzt paYg1NbH4YJOUdagnvYmJ2rnHU+DyeW0kTVsAHDH5mrfxbw/pRR4aFBLwtjzdnOFRKX5 LjrA== X-Received: by 10.66.224.165 with SMTP id rd5mr10305462pac.119.1390510842706; Thu, 23 Jan 2014 13:00:42 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id pq1sm40975268pbc.8.2014.01.23.13.00.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 13:00:42 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: Date: Thu, 23 Jan 2014 13:00:40 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:00:43 -0000 On Jan 23, 2014, at 12:56 PM, Alan Somers wrote: > On Thu, Jan 23, 2014 at 1:53 PM, Garrett Cooper = wrote: >> On Jan 22, 2014, at 8:34 AM, Garrett Cooper = wrote: >>=20 >>>=20 >>>> On Jan 22, 2014, at 7:42, Alan Somers wrote: >>>>=20 >>>>> On Tue, Jan 21, 2014 at 3:17 PM, Garrett Cooper = wrote: >>>>> Hi all, >>>>> The attached patch fixes/enhances the build so -DNO_TESTS is = passed properly through various sections of the build that don't require = tests in order to build/install. This should also fix Alan's recent = reported issue with -DWITH_TESTS failing on amd64. >>>>> Cheers! >>>>> -Garrett >>>>=20 >>>> The patch doesn't work for me. I get the following error during >>>> buildworld during stage 4.4. >>>=20 >>> ... >>>=20 >>> Oh yeah... It's that chicken and egg issue due to ATF now being = controlled by MK_TESTS=3D=3Dyes... There's more to this patch I need to = provide you in order for things to work... >>=20 >> Here=92s the working patch. The difference between this one = and the prior version is that you have to explicitly override -DNO_TESTS = when building lib/atf* (Simon: do you have any comments?): >>=20 >> -.if ${MK_TESTS} !=3D "no" >> +.if defined(WITH_ATF) || ${MK_TESTS} !=3D "no" >> +# Make sure WITH_ATF overrules -DNO_TESTS >> +.if !defined(WITH_ATF) >> +MAKE+=3D -DWITH_ATF >> +.endif >>=20 >> I unrolled most of the local changes to Makefile.inc1 on my = github fork so it=92ll be easier to spot if you diff it against my tree. >> Thanks! >> -Garrett >=20 > Shouldn't that be "WITH_TESTS" instead of "WITH_ATF"? Those options > got combined awhile back. Unfortunately no. The chicken and the egg problem I ran into is = like so: 1. make libraries shouldn=92t build tests (in particular it will bomb if = it tries to build something that needs to link against libatf*.so). This = isn=92t a problem today, but will become a problem when lib/libc/tests = is pushed back to FreeBSD. 2. lib/atf needs to be installed sooner so atf-check, etc link properly. This wasn=92t a chicken and egg problem until the two knobs were = merged=85 Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:11:36 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA122895; Thu, 23 Jan 2014 21:11:35 +0000 (UTC) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com [IPv6:2a00:1450:400c:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4A0861C7A; Thu, 23 Jan 2014 21:11:35 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id y10so2163878wgg.20 for ; Thu, 23 Jan 2014 13:11:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=aTU/MFqrhWPT6m4dvRGL3K3xGyye1tv+y3mTSVdW5lM=; b=Ehvf5oCLmFkoaJLY+ZmJHvGjnRnksnTM86kjOU17wNBLeDEHwXnsGnZiG2EQlNJfv+ h8K99aufgP7Y+0Y+PTpe5VIsf9jjeTMOSbJG3JPvl52VUCsBB4TrD1gYKe35vD2/PbM/ bTB0L16XQYweV0nXYkXK58pC3BS8kWKWv2BLCaGEknY6xptjEND+YkY7lMgaaKn8rawQ bKXeR4sNL+r+kO53VcNPpW2nPffOcULDspK4v6PTNRpaF11exsSpJ7RKEEC+5cHsohkc NwOqbpDJpeWaEag7dwfbhFf3CpQ2wYfSfj1qdJ+e32c9QnqyIdvCMpDhLnxyWYAV9fK/ vvOg== MIME-Version: 1.0 X-Received: by 10.194.185.113 with SMTP id fb17mr8059686wjc.29.1390511493757; Thu, 23 Jan 2014 13:11:33 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Thu, 23 Jan 2014 13:11:33 -0800 (PST) In-Reply-To: References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> Date: Thu, 23 Jan 2014 14:11:33 -0700 X-Google-Sender-Auth: UPrc9n1p3L3QuZ-x1k1KsYXJZ4Q Message-ID: Subject: Re: Makefile.inc1.patch From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:11:36 -0000 On Thu, Jan 23, 2014 at 2:00 PM, Garrett Cooper wro= te: > On Jan 23, 2014, at 12:56 PM, Alan Somers wrote: > >> On Thu, Jan 23, 2014 at 1:53 PM, Garrett Cooper = wrote: >>> On Jan 22, 2014, at 8:34 AM, Garrett Cooper wro= te: >>> >>>> >>>>> On Jan 22, 2014, at 7:42, Alan Somers wrote: >>>>> >>>>>> On Tue, Jan 21, 2014 at 3:17 PM, Garrett Cooper wrote: >>>>>> Hi all, >>>>>> The attached patch fixes/enhances the build so -DNO_TESTS is passed= properly through various sections of the build that don't require tests in= order to build/install. This should also fix Alan's recent reported issue = with -DWITH_TESTS failing on amd64. >>>>>> Cheers! >>>>>> -Garrett >>>>> >>>>> The patch doesn't work for me. I get the following error during >>>>> buildworld during stage 4.4. >>>> >>>> ... >>>> >>>> Oh yeah... It's that chicken and egg issue due to ATF now being contro= lled by MK_TESTS=3D=3Dyes... There's more to this patch I need to provide y= ou in order for things to work... >>> >>> Here=92s the working patch. The difference between this one and = the prior version is that you have to explicitly override -DNO_TESTS when b= uilding lib/atf* (Simon: do you have any comments?): >>> >>> -.if ${MK_TESTS} !=3D "no" >>> +.if defined(WITH_ATF) || ${MK_TESTS} !=3D "no" >>> +# Make sure WITH_ATF overrules -DNO_TESTS >>> +.if !defined(WITH_ATF) >>> +MAKE+=3D -DWITH_ATF >>> +.endif >>> >>> I unrolled most of the local changes to Makefile.inc1 on my gith= ub fork so it=92ll be easier to spot if you diff it against my tree. >>> Thanks! >>> -Garrett >> >> Shouldn't that be "WITH_TESTS" instead of "WITH_ATF"? Those options >> got combined awhile back. > > Unfortunately no. The chicken and the egg problem I ran into is l= ike so: > 1. make libraries shouldn=92t build tests (in particular it will bomb if = it tries to build something that needs to link against libatf*.so). This is= n=92t a problem today, but will become a problem when lib/libc/tests is pus= hed back to FreeBSD. > 2. lib/atf needs to be installed sooner so atf-check, etc link properly. > This wasn=92t a chicken and egg problem until the two knobs were = merged=85 > Thanks! > -Garrett In that case, I'm missing something. I can't find any makefiles that reference MK_ATF or a related variable. What is the effect of setting WITH_ATF ? -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:24:00 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82843CB7; Thu, 23 Jan 2014 21:24:00 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3F9511D70; Thu, 23 Jan 2014 21:24:00 +0000 (UTC) Received: by mail-pd0-f170.google.com with SMTP id p10so2283260pdj.29 for ; Thu, 23 Jan 2014 13:23:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=GN4hfrDbAqy/yQI3FhpwVkP7aHkzJuS5Vhmu+hqlO8w=; b=m0SP7jtO7B+sT1kQH4FKLzDIG7I2wizH6txiivryW3Mi6mAclAJOD4mQupsfFM80jG 2cxieq0l7bLKfnaYZOJPoaZFSVIe1cX1J+MhSTtsq+8+QdoL8mfw3jHODxehtbFZ4HxD JQ7Fykf4K9pqE+ZpppRSvTVtIxqH/781TFxuM1SYTRgDgJttrsddqoQKdnK0C2DnrKtq WNCeZ23aWp7dKBFYeUMVbJqyFtFATJw/qkIKI9lE5ZLRsW0tGpAuBr2irO/Aj95aOctu M1MzTgGhqZBjJrqiniNRXoXDBos0PA08EK+CN42cfZs2wBwh3piRgxRNSpyXXD6UccF/ 3Rpg== X-Received: by 10.68.89.162 with SMTP id bp2mr10163314pbb.151.1390512239824; Thu, 23 Jan 2014 13:23:59 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id om6sm41247771pbc.43.2014.01.23.13.23.58 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 13:23:59 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: Date: Thu, 23 Jan 2014 13:23:57 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:24:00 -0000 On Jan 23, 2014, at 1:11 PM, Alan Somers wrote: > In that case, I'm missing something. I can't find any makefiles that > reference MK_ATF or a related variable. What is the effect of setting > WITH_ATF ? >=20 > -Alan NO_TESTS forces WITHOUT_TESTS to be set. So, if I set NO_TESTS = in the various build steps it will force ATF to not be built. For that = reason (and that reason alone) I reintroduced WITH_ATF just for = Makefile.inc1 (but you could replace it with something else like = WITH_ATF_LIBS, etc, if the naming is too confusing). Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:29:49 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBF1AE8D; Thu, 23 Jan 2014 21:29:49 +0000 (UTC) Received: from mail-we0-x22f.google.com (mail-we0-x22f.google.com [IPv6:2a00:1450:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 377C51DB2; Thu, 23 Jan 2014 21:29:49 +0000 (UTC) Received: by mail-we0-f175.google.com with SMTP id p61so1864888wes.34 for ; Thu, 23 Jan 2014 13:29:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=2AnP4g5ln+rOClkn6td59ynojsCo5KOyV+cviNNVDS4=; b=0TpkZvKo5xndGK3+HYAxuTF1gGe78snFo2Yl3wR4kYRIUyR4RlrWqUofOPkrwy0Qof BBxJ6H6oAq/Smr+qo/+5jpX/KC51o/MJ9yh2MLXZE8wqINnHyG+hGLhwuOxvhHTfs3jP 67P3zZ8RAHyikolrcw5rl2Ir+eEFcoZ0MPto0TQINC30fld7lBO43irbZGdzWxDdrvG/ OANL1ODhg5KX58wF3kU04u/HQBvCJN1DpE428eSsP1l4HVb/No7YyIUputuj8nrDlvQT f5rgB/3uJP0LZvsHiXd4uBecPL8SK1FwgdhRKV1FVPABaZOZva44d+O8dseHhkR3hbJy LW+g== MIME-Version: 1.0 X-Received: by 10.194.187.101 with SMTP id fr5mr5937wjc.76.1390512587732; Thu, 23 Jan 2014 13:29:47 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Thu, 23 Jan 2014 13:29:47 -0800 (PST) In-Reply-To: <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> Date: Thu, 23 Jan 2014 14:29:47 -0700 X-Google-Sender-Auth: e5K0vOWBjtw8UpQk2UtwqfAxKr4 Message-ID: Subject: Re: Makefile.inc1.patch From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:29:50 -0000 On Thu, Jan 23, 2014 at 2:23 PM, Garrett Cooper wro= te: > On Jan 23, 2014, at 1:11 PM, Alan Somers wrote: > >> In that case, I'm missing something. I can't find any makefiles that >> reference MK_ATF or a related variable. What is the effect of setting >> WITH_ATF ? >> >> -Alan > > NO_TESTS forces WITHOUT_TESTS to be set. So, if I set NO_TESTS in= the various build steps it will force ATF to not be built. For that reason= (and that reason alone) I reintroduced WITH_ATF just for Makefile.inc1 (bu= t you could replace it with something else like WITH_ATF_LIBS, etc, if the = naming is too confusing). > Thanks! > -Garrett I get that much, but what I don't understand is what direct affect WITH_ATF has. Did you forget to reintroduce a ".if defined(MK_ATF)" in some other file? -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:30:15 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09F38ED7; Thu, 23 Jan 2014 21:30:15 +0000 (UTC) Received: from mail-pa0-x233.google.com (mail-pa0-x233.google.com [IPv6:2607:f8b0:400e:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C45C41DBE; Thu, 23 Jan 2014 21:30:14 +0000 (UTC) Received: by mail-pa0-f51.google.com with SMTP id ld10so2373889pab.24 for ; Thu, 23 Jan 2014 13:30:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6OnRr9kII+QFN1pbQfh4WNnnm33Ez6ZpmsSVH8be9LI=; b=vK2w+Fbb3KUasJKG9YEsHaEoMLXWLU/0MoSS6D7F/Pwwb1ih1mjOIVZcW90h+1krIL us4odzopxFTYwleg7GpUJGRLR4Quyhoa11rtW+SQCBzYlV4XFpypeBTs8spBfhqfmMRw PiTMbSK1DXgj0WRYoqq8/I/9aPR0k4IY/XHrgU1Oe2/sujM+EjTy/FpPSp+3gwUay5L7 3F7NC1hr35ZQtwLxYSg2GpmYshyCEiHOkwMQq0Y4krANvuuKcTeqDSOlzm9ypcGWXgXi 6bRn5ObKNo6nLhCJX2ksR+7M3jHHkEDaEd/uYdBpo3pDv1bptk4mQxYo0wtT9hTljrMU 3YKA== X-Received: by 10.68.96.99 with SMTP id dr3mr10482701pbb.40.1390512614468; Thu, 23 Jan 2014 13:30:14 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id de1sm41229869pbc.7.2014.01.23.13.30.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 13:30:13 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: <20140123210308.0E1D65807E@chaos.jnpr.net> Date: Thu, 23 Jan 2014 13:30:12 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <20140123210308.0E1D65807E@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , brooks@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:30:15 -0000 On Jan 23, 2014, at 1:03 PM, Simon J. Gerraty wrote: > [+brooks] >=20 > On Thu, 23 Jan 2014 12:53:31 -0800, Garrett Cooper writes: >> Here=3D92s the working patch. The difference between this one = and =3D >> the prior version is that you have to explicitly override -DNO_TESTS = =3D >> when building lib/atf* (Simon: do you have any comments?): >=20 > Not crazy about frobbing ${MAKE} Neither am I, but .export is a bmake only feature. I=92m still using = fmake :(. >> -.if ${MK_TESTS} !=3D3D "no" >> +.if defined(WITH_ATF) || ${MK_TESTS} !=3D3D "no" >> +# Make sure WITH_ATF overrules -DNO_TESTS >> +.if !defined(WITH_ATF) >> +MAKE+=3D3D -DWITH_ATF >> +.endif >=20 > Would it make sense to have ATF or TESTS depend on the other? This functionality was removed several months ago.. otherwise I would = totally agree :/. > As is that can't be done, since one cannot always safely include > bsd.own.mk from the tree. >=20 > I'd really like to see the WITH[OUT]_ processing separated to its own > makefile (I use options.mk) so that it can always be safely used - = even > with an option list specific to a given makefile. Yeah, I would too [in an ideal world], but a lot of stuff is dependent = per-TARGET/TARGET_ARCH and compiler, so things get super complicated = really quickly in that regard. >=20 > The semantics in bsd.own.mk are quite broken and result in a lot of = complex > dancing to keep things working. In this case though, this is complex dancing due to how the different = stages stack upon one another in the build process and the fact that = meta make isn=92t here (yet), so things have to be built in the right = order. This method is one that I=92ve been using for quite some time = without any side effects on multiple machines... >>=20 >> I unrolled most of the local changes to Makefile.inc1 on my =3D >> github fork so it=3D92ll be easier to spot if you diff it against my = tree. Thanks, -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:32:31 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C1AC126; Thu, 23 Jan 2014 21:32:31 +0000 (UTC) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3F2A51E31; Thu, 23 Jan 2014 21:32:31 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id q10so2269845pdj.24 for ; Thu, 23 Jan 2014 13:32:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=u6d6ULHVpk8xtW+ulTWcyWRJx0IwMZtZJYztfD0wk8k=; b=HtXi/KqVB3G8us/Hwk3Qxk7flYoD5ciCssxKSjppNeIqcONeuXjaNhyAfyU82Y0PTE BSd9mcOiHcMMYAURXZ91rblJyTKj57mPYSYJmeyt0tuyjBWRE/LJKeFYfpPazF5r1D5J IIXM02Vnz82wM+XqEZlRUlAJXG1MW3WnC//hwhOgqiZJcAIVe2zRd9enaFCY4oOmchhj iAI4wB+i3H4LrSDaDMmjnwPs/fbdklY4uaWLAsqNO/1atknnlde03DV4ePG6IHeoyMh6 jyfBHaoVlyAFvebe35t1PV26da65k9W3VRK7pfK2G78n+AZxIj3hqE6+YI5mJN2UGzAN SD/w== X-Received: by 10.66.164.70 with SMTP id yo6mr10491990pab.85.1390512750933; Thu, 23 Jan 2014 13:32:30 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id nw11sm66148624pab.13.2014.01.23.13.32.29 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 13:32:30 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: Date: Thu, 23 Jan 2014 13:32:28 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:32:31 -0000 On Jan 23, 2014, at 1:29 PM, Alan Somers wrote: > On Thu, Jan 23, 2014 at 2:23 PM, Garrett Cooper = wrote: >> On Jan 23, 2014, at 1:11 PM, Alan Somers wrote: >>=20 >>> In that case, I'm missing something. I can't find any makefiles = that >>> reference MK_ATF or a related variable. What is the effect of = setting >>> WITH_ATF ? >>>=20 >>> -Alan >>=20 >> NO_TESTS forces WITHOUT_TESTS to be set. So, if I set NO_TESTS = in the various build steps it will force ATF to not be built. For that = reason (and that reason alone) I reintroduced WITH_ATF just for = Makefile.inc1 (but you could replace it with something else like = WITH_ATF_LIBS, etc, if the naming is too confusing). >> Thanks! >> -Garrett >=20 > I get that much, but what I don't understand is what direct affect > WITH_ATF has. Did you forget to reintroduce a ".if defined(MK_ATF)" > in some other file? No, the purpose of WITH_ATF is to override NO_TESTS, so building the ATF = libs now has two conditions: build_atf_libs =3D (is WITH_ATF defined?) && (is WITHOUT_TESTS defined?) versus one: build_atf_libs =3D (is WITHOUT_TESTS defined?) This allows us pepper NO_TESTS around and thus not build tests in the = build process unless they=92re _really_ needed (e.g. in make = everything). Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:34:22 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E15ED21D; Thu, 23 Jan 2014 21:34:21 +0000 (UTC) Received: from mail-pb0-x22a.google.com (mail-pb0-x22a.google.com [IPv6:2607:f8b0:400e:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9D50D1E51; Thu, 23 Jan 2014 21:34:21 +0000 (UTC) Received: by mail-pb0-f42.google.com with SMTP id jt11so2366769pbb.15 for ; Thu, 23 Jan 2014 13:34:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=cvD0W8Gp3rwCQL/fICLqKH3ZgKvowj78zIiA0KrUNFY=; b=cRY13uZLzrLPBd9nufRikwQ6AfJBzLcjyFeLZTn5Yiolo/hrqOWU1cZNxMmPW6no58 WuSRJzV54Hid0UPK4kEVLaKKELt+R3eqs69dmHMeTGqCnUpr11duutQJW+veeKCvHtom n0sfrKQbqIqRCUoeQMYY4CHogrR5jpeIe4/FtVQA/kzd2w8vvkdQFebrllOGNE3OALr/ QapyhMM5VIWV2ViDDEAbvoUAv41ox7UtEv5TPdWCw0GPRQ62Z3LT1uPubpQL/eJQc3q0 o1W4qt0ZYIvThOfiD7LC70ZVvqXlF3JbLPeCP96YMJI7D8aaLu3uWyP4UqUb2CYa8tqU XsRA== X-Received: by 10.66.232.7 with SMTP id tk7mr10325881pac.94.1390512861243; Thu, 23 Jan 2014 13:34:21 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id dq3sm41316658pbc.35.2014.01.23.13.34.20 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 13:34:20 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: Date: Thu, 23 Jan 2014 13:34:19 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <321F5F80-8195-4D06-8C43-3CC696C7BA01@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:34:22 -0000 Ugh. Backwards logic (sorry)... On Jan 23, 2014, at 1:32 PM, Garrett Cooper = wrote: > On Jan 23, 2014, at 1:29 PM, Alan Somers wrote: >=20 >> On Thu, Jan 23, 2014 at 2:23 PM, Garrett Cooper = wrote: >>> On Jan 23, 2014, at 1:11 PM, Alan Somers = wrote: >>>=20 >>>> In that case, I'm missing something. I can't find any makefiles = that >>>> reference MK_ATF or a related variable. What is the effect of = setting >>>> WITH_ATF ? >>>>=20 >>>> -Alan >>>=20 >>> NO_TESTS forces WITHOUT_TESTS to be set. So, if I set NO_TESTS = in the various build steps it will force ATF to not be built. For that = reason (and that reason alone) I reintroduced WITH_ATF just for = Makefile.inc1 (but you could replace it with something else like = WITH_ATF_LIBS, etc, if the naming is too confusing). >>> Thanks! >>> -Garrett >>=20 >> I get that much, but what I don't understand is what direct affect >> WITH_ATF has. Did you forget to reintroduce a ".if defined(MK_ATF)" >> in some other file? >=20 > No, the purpose of WITH_ATF is to override NO_TESTS, so building the = ATF libs now has two conditions: >=20 > build_atf_libs =3D (is WITH_ATF defined?) && (is WITHOUT_TESTS = defined?) build_atf_libs =3D (is WITH_ATF defined?) && (is WITH_TESTS defined?) > versus one: >=20 > build_atf_libs =3D (is WITHOUT_TESTS defined?) build_atf_libs =3D (is WITH_TESTS defined?) > This allows us pepper NO_TESTS around and thus not build tests in the = build process unless they=92re _really_ needed (e.g. in make = everything). From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:39:39 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B58EE4E7; Thu, 23 Jan 2014 21:39:39 +0000 (UTC) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com [IPv6:2a00:1450:400c:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F2F8E1E98; Thu, 23 Jan 2014 21:39:38 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id y10so2140179wgg.8 for ; Thu, 23 Jan 2014 13:39:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=1yuOAjxXzm/DwGI4WHHdOTQWI+fIIJdIQbiiYm4ruVY=; b=E3YQMFbZDT9Vl2QLyDT7kqcRezCC9KA6eywvuXglwH3/QrvJD1K4iOsWdsZAnp+YZF +6PwK+yi4W59JiQC03qE5aR76qL6ar8WhfG0MXxbA05QdbOhf1dvx8AI0nAhDCG4uyoR cHeAeO3GU5Z762M+y1fOWuROskjxSxjoncZy8Ol8dvkmXgCZ4kceLVL9I3jc5Hdkh/CC XQqDjZKUsvbYW/bO2MLR1HOw2EIvX/ks7GZc/17zwS4PXJ6BHvmuPJSPgRNd4enRfmWp KOhDIY8hFQWjzwoMidr7mmfNadRjELHYwhD3kdycze8g51C/8av+1h+z+IF3SoOxeQzG TR0Q== MIME-Version: 1.0 X-Received: by 10.180.73.19 with SMTP id h19mr836897wiv.40.1390513177436; Thu, 23 Jan 2014 13:39:37 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Thu, 23 Jan 2014 13:39:37 -0800 (PST) In-Reply-To: <321F5F80-8195-4D06-8C43-3CC696C7BA01@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> <321F5F80-8195-4D06-8C43-3CC696C7BA01@gmail.com> Date: Thu, 23 Jan 2014 14:39:37 -0700 X-Google-Sender-Auth: Nn9MD_s_vm6yTQFYg93fNlsrYCw Message-ID: Subject: Re: Makefile.inc1.patch From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:39:39 -0000 On Thu, Jan 23, 2014 at 2:34 PM, Garrett Cooper wro= te: > Ugh. Backwards logic (sorry)... > > On Jan 23, 2014, at 1:32 PM, Garrett Cooper wrote= : > >> On Jan 23, 2014, at 1:29 PM, Alan Somers wrote: >> >>> On Thu, Jan 23, 2014 at 2:23 PM, Garrett Cooper = wrote: >>>> On Jan 23, 2014, at 1:11 PM, Alan Somers wrote: >>>> >>>>> In that case, I'm missing something. I can't find any makefiles that >>>>> reference MK_ATF or a related variable. What is the effect of settin= g >>>>> WITH_ATF ? >>>>> >>>>> -Alan >>>> >>>> NO_TESTS forces WITHOUT_TESTS to be set. So, if I set NO_TESTS i= n the various build steps it will force ATF to not be built. For that reaso= n (and that reason alone) I reintroduced WITH_ATF just for Makefile.inc1 (b= ut you could replace it with something else like WITH_ATF_LIBS, etc, if the= naming is too confusing). >>>> Thanks! >>>> -Garrett >>> >>> I get that much, but what I don't understand is what direct affect >>> WITH_ATF has. Did you forget to reintroduce a ".if defined(MK_ATF)" >>> in some other file? >> >> No, the purpose of WITH_ATF is to override NO_TESTS, so building the ATF= libs now has two conditions: >> >> build_atf_libs =3D (is WITH_ATF defined?) && (is WITHOUT_TESTS defined?) > > build_atf_libs =3D (is WITH_ATF defined?) && (is WITH_TESTS defined?) Should that be || instead of && ? > >> versus one: >> >> build_atf_libs =3D (is WITHOUT_TESTS defined?) > > build_atf_libs =3D (is WITH_TESTS defined?) > >> This allows us pepper NO_TESTS around and thus not build tests in the bu= ild process unless they=92re _really_ needed (e.g. in make everything). Ok, I think I get it now. It's a recursive thing. At the top level, your patch adds WITH_ATF to MAKE. Then, in a child make process, the presence of WITH_ATF causes _lib_atf to be defined. Is that correct? -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:40:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04B596D4; Thu, 23 Jan 2014 21:40:38 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B34971EBA; Thu, 23 Jan 2014 21:40:37 +0000 (UTC) Received: by mail-pd0-f175.google.com with SMTP id w10so2277019pde.34 for ; Thu, 23 Jan 2014 13:40:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=YjNf3c5txTAKrTGyiwRbWLbxNTSnUPq8R4+Cuiiastc=; b=ZmnAr+pHSztZdGLV2bEG4GM+S6pvdqxGiUoc3w09gQOVTOBw3gbhui94lEe+Ze8aq5 sPJTkY4aOqfO+gp6f/y0scT0Rs8rwSqv/2PEhcIA9yQyOjJmUtpFx2FDN8G4cLUdZ6ue sCljBMUg8PymVPBbDUhqHUcH2BjSJpfRtfBCmtsbjKGYnhBN4npims/ojFLiKx20x9rb p3GZKKUM3i4AaoN5Au9YfX8KfPRLaDkfljfrR7KKNXRKEMXNQvJv9wuaq3ilfkuxdfUu w5JFvRP0gwg2XRnc4eVD8TZUPcuzBP2kJvr6HKR2HM99NBWJGK6K8ViO8iKqemqFqP6i bLFQ== X-Received: by 10.68.170.66 with SMTP id ak2mr10393321pbc.5.1390513237399; Thu, 23 Jan 2014 13:40:37 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id tu3sm41386476pbc.40.2014.01.23.13.40.35 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 13:40:36 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: Date: Thu, 23 Jan 2014 13:40:34 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <8541BFC7-9A0B-4199-97CB-2B29530C8D40@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> <321F5F80-8195-4D06-8C43-3CC696C7BA01@gmail.com> To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:40:38 -0000 On Jan 23, 2014, at 1:39 PM, Alan Somers wrote: > On Thu, Jan 23, 2014 at 2:34 PM, Garrett Cooper = wrote: >> Ugh. Backwards logic (sorry)... >>=20 >> On Jan 23, 2014, at 1:32 PM, Garrett Cooper = wrote: >>=20 >>> On Jan 23, 2014, at 1:29 PM, Alan Somers = wrote: >>>=20 >>>> On Thu, Jan 23, 2014 at 2:23 PM, Garrett Cooper = wrote: >>>>> On Jan 23, 2014, at 1:11 PM, Alan Somers = wrote: >>>>>=20 >>>>>> In that case, I'm missing something. I can't find any makefiles = that >>>>>> reference MK_ATF or a related variable. What is the effect of = setting >>>>>> WITH_ATF ? >>>>>>=20 >>>>>> -Alan >>>>>=20 >>>>> NO_TESTS forces WITHOUT_TESTS to be set. So, if I set = NO_TESTS in the various build steps it will force ATF to not be built. = For that reason (and that reason alone) I reintroduced WITH_ATF just for = Makefile.inc1 (but you could replace it with something else like = WITH_ATF_LIBS, etc, if the naming is too confusing). >>>>> Thanks! >>>>> -Garrett >>>>=20 >>>> I get that much, but what I don't understand is what direct affect >>>> WITH_ATF has. Did you forget to reintroduce a ".if = defined(MK_ATF)" >>>> in some other file? >>>=20 >>> No, the purpose of WITH_ATF is to override NO_TESTS, so building the = ATF libs now has two conditions: >>>=20 >>> build_atf_libs =3D (is WITH_ATF defined?) && (is WITHOUT_TESTS = defined?) >>=20 >> build_atf_libs =3D (is WITH_ATF defined?) && (is WITH_TESTS defined?) >=20 > Should that be || instead of && ? Yeah, I=92m really allowing myself to get distracted after lunch = :(.. >>=20 >>> versus one: >>>=20 >>> build_atf_libs =3D (is WITHOUT_TESTS defined?) >>=20 >> build_atf_libs =3D (is WITH_TESTS defined?) >>=20 >>> This allows us pepper NO_TESTS around and thus not build tests in = the build process unless they=92re _really_ needed (e.g. in make = everything). >=20 > Ok, I think I get it now. It's a recursive thing. At the top level, > your patch adds WITH_ATF to MAKE. Then, in a child make process, the > presence of WITH_ATF causes _lib_atf to be defined. Is that correct? Correct :)! Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:45:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 780C4A8A; Thu, 23 Jan 2014 21:45:59 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3A4431F72; Thu, 23 Jan 2014 21:45:59 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id x10so2302123pdj.39 for ; Thu, 23 Jan 2014 13:45:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=5C5lo2Xm1rwkAnT9Tgqkj9S+Lv354sDjJr4fuhxWy4Y=; b=nxjk+a6ZC8vRHaQ6KxrPkGD0NOuDFEu0yLzfMZCrV/nBh/+0zfjKcEAFAiHrSmcdLP bY/XPIittXp2sIPlPPKDUx7TWn77WmqkjJSaIp1RDD+33GmZCbIq6ZpMI6Cm/9qedE0j nKh0RTx8hZDMSPMvqTaz39Qsb4Qc12bWglqVusoKqPyEvvk3+rSktOl1x/JrvjV3YVWd x8RlzrlNJDA/wt3lzviWWU7zEzWmBI2EtwW5HN5nDErcBI0FiRgh/+SYYCyqAyOuqyxS hmpG9/jwnburwik+aTsyyC7wmMmJNdnLdK5Hc2uJYhX52JkCAn4bFP/FPHm0O/yNOQnL JfZg== X-Received: by 10.66.150.69 with SMTP id ug5mr10460513pab.55.1390513558804; Thu, 23 Jan 2014 13:45:58 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id da3sm41395250pbc.30.2014.01.23.13.45.57 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 13:45:58 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: <20140123214335.DD4255807E@chaos.jnpr.net> Date: Thu, 23 Jan 2014 13:45:55 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <949DAC82-A5AA-42FE-81A5-385A84718166@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> <20140123214335.DD4255807E@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , Brooks Davis X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:45:59 -0000 On Jan 23, 2014, at 1:43 PM, Simon J. Gerraty wrote: >=20 > On Thu, 23 Jan 2014 13:23:57 -0800, Garrett Cooper writes: >> NO_TESTS forces WITHOUT_TESTS to be set. So, if I set NO_TESTS =3D= >> in the various build steps it will force ATF to not be built. For = that =3D >> reason (and that reason alone) I reintroduced WITH_ATF just for =3D >> Makefile.inc1 (but you could replace it with something else like =3D >> WITH_ATF_LIBS, etc, if the naming is too confusing). >=20 > Is there a need to have a knob at all? To build atf libs I mean. For right now yeah, it=92s distinguished that way :/.. but it = seems lightweight enough (2 mins compile time?) to include in standard = builds. Maybe have a well-documented NO_ATF_LIBS knob for those that = need the extra compile time? Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 21:48:43 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 277F4CA4; Thu, 23 Jan 2014 21:48:43 +0000 (UTC) Received: from ch1outboundpool.messaging.microsoft.com (ch1ehsobe005.messaging.microsoft.com [216.32.181.185]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BF6BB1F99; Thu, 23 Jan 2014 21:48:42 +0000 (UTC) Received: from mail123-ch1-R.bigfish.com (10.43.68.228) by CH1EHSOBE010.bigfish.com (10.43.70.60) with Microsoft SMTP Server id 14.1.225.22; Thu, 23 Jan 2014 21:03:16 +0000 Received: from mail123-ch1 (localhost [127.0.0.1]) by mail123-ch1-R.bigfish.com (Postfix) with ESMTP id 866F040016A; Thu, 23 Jan 2014 21:03:16 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 0 X-BigFish: VPS0(zz103dK1432Idf9Izz1f42h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kz8dhz1de097h74efjz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h1155h) Received-SPF: softfail (mail123-ch1: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail123-ch1 (localhost.localdomain [127.0.0.1]) by mail123-ch1 (MessageSwitch) id 1390510994895815_17923; Thu, 23 Jan 2014 21:03:14 +0000 (UTC) Received: from CH1EHSMHS005.bigfish.com (snatpool2.int.messaging.microsoft.com [10.43.68.236]) by mail123-ch1.bigfish.com (Postfix) with ESMTP id D5D5EC0047; Thu, 23 Jan 2014 21:03:14 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by CH1EHSMHS005.bigfish.com (10.43.70.5) with Microsoft SMTP Server (TLS) id 14.16.227.3; Thu, 23 Jan 2014 21:03:14 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 23 Jan 2014 13:03:13 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s0NL38L38198; Thu, 23 Jan 2014 13:03:08 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 0E1D65807E; Thu, 23 Jan 2014 13:03:08 -0800 (PST) To: Garrett Cooper Subject: Re: Makefile.inc1.patch In-Reply-To: <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> Comments: In-reply-to: Garrett Cooper message dated "Thu, 23 Jan 2014 12:53:31 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 23 Jan 2014 13:03:08 -0800 Message-ID: <20140123210308.0E1D65807E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: "freebsd-testing@freebsd.org" , brooks@freebsd.org, sjg@juniper.net X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 21:48:43 -0000 [+brooks] On Thu, 23 Jan 2014 12:53:31 -0800, Garrett Cooper writes: > Here=92s the working patch. The difference between this one and = >the prior version is that you have to explicitly override -DNO_TESTS = >when building lib/atf* (Simon: do you have any comments?): Not crazy about frobbing ${MAKE} > >-.if ${MK_TESTS} !=3D "no" >+.if defined(WITH_ATF) || ${MK_TESTS} !=3D "no" >+# Make sure WITH_ATF overrules -DNO_TESTS >+.if !defined(WITH_ATF) >+MAKE+=3D -DWITH_ATF >+.endif Would it make sense to have ATF or TESTS depend on the other? As is that can't be done, since one cannot always safely include bsd.own.mk from the tree. I'd really like to see the WITH[OUT]_ processing separated to its own makefile (I use options.mk) so that it can always be safely used - even with an option list specific to a given makefile. The semantics in bsd.own.mk are quite broken and result in a lot of complex dancing to keep things working. > > I unrolled most of the local changes to Makefile.inc1 on my = >github fork so it=92ll be easier to spot if you diff it against my tree. >Thanks! >-Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 22:01:10 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00BA9520; Thu, 23 Jan 2014 22:01:09 +0000 (UTC) Received: from mail-pb0-x22b.google.com (mail-pb0-x22b.google.com [IPv6:2607:f8b0:400e:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B73091145; Thu, 23 Jan 2014 22:01:09 +0000 (UTC) Received: by mail-pb0-f43.google.com with SMTP id md12so2387050pbc.16 for ; Thu, 23 Jan 2014 14:01:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=P7IamqrDDN5ibZGCDp2bdV3au/9mfKYoeuFxYmnq45U=; b=bWswiiLpU8vEjElsr+DLdYftl9z8XVNT05FXZ2eSxBGZJb5tx8UgCVhbBHxxgJN7cm snFB5Uop66nLdrQfQVlQNj79oTa5F7b50RMZoq3CFh37GNjbk5tPSFje7Dt1p/bOk57j LvhpDeT4qhxNoubxLBMonzQrEo/SpyfFCtlPL0EG9Qo0/VyO/b2B1/vSNYO7InNfEiFf e/2l+NGT+U2EsPDoGnzbQnRak06prZ5I/shMqKHDGZjWy/VLnZ7baSgWo2xtkSn9l+9b v4RfcyTZkEdiJnA+wZUK6GdsdUzB/PLk0gQ3OK024EgFj8SfNCLwCAdbOvkhrZjolZxZ d3wg== X-Received: by 10.68.189.132 with SMTP id gi4mr10535859pbc.57.1390514469368; Thu, 23 Jan 2014 14:01:09 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id sq7sm41488039pbc.19.2014.01.23.14.01.08 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 14:01:08 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: <20140123215430.4B7B15807E@chaos.jnpr.net> Date: Thu, 23 Jan 2014 14:01:07 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <8D80A156-F649-4CA1-846A-DBAE9CC30627@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <20140123210308.0E1D65807E@chaos.jnpr.net> <20140123215430.4B7B15807E@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , brooks@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 22:01:10 -0000 On Jan 23, 2014, at 1:54 PM, Simon J. Gerraty wrote: > On Thu, 23 Jan 2014 13:30:12 -0800, Garrett Cooper writes: >>> Not crazy about frobbing ${MAKE} >>=20 >> Neither am I, but .export is a bmake only feature. I=3D92m still = using =3D >> fmake :(. >=20 > Not necessarily a good excuse ;-) fmake: "http://www.youtube.com/watch?v=3DdGFXGwHsD_A !!!!=94 :D It needs to die sometime, but Makefile.inc1 needs to continue to work = with it (for the time being). >>> The semantics in bsd.own.mk are quite broken and result in a lot of = =3D >> complex >>> dancing to keep things working. >>=20 >> In this case though, this is complex dancing due to how the different = =3D >> stages stack upon one another in the build process and the fact that = =3D >> meta make isn=3D92t here (yet), so things have to be built in the = right =3D >> order. This method is one that I=3D92ve been using for quite some = time =3D >> without any side effects on multiple machines... >=20 > Actually a simple tweak to the bsd.own.mk semantics would alleviate of > lot of the pain. Throwing errors if MK_* is already set, or if both > WITH_* and WITHOUT_* are set makes the usage very messy indeed. Yeah... > For options.mk I allow MK_* to already be set and WITHOUT_* to take > precedence over WITH_*. I also allow makefiles to have their own = lists > of options - separate the policy from the mechanism. Would that fix this case though? > I guess you could even allow a per-knob setting as to which takes > precedence.=20 You mean override the default so WITH_* overrides WITHOUT_*? > By simply allowing WITHOUT_* to overrule WITH_*, the Makefile.inc1 = usage > would be greatly simplified. Maybe=85 the -DNO_* logic is a bit messy=85 Curious to see what you have in mind :).. Thanks for the input :)! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 22:15:06 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1534DD2; Thu, 23 Jan 2014 22:15:05 +0000 (UTC) Received: from mail-pa0-x22e.google.com (mail-pa0-x22e.google.com [IPv6:2607:f8b0:400e:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A4F61124B; Thu, 23 Jan 2014 22:15:05 +0000 (UTC) Received: by mail-pa0-f46.google.com with SMTP id rd3so2440177pab.33 for ; Thu, 23 Jan 2014 14:15:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=CVqIDjuTKM1nbnS7pA/OpxjTchBu9vKoeWdzXS8ya3w=; b=M0UO2LeVLS7NhdeiXddzJ+OxOYNrTDuNKJY/r1QBBNWKxNW3KYVOOY7Nocp4och0MO LdnEAF7l4N6TEBQgKMM6UjHNC9w0OeWSg/KC1yIMBaVCExy5k1X4P9C0ufwI/v6J2pC9 3iXF3lj801JAuPUJOzv4Jp+T1xjG6nHwgnrOH2CpBwcpNBZmXhf6QSF9WQ9wvev4yG2N +I4r28p5NkH42kADVTdXpkQ/pS3kRnlfPOLsWDPV1lQurEGYVjc67yryai3Ey7yftq67 Fw/wVgumlviXM8RUh2XFBRUFY47CmGyrAg+9/n2mnA22+tatDR6VoArIl5IAXg+Id2ig ooqA== X-Received: by 10.68.35.129 with SMTP id h1mr10492260pbj.163.1390515305265; Thu, 23 Jan 2014 14:15:05 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id iq10sm41594695pbc.14.2014.01.23.14.15.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 14:15:04 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Makefile.inc1.patch From: Garrett Cooper In-Reply-To: <20140123221142.814FC5807E@chaos.jnpr.net> Date: Thu, 23 Jan 2014 14:15:02 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <7CB7D14F-7848-46B7-AB85-7E16145268D9@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <20140123210308.0E1D65807E@chaos.jnpr.net> <20140123215430.4B7B15807E@chaos.jnpr.net> <8D80A156-F649-4CA1-846A-DBAE9CC30627@gmail.com> <20140123221142.814FC5807E@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , brooks@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 22:15:06 -0000 On Jan 23, 2014, at 2:11 PM, Simon J. Gerraty wrote: >>> For options.mk I allow MK_* to already be set and WITHOUT_* to take >>> precedence over WITH_*. I also allow makefiles to have their own =3D >> lists >>> of options - separate the policy from the mechanism. >>=20 >> Would that fix this case though? >=20 > I imagine it would make fixing it easier. >=20 >>> I guess you could even allow a per-knob setting as to which takes >>> precedence.=3D20 >>=20 >> You mean override the default so WITH_* overrides WITHOUT_*? >=20 > Yes - I expect that would be rare, but worth it for completness. > The important thing is a simple precidence rule. >=20 >>> By simply allowing WITHOUT_* to overrule WITH_*, the Makefile.inc1 =3D= >> usage >>> would be greatly simplified. >>=20 >> Maybe=3D85 the -DNO_* logic is a bit messy=3D85 >=20 > NO_* always wins, it allows a makefile to say "I don't care what you > want I cannot do that". >=20 > Most places you see -DNO_* used could be -DWITHOUT_* if the semantics > were not broken as previously described. > NO_* should be mainly for makefiles to set - like NO_MAN=3D (i don't = got > no man page man) >=20 >> Curious to see what you have in mind :).. >=20 > Look at contrib/bmake/Makefile Ok, I=92ll definitely look at that. Alan, As far as fixing your issue is concerned though, has a fix already been = committed or does one still need to be committed? If the latter, does = this suffice for today =97 with the intent that it will get ripped out = in favor of something cleaner in the [near] future? Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 22:16:35 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5DF6FF2F; Thu, 23 Jan 2014 22:16:35 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2C0521269; Thu, 23 Jan 2014 22:16:35 +0000 (UTC) Received: by mail-pd0-f175.google.com with SMTP id w10so2309370pde.6 for ; Thu, 23 Jan 2014 14:16:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Jk1QSaBZKqWqWSoY1H+HoD56xdMOZyEbWKdIZLbBMGY=; b=un6ZvCW59iELQDzpxJvxcKM2WSoQofUkxOWO201l0E3Gb4k0bd0qNF/K9eN9Z0z4ml 0AKdeA7cQRod0jUDZVMgQfWQDm+xgOuHUaEBW7KU0Kjlu50TpnRbdZM78i2BOEMdtlt5 w64c9jiovor9Mww3jz3uS4vldSiHHnvZDUVIlHOLVBRV7Esg88kSLEgUmtxB3BUE7Abv uHwSWQE/LFwBm6MrreEv3acKVCueK8v/BpjtDocjfV2gSKmxzMga5gqWi6i64w34+L4t Ahg6T3hYsPtk3OrGHY4LgSO1iqJNxFA1LvAauUE10DzAhS1LAQXn7j4iV7MGDsV7uKXf iA6w== X-Received: by 10.68.89.162 with SMTP id bp2mr10369125pbb.151.1390515394877; Thu, 23 Jan 2014 14:16:34 -0800 (PST) Received: from fuji.zcorp.zonarsystems.com ([64.14.143.130]) by mx.google.com with ESMTPSA id de3sm41678389pbb.33.2014.01.23.14.16.33 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 14:16:34 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: [PATCH] convert bin/date over to ATF From: Garrett Cooper In-Reply-To: Date: Thu, 23 Jan 2014 14:16:32 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> To: Julio Merino X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@FreeBSD.org, Giorgos Keramidas X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 22:16:35 -0000 On Jan 22, 2014, at 3:56 PM, Julio Merino wrote: > On Jan 20, 2014, at 13:40, Garrett Cooper = wrote: >=20 >> This is based on work done by Giorgos a couple years ago. >> Thanks! >> -Garrett >=20 > Pasting patch contents and commenting inline: >=20 >> diff --git a/bin/date/tests/Makefile b/bin/date/tests/Makefile >> index 540008b..459d019 100644 >> --- a/bin/date/tests/Makefile >> +++ b/bin/date/tests/Makefile >> @@ -4,6 +4,6 @@ >>=20 >> TESTSDIR=3D ${TESTSBASE}/bin/date >>=20 >> -TAP_TESTS_SH=3D legacy_test >> +ATF_TESTS_SH=3D regress >=20 > Tests ought to end with _test per the description in = https://wiki.freebsd.org/TestSuite/Structure Ok! Is `_tests` ok? > Also, "regress_test" is not a very indicative name. Will this only = contain test cases for bugs to prevent regressions? It should be =93date_functional_tests=94, because that=92s what they=92re = doing. > date_test or integration_test (due to the lack of unit tests for the = code) may be better choices. ... >> --- /dev/null >> +++ b/bin/date/tests/regress.sh >> @@ -0,0 +1,557 @@ >> +#!/bin/sh >=20 > Remove. This comes from atf.test.mk automatically. >=20 >> + >> +# >> +# Regression tests for date(1) >> +# >> +# Submitted by Edwin Groothuis >> +# >> +# $FreeBSD: src/tools/regression/bin/date/regress.sh,v 1.4 = 2011/01/09 22:05:09 keramida Exp $ >> +# >> + >> +# >> +# These two date/times have been chosen carefully, they >> +# create both the single digit and double/multidigit version of >> +# the values. >> +# >> +# To create a new one, make sure you are using the UTC timezone! >> +# >> + >> +TEST1=3D3222243 # 1970-02-07 07:04:03 >> +TEST2=3D1005600000 # 2001-11-12 21:11:12 >> + >> +export LC_ALL=3DC >> +export TZ=3DUTC >=20 > Kyua does this already as part of the contract between atf and the = runtime engine. You should not be resetting these. Ok. This was code originally written with ATF in mind, but I totally = agree. >> + >> +check() >> +{ >> + S=3D$1 >> + A1=3D$2 >> + A2=3D$3 >=20 > S, A1, A2? What do these mean? Not sure=85 giorgos wrote these (but I assume it=92s format_string, = string, string_post_manipulation based on the comments=85). > Also, make local. Totally agree (will do)! >> + >> + # If the second sample text for formatted output has not been >> + # passed, assume it should match exactly the first one. >> + if [ -z "${A2}" ]; then >> + A2=3D${A1} >> + fi >> + >> + R=3D`date -r ${TEST1} +%${S}` >=20 > Prefer $() over ``. Me too (will change). >> + atf_check test "${R}" =3D "${A1}" >> + >> + R=3D`date -r ${TEST2} +%${S}` >> + atf_check test "${R}" =3D "${A2}" >> +} >> + >> +# = ---------------------------------------------------------------------- >> + >> +atf_test_case A >> +A_head() >> +{ >> + atf_set "descr" "Verifies that 'A' formatting spec works" >=20 > These test case names are truly non-descriptive. I'd recommend = renaming them to something like formatting_spec__A, etc. and omitting = the definition of head() altogether. Will result on a much shorter test = program, and being concise here for readability matters a lot. >=20 > ... and then you can just create a "macro" to define test cases. Like = this, but untested: >=20 > formatting_spec_test_case() { > local subname=3D"${1}"; shift > local name=3D"formatting_spec__${subname}" >=20 > atf_test_case "${name}" > eval "${name}_body() { check ${*} }" > } >=20 > formatting_spec_test_case a a Sat Mon > formatting_spec_test_case B B February November > ... > formatting_spec_test_case plus + "Sat Feb 7 07:04:03 UTC 1970" "Mon = Nov 12 21:20:00 UTC 2001=94 Yeah. I=92ll think about that further and get back to you after I figure = out how the code works a bit more (this was just a straight carryover = from giorgos). =85 Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 22:21:04 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84DEE16A; Thu, 23 Jan 2014 22:21:04 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E8E6D12F8; Thu, 23 Jan 2014 22:21:03 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id x12so2190800wgg.13 for ; Thu, 23 Jan 2014 14:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=px5wLFP+3kUEGZuZYJZCIbNKzTBRIR0Yx5b7tsD/ISM=; b=SfWOsdYOXz5rb8zCLhU5xu0OowiJKv9jsSUFZ+gJbJ3HPfXp4KdGLOudjJAMuTZEbC UABs42U7Y0OUg2ltb4l5h6WNVgiEsKB7txdLVSGK2yWOkGdi5kyNI7BS2KMmmdYVeEvL XeJF0ZArgMT0UjsRoqO3I9fbki0ZKB6n8kLdBFOqwHOd8PQjcq1xz8xTjT9P/SwmJYCx aCQ1qQjW2WLpU5sHmeGvjEjXkW4s1e2+znhCjcrsmEIaETjt7ZQpiy0R7Vaty53KleIG uLtnHqKcL3MZuzrwip2wRN+5yAoIObJ9lOyDTZAXGuKwaT1D5MK+j/hZL3V+qLz7QJyv AZtA== MIME-Version: 1.0 X-Received: by 10.180.76.168 with SMTP id l8mr918928wiw.40.1390515662264; Thu, 23 Jan 2014 14:21:02 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Thu, 23 Jan 2014 14:21:02 -0800 (PST) In-Reply-To: <7CB7D14F-7848-46B7-AB85-7E16145268D9@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <20140123210308.0E1D65807E@chaos.jnpr.net> <20140123215430.4B7B15807E@chaos.jnpr.net> <8D80A156-F649-4CA1-846A-DBAE9CC30627@gmail.com> <20140123221142.814FC5807E@chaos.jnpr.net> <7CB7D14F-7848-46B7-AB85-7E16145268D9@gmail.com> Date: Thu, 23 Jan 2014 15:21:02 -0700 X-Google-Sender-Auth: bANUA9YFVYyWNhf2iPXKAF9K-NM Message-ID: Subject: Re: Makefile.inc1.patch From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 22:21:04 -0000 On Thu, Jan 23, 2014 at 3:15 PM, Garrett Cooper wro= te: > On Jan 23, 2014, at 2:11 PM, Simon J. Gerraty wrote: > >>>> For options.mk I allow MK_* to already be set and WITHOUT_* to take >>>> precedence over WITH_*. I also allow makefiles to have their own =3D >>> lists >>>> of options - separate the policy from the mechanism. >>> >>> Would that fix this case though? >> >> I imagine it would make fixing it easier. >> >>>> I guess you could even allow a per-knob setting as to which takes >>>> precedence.=3D20 >>> >>> You mean override the default so WITH_* overrides WITHOUT_*? >> >> Yes - I expect that would be rare, but worth it for completness. >> The important thing is a simple precidence rule. >> >>>> By simply allowing WITHOUT_* to overrule WITH_*, the Makefile.inc1 =3D >>> usage >>>> would be greatly simplified. >>> >>> Maybe=3D85 the -DNO_* logic is a bit messy=3D85 >> >> NO_* always wins, it allows a makefile to say "I don't care what you >> want I cannot do that". >> >> Most places you see -DNO_* used could be -DWITHOUT_* if the semantics >> were not broken as previously described. >> NO_* should be mainly for makefiles to set - like NO_MAN=3D (i don't got >> no man page man) >> >>> Curious to see what you have in mind :).. >> >> Look at contrib/bmake/Makefile > > Ok, I'll definitely look at that. > > Alan, > > As far as fixing your issue is concerned though, has a fix already been c= ommitted or does one still need to be committed? If the latter, does this s= uffice for today -- with the intent that it will get ripped out in favor of= something cleaner in the [near] future? Umm, I accidentally committed my earlier patch along with a different change today. Oops. I'm currently testing your latest patch. I'm happy with committing it if it works. -Alan > > Thanks! > -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 22:26:58 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 054EE32B; Thu, 23 Jan 2014 22:26:58 +0000 (UTC) Received: from co1outboundpool.messaging.microsoft.com (co1ehsobe006.messaging.microsoft.com [216.32.180.189]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B009A1337; Thu, 23 Jan 2014 22:26:57 +0000 (UTC) Received: from mail178-co1-R.bigfish.com (10.243.78.243) by CO1EHSOBE036.bigfish.com (10.243.66.101) with Microsoft SMTP Server id 14.1.225.22; Thu, 23 Jan 2014 22:11:47 +0000 Received: from mail178-co1 (localhost [127.0.0.1]) by mail178-co1-R.bigfish.com (Postfix) with ESMTP id EA4ACD002A3; Thu, 23 Jan 2014 22:11:46 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 2 X-BigFish: VPS2(zz1432Izz1f42h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kz8dhz1de097hz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h1155h) Received-SPF: softfail (mail178-co1: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail178-co1 (localhost.localdomain [127.0.0.1]) by mail178-co1 (MessageSwitch) id 1390515105242841_25588; Thu, 23 Jan 2014 22:11:45 +0000 (UTC) Received: from CO1EHSMHS017.bigfish.com (unknown [10.243.78.251]) by mail178-co1.bigfish.com (Postfix) with ESMTP id 365D1500047; Thu, 23 Jan 2014 22:11:45 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by CO1EHSMHS017.bigfish.com (10.243.66.27) with Microsoft SMTP Server (TLS) id 14.16.227.3; Thu, 23 Jan 2014 22:11:44 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 23 Jan 2014 14:11:43 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s0NMBgL86536; Thu, 23 Jan 2014 14:11:42 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 814FC5807E; Thu, 23 Jan 2014 14:11:42 -0800 (PST) To: Garrett Cooper Subject: Re: Makefile.inc1.patch In-Reply-To: <8D80A156-F649-4CA1-846A-DBAE9CC30627@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <20140123210308.0E1D65807E@chaos.jnpr.net> <20140123215430.4B7B15807E@chaos.jnpr.net> <8D80A156-F649-4CA1-846A-DBAE9CC30627@gmail.com> Comments: In-reply-to: Garrett Cooper message dated "Thu, 23 Jan 2014 14:01:07 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 23 Jan 2014 14:11:42 -0800 Message-ID: <20140123221142.814FC5807E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: "freebsd-testing@freebsd.org" , brooks@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 22:26:58 -0000 >> For options.mk I allow MK_* to already be set and WITHOUT_* to take >> precedence over WITH_*. I also allow makefiles to have their own = >lists >> of options - separate the policy from the mechanism. > >Would that fix this case though? I imagine it would make fixing it easier. >> I guess you could even allow a per-knob setting as to which takes >> precedence.=20 > >You mean override the default so WITH_* overrides WITHOUT_*? Yes - I expect that would be rare, but worth it for completness. The important thing is a simple precidence rule. >> By simply allowing WITHOUT_* to overrule WITH_*, the Makefile.inc1 = >usage >> would be greatly simplified. > >Maybe=85 the -DNO_* logic is a bit messy=85 NO_* always wins, it allows a makefile to say "I don't care what you want I cannot do that". Most places you see -DNO_* used could be -DWITHOUT_* if the semantics were not broken as previously described. NO_* should be mainly for makefiles to set - like NO_MAN= (i don't got no man page man) >Curious to see what you have in mind :).. Look at contrib/bmake/Makefile --sjg From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 22:29:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E1543AA; Thu, 23 Jan 2014 22:29:18 +0000 (UTC) Received: from tx2outboundpool.messaging.microsoft.com (tx2ehsobe002.messaging.microsoft.com [65.55.88.12]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B9C761353; Thu, 23 Jan 2014 22:29:17 +0000 (UTC) Received: from mail174-tx2-R.bigfish.com (10.9.14.226) by TX2EHSOBE006.bigfish.com (10.9.40.26) with Microsoft SMTP Server id 14.1.225.22; Thu, 23 Jan 2014 21:43:47 +0000 Received: from mail174-tx2 (localhost [127.0.0.1]) by mail174-tx2-R.bigfish.com (Postfix) with ESMTP id 83C931001EA; Thu, 23 Jan 2014 21:43:47 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.16; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 3 X-BigFish: VPS3(zzzz1f42h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kz8dhzz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h1155h) Received-SPF: softfail (mail174-tx2: transitioning domain of juniper.net does not designate 66.129.239.16 as permitted sender) client-ip=66.129.239.16; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail174-tx2 (localhost.localdomain [127.0.0.1]) by mail174-tx2 (MessageSwitch) id 1390513424894285_15512; Thu, 23 Jan 2014 21:43:44 +0000 (UTC) Received: from TX2EHSMHS019.bigfish.com (unknown [10.9.14.227]) by mail174-tx2.bigfish.com (Postfix) with ESMTP id D47903C0049; Thu, 23 Jan 2014 21:43:44 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.16) by TX2EHSMHS019.bigfish.com (10.9.99.119) with Microsoft SMTP Server (TLS) id 14.16.227.3; Thu, 23 Jan 2014 21:43:39 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 23 Jan 2014 13:43:38 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s0NLhaL67857; Thu, 23 Jan 2014 13:43:36 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id DD4255807E; Thu, 23 Jan 2014 13:43:35 -0800 (PST) To: Garrett Cooper Subject: Re: Makefile.inc1.patch In-Reply-To: <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <4DB8E40F-6D7B-41A9-A0FA-B2E241E9A180@gmail.com> Comments: In-reply-to: Garrett Cooper message dated "Thu, 23 Jan 2014 13:23:57 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 23 Jan 2014 13:43:35 -0800 Message-ID: <20140123214335.DD4255807E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: "freebsd-testing@freebsd.org" , Brooks Davis , sjg@juniper.net X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 22:29:18 -0000 On Thu, 23 Jan 2014 13:23:57 -0800, Garrett Cooper writes: > NO_TESTS forces WITHOUT_TESTS to be set. So, if I set NO_TESTS = >in the various build steps it will force ATF to not be built. For that = >reason (and that reason alone) I reintroduced WITH_ATF just for = >Makefile.inc1 (but you could replace it with something else like = >WITH_ATF_LIBS, etc, if the naming is too confusing). Is there a need to have a knob at all? To build atf libs I mean. From owner-freebsd-testing@FreeBSD.ORG Thu Jan 23 22:40:04 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C783B66; Thu, 23 Jan 2014 22:40:04 +0000 (UTC) Received: from co1outboundpool.messaging.microsoft.com (co1ehsobe005.messaging.microsoft.com [216.32.180.188]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C9232148D; Thu, 23 Jan 2014 22:40:03 +0000 (UTC) Received: from mail204-co1-R.bigfish.com (10.243.78.226) by CO1EHSOBE001.bigfish.com (10.243.66.64) with Microsoft SMTP Server id 14.1.225.22; Thu, 23 Jan 2014 21:54:41 +0000 Received: from mail204-co1 (localhost [127.0.0.1]) by mail204-co1-R.bigfish.com (Postfix) with ESMTP id 71C801C0089; Thu, 23 Jan 2014 21:54:41 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 2 X-BigFish: VPS2(zz1432Izz1f42h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kz8dhz1de097hz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h1155h) Received-SPF: softfail (mail204-co1: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail204-co1 (localhost.localdomain [127.0.0.1]) by mail204-co1 (MessageSwitch) id 139051407923561_20197; Thu, 23 Jan 2014 21:54:39 +0000 (UTC) Received: from CO1EHSMHS001.bigfish.com (unknown [10.243.78.232]) by mail204-co1.bigfish.com (Postfix) with ESMTP id 012E1A80047; Thu, 23 Jan 2014 21:54:39 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by CO1EHSMHS001.bigfish.com (10.243.66.11) with Microsoft SMTP Server (TLS) id 14.16.227.3; Thu, 23 Jan 2014 21:54:38 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 23 Jan 2014 13:54:37 -0800 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s0NLsWL74407; Thu, 23 Jan 2014 13:54:32 -0800 (PST) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 4B7B15807E; Thu, 23 Jan 2014 13:54:30 -0800 (PST) To: Garrett Cooper Subject: Re: Makefile.inc1.patch In-Reply-To: References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <20140123210308.0E1D65807E@chaos.jnpr.net> Comments: In-reply-to: Garrett Cooper message dated "Thu, 23 Jan 2014 13:30:12 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 23 Jan 2014 13:54:30 -0800 Message-ID: <20140123215430.4B7B15807E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: "freebsd-testing@freebsd.org" , brooks@freebsd.org, sjg@juniper.net X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2014 22:40:04 -0000 On Thu, 23 Jan 2014 13:30:12 -0800, Garrett Cooper writes: >> Not crazy about frobbing ${MAKE} > >Neither am I, but .export is a bmake only feature. I=92m still using = >fmake :(. Not necessarily a good excuse ;-) >> The semantics in bsd.own.mk are quite broken and result in a lot of = >complex >> dancing to keep things working. > >In this case though, this is complex dancing due to how the different = >stages stack upon one another in the build process and the fact that = >meta make isn=92t here (yet), so things have to be built in the right = >order. This method is one that I=92ve been using for quite some time = >without any side effects on multiple machines... Actually a simple tweak to the bsd.own.mk semantics would alleviate of lot of the pain. Throwing errors if MK_* is already set, or if both WITH_* and WITHOUT_* are set makes the usage very messy indeed. For options.mk I allow MK_* to already be set and WITHOUT_* to take precedence over WITH_*. I also allow makefiles to have their own lists of options - separate the policy from the mechanism. I guess you could even allow a per-knob setting as to which takes precedence. By simply allowing WITHOUT_* to overrule WITH_*, the Makefile.inc1 usage would be greatly simplified. From owner-freebsd-testing@FreeBSD.ORG Fri Jan 24 03:38:30 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B747DD10 for ; Fri, 24 Jan 2014 03:38:30 +0000 (UTC) Received: from mail-pb0-x22f.google.com (mail-pb0-x22f.google.com [IPv6:2607:f8b0:400e:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 892DB1E00 for ; Fri, 24 Jan 2014 03:38:30 +0000 (UTC) Received: by mail-pb0-f47.google.com with SMTP id rp16so2690087pbb.34 for ; Thu, 23 Jan 2014 19:38:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=9Pkkm/y9F/oK0sXG5evGYoL+2l5+/ATo2aLGm1D2YnQ=; b=WCiB9DjSAawaM3BUC+ZTSbhWfHxpsi6mXQYeRVr3+xN2iA+OS3/keXSzlfS66EotmR h3tRaSg53+nWYF8iOmKF+2UNGKKlOzXdq5DWYk0hED8zAyt4d000hznWU2KdE1tZ763c XOZ64/gP8g1byxCeLiZg95upu/o3R/Ot6/2Yuv1yloKCB40mc2iLD0XVxJbXYdj1qljO zbuSo17FVKUSMLImtULA2W8NLv8GRFD9jYiSJZ5RUy+2BQJovzpEnUIPjdVapwgRGw/i JjLegfGAXq35MkoOqynPEtYcZZIBzL1okjIGABKI1hyfXMWfeA3hVQATZ9EArMcNzpV2 pNSg== X-Received: by 10.66.156.137 with SMTP id we9mr11971519pab.30.1390534710256; Thu, 23 Jan 2014 19:38:30 -0800 (PST) Received: from [192.168.20.5] (c-50-181-163-89.hsd1.wa.comcast.net. [50.181.163.89]) by mx.google.com with ESMTPSA id qq5sm44611444pbb.24.2014.01.23.19.38.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 23 Jan 2014 19:38:29 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: [PATCH] convert /bin/sh tests over to ATF From: Garrett Cooper In-Reply-To: <2222060C-CA92-41E5-9B10-9EA075A2B96B@meroh.net> Date: Thu, 23 Jan 2014 19:38:26 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <2F20BCCB-1464-495B-B08A-B2C538B27D18@gmail.com> References: <2222060C-CA92-41E5-9B10-9EA075A2B96B@meroh.net> To: Julio Merino X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@FreeBSD.org, Jilles Tjoelker X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 03:38:30 -0000 On Jan 22, 2014, at 3:36 PM, Julio Merino wrote: > On Jan 20, 2014, at 11:44, Garrett Cooper = wrote: >=20 >> The attached patch converts the /bin/sh tests using the existing = framework bin/sh/tests. >> All of the tests pass except locale.0, and that=92s due to some = [currently] unknown issue with my environment =97 see misc/181151). I = could make some of the entries more permanent, but I was trying to make = the tests easier to backport by using shell globs. >> Also, I added some functionality so the tests expect failures on = certain versions of FreeBSD due to missing functionality or conformance = modifications (see the .valid_osreldate files). >> Thoughts? Comments? >> Thanks! >> -Garrett >=20 > Pasted your diff inline for easier commenting but trimmed stuff. =85 > KYUAFILE=3Dauto is the default; remove here and anywhere else. Fixed =97 thanks :)! >> +SH_TESTS_FILES!=3D cd ${.CURDIR} && find . \( -type f -and \! -name = Makefile\* \) | \ >> + sed -e 's,^\./,,g' >=20 > This won't work well if you don't use an OBJDIR, will it? Yeah, it probably wouldn=92t work too well with ${SH_TESTS} being = defined the way it is. I overlooked the point when I last updated things = by continuing to use that variable... >> + >> +TESTSDIR=3D ${TESTSBASE}/bin/sh/${.CURDIR:T} >> + >> +FILES+=3D ${SH_TESTS_FILES} >> +FILESDIR=3D ${TESTSDIR} >> + >> +regress.sh: ../Makefile.inc ${SH_TESTS_FILES} >> + @echo '' > ${.TARGET} >=20 > Don't write directly to ${.TARGET} if you have more than one command = doing so. Otherwise, if the target fails half-way through (or is = interrupted), a second invocation of make won't properly regenerate the = file. >=20 > You can either do: >=20 > @{ \ > echo ... \ > echo ... \ > ... \ > } >${.TARGET} >=20 > or create a .tmp file and move "atomically" to the target at the end. Good point. I=92d use the latter method (in this case) because it would = be too hairy/convoluted if I tried to use inline shell logic. ... > All this logic in a Makefile is too complex and should really not be = there. >=20 > Just create a simple shell script that defines test cases dynamically = based on the available files and install that instead. It's shell after = all, so you can define functions at run time with ease! The results = will be much easier to digest. >=20 > ---- >=20 > But now, on a more abstract level... >=20 > This approach does not fit the "model" of atf-sh -- nor the model of = most unit testing libraries for that matter. An individual test program = for every test case is overkill and keeping all these tiny files around = is suboptimal. The following page has more details on this and the = corresponding rationales: >=20 > https://wiki.freebsd.org/TestSuite/Structure >=20 > I recommend to not do this in this manner. Instead, what I'd do is = the following: >=20 > First, define one test program for every directory that currently = exists. This would leave you with builtins_test errors_test = execution_test expansion_test parameters_test parser_test set-e_test, = all in /usr/tests/bin/sh. There is no reason to put them in separate = subdirectories. >=20 > Second, make each of these top-level test programs "auto-discover" the = various data files in their corresponding subdirectories, define an = individual test case for each, and run them "as is". For example, = builtins_test would inspect srcdir/builtins/*.0, dynamically define an = atf_test_case, and run the test code from within it. >=20 > These two steps change the "glue code" to be atf and expose the = individual test cases to kyua with minimal changes to the existing code. >=20 > And third, separately once the above works, consider merging the = contents of the individual files into the main test program. I'm not = sure keeping tons of small files is worth it, but that's a separate = discussion once you have got the basic atf structure in place. >=20 > This is all very similar to what was done with the ipf tests in = NetBSD. You can take a look at them here: >=20 > = http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/ipf/?only_with_tag=3DMAIN >=20 > (Before this was done, something similar to what you did above was = attempted by using awk to generate test programs... it was a nightmare = to maintain.) >=20 > Hope this helps and thanks for looking into this! All valuable input. I=92ll definitely think about it and apply your = comments appropriately :). Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Fri Jan 24 16:28:04 2014 Return-Path: Delivered-To: freebsd-testing@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 439C0C95 for ; Fri, 24 Jan 2014 16:28:04 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 093131ECC for ; Fri, 24 Jan 2014 16:28:00 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id A6DC4B8439; Fri, 24 Jan 2014 17:27:59 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 93ED128497; Fri, 24 Jan 2014 17:27:59 +0100 (CET) Date: Fri, 24 Jan 2014 17:27:59 +0100 From: Jilles Tjoelker To: Garrett Cooper Subject: Re: [PATCH] convert /bin/sh tests over to ATF Message-ID: <20140124162759.GB90996@stack.nl> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-testing@FreeBSD.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 16:28:04 -0000 On Mon, Jan 20, 2014 at 08:44:58AM -0800, Garrett Cooper wrote: > Also, I added some functionality so the tests expect failures on > certain versions of FreeBSD due to missing functionality or > conformance modifications (see the .valid_osreldate files). I think this part is a bit too simplistic and inflexible. I have always dealt with changes affecting the tests by MFCing the changes to the tests along with the changes to sh itself, and this has worked well enough. On the other hand, I also run -current's sh and tests on my -stable machine, without installing the tests (and often also without installing the new sh). Therefore, the osreldate of the machine does not imply that the sh is also of that version. There are also changes like r222684 that need an adjustment to the tests which are hard to handle properly using a .valid_osreldate file. There may be some point in marking tests that are supposed to work on any POSIX-compliant sh and ones that are inherently FreeBSD-specific, or possibly more granular. However, this would be most useful when someone who cares about another sh implementation works on it. -- Jilles Tjoelker From owner-freebsd-testing@FreeBSD.ORG Fri Jan 24 16:58:53 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 792E010C for ; Fri, 24 Jan 2014 16:58:53 +0000 (UTC) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 095B61291 for ; Fri, 24 Jan 2014 16:58:52 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id w61so2979961wes.30 for ; Fri, 24 Jan 2014 08:58:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:cc:content-type:content-transfer-encoding; bh=MymXDn+HnsUp2weNHn79YD0KrrGRuhPLM6oro/O8Vf8=; b=jmmfoV09ZwcgCNw4PPgMWUfX3T5Hkdx18fLwZS+KzfhLcomZbklVbeHGDAeEFlNqSi c0b8h3Vw9xtxYXtFQlsaFO9dYKU611I/YKahq73f2g28zHy6NcIcmQUUdkBV8MGjf+GJ s0ZJOJDPJQCvOap2Skq3LDfg5g2tnjjVsfF5COlT99fw7lvPojrhMSDvrx/KDFf5drZm XOPfQHiWBFJ+ZipmZBFZVoQkvfE8K2hibk4tg90dkr5FHV9RaluGiFRi4XfwIp1JJvPi 2KKpwOgGBymUQvCBcaftjrZ66+jelErwL/K09d+RwipwjgjyfVlbSm1Fs7/qU/zDEXV4 Qj0w== MIME-Version: 1.0 X-Received: by 10.194.202.230 with SMTP id kl6mt14700970wjc.9.1390582731375; Fri, 24 Jan 2014 08:58:51 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Fri, 24 Jan 2014 08:58:51 -0800 (PST) In-Reply-To: References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <20140123210308.0E1D65807E@chaos.jnpr.net> <20140123215430.4B7B15807E@chaos.jnpr.net> <8D80A156-F649-4CA1-846A-DBAE9CC30627@gmail.com> <20140123221142.814FC5807E@chaos.jnpr.net> <7CB7D14F-7848-46B7-AB85-7E16145268D9@gmail.com> Date: Fri, 24 Jan 2014 09:58:51 -0700 X-Google-Sender-Auth: 4vIJ4Gp7xunPKOXNPJRCBGUx7R8 Message-ID: Subject: Re: Makefile.inc1.patch From: Alan Somers Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 16:58:53 -0000 On Thu, Jan 23, 2014 at 3:21 PM, Alan Somers wrote: > On Thu, Jan 23, 2014 at 3:15 PM, Garrett Cooper w= rote: >> On Jan 23, 2014, at 2:11 PM, Simon J. Gerraty wrote: >> >>>>> For options.mk I allow MK_* to already be set and WITHOUT_* to take >>>>> precedence over WITH_*. I also allow makefiles to have their own =3D >>>> lists >>>>> of options - separate the policy from the mechanism. >>>> >>>> Would that fix this case though? >>> >>> I imagine it would make fixing it easier. >>> >>>>> I guess you could even allow a per-knob setting as to which takes >>>>> precedence.=3D20 >>>> >>>> You mean override the default so WITH_* overrides WITHOUT_*? >>> >>> Yes - I expect that would be rare, but worth it for completness. >>> The important thing is a simple precidence rule. >>> >>>>> By simply allowing WITHOUT_* to overrule WITH_*, the Makefile.inc1 = =3D >>>> usage >>>>> would be greatly simplified. >>>> >>>> Maybe=3D85 the -DNO_* logic is a bit messy=3D85 >>> >>> NO_* always wins, it allows a makefile to say "I don't care what you >>> want I cannot do that". >>> >>> Most places you see -DNO_* used could be -DWITHOUT_* if the semantics >>> were not broken as previously described. >>> NO_* should be mainly for makefiles to set - like NO_MAN=3D (i don't go= t >>> no man page man) >>> >>>> Curious to see what you have in mind :).. >>> >>> Look at contrib/bmake/Makefile >> >> Ok, I=92ll definitely look at that. >> >> Alan, >> >> As far as fixing your issue is concerned though, has a fix already been = committed or does one still need to be committed? If the latter, does this = suffice for today =97 with the intent that it will get ripped out in favor = of something cleaner in the [near] future? > > Umm, I accidentally committed my earlier patch along with a different > change today. Oops. I'm currently testing your latest patch. I'm > happy with committing it if it works. > > -Alan Works for me. Do you want me to commit it for you? > >> >> Thanks! >> -Garrett From owner-freebsd-testing@FreeBSD.ORG Fri Jan 24 17:39:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA69FEA9; Fri, 24 Jan 2014 17:39:38 +0000 (UTC) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7908E1743; Fri, 24 Jan 2014 17:39:38 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id lx4so3209157iec.27 for ; Fri, 24 Jan 2014 09:39:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=RHOA2PRmJjD6BNk/IYVZXKA3jSBgytcD1QhFjM+RWmg=; b=TmCR1ccuy8JbsMQErAKD3KVft0+X55qIYuZuQmwulgSYikbP/IQ2M15YVikmQ9acsh EVdhkQZiftUKYj7UmxOhWtwYnJBHcReR0sP5CInQUIBL76t727hYQY6hm8SobgRXclDK o/nW/5euHldhH1MmwLUFeqpmv5WIdeZq0siZj6GV7HyQmJQz2y8RRB+jJqOPEKNO0pPD GiRrXoTF2NQupMtOV2vPL2sDsXqyntCCkYijSengIRasKxQxgm+Tm2ElF3unz2I0PFHZ aoY3AkL3lxvsuD/CYC/gubzYuxMnWH3XYrc3iRsjB/LOG5oyjuKNm7beb5fxXW464uzo aw6w== X-Received: by 10.50.25.129 with SMTP id c1mr5700274igg.23.1390585177818; Fri, 24 Jan 2014 09:39:37 -0800 (PST) Received: from [10.0.2.128] ([64.14.143.130]) by mx.google.com with ESMTPSA id ac3sm12040741igd.4.2014.01.24.09.39.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 24 Jan 2014 09:39:36 -0800 (PST) References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> <20140123210308.0E1D65807E@chaos.jnpr.net> <20140123215430.4B7B15807E@chaos.jnpr.net> <8D80A156-F649-4CA1-846A-DBAE9CC30627@gmail.com> <20140123221142.814FC5807E@chaos.jnpr.net> <7CB7D14F-7848-46B7-AB85-7E16145268D9@gmail.com> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=cp932 Content-Transfer-Encoding: quoted-printable Message-Id: <31B88FF3-A392-4467-9D95-1414B9583B79@gmail.com> X-Mailer: iPhone Mail (11B554a) From: Garrett Cooper Subject: Re: Makefile.inc1.patch Date: Fri, 24 Jan 2014 09:39:35 -0800 To: Alan Somers Cc: "freebsd-testing@freebsd.org" , Brooks Davis , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 17:39:39 -0000 > On Jan 24, 2014, at 8:58, Alan Somers wrote: >=20 >> On Thu, Jan 23, 2014 at 3:21 PM, Alan Somers wrote:= >>> On Thu, Jan 23, 2014 at 3:15 PM, Garrett Cooper w= rote: >>> On Jan 23, 2014, at 2:11 PM, Simon J. Gerraty wrote: >>>=20 >>>>>> For options.mk I allow MK_* to already be set and WITHOUT_* to take >>>>>> precedence over WITH_*. I also allow makefiles to have their own =3D= >>>>> lists >>>>>> of options - separate the policy from the mechanism. >>>>>=20 >>>>> Would that fix this case though? >>>>=20 >>>> I imagine it would make fixing it easier. >>>>=20 >>>>>> I guess you could even allow a per-knob setting as to which takes >>>>>> precedence.=3D20 >>>>>=20 >>>>> You mean override the default so WITH_* overrides WITHOUT_*? >>>>=20 >>>> Yes - I expect that would be rare, but worth it for completness. >>>> The important thing is a simple precidence rule. >>>>=20 >>>>>> By simply allowing WITHOUT_* to overrule WITH_*, the Makefile.inc1 =3D= >>>>> usage >>>>>> would be greatly simplified. >>>>>=20 >>>>> Maybe=3D85 the -DNO_* logic is a bit messy=3D85 >>>>=20 >>>> NO_* always wins, it allows a makefile to say "I don't care what you >>>> want I cannot do that". >>>>=20 >>>> Most places you see -DNO_* used could be -DWITHOUT_* if the semantics >>>> were not broken as previously described. >>>> NO_* should be mainly for makefiles to set - like NO_MAN=3D (i don't go= t >>>> no man page man) >>>>=20 >>>>> Curious to see what you have in mind :).. >>>>=20 >>>> Look at contrib/bmake/Makefile >>>=20 >>> Ok, I=81fll definitely look at that. >>>=20 >>> Alan, >>>=20 >>> As far as fixing your issue is concerned though, has a fix already been c= ommitted or does one still need to be committed? If the latter, does this su= ffice for today =81\ with the intent that it will get ripped out in favor of= something cleaner in the [near] future? >>=20 >> Umm, I accidentally committed my earlier patch along with a different >> change today. Oops. I'm currently testing your latest patch. I'm >> happy with committing it if it works. >>=20 >> -Alan >=20 > Works for me. Do you want me to commit it for you? Please! The only thing I might do is add a comment stating that WITH_ATF is there ju= st to ensure that lib/atf is built in make libraries, so any code which need= s to link against it later on when -DNO_TESTS, can. Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Fri Jan 24 23:36:16 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44DA2D68 for ; Fri, 24 Jan 2014 23:36:16 +0000 (UTC) Received: from mail-la0-f48.google.com (mail-la0-f48.google.com [209.85.215.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BDE6314A5 for ; Fri, 24 Jan 2014 23:36:15 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id mc6so3056421lab.35 for ; Fri, 24 Jan 2014 15:36:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=J6k0HXBjW/XTDeMt5VqsC/RLL+NloHCBJxkAK5CW8Qk=; b=hhunBvHqDH3Zgv6Z8IAmv2CPsfkFV3FB3SFvxHNgBqeuVYAMYxxxjRnrHIutGXm7oD b2xb1fVf2NhIN6DSvdPqAfJ86Ctfy1RBhl5tGbe/Fd9rjP5QDzA1WUQhcg4x+RIkppnJ tGmxaar20yRc7QJE+PNwPWIVLZJp7W93JOki9LS7N8uwB6rtUSgBPNgQUdbdBRt3UiGv VYweJ6CRYBHvqzWxM+fySBdNghTCceVauibRAefoJuRZD9V7Sm1VgLpZby1CeFzobVDh i+rCOXuM7e1lBafN0RmLSskbMA+Vk06n92AKOMzMYSNtH4v83uHXUP9zLYIoyFkP+9rE q3/Q== X-Gm-Message-State: ALoCoQnxjIc7c3RK9yKZ3JsBEDb1KTg4GC2E9DCTZxzs3uaOvG7JsSlyXt6pN29/++z8v9bI1ZTL X-Received: by 10.152.229.225 with SMTP id st1mr10660392lac.2.1390606573786; Fri, 24 Jan 2014 15:36:13 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.129.101 with HTTP; Fri, 24 Jan 2014 15:35:53 -0800 (PST) X-Originating-IP: [2620:0:1003:100e:15ba:18e2:3e40:cdd2] In-Reply-To: <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> References: <4A3E3984-73D3-4441-97A7-D58679EFF978@gmail.com> <9775878D-91AB-4BE4-ADFA-32D8DB582AA6@gmail.com> From: Julio Merino Date: Fri, 24 Jan 2014 18:35:53 -0500 Message-ID: Subject: Re: Makefile.inc1.patch To: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , "Simon J. Gerraty" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 23:36:16 -0000 On Thu, Jan 23, 2014 at 3:53 PM, Garrett Cooper wro= te: > Here=92s the working patch. The difference between this one and t= he prior version is that you have to explicitly override -DNO_TESTS when bu= ilding lib/atf* (Simon: do you have any comments?): > > -.if ${MK_TESTS} !=3D "no" > +.if defined(WITH_ATF) || ${MK_TESTS} !=3D "no" > +# Make sure WITH_ATF overrules -DNO_TESTS > +.if !defined(WITH_ATF) > +MAKE+=3D -DWITH_ATF > +.endif Hmm... but this is only necessary to support building the lib32 versions of libatf, correct? Because otherwise, the libraries are already being built fine as part of the library building stage. Also, I'd recommend not using the WITH_ATF name here because this is not a variable supposed to be set by the users in src.conf and can be easily confused by others. It's an internal knob to support the recursion done by Makefile.inc1, so a more obscure name would suit. Anyway, I see this has been committed now. Please don't forget to pull any changes up to stable/10! --=20 Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Fri Jan 24 23:42:05 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED10BE9 for ; Fri, 24 Jan 2014 23:42:05 +0000 (UTC) Received: from mail-lb0-f170.google.com (mail-lb0-f170.google.com [209.85.217.170]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 762A2154D for ; Fri, 24 Jan 2014 23:42:04 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id u14so3054204lbd.15 for ; Fri, 24 Jan 2014 15:41:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=1leV5VRbJO8OKMyV6bbDc2A8Wd8gVJXLvjXURQCz7L8=; b=CW7vwTA3eHZAs/+VknQFwzY9uSTGTCfHBKziL9eQBaRhN0MSVgDSbclOLpeT8Ida2h JLe/9X6g6LUeCRc2B8op6DGCSYHZzLDW7tFlMWpLqTduMdbFNn9HNBHB2TPPtjrxj0I4 4WJxVziDDxxS8RUhzL6siYDfy7SeNIF4S6VmUwr0RFAxza+fbGtZxB2k+qY4/X9WWo0G yEIhf0LmwwtEER3bLeZGGRGdwN8zSIFvRb2dNM+acGCPFdukHmrsgyOGJNgHNMnG8v0Q G9q/BVk97bvDaFvv8TT0Kz8CEAf7+Ugw5eF4R+OaMgounqKFcXB1AI85xqPivS03xlGn V3Tw== X-Gm-Message-State: ALoCoQmAjVvFB6fp+dUyZNlqYzUQFd63GgR9lNna8jb/2MaA0qo9nS1CDHuKqJyhdRA2g+iIBdWB X-Received: by 10.112.78.4 with SMTP id x4mr17784lbw.81.1390606433423; Fri, 24 Jan 2014 15:33:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.129.101 with HTTP; Fri, 24 Jan 2014 15:33:33 -0800 (PST) X-Originating-IP: [2620:0:1003:100e:15ba:18e2:3e40:cdd2] In-Reply-To: <20140124162759.GB90996@stack.nl> References: <20140124162759.GB90996@stack.nl> From: Julio Merino Date: Fri, 24 Jan 2014 18:33:33 -0500 Message-ID: Subject: Re: [PATCH] convert /bin/sh tests over to ATF To: Jilles Tjoelker Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 23:42:06 -0000 On Fri, Jan 24, 2014 at 11:27 AM, Jilles Tjoelker wrote: > > There may be some point in marking tests that are supposed to work on > any POSIX-compliant sh and ones that are inherently FreeBSD-specific, or > possibly more granular. However, this would be most useful when someone > who cares about another sh implementation works on it. That's a worthy goal, but only if the test suite was an "external component" that you shipped outside of FreeBSD and was easily applicable to other systems. As it is now, the test suite lives in the FreeBSD tree and therefore it can be considered to be FreeBSD-specific. Special-casing things out as only applying to FreeBSD will be difficult to maintain and make little sense in the current context. Now... all this can (and should) easily turn into the more generic discussion of: how to define tests that can be shared across the various BSDs and how to better maintain them? I currently have no answer for that, but it's a pretty large topic to cover. -- Julio Merino / @jmmv From owner-freebsd-testing@FreeBSD.ORG Sat Jan 25 20:36:47 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A0A5939 for ; Sat, 25 Jan 2014 20:36:47 +0000 (UTC) Received: from mail-lb0-f178.google.com (mail-lb0-f178.google.com [209.85.217.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 111731445 for ; Sat, 25 Jan 2014 20:36:45 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id u14so3410553lbd.23 for ; Sat, 25 Jan 2014 12:36:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=0sf01asixfcem68Ypwqm+ENnpWC9BjaaFhbLNWRfsF4=; b=MH0JyhOfEWwNzyBXQy4RBxnHeAtO0t3gYqiiqzA5Y0QGVYWFNJSPkDVQy4inXAfQWP KtePwFhoCNUvna77mvx5xIigG1Gx4Fy5/b10JozEllhDu/wlw8yRfwxyYBPeuZhWJdxb Nba+ihqFB5mSa38vH+8/iGgvGCeEjSOwQ6UUqsZ5iY96HrmwxHE09vz6gWfivmzicDrB u7Xla9kF7xSHiu7vJgSNdorJKlOjpWrRsNWEOBkylUhiQNCC+6pSIGuVBpW8gJOm6GzJ jrWMZkURpksEdhO+dmVJQ95nR0r7llqXZL82oRKuqtNnNO6Bu0iI4k9JCTny8dxrDqeW ROZA== X-Gm-Message-State: ALoCoQnGlm58hFvMQxjFnXufAthDWXFivdd3SQM0b8G58Xkqj9k3uiooHEflRXIj341h5yN3pVc3 X-Received: by 10.152.3.10 with SMTP id 10mr63037lay.35.1390682204036; Sat, 25 Jan 2014 12:36:44 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.129.101 with HTTP; Sat, 25 Jan 2014 12:36:22 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> From: Julio Merino Date: Sat, 25 Jan 2014 15:36:22 -0500 Message-ID: Subject: Re: [PATCH] convert bin/date over to ATF To: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: freebsd-testing@freebsd.org, Giorgos Keramidas X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 20:36:47 -0000 On Thu, Jan 23, 2014 at 5:16 PM, Garrett Cooper wro= te: > On Jan 22, 2014, at 3:56 PM, Julio Merino wrote: > >> On Jan 20, 2014, at 13:40, Garrett Cooper wrote: >> >>> This is based on work done by Giorgos a couple years ago. >>> Thanks! >>> -Garrett >> >> Pasting patch contents and commenting inline: >> >>> diff --git a/bin/date/tests/Makefile b/bin/date/tests/Makefile >>> index 540008b..459d019 100644 >>> --- a/bin/date/tests/Makefile >>> +++ b/bin/date/tests/Makefile >>> @@ -4,6 +4,6 @@ >>> >>> TESTSDIR=3D ${TESTSBASE}/bin/date >>> >>> -TAP_TESTS_SH=3D legacy_test >>> +ATF_TESTS_SH=3D regress >> >> Tests ought to end with _test per the description in https://wiki.freebs= d.org/TestSuite/Structure > > Ok! Is `_tests` ok? _test is a very common convention "out there"; I've seen it used pretty much everywhere plus we already have a bunch of _test programs. I don't think diverging to _tests is a good idea. (The rationale for _test, I believe, is that this refers to the "test program", not its contents. But who knows -- I'm just making that up.) >> Also, "regress_test" is not a very indicative name. Will this only cont= ain test cases for bugs to prevent regressions? > > It should be =93date_functional_tests=94, because that=92s what they=92re= doing. Sure; the more descriptive, the better. (But note that this will live in a "date" subdirectory, so arguably functional_test is enough.) --=20 Julio Merino / @jmmv