From owner-freebsd-current@FreeBSD.ORG Fri Jun 4 12:49:50 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D3731065686 for ; Fri, 4 Jun 2010 12:49:50 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id B84208FC0C for ; Fri, 4 Jun 2010 12:49:49 +0000 (UTC) Received: from [10.0.1.30] (unknown [85.82.25.93]) by csmtp2.one.com (Postfix) with ESMTP id 198911B003B8B; Fri, 4 Jun 2010 12:49:48 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: multipart/signed; boundary=Apple-Mail-679-995844465; protocol="application/pkcs7-signature"; micalg=sha1 From: Erik Cederstrand In-Reply-To: <20100603181316.A65565@mp2.macomnet.net> Date: Fri, 4 Jun 2010 14:49:47 +0200 Message-Id: <54C57DCD-9EEE-49A6-8AED-8DCF6A1AB2D1@cederstrand.dk> References: <97382526-EC81-4660-B494-315A73DD5783@cederstrand.dk> <20100603140951.J65565@mp2.macomnet.net> <1E5F7C3C-0502-4503-B168-9CF564FA51F8@cederstrand.dk> <20100603181316.A65565@mp2.macomnet.net> To: Maxim Konovalov X-Mailer: Apple Mail (2.1078) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: current@freebsd.org Subject: Re: Running all regression tests X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 12:49:50 -0000 --Apple-Mail-679-995844465 Content-Type: multipart/mixed; boundary=Apple-Mail-678-995844450 --Apple-Mail-678-995844450 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Den 03/06/2010 kl. 16.14 skrev Maxim Konovalov: > On Thu, 3 Jun 2010, 15:15+0200, Erik Cederstrand wrote: >=20 >> I just wrote a shell script to recurse into the subdirectories and >> run make on the Makefiles found. Unfortunately, some of the >> Makefiles start running tests immediately, some have syntax errors >> etc., so I'll have to add some more logic. >>=20 > It would be nice to get patches while you are there. A couple of patches to Makefiles were already committed (thanks, = Maxim!). Here's a cleanup patch for some more Makefiles. It enforces the = following convention for make targets: .all: builds any code needed to run the tests .test: runs any test cases defined .clean: cleans up any files Not all Makefiles define these targets, but it changes the Makefiles = that do have them. The patch allows to recursively run 'make all' on all Makefiles first, = then run 'make test' on all Makefiles to run the regression tests. The = patch doesn't attempt to rewrite existing tests to .t format. Also, I = know next to nothing about Makefiles. If it looks OK, is somebody willing to commit this? Thanks, Erik --Apple-Mail-678-995844450 Content-Disposition: attachment; filename=regression.patch Content-Type: application/octet-stream; name="regression.patch" Content-Transfer-Encoding: 7bit Index: tools/regression/execve/Makefile =================================================================== --- tools/regression/execve/Makefile (revision 208799) +++ tools/regression/execve/Makefile (working copy) @@ -18,7 +18,7 @@ chmod +x ${x} .endfor -regress: test-empty test-nonexist test-nonexistshell \ +test: test-empty test-nonexist test-nonexistshell \ test-devnullscript test-badinterplen test-goodscript \ test-scriptarg test-scriptarg-nospace test-goodaout \ test-truncaout test-sparseaout Index: tools/regression/acct/Makefile =================================================================== --- tools/regression/acct/Makefile (revision 208799) +++ tools/regression/acct/Makefile (working copy) @@ -9,7 +9,7 @@ CFLAGS= -include /usr/include/sys/limits.h \ -include /usr/include/sys/syslog.h -all: regress +all: pack: pack.c convert.c @@ -17,7 +17,7 @@ sed -n -e 's/log(/syslog(/g' \ -e '/FLOAT_CONVERSION_START/,/FLOAT_CONVERSION_END/p' $? >$@ -regress: pack regress.t +test: pack regress.t ./regress.t clean: Index: tools/regression/file/closefrom/Makefile =================================================================== --- tools/regression/file/closefrom/Makefile (revision 208799) +++ tools/regression/file/closefrom/Makefile (working copy) @@ -7,3 +7,4 @@ LDADD= -lutil .include + Index: tools/regression/usr.bin/lastcomm/Makefile =================================================================== --- tools/regression/usr.bin/lastcomm/Makefile (revision 208799) +++ tools/regression/usr.bin/lastcomm/Makefile (working copy) @@ -1,6 +1,6 @@ # $FreeBSD$ -all: regress +all: -regress: regress.t +test: regress.t ./regress.t Index: tools/regression/usr.bin/m4/Makefile =================================================================== --- tools/regression/usr.bin/m4/Makefile (revision 208799) +++ tools/regression/usr.bin/m4/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @sh ${.CURDIR}/regress.sh ${.CURDIR} Index: tools/regression/usr.bin/xargs/Makefile =================================================================== --- tools/regression/usr.bin/xargs/Makefile (revision 208799) +++ tools/regression/usr.bin/xargs/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/apply/Makefile =================================================================== --- tools/regression/usr.bin/apply/Makefile (revision 208799) +++ tools/regression/usr.bin/apply/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/tr/Makefile =================================================================== --- tools/regression/usr.bin/tr/Makefile (revision 208799) +++ tools/regression/usr.bin/tr/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/comm/Makefile =================================================================== --- tools/regression/usr.bin/comm/Makefile (revision 208799) +++ tools/regression/usr.bin/comm/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/jot/Makefile =================================================================== --- tools/regression/usr.bin/jot/Makefile (revision 208799) +++ tools/regression/usr.bin/jot/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/uudecode/Makefile =================================================================== --- tools/regression/usr.bin/uudecode/Makefile (revision 208799) +++ tools/regression/usr.bin/uudecode/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/join/Makefile =================================================================== --- tools/regression/usr.bin/join/Makefile (revision 208799) +++ tools/regression/usr.bin/join/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/printf/Makefile =================================================================== --- tools/regression/usr.bin/printf/Makefile (revision 208799) +++ tools/regression/usr.bin/printf/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/file2c/Makefile =================================================================== --- tools/regression/usr.bin/file2c/Makefile (revision 208799) +++ tools/regression/usr.bin/file2c/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/uuencode/Makefile =================================================================== --- tools/regression/usr.bin/uuencode/Makefile (revision 208799) +++ tools/regression/usr.bin/uuencode/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/calendar/Makefile =================================================================== --- tools/regression/usr.bin/calendar/Makefile (revision 208799) +++ tools/regression/usr.bin/calendar/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/sed/Makefile =================================================================== --- tools/regression/usr.bin/sed/Makefile (revision 208799) +++ tools/regression/usr.bin/sed/Makefile (working copy) @@ -1,5 +1,5 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} @sh multitest.t Index: tools/regression/usr.bin/ncal/Makefile =================================================================== --- tools/regression/usr.bin/ncal/Makefile (revision 208799) +++ tools/regression/usr.bin/ncal/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: @m4 ${.CURDIR}/../regress.m4 ${.CURDIR}/regress.sh | sh /dev/stdin ${.CURDIR} Index: tools/regression/usr.bin/make/archives/fmt_44bsd_mod/Makefile =================================================================== --- tools/regression/usr.bin/make/archives/fmt_44bsd_mod/Makefile (revision 208799) +++ tools/regression/usr.bin/make/archives/fmt_44bsd_mod/Makefile (working copy) @@ -5,6 +5,8 @@ # The modification of the archive consists in slashes appended to short names. # +test: test1 test2 test3 test4 test5 test6 test7 + ############################################################################# # must be ok Index: tools/regression/usr.bin/make/archives/fmt_44bsd/Makefile =================================================================== --- tools/regression/usr.bin/make/archives/fmt_44bsd/Makefile (revision 208799) +++ tools/regression/usr.bin/make/archives/fmt_44bsd/Makefile (working copy) @@ -2,6 +2,8 @@ # This test checks the code reading archive files. The archive file # is a BSD4.4 file with __.SYMTAB and #1/N long file names. +test: test1 test2 test3 test4 test5 test6 test7 + ############################################################################# # should be ok Index: tools/regression/usr.bin/make/archives/fmt_oldbsd/Makefile =================================================================== --- tools/regression/usr.bin/make/archives/fmt_oldbsd/Makefile (revision 208799) +++ tools/regression/usr.bin/make/archives/fmt_oldbsd/Makefile (working copy) @@ -3,6 +3,8 @@ # is an old BSD file with __.SYMTAB and no long file names. Filenames # are truncated to 16 characters. +test: test1 test2 test3 test4 test5 test6 test7 + ############################################################################# # must be ok Index: tools/regression/usr.bin/make/syntax/semi/Makefile =================================================================== --- tools/regression/usr.bin/make/syntax/semi/Makefile (revision 208799) +++ tools/regression/usr.bin/make/syntax/semi/Makefile (working copy) @@ -3,6 +3,8 @@ # Test handling of escaped newlines. # +test: test1 test2 + .ifmake test1 FOO= Index: tools/regression/usr.bin/make/sysmk/t0/2/1/Makefile =================================================================== --- tools/regression/usr.bin/make/sysmk/t0/2/1/Makefile (revision 208799) +++ tools/regression/usr.bin/make/sysmk/t0/2/1/Makefile (working copy) @@ -2,5 +2,5 @@ # # Can we traverse up to / and find a 'mk/sys.mk'? # -all: +test: @echo ${DASH_M_DOTDOTDOT} Index: tools/regression/usr.bin/make/shell/builtin/Makefile =================================================================== --- tools/regression/usr.bin/make/shell/builtin/Makefile (revision 208799) +++ tools/regression/usr.bin/make/shell/builtin/Makefile (working copy) @@ -11,6 +11,8 @@ .PHONY: builtin no-builtin +test: builtin no-builtin + builtin: @exec ls -d . Index: tools/regression/usr.bin/make/shell/meta/Makefile =================================================================== --- tools/regression/usr.bin/make/shell/meta/Makefile (revision 208799) +++ tools/regression/usr.bin/make/shell/meta/Makefile (working copy) @@ -11,6 +11,8 @@ .PHONY: meta no-meta +test: meta no-meta + meta: @ls * Index: tools/regression/usr.bin/make/shell/replace/Makefile =================================================================== --- tools/regression/usr.bin/make/shell/replace/Makefile (revision 208799) +++ tools/regression/usr.bin/make/shell/replace/Makefile (working copy) @@ -15,7 +15,7 @@ echoFlag="x" errFlag="y" \ hasErrCtl=y check="check errors" ignore="ignore errors" -.PHONY: test1 -test1: +.PHONY: test +test: -@funny $$ funnier $$ Index: tools/regression/usr.bin/make/variables/t0/Makefile =================================================================== --- tools/regression/usr.bin/make/variables/t0/Makefile (revision 208799) +++ tools/regression/usr.bin/make/variables/t0/Makefile (working copy) @@ -4,7 +4,7 @@ # A = 0 AV = 1 -test1: +test: @echo $A @echo ${AV} @echo ${A} Index: tools/regression/usr.bin/make/variables/modifier_M/Makefile =================================================================== --- tools/regression/usr.bin/make/variables/modifier_M/Makefile (revision 208799) +++ tools/regression/usr.bin/make/variables/modifier_M/Makefile (working copy) @@ -8,7 +8,7 @@ map.c map.h \ parser.y lexer.l \ cmdman.1 format.5 -test1: +test: @echo "all files: ${FILES}" @echo "cfiles: ${FILES:M*.c}" @echo "hfiles: ${FILES:M*.h}" Index: tools/regression/usr.bin/make/suffixes/basic/Makefile =================================================================== --- tools/regression/usr.bin/make/suffixes/basic/Makefile (revision 208799) +++ tools/regression/usr.bin/make/suffixes/basic/Makefile (working copy) @@ -2,7 +2,7 @@ # # Check that a simple suffix rule is correctly processed. -test1: TEST1.b +test: TEST1.b diff -u TEST1.a TEST1.b .SUFFIXES: Index: tools/regression/usr.bin/make/suffixes/src_wild1/Makefile =================================================================== --- tools/regression/usr.bin/make/suffixes/src_wild1/Makefile (revision 208799) +++ tools/regression/usr.bin/make/suffixes/src_wild1/Makefile (working copy) @@ -1,6 +1,6 @@ # $FreeBSD$ -test1: TEST1.b +test: TEST1.b cat TEST1.a TEST2.a | diff -u - TEST1.b TEST1.b: *.a Index: tools/regression/usr.bin/make/suffixes/src_wild2/Makefile =================================================================== --- tools/regression/usr.bin/make/suffixes/src_wild2/Makefile (revision 208799) +++ tools/regression/usr.bin/make/suffixes/src_wild2/Makefile (working copy) @@ -5,7 +5,7 @@ SRC=*.a -test1: TEST1.b +test: TEST1.b cat TEST1.a TEST2.a | diff -u - TEST1.b TEST1.b: ${SRC} Index: tools/regression/usr.sbin/sa/Makefile =================================================================== --- tools/regression/usr.sbin/sa/Makefile (revision 208799) +++ tools/regression/usr.sbin/sa/Makefile (working copy) @@ -1,6 +1,6 @@ # $FreeBSD$ -all: regress +all: -regress: regress.t +test: regress.t ./regress.t Index: tools/regression/usr.sbin/Makefile =================================================================== --- tools/regression/usr.sbin/Makefile (revision 208799) +++ tools/regression/usr.sbin/Makefile (working copy) @@ -1,5 +1,4 @@ # $FreeBSD$ -SUBDIR= sa .include Index: tools/regression/lib/msun/Makefile =================================================================== --- tools/regression/lib/msun/Makefile (revision 208799) +++ tools/regression/lib/msun/Makefile (working copy) @@ -6,7 +6,7 @@ CFLAGS+= -O0 -lm .PHONY: tests -tests: ${TESTS} +test: ${TESTS} for p in ${TESTS}; do ${.OBJDIR}/$$p; done .PHONY: clean Index: tools/regression/lib/libmp/Makefile =================================================================== --- tools/regression/lib/libmp/Makefile (revision 208799) +++ tools/regression/lib/libmp/Makefile (working copy) @@ -4,7 +4,7 @@ CFLAGS+= -g -Wall -lcrypto -lmp .PHONY: tests -tests: ${TESTS} +test: ${TESTS} for p in ${TESTS}; do ${.OBJDIR}/$$p; done .PHONY: clean Index: tools/regression/lib/libutil/Makefile =================================================================== --- tools/regression/lib/libutil/Makefile (revision 208799) +++ tools/regression/lib/libutil/Makefile (working copy) @@ -4,7 +4,7 @@ CFLAGS+= -g -Wall -Wextra -Werror -lutil .PHONY: tests -tests: ${TESTS} +test: ${TESTS} for p in ${TESTS}; do ${.OBJDIR}/$$p; done .PHONY: clean Index: tools/regression/lib/libc/locale/Makefile =================================================================== --- tools/regression/lib/libc/locale/Makefile (revision 208799) +++ tools/regression/lib/libc/locale/Makefile (working copy) @@ -17,7 +17,7 @@ test-towctrans .PHONY: tests -tests: ${TESTS} +test: ${TESTS} for p in ${TESTS}; do ${.OBJDIR}/$$p; done .PHONY: clean Index: tools/regression/lib/libc/stdlib/Makefile =================================================================== --- tools/regression/lib/libc/stdlib/Makefile (revision 208799) +++ tools/regression/lib/libc/stdlib/Makefile (working copy) @@ -5,7 +5,7 @@ test-mergesort .PHONY: tests -tests: ${TESTS} +test: ${TESTS} for p in ${TESTS}; do ${.OBJDIR}/$$p; done .PHONY: clean Index: tools/regression/lib/libc/stdio/Makefile =================================================================== --- tools/regression/lib/libc/stdio/Makefile (revision 208799) +++ tools/regression/lib/libc/stdio/Makefile (working copy) @@ -4,7 +4,7 @@ CFLAGS+= -lm .PHONY: tests -tests: ${TESTS} +test: ${TESTS} for p in ${TESTS}; do ${.OBJDIR}/$$p; done .PHONY: clean Index: tools/regression/lib/libc/nss/Makefile =================================================================== --- tools/regression/lib/libc/nss/Makefile (revision 208799) +++ tools/regression/lib/libc/nss/Makefile (working copy) @@ -5,7 +5,7 @@ CFLAGS+= -g -Wall .PHONY: tests -tests: ${TESTS} +test: ${TESTS} .PHONY: clean clean: Index: tools/regression/lib/libc/net/Makefile =================================================================== --- tools/regression/lib/libc/net/Makefile (revision 208799) +++ tools/regression/lib/libc/net/Makefile (working copy) @@ -4,7 +4,7 @@ CFLAGS+= -g -Wall .PHONY: tests -tests: ${TESTS} +test: ${TESTS} for p in ${TESTS}; do ${.OBJDIR}/$$p; done .PHONY: clean Index: tools/regression/lib/libc/string/Makefile =================================================================== --- tools/regression/lib/libc/string/Makefile (revision 208799) +++ tools/regression/lib/libc/string/Makefile (working copy) @@ -7,7 +7,7 @@ TESTS= test-stpncpy test-strerror test-wcscasecmp test-wcsnlen .PHONY: tests -tests: ${TESTS} +test: ${TESTS} for p in ${TESTS}; do ${.OBJDIR}/$$p; done .PHONY: clean Index: tools/regression/lib/libc/resolv/Makefile =================================================================== --- tools/regression/lib/libc/resolv/Makefile (revision 208799) +++ tools/regression/lib/libc/resolv/Makefile (working copy) @@ -9,7 +9,7 @@ DPADD+= ${LIBPTHREAD} LDADD+= -lpthread -regress: ${PROG} +test: ${PROG} ./${PROG} -r ${.CURDIR}/mach .include Index: tools/regression/bin/date/Makefile =================================================================== --- tools/regression/bin/date/Makefile (revision 208799) +++ tools/regression/bin/date/Makefile (working copy) @@ -1,4 +1,4 @@ # $FreeBSD$ -all: +test: sh regress.sh Index: tools/regression/pipe/Makefile =================================================================== --- tools/regression/pipe/Makefile (revision 208799) +++ tools/regression/pipe/Makefile (working copy) @@ -6,7 +6,7 @@ PROG= bigpipetest NO_MAN= -regress: +test: @if ./bigpipetest; then \ echo "PASS"; \ else \ Index: tools/regression/sockets/accept_fd_leak/Makefile =================================================================== --- tools/regression/sockets/accept_fd_leak/Makefile (revision 208799) +++ tools/regression/sockets/accept_fd_leak/Makefile (working copy) @@ -5,7 +5,7 @@ PROG= accept_fd_leak NO_MAN= -regress: +test: ./accept_fd_leak .include Index: tools/regression/sockets/accf_data_attach/Makefile =================================================================== --- tools/regression/sockets/accf_data_attach/Makefile (revision 208799) +++ tools/regression/sockets/accf_data_attach/Makefile (working copy) @@ -5,7 +5,7 @@ PROG= accf_data_attach NO_MAN= -accf_data_attach: accf_data_attach.c +all: accf_data_attach.c gcc -Wall -o accf_data_attach accf_data_attach.c .include --Apple-Mail-678-995844450-- --Apple-Mail-679-995844465--