From owner-freebsd-testing@freebsd.org Sun Oct 18 02:21:55 2015 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A5EBA10E84 for ; Sun, 18 Oct 2015 02:21:55 +0000 (UTC) (envelope-from jmmv@meroh.net) Received: from mail-qk0-f181.google.com (mail-qk0-f181.google.com [209.85.220.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3BCF4201 for ; Sun, 18 Oct 2015 02:21:54 +0000 (UTC) (envelope-from jmmv@meroh.net) Received: by qkcy65 with SMTP id y65so1947490qkc.0 for ; Sat, 17 Oct 2015 19:21:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=UEEu+hBbCjO6PWvqgaDtaP7J7dkJ6KKGoOHHhUM7BcI=; b=d5cuZx2J8XBlD62/0+6OWfwe2DZqb4rwKn/zzbE4happgb533ahebQ0BOwvchE/OCS gZF4SEYcUaFlKcIGMvEgkevYO3eBnRDUBKWdX608BafEdC4qzh07ZABCwGOixm0KGeo3 kBb939ISLnnWxR54Zzyu7SwTo/YcVlPIfubdIt/Gn/f3C6oG2fP/2vxd3MgWzMgBytkX ok5HiE6jUifHQUnrt6yNRyiIkcLXHQlPy494dn5njuQrBnXni6UxLHgm/G+aZTe8Gd5k XNvVQu4kIPypsAySq0SCXYw6Uc46ELa490Gs77reN8cgIQBpgi89zaMBilBfFaM6MjaS 979w== X-Gm-Message-State: ALoCoQmDcFFtBy+DrsezihDXxCigB4YjjEqdfWy9oLlT+xa+59HEEA3L51lBJ7oAHkPUVaUaSlgl X-Received: by 10.55.200.71 with SMTP id c68mr27964476qkj.72.1445134912997; Sat, 17 Oct 2015 19:21:52 -0700 (PDT) Received: from ?IPv6:2604:2000:c667:7600:4815:f07:b75a:45a7? ([2604:2000:c667:7600:4815:f07:b75a:45a7]) by smtp.gmail.com with ESMTPSA id l91sm11364651qkh.46.2015.10.17.19.21.52 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 17 Oct 2015 19:21:52 -0700 (PDT) Sender: Julio Merino Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.0 \(3094\)) Subject: Re: [RFC] Rename `make test` in suite.test.mk with `make regress` From: Julio Merino In-Reply-To: Date: Sat, 17 Oct 2015 22:21:49 -0400 Cc: "freebsd-testing@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <5C39F495-5717-450F-8460-14B69DD48AD6@freebsd.org> To: NGie Cooper X-Mailer: Apple Mail (2.3094) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2015 02:21:55 -0000 > On Oct 17, 2015, at 21:56, NGie Cooper wrote: >=20 >>=20 >> On Oct 17, 2015, at 18:50, Julio Merino wrote: >>=20 >> On Oct 17, 2015, at 18:03, NGie Cooper wrote: >>>=20 >>> Hi all, >>> There=E2=80=99s a lesser known target in suite.test.mk that runs = `kyua test` in a similar manner to how Jenkins and other groups have = integrated kyua into their test infrastructures. >>> The legacy target on FreeBSD was `regress`, but the target = created with the bsd.test.mk creation back a few years ago was `test`. = Why change from `test` to `regress`? There are places in the tree = (bin/test for example) that have targets named test, so in order to = avoid clashing with a common target (name), it=E2=80=99s best to use the = legacy target name. >>> Would anyone have any serious heartburn over the change? >>=20 >> Is this only because of bin/test? Seems like renaming a target to = avoid that one collision is just moving the problem around. It is = possible that some other directory could later grow a target that = conflicts with your new name. >=20 > Yes, I realize that. The goal though is I want to be able to call = `make ` from the top and it would iterate down each and every = subdirectory and run tests. Other for the bin/test naming collision, that should work already, = shouldn't it? Not sure how ugly this would be, but you'd also define an internal = "run-tests" target or similar that is used exclusively by the recursion = code. Then, in the top-level Makefile, you define a bare "test" target = that starts the recursion so that "make test" works. And then, in = bsd.test.mk, you define a "test" target if - and only if - there is no = "test" target yet. bin/test would behave differently than the rest if = you ever went in there and ran "make test", but I think that's a minor = problem. This is offtopic, but do you have a plan anywhere on how to make the = tests invocation this way reliable? Without installed binaries, the = tests might not do the right thing, and I'm not sure if the object tree = is enough for this to work properly. That's the reason why "make test" = from the top-level directory is currently disallowed. > Sadly, most people don=E2=80=99t care to figure out how kyua works = enough to run `kyua test`=E2=80=A6 so I=E2=80=99m just trying to lower = the barrier of entry in a way that will work 100% of the time. >=20 >> Strictly speaking, "regress" is wrong. We do not have regression = tests only. Also, "regress" is a pretty obscure name for a target; it = does not appear in any other projects nor in any other build systems = that I know of. >=20 > Unfortunately it=E2=80=99s been in place a lot longer than anything = else, so it=E2=80=99s kind of the =E2=80=9Cdefacto standard=E2=80=9D. = That said... I think I see what you mean: that because "regress" already existed in = the FreeBSD tree, no other targets were introduced to collide with it? = If yes, I wouldn't expect that to be the case. Tests were not = interleaved with the source tree in the past, so the "regress" target = only existed in a subset of the tree, didn't it? >> Have you considered "check"? That'd be in line with what automake = does, for example, which would homogenize the target name with a ton of = other projects out there. >=20 > =E2=80=A6 `make check` would be ok too. I just have to comb the tree = for binaries that don=E2=80=99t match `check`. >=20 > Thanks! > -NGie