From owner-freebsd-testing@FreeBSD.ORG Tue Oct 15 02:44:54 2013 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BAC79185 for ; Tue, 15 Oct 2013 02:44:54 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-la0-f45.google.com (mail-la0-f45.google.com [209.85.215.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4724A290A for ; Tue, 15 Oct 2013 02:44:53 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id eh20so6343287lab.18 for ; Mon, 14 Oct 2013 19:44:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-type; bh=kSVimKYCohdsgBkr3QnV/9nGzL5y53onkpO7e6GEUI0=; b=PucwUP8Rr/pl1yEtFGE0qnjbpMpkGbRMb1XjhgI3N3/rVGuMAtUPAi3OIrNuO3Nmqb LgnI1cq2i6GJ4i4W62aIedUkgw2yjrtcJubXjKTEmjxArPM5x8iSi66stP8b6X4L6YLk HjB0dyzI/geFqwsjMBZeDFGRvOjBuJVOK1zQhPiXMV68QGRH7CKupeEcrpNkE1+Qq0IF OIcUWdSyJu8BaRWRYxupKL5OyD71cnk+m+hgMYHoLHJv2nXqBG4+LHrT2FQZsCUX3Cdc 4jjus90z8dUPbO8fHUIaSp5hTMC+XQFQ3LvBC8qgQX/bPSbusK528fzAZZx/epvzId5Q GsSQ== X-Gm-Message-State: ALoCoQlicn8FjBH78XJT+c5fpi1sCaz/cL3x2rfJICcIzrZLYRFqSGnxbSDhA/+ZNOic9IL2QI25 X-Received: by 10.112.146.200 with SMTP id te8mr4678174lbb.32.1381805085682; Mon, 14 Oct 2013 19:44:45 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.132.135 with HTTP; Mon, 14 Oct 2013 19:44:25 -0700 (PDT) X-Originating-IP: [108.176.158.82] From: Julio Merino Date: Mon, 14 Oct 2013 22:44:25 -0400 Message-ID: Subject: Refactoring of *.test.mk To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Rui Paulo , "yaneurabeya ." , Simon Gerraty X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 02:44:54 -0000 Hello all, [ CCing a few people directly that are probably interested. ] After the recent removal of the deprecated ATF tools from the tree, I have been working on a refactoring of the FreeBSD *.test.mk Makefiles to add various features that spilled from that change. The highlights of this work so far: - Clearly documented the interface of each *.test.mk file by putting the list of public variables at the top and adding comments where due. - Split the logic to build framework-less test programs into a new plain.test.mk file. This is in accordance with atf.test.mk, but also helps clarifying the code, defining a proper interfac in bsd.test.mk and makes some of the other changes simpler. - Added support to generate both Atffile and Kyuafile files. - Added support to run the tests using both atf-run and kyua (from ports). You can find the relevant patches in: http://portal.meroh.net/~jmmv/freebsd-testing/ . The patches have been generated using quilt so take a look at the contents of the 'series' file to see in which order they would need to be applied. Each patch has a description at the top which can later be used as the commit log. Note that, at the moment, this can only be exercised in lib/libcrypt/tests/ . I actually wanted to write some tests for *.test.mk themselves (!) but I wouldn't want to spend time on that before agreeing that this structure is sensible. If it is, I might put the effort ;-) See the ones here http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/share/mk/?only_with_tag=MAIN to get an idea of what they could look like. Anybody up for reviewing these patches and sharing their comments? Thanks! -- Julio Merino / @jmmv