From owner-freebsd-doc@FreeBSD.ORG Sat Dec 11 14:28:19 2004 Return-Path: Delivered-To: freebsd-doc@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 59E6916A4CE; Sat, 11 Dec 2004 14:28:19 +0000 (GMT) Received: from jc.ngo.org.uk (jc.ngo.org.uk [69.55.225.33]) by mx1.FreeBSD.org (Postfix) with ESMTP id A29F443D55; Sat, 11 Dec 2004 14:28:18 +0000 (GMT) (envelope-from nik@crf-consulting.co.uk) Received: from crf-consulting.co.uk (82-44-220-218.cable.ubr10.haye.blueyonder.co.uk [82.44.220.218]) by jc.ngo.org.uk (8.12.11/8.13.1) with ESMTP id iBBESEx6005240; Sat, 11 Dec 2004 06:28:14 -0800 (PST) (envelope-from nik@crf-consulting.co.uk) Received: from clan.nothing-going-on.org (clan.nothing-going-on.org [192.168.1.20]) by crf-consulting.co.uk (8.13.1/8.12.9) with ESMTP id iBBESDjq084799; Sat, 11 Dec 2004 14:28:13 GMT (envelope-from nik@catkin) Received: from clan.nothing-going-on.org (localhost [127.0.0.1]) iBBESCku087814; Sat, 11 Dec 2004 14:28:13 GMT (envelope-from nik@clan.nothing-going-on.org) Received: (from nik@localhost) by clan.nothing-going-on.org (8.13.1/8.13.1/Submit) id iBBESCBa087813; Sat, 11 Dec 2004 14:28:12 GMT (envelope-from nik) Date: Sat, 11 Dec 2004 14:28:11 +0000 From: Nik Clayton To: Giorgos Keramidas Message-ID: <20041211142811.GB34046@clan.nothing-going-on.org> References: <20041207221810.GC79404@clan.nothing-going-on.org> <20041210035853.GA38277@gothmog.gr> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2B/JsCI69OhZNC5r" Content-Disposition: inline In-Reply-To: <20041210035853.GA38277@gothmog.gr> User-Agent: Mutt/1.4.2.1i Organization: FreeBSD Project cc: doc@freebsd.org cc: Nik Clayton Subject: Re: Testing the build infrastructure X-BeenThere: freebsd-doc@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Documentation project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Dec 2004 14:28:19 -0000 --2B/JsCI69OhZNC5r Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 10, 2004 at 05:58:53AM +0200, Giorgos Keramidas wrote: > On 2004-12-07 22:18, Nik Clayton wrote: > > Guys, > > Given the comments that have been bandied about recently relating to > > breakage of various parts of the build infrastructure, and functionality > > that used to work no longer working, I'd like to make some concrete > > progress towards fixing things. [...] > > > > Thoughts? >=20 > That looks like a good plan :) >=20 > Any details on how the tests should be written? Can we (re)use the > style of the tests of src/ at all? Ideally that's exactly what we'll do. I'm a big fan of prove(1), which will run tests and generate a report of which succeeded, which failed, and so on. The output format that tests need to generate is listed in src/tools/regression/README. It's easy to write tests in Perl and /bin/sh, and I'm working on a C implementation. I strongly suspect that most of our tests will be written in /bin/sh -- a lot of them will be: * Did this file get created? * Does the output match the canonical output which we've stored in /this/file? * Was the exit code the one we expected? For example, I don't (because I've not looked in to it in any detail) understand exactly what's broken in the obj/ building case. But with tests that describe how things are supposed to work I don't need to. All I need to do (prior to making a commit) is run the tests. As long as they all pass everything's fine. The only extra work (after this is in place) is when we make infrastructure changes. There need to be corresponding tests in place. So, when I committed the .sgml -> .xml conversion code, if I'd committed a test for it as well, we'd know exactly when it broke, and why. N --=20 FreeBSD: The Power to Serve http://www.freebsd.org/ (__) FreeBSD Documentation Project http://www.freebsd.org/docproj/ \\\'',) \/ \= ^ --- 15B8 3FFC DDB4 34B0 AA5F 94B7 93A8 0764 2C37 E375 --- .\._/= _) --2B/JsCI69OhZNC5r Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFBuwP6k6gHZCw343URAmZXAJ9QL2PBVcx307zPx0XfXJBt7eOBNgCghgYv r3KjNAY9p+XETC9ob5DtUsc= =9MXw -----END PGP SIGNATURE----- --2B/JsCI69OhZNC5r--