From owner-freebsd-testing@FreeBSD.ORG Tue Mar 3 01:29:42 2015 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 248CF18E for ; Tue, 3 Mar 2015 01:29:42 +0000 (UTC) Received: from mail-yk0-f175.google.com (mail-yk0-f175.google.com [209.85.160.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D35AB82D for ; Tue, 3 Mar 2015 01:29:41 +0000 (UTC) Received: by yks20 with SMTP id 20so15217792yks.8 for ; Mon, 02 Mar 2015 17:29:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=d+uPB94hKCTRS1zSqLLWFhMDA2O0GafHSUANgvCFLTY=; b=lfd91KY48dA19I2n9qttkKSwalq3eQh4YI/AGoDp/oHWaoP8fhPuMKMYPMbYKoFKPd f7Dzp+bc2V0RbqwGA9PLk+uN3QkNJ8e5jXmuVingWfurTuJC3zJN7lB+vapp31FoW96g Hyvvmu71whPG9f890NU1u6gIY05Xc8bWg63V80aMPuO9iHbgBtP14FqE5AN8x6VQr4OA PD3eQXtHtWGzOh4m4mLky/EeQsim+NqOanZZvj2BBdu09wjw6eM6/lOOE1RjnjTrGsmS NrmVW/1GFYfvTS6XDFDHLw8PzMtOQUuGrAiSuR6PX20aL9XOP/EpLaLajpKKIUOOWgei q+og== X-Gm-Message-State: ALoCoQkD405XJMMVX64ezQ3sK8WHFUN6DMSFagXftahUjd2Semdq3hzvf2kktVhBWWJVkH9tkqNh X-Received: by 10.170.199.137 with SMTP id q131mr30163397yke.113.1425340039964; Mon, 02 Mar 2015 15:47:19 -0800 (PST) Received: from ?IPv6:2620::1003:1007:3182:e30f:78b:a627? ([2620:0:1003:1007:3182:e30f:78b:a627]) by mx.google.com with ESMTPSA id t28sm11253450yhg.27.2015.03.02.15.47.19 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 02 Mar 2015 15:47:19 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Running tests as a developer prior to commit From: Julio Merino In-Reply-To: Date: Mon, 2 Mar 2015 18:47:18 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Craig Rodrigues X-Mailer: Apple Mail (2.2070.6) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 01:29:42 -0000 On Feb 26, 2015, at 22:17, Craig Rodrigues wrote: >=20 > On Mon, Feb 23, 2015 at 7:20 PM, Ed Maste wrote: >=20 >>=20 >> This is precisely why I'd like to find a way to make it easy to run a >> (rather small) subset of tests, if a developer desires to do so. The >> alternative is that they decide it's too awkward to run the tests, = and >> just don't bother. Even in that case we're still better off today >> (since the Jenkins task will catch it) than we were some time ago >> before consistent test infrastructure existed. But it seems we ought >> to avoid the latency, mailing list churn, and masking effect of a >> broken test run if we can facilitate the developer running a set of >> tests that they've identified. >>=20 >=20 > If I could do: > cd /usr/src/bin/ls/ > make test >=20 > and have it run the unit tests associated with /bin/ls, that would be = a > good start. That already exists but is apparently broken as Garrett mentions. = Should be fixed. I may take a look if I have time. > It wouldn't be perfect, because if a change to /bin/ls, affects some = other > test > in /usr/src/usr.bin/foobar/ , this wouldn't catch it. You would have = to do > something > more complicated by building an expert system which can compute the > dependencies > in the tree and run the appropriate tests. But having that is better = than > nothing. Right, and this is why the existing test target outputs a fat warning = before startup. But as you say, having this working in some form would = be sufficient to validate most changes quickly. >=20 > As far as I understand things, the way the unit tests are integrated = in the > Makefile > infrastructure, the tests need to be staged to /usr/tests/ in order to = be > run. Not really. The programs _and_ their corresponding Kyuafile need to be = built, but once built, the majority of them can be run from the object = directory (be it separate from the source directory or not).=