From owner-freebsd-testing@FreeBSD.ORG Sun Oct 26 23:00:28 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D8E55F6; Sun, 26 Oct 2014 23:00:28 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id B955F1F9; Sun, 26 Oct 2014 23:00:27 +0000 (UTC) Received: from [100.104.137.101] (167.sub-70-197-7.myvzw.com [70.197.7.167]) by elvis.mu.org (Postfix) with ESMTPSA id B0A65341F83D; Sun, 26 Oct 2014 16:00:21 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Panicking UUTs and integrating the ZFS test suite into the default run From: Alfred Perlstein X-Mailer: iPhone Mail (12B411) In-Reply-To: <86E89F83-179D-4301-9EC0-0ECC3909B924@gmail.com> Date: Sun, 26 Oct 2014 16:00:19 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20141024053636.GH11222@dft-labs.eu> <81030948-E60F-4AAD-AAF1-16349607917D@gmail.com> <544B46BA.4000008@freebsd.org> <86E89F83-179D-4301-9EC0-0ECC3909B924@gmail.com> To: Garrett Cooper Cc: "freebsd-testing@freebsd.org" , Alfred Perlstein X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Oct 2014 23:00:28 -0000 > On Oct 25, 2014, at 11:59 AM, Garrett Cooper wrote= : >=20 > (moving to -testing and renaming because this is turning into a bikeshed d= iscussion) I believe this is a strong case to illustrate the axiom "he who smelt it, de= alt it." -Alfred.=20= From owner-freebsd-testing@FreeBSD.ORG Mon Oct 27 15:10:40 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61386E86 for ; Mon, 27 Oct 2014 15:10:40 +0000 (UTC) Received: from mail-qa0-f53.google.com (mail-qa0-f53.google.com [209.85.216.53]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1ADA2289 for ; Mon, 27 Oct 2014 15:10:39 +0000 (UTC) Received: by mail-qa0-f53.google.com with SMTP id n8so1571961qaq.40 for ; Mon, 27 Oct 2014 08:10:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=TBl6wMCDbY3weub38DtEmRd0Rf1UxjK1xrapWDvlc9M=; b=KPyIZYeCEwh5jXhyDAwV0JSPdY+LvJfqDy2aWQk0Rk+k+Imc2UyUNNXK3VNwYvX0Eo M9F6YWxQVAxCiaU2kMBNJtWQc4KsG9/EFce8n/Vot+4JoZixqglsd0El3NIt0Rm8WHD8 zyqHXTAaI1rnp/52ywUFMWQchDo0pFKWfRpdIKd16EFVcPGYDZn9foOmRKWe4M2MPUPC aN2NnTConNYfqQLU1DQD5Ld1Fe9WuEhQ5jx5KxTg785esOxaZmshU9N++I9oHfKtF55V OJn7LFcNzfbuHc7D/l8Pgo+GKd3of2Uiht54f71vsZKcBP+Oz5J6HBpN+YTp8GqPXWjR T0JQ== X-Gm-Message-State: ALoCoQnJmdq1HwuOjOjd+562FXYmXRa9qKwsJECh489DI17yJrSJnrG5ptePCfznCu8Dv/Tny5Zo X-Received: by 10.224.61.7 with SMTP id r7mr34364552qah.9.1414422298155; Mon, 27 Oct 2014 08:04:58 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.66.69 with HTTP; Mon, 27 Oct 2014 08:04:37 -0700 (PDT) X-Originating-IP: [81.184.188.119] In-Reply-To: <86E89F83-179D-4301-9EC0-0ECC3909B924@gmail.com> References: <20141024053636.GH11222@dft-labs.eu> <81030948-E60F-4AAD-AAF1-16349607917D@gmail.com> <544B46BA.4000008@freebsd.org> <86E89F83-179D-4301-9EC0-0ECC3909B924@gmail.com> From: Julio Merino Date: Mon, 27 Oct 2014 16:04:37 +0100 X-Google-Sender-Auth: q2XxFKlkoMlXxB2tFCBTHuVOrFg Message-ID: Subject: Re: Panicking UUTs and integrating the ZFS test suite into the default run To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , Alfred Perlstein X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2014 15:10:40 -0000 On Sat, Oct 25, 2014 at 8:59 PM, Garrett Cooper wrote: > (moving to -testing and renaming because this is turning into a bikeshed discussion) > > On Oct 25, 2014, at 9:49, Craig Rodrigues wrote: > >> On Fri, Oct 24, 2014 at 11:44 PM, Alfred Perlstein wrote: > > 6 > 7 [ "${os}" = "FreeBSD" ] && die "panics FreeBSD; see bug # 194589" > 8 > > 1. The issue is documented, so developers and testers know where to get the bug information > 2. The test is marked as a failure. > > Both of these items encourage testers and other developers to go prod developers with knowledge and cycles to go fix the bugs so things eventually turn green. That's bad as well. Marking tests as broken trains developers to ignore failures and to be more lax in tolerating breakage, and thus makes the signal from the test suite useless. Developers must be able to trust the results of the test suite, and to understand that a flip from green to red is really bad. Thinking "Hey, the build has been red like, forever, so one more broken test from me is irrelevant!" is quite common. These tests must be kept green if they are for known problems. A way to do so is with "expected failures", which I think you can denote in TAP with the "TODO" keyword and with expect_fail in ATF. Regarding the panics: making the box panic every time the test runs is probably not a great idea... but this is a shortcoming caused by our inability to properly test the kernel in an isolated environment. (In NetBSD, we'd use rump for this and thus the machine would never panic.) From owner-freebsd-testing@FreeBSD.ORG Mon Oct 27 21:51:00 2014 Return-Path: Delivered-To: freebsd-testing@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06E7AE5D; Mon, 27 Oct 2014 21:50:59 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A2D409C0; Mon, 27 Oct 2014 21:50:59 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s9RLowte075571 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 27 Oct 2014 14:50:58 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s9RLowHq075570; Mon, 27 Oct 2014 14:50:58 -0700 (PDT) (envelope-from jmg) Date: Mon, 27 Oct 2014 14:50:58 -0700 From: John-Mark Gurney To: freebsd-testing@FreeBSD.org Subject: issues w/ installing stuff multiple times... Message-ID: <20141027215058.GN82214@funkthat.com> Mail-Followup-To: freebsd-testing@FreeBSD.org, current@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 27 Oct 2014 14:50:58 -0700 (PDT) Cc: current@FreeBSD.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2014 21:51:00 -0000 There are issues w/ installing tests where the test files get installed multiple times. To reproduce this, use the following steps: make installworld -j 8 DESTDIR= -DNO_ROOT Once you have done the above, in there will be the file METALOG, run: grep -v type=dir /METALOG | awk '{ print $1 }' | sort | uniq -d This will print out the current list if files that get installed multiple times.... Currently, it looks like all the tests subdirs are installed a second time... Could someone look at making it so that they don't get installed multiple times? Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-testing@FreeBSD.ORG Mon Oct 27 21:55:11 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8FC51161; Mon, 27 Oct 2014 21:55:11 +0000 (UTC) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [IPv6:2607:f8b0:400e:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F5ADA84; Mon, 27 Oct 2014 21:55:11 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id eu11so6317086pac.30 for ; Mon, 27 Oct 2014 14:55:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=UoSs3NeILJoGF31mXN0WJtxfAeuBVf20wkGVM/KPKMY=; b=jIugk8KTMjkPruL8e3ps3A96jJehuiXpGlEAnHhuCDh5Z5a32ucxi3a6BiVCANdowg LIY+WfMioIf3x23nOT8m4kWKgZOsm1ZLTiKA2JxQuTEyLx0vkb+cV/X7NdqLcheomp8I BUZXQAOSkxx0Swz5crs5uQaAZk6eggCsWuvFoatSviwf06+AQ+e1Cz3tUC4Q9C+yAgSj jD8mkcmS9531agtXHQjih+AllcW/c5TrJn5k9aL0tIGovJ1bbkjDX2cQ42SNTXL1/UyH To4rlDb6eMuRVUaeLep4WfVLyl3Y4sxCUQg7xXvKtTU1A7DSL+Kbb+K8ap9tvqONMJBr VmJg== X-Received: by 10.66.228.35 with SMTP id sf3mr26682190pac.110.1414446910542; Mon, 27 Oct 2014 14:55:10 -0700 (PDT) Received: from [192.168.242.55] (c-67-182-131-225.hsd1.wa.comcast.net. [67.182.131.225]) by mx.google.com with ESMTPSA id x7sm11749639pdj.36.2014.10.27.14.55.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 27 Oct 2014 14:55:09 -0700 (PDT) References: <20141027215058.GN82214@funkthat.com> Mime-Version: 1.0 (1.0) In-Reply-To: <20141027215058.GN82214@funkthat.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (12B411) From: Garrett Cooper Subject: Re: issues w/ installing stuff multiple times... Date: Mon, 27 Oct 2014 14:55:09 -0700 To: John-Mark Gurney Cc: "freebsd-testing@FreeBSD.org" , "current@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2014 21:55:11 -0000 > On Oct 27, 2014, at 14:50, John-Mark Gurney wrote: >=20 > There are issues w/ installing tests where the test files get installed > multiple times. >=20 > To reproduce this, use the following steps: > make installworld -j 8 DESTDIR=3D -DNO_ROOT >=20 > Once you have done the above, in there will be the file > METALOG, run: > grep -v type=3Ddir /METALOG | awk '{ print $1 }' | sort | un= iq -d >=20 > This will print out the current list if files that get installed multiple > times.... >=20 > Currently, it looks like all the tests subdirs are installed a second > time... >=20 > Could someone look at making it so that they don't get installed > multiple times? Hi jmg! I have a patch out for this that I need to commit today. Thank you for t= he reminder. Cheers!= From owner-freebsd-testing@FreeBSD.ORG Mon Oct 27 22:10:49 2014 Return-Path: Delivered-To: freebsd-testing@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2AE70593; Mon, 27 Oct 2014 22:10:49 +0000 (UTC) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) by mx1.freebsd.org (Postfix) with ESMTP id 04751BC7; Mon, 27 Oct 2014 22:10:48 +0000 (UTC) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 5BEE15A9F29; Mon, 27 Oct 2014 22:10:48 +0000 (UTC) Date: Mon, 27 Oct 2014 22:10:48 +0000 From: Brooks Davis To: Garrett Cooper Subject: Re: issues w/ installing stuff multiple times... Message-ID: <20141027221048.GF59119@spindle.one-eyed-alien.net> References: <20141027215058.GN82214@funkthat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="11Y7aswkeuHtSBEs" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@FreeBSD.org" , John-Mark Gurney , "current@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2014 22:10:49 -0000 --11Y7aswkeuHtSBEs Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 27, 2014 at 02:55:09PM -0700, Garrett Cooper wrote: >=20 > > On Oct 27, 2014, at 14:50, John-Mark Gurney wrote: > >=20 > > There are issues w/ installing tests where the test files get installed > > multiple times. > >=20 > > To reproduce this, use the following steps: > > make installworld -j 8 DESTDIR=3D -DNO_ROOT > >=20 > > Once you have done the above, in there will be the file > > METALOG, run: > > grep -v type=3Ddir /METALOG | awk '{ print $1 }' | sort |= uniq -d > >=20 > > This will print out the current list if files that get installed multip= le > > times.... > >=20 > > Currently, it looks like all the tests subdirs are installed a second > > time... > >=20 > > Could someone look at making it so that they don't get installed > > multiple times? >=20 > Hi jmg! > I have a patch out for this that I need to commit today. Thank you fo= r the reminder. Great to hear this will be fixed. Once we've fixed them all, it would be really good to have a test in Jenkins looking out for new duplicate files since they are always bugs. -- Brooks --11Y7aswkeuHtSBEs Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlROwucACgkQXY6L6fI4GtSsTQCg1n09XFf1wBXxJbfcQkidIsGm YL4An07FfBIdexR14oYGbG/YPU2mzRbX =CFgn -----END PGP SIGNATURE----- --11Y7aswkeuHtSBEs-- From owner-freebsd-testing@FreeBSD.ORG Mon Oct 27 22:47:34 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F273A337; Mon, 27 Oct 2014 22:47:33 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B32CBF90; Mon, 27 Oct 2014 22:47:33 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s9RMlW16076294 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 27 Oct 2014 15:47:32 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s9RMlWXQ076293; Mon, 27 Oct 2014 15:47:32 -0700 (PDT) (envelope-from jmg) Date: Mon, 27 Oct 2014 15:47:32 -0700 From: John-Mark Gurney To: Brooks Davis Subject: Re: issues w/ installing stuff multiple times... Message-ID: <20141027224732.GP82214@funkthat.com> Mail-Followup-To: Brooks Davis , Garrett Cooper , "freebsd-testing@FreeBSD.org" , "current@FreeBSD.org" References: <20141027215058.GN82214@funkthat.com> <20141027221048.GF59119@spindle.one-eyed-alien.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20141027221048.GF59119@spindle.one-eyed-alien.net> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 27 Oct 2014 15:47:32 -0700 (PDT) Cc: "freebsd-testing@FreeBSD.org" , "current@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2014 22:47:34 -0000 Brooks Davis wrote this message on Mon, Oct 27, 2014 at 22:10 +0000: > On Mon, Oct 27, 2014 at 02:55:09PM -0700, Garrett Cooper wrote: > > > > > On Oct 27, 2014, at 14:50, John-Mark Gurney wrote: > > > > > > There are issues w/ installing tests where the test files get installed > > > multiple times. > > > > > > To reproduce this, use the following steps: > > > make installworld -j 8 DESTDIR= -DNO_ROOT > > > > > > Once you have done the above, in there will be the file > > > METALOG, run: > > > grep -v type=dir /METALOG | awk '{ print $1 }' | sort | uniq -d > > > > > > This will print out the current list if files that get installed multiple > > > times.... > > > > > > Currently, it looks like all the tests subdirs are installed a second > > > time... > > > > > > Could someone look at making it so that they don't get installed > > > multiple times? > > > > Hi jmg! > > I have a patch out for this that I need to commit today. Thank you for the reminder. > > Great to hear this will be fixed. Once we've fixed them all, it would be > really good to have a test in Jenkins looking out for new duplicate files > since they are always bugs. I agree.... I've given the meat of the test above... Any output from that pipeline means that there is a bug... Also, it'd be good if Jenkins would do an installworld in addition to a buildworld... We've recently had breakages that are only uncovered when installworld is done... Current items (other than tests) installed multiple times: /etc/mail/mailer.conf /usr/libdata/pkgconfig/* various man pages: nvlist_freef, various loader So, shouldn't be too much work to knock the rest out... If someone adds the jenkins test, I'll fix the remaining issues (others are free to fix these too)... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-testing@FreeBSD.ORG Mon Oct 27 23:43:03 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F469138; Mon, 27 Oct 2014 23:43:03 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E90C839; Mon, 27 Oct 2014 23:43:02 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id f15so3035039lbj.23 for ; Mon, 27 Oct 2014 16:42:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jLxlJImGK/mdvrc4s9ZJgf2hzd0Z2XnjfAWa86+8krc=; b=gIPoI9aFxNzvmlJVRB6hHK2bTw7kF0QKYkB0K7f5c/NpOG9KNvsvdfju45yMabQhtf iKCxTd4d+KqYWQjSuS2fDufNU0mvb1/DQW4DOWKhd5wr33Q795leBGPSCiEAcT1ZsMKe sGmsklN0QdIzpsXpOVBAIb28qfT3MhofkzjUqZw8GV5zZWSJzIIOpM++4u1M0EH89j6e gov2o2ftHn8ZQInXE+OzDoNybkbmnaeSq81098ZnCrOuh+oK+juryxFiUl6k0BdAeuod BkLozhE+lxO9B2y/SS5dYlrvOqV5X3b+ovoLOJALQ2ilWWteB8OGlhMyr7cZFSbOSSRL sPFQ== MIME-Version: 1.0 X-Received: by 10.112.225.225 with SMTP id rn1mr6640441lbc.98.1414453379125; Mon, 27 Oct 2014 16:42:59 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.84.197 with HTTP; Mon, 27 Oct 2014 16:42:58 -0700 (PDT) In-Reply-To: <886660362.1106176.1414183052149.JavaMail.zimbra@xinuos.com> References: <5449C81B.8080008@freebsd.org> <5449DB7C.9080107@xinuos.com> <06ec01cfef62$3776a6b0$a663f410$@Xinuos.com> <886660362.1106176.1414183052149.JavaMail.zimbra@xinuos.com> Date: Mon, 27 Oct 2014 16:42:58 -0700 X-Google-Sender-Auth: on84ll1GpO2p1hbmo-9AeDvZW3E Message-ID: Subject: Re: Automatically running /usr/tests on stable/10 branch under Jenkins From: Craig Rodrigues To: Susan Stanziano Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: John Wolfe , "freebsd-virtualization@freebsd.org" , "freebsd-testing@freebsd.org" , Eric@xinuos.com, Allan Jude , Cheryl Blain , Andy Nagle X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2014 23:43:03 -0000 Susan, In addition to the slides which I previously mentioned, you and your colleagues at Xinuos may wish to take a quick read of this document which I wrote, the "Kyua Quickstart Guide": https://github.com/rodrigc/kyua/wiki/Quickstart-Guide kyua is the test harness that is used to run the tests in the /usr/tests directory in FreeBSD. That doc is a very quick guide on how to run tests under kyua, and also how to write tests which can be run under kyua. Getting any feedback on this would be very useful. Does that doc make sense to a newcomer? Are there typos? I want the FreeBSD project to get to the point where: (1) anybody can download the FreeBSD tests (2) anybody can run the tests (3) people can easily figure out how to write new tests and contribute them back (if they wish) to FreeBSD Better documentation will hopefully help with this. Thanks. -- Craig On Fri, Oct 24, 2014 at 1:37 PM, Susan Stanziano wrote: > Hello Craig, > > I have been reading through your slides and your website announcement, as > requested, and have a few questions. (I have also subscribed to the > mailing list as suggested). > > (1) The slides indicate that with a FreeBSD10 and lower OS, I should be > able to build tests already included with FreeBSD. I have created a VM on > FreeBSD10.0 but do not have the /etc/make.conf file necessary to build and > install the tests. This may be that I am missing some set-up steps > required but not covered in the slide presentation. > > (2) Does the test build require the use of the "Build Flow Plugin" or can > the tests be run independent of the "continuous build" model? > > (3) Can the tests only be run under Bhyve as described in the announcement? > > I would be happy to exchange emails with you or skype as I investigate > this work further. > > Regards, > > Susan > > ------------------------------ > *From: *"Craig Rodrigues" > *To: *Eric@xinuos.com > *Cc: *"Allan Jude" , "John Wolfe" , > freebsd-virtualization@freebsd.org, freebsd-testing@freebsd.org, "Susan > Stanziano" , "Cheryl Blain" < > cheryl.blain@xinuos.com>, "Andy Nagle" > *Sent: *Friday, October 24, 2014 7:09:04 AM > *Subject: *Re: Re: Automatically running /usr/tests on stable/10 branch > under Jenkins > > > > > On Fri, Oct 24, 2014 at 1:11 AM, Eric le Blan wrote: > >> Allan, >> >> >> >> Always happy to discuss testing. I believe that Susan Stanziono of Xinuos >> system test team already spoke to some of you. Please do not hesitate to >> reach her for any specific needs. Also, Cheryl Blain - Xinuos VP of >> operations - will be at the MeetBSD vendor summit on Nov 3-4 and can reach >> to Craig. > > > > It would be helpful if someone (or some people) from Xinuous could do the > following: > > (1) Subscribe to freebsd-testing@freebsd.org at > https://lists.freebsd.org/mailman/listinfo/freebsd-testing to start > monitoring the discussions on that list. > > (2) Review this announcement: > http://jenkins-ci.org/content/freebsd-project-use-jenkins-os-testing > > (3) Review these slides: > http://www.slideshare.net/CraigRodrigues1/kyua-jenkins > > (4) Talk to me at MeetBSD Vendor Summit > > I have not been given a long timeslot to give a presentation, so reviewing > those links before MeetBSD would > be helpful. Long term, any testing of FreeBSD will only succeed if there > is > an ecosystem of people working together and sharing common scripts and > information for testing FreeBSD. > > Any help that Xinuous (or any other company) can provide would be welcome. > > -- > Craig > > From owner-freebsd-testing@FreeBSD.ORG Wed Oct 29 03:01:21 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 453AB7DD for ; Wed, 29 Oct 2014 03:01:21 +0000 (UTC) Received: from mail-yh0-x22f.google.com (mail-yh0-x22f.google.com [IPv6:2607:f8b0:4002:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 024A5F5D for ; Wed, 29 Oct 2014 03:01:21 +0000 (UTC) Received: by mail-yh0-f47.google.com with SMTP id i57so161847yha.34 for ; Tue, 28 Oct 2014 20:01:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=CcHcihn7O2j2J0VcgcA5PQdMQzlU4xm5fnnGabxVBUw=; b=QW4wYN+ihR7ZbxBqr4iH2dJJvvvGM0du4U2RqYNmETaZv1F7lZp9QUV7++gcvvOgDv JpKVsu3zcZNqAoQqItOZUjDL8EPURaBM3f5wMf6tudOGjBlLj0Bm7yg1KkVzHPu/SiPk ETUAWBV3ZTOUWKros2/aCNc2TjOh7Br0yDAOa+xbumou4/i5LGz3UQtnj8klFFxlNdcp fPPj2R7NLirHq0vt88pg6rwCOPWzo4+dkVWIm6S1xZH4+g22XRtXYSYbToKioLMvgEan 0a8umCFtUg8/COhZTPLoWx9MPVN9iH722Tyth4aSPd7k8tCPECRhQQLYRxVRv98hLVpc T6vw== MIME-Version: 1.0 X-Received: by 10.170.159.2 with SMTP id a2mr8576643ykd.82.1414551680212; Tue, 28 Oct 2014 20:01:20 -0700 (PDT) Received: by 10.170.118.21 with HTTP; Tue, 28 Oct 2014 20:01:20 -0700 (PDT) Date: Tue, 28 Oct 2014 20:01:20 -0700 Message-ID: Subject: FreeBSD Testing Facility From: Mehmet Erol Sanliturk To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Oct 2014 03:01:21 -0000 Dear Craig , At the past , there was the following thread : https://lists.freebsd.org/pipermail/freebsd-current/2013-February/040102.ht= ml FreeBSD Testing Facility When I read your messages , I am remembering this thread which I consider it being forgotten . Now it may be possible to consider such a distributed testing community . In these days , a very good thread is continuing by Jean-S=C3=A9bastien P= =C3=A9dron : https://lists.freebsd.org/pipermail/freebsd-ports/2014-October/096161.html CFT: Update to xf86-video-ati 7.5.0 with very easily understandable and applicable steps explained to the user = . By using such mail threads or other sources it may be possible to obtain a good collection of tests and algorithms to apply them . Assuming that the above my message may be a useful reminder of issues , and with my best wishes for you , Mehmet Erol Sanliturk From owner-freebsd-testing@FreeBSD.ORG Wed Oct 29 03:47:47 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 789053F6 for ; Wed, 29 Oct 2014 03:47:47 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFEEC74C for ; Wed, 29 Oct 2014 03:47:46 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id f15so1968686lbj.9 for ; Tue, 28 Oct 2014 20:47:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=a3VxPHmw/DLXit3+k3wW7DGKkp0mElE+/yvdbXWoCWc=; b=oUI9uFgY+o5My6d07MUR89BNNTLQCMDvpaO2cu9nT8+hdY+1/sq1PW0erkSmoQFmTx e1vfo2GDMDtBRl8iLNiUhwSugBfXFqbIgLEg5NVubqEMvUzfxznVIKCqxLyt7x3m8Xdn M+YBPpiU7q+/Qao+nI4hKvOzjDyZfARNelzznUNv5xOVbqkHaqwpVXt30seNQmaSoXC0 s3CFlb9V+iUy5dkbE7DYVxQwt5Xnuwegjh+Wbp+JoD8EnU89bhDto6YzBj4C/kkM6hVi MhbNEZ9i9yxsj8KwQywP5qkdXco6rX1lLWz6mOrJmzc8/5EoS/4x5Fj8vzywsruQsm/B EnZQ== MIME-Version: 1.0 X-Received: by 10.152.203.231 with SMTP id kt7mr8311390lac.84.1414554461067; Tue, 28 Oct 2014 20:47:41 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.84.197 with HTTP; Tue, 28 Oct 2014 20:47:41 -0700 (PDT) In-Reply-To: References: Date: Tue, 28 Oct 2014 20:47:41 -0700 X-Google-Sender-Auth: p-tXQPUjwag_qjsmhADA14CL8Z8 Message-ID: Subject: Re: FreeBSD Testing Facility From: Craig Rodrigues To: Mehmet Erol Sanliturk Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Oct 2014 03:47:47 -0000 On Tue, Oct 28, 2014 at 8:01 PM, Mehmet Erol Sanliturk < m.e.sanliturk@gmail.com> wrote: > > Dear Craig , > > > At the past , there was the following thread : > > > https://lists.freebsd.org/pipermail/freebsd-current/2013-February/040102.html Thanks, Mehmet. I do not remember reading that posting on freebsd-current. I invite you to look at the presentations I have given over the past year: https://wiki.freebsd.org/Jenkins/#Presentations_and_Working_Groups http://jenkins-ci.org/content/freebsd-project-use-jenkins-os-testing If you look at those presentations, you will see that we are accomplishing many of the goals listed in the posting that you mentioned. That is because we are automatically running the tests in the /usr/tests directory in FreeBSD every day, and providing the results of the tests on the web. Over time, hopefully we will be able to get more people to help write tests and expand the testing infrastructure for FreeBSD. Thanks. -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Oct 29 04:39:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7A12D35; Wed, 29 Oct 2014 04:39:18 +0000 (UTC) Received: from mail-pd0-x235.google.com (mail-pd0-x235.google.com [IPv6:2607:f8b0:400e:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BF27B81; Wed, 29 Oct 2014 04:39:18 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id y10so2172692pdj.40 for ; Tue, 28 Oct 2014 21:39:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=LnzvuLvFYcPRrpWob5qzTNv5D1CmiRPmzlaGweo+VzQ=; b=olYpVqS0slq3+KiF7yx6GSf3cesQPcT9ctlTWQ+UAs0ojtmryVIG5JCYZOIVCT+q1v 3lCvldpuK3jzgX5zUtwHBbJAD04teKTkqawU2l/XPz8HiET0drI5025M4RdJVM6iT5JA 1MY9+S7p5cRGZGg16eDRHoFJMRr9aFkl5qxVrgumF5Wyzb/E3f82yPyy55FdEAU0hwyL TCWZB3Ir/p8EFpKirmWzrVm0jzIJVLpDmm6onqBfvP69V5yB9RnJn7YPfm7xu+IWwqH7 FnR09Cz6bBU3nvb/bnd2ap+1G2IVzQ2rO87ftj6U1T3BVK59/hN3FDDrl04cjkwvJ3df 45CQ== X-Received: by 10.70.42.15 with SMTP id j15mr8052198pdl.17.1414557558134; Tue, 28 Oct 2014 21:39:18 -0700 (PDT) Received: from [192.168.242.58] (c-67-182-131-225.hsd1.wa.comcast.net. [67.182.131.225]) by mx.google.com with ESMTPSA id ra4sm3047509pab.33.2014.10.28.21.39.16 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 28 Oct 2014 21:39:17 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_6C4B31FC-B77F-4AA8-91F3-794F23D2AC67"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: issues w/ installing stuff multiple times... From: Garrett Cooper In-Reply-To: <20141027221048.GF59119@spindle.one-eyed-alien.net> Date: Tue, 28 Oct 2014 21:39:14 -0700 Message-Id: <80D41F31-3CF7-4537-8D72-30A0324F0F59@gmail.com> References: <20141027215058.GN82214@funkthat.com> <20141027221048.GF59119@spindle.one-eyed-alien.net> To: Brooks Davis X-Mailer: Apple Mail (2.1878.6) Cc: "freebsd-testing@FreeBSD.org" , John-Mark Gurney , "current@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Oct 2014 04:39:19 -0000 --Apple-Mail=_6C4B31FC-B77F-4AA8-91F3-794F23D2AC67 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Oct 27, 2014, at 15:10, Brooks Davis wrote: > On Mon, Oct 27, 2014 at 02:55:09PM -0700, Garrett Cooper wrote: >>=20 >>> On Oct 27, 2014, at 14:50, John-Mark Gurney = wrote: >>>=20 >>> There are issues w/ installing tests where the test files get = installed >>> multiple times. >>>=20 >>> To reproduce this, use the following steps: >>> make installworld -j 8 DESTDIR=3D -DNO_ROOT >>>=20 >>> Once you have done the above, in there will be the = file >>> METALOG, run: >>> grep -v type=3Ddir /METALOG | awk '{ print $1 }' | = sort | uniq -d >>>=20 >>> This will print out the current list if files that get installed = multiple >>> times.... >>>=20 >>> Currently, it looks like all the tests subdirs are installed a = second >>> time... >>>=20 >>> Could someone look at making it so that they don't get installed >>> multiple times? >>=20 >> Hi jmg! >> I have a patch out for this that I need to commit today. Thank you = for the reminder. >=20 > Great to hear this will be fixed. Once we've fixed them all, it would = be > really good to have a test in Jenkins looking out for new duplicate = files > since they are always bugs. I got swamped at $work yet again. I need to do a bit more testing for = the patch, but I=92ll try to get it out this week after I=92ve done the = necessary testing. Thank you, --Apple-Mail=_6C4B31FC-B77F-4AA8-91F3-794F23D2AC67 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUUG9yAAoJEMZr5QU6S73eNQcH/1dWKs7kczw9gtFxcESW13oa jE3doMHU+ogbueqKF73vXG3bJJlFJfC355U9qgF+Capxa0O6eVV+B+9/hKBjegnK rdJTrn2eW+WvTIO6MMgGbhM0ZwWFmnObmYOdLFUk+LL7EgkdZfYvV3lU6Hzf8ORf qNZ2xIg/7bqLvmBsKcJhQiVFnnHs4QyVspeBbKB90jLNH560oNUNuze/p5YinAum lY8Uex0Z+cdVngS0+1zpol0RNRF3WA1dBqHd28IwoIDCpgAV5fi/JbSXtXB0vBV3 JHpXI+KRy+VcwRbMaYX3JIZDRPRVhfuwTNqsfUnq5GWZeachOJ+5sMlIBP9bMfI= =Rk23 -----END PGP SIGNATURE----- --Apple-Mail=_6C4B31FC-B77F-4AA8-91F3-794F23D2AC67-- From owner-freebsd-testing@FreeBSD.ORG Wed Oct 29 05:44:08 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3127B5C8; Wed, 29 Oct 2014 05:44:08 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41AA9199; Wed, 29 Oct 2014 05:44:07 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id z12so2504578wgg.9 for ; Tue, 28 Oct 2014 22:44:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=wD9/ZBSYbsqHP+ml+7rtKw/NTuD/0yHRNj0snrwI1sU=; b=CdzH2wYy6BOns1gDm4ruWrqO6R/xnGVodAa+B1PtpkI43BXb1h3XPr7Uvmv+rQbWbl 8s/02edfnka8NFjgmG3w4GPLoAr9wQVDvh2r1t0X4YRn+jAmlCA+1llSIbyz02vq50BA j/xwmHJhtVd/tXDjNF6CVKBhnXgwHIMJFi7eWNKPdadwzEHBGL6fVaPKy0yN/W3B9s4f KpKFpGOiW8tfTOKu/klCegsi1JhCmqotxFQN4zK/tkaBsJjfg2lzjGDQncBqk6ttsKfz hX1bv9lFNZG2yvgPy+FuJfC5ehysyz+ciG42KNjI/bA0nrj0O6B1HArAoQmSe7M5kOYi emmg== X-Received: by 10.180.221.229 with SMTP id qh5mr21827079wic.25.1414561445403; Tue, 28 Oct 2014 22:44:05 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id n4sm7490100wiz.17.2014.10.28.22.44.03 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 28 Oct 2014 22:44:04 -0700 (PDT) Date: Wed, 29 Oct 2014 06:44:02 +0100 From: Mateusz Guzik To: Craig Rodrigues Subject: Re: Automatically running /usr/tests on stable/10 branch under Jenkins Message-ID: <20141029054401.GC12014@dft-labs.eu> References: <20141024053636.GH11222@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" , FreeBSD stable , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Oct 2014 05:44:08 -0000 On Fri, Oct 24, 2014 at 09:09:48PM -0700, Craig Rodrigues wrote: > On Thu, Oct 23, 2014 at 10:36 PM, Mateusz Guzik wrote: > > > > (1) does a buildworld/buildkernel on amd64 when someone checks new > > > code into the stable/10 branch > > > > Is not this excessive? > > > It has not been a problem. For example, when a build occurs on the HEAD in > svn, > if further commits come in on HEAD, we have Jenkins configured so that it > will not trigger another build on HEAD until the build in progress is done. > It has been working fine. > Ok. > If no commits > > > > > (2) Creates a bootable UFS image with makefs > > > > any chance zfs will be used as well? > > > > Sure, we can look at that as well, but as I said earlier, > there need to be more bodies working on setting up builds and > configurations for this > to happen. > I can't help with that, sorry. > > > > > would be nice to run some kind of stress testing. buildworld with a high > > -j is an example of a general purpose test. This could be done with > > different frequency than regular tests. > > > > > Are you volunteering to write the scripts that incorporate any stress > testing > that you think should be done? We would welcome any contributions. > > Are you aware of e.g. stress2 by pho? https://people.freebsd.org/~pho/stress/index.html Plugging this in should not be hard for someone who knows jenkins. That's not me though. :) > > Do you have crashdumps configured in case stuff goes wrong? > > > > > > No. > > You can look at our scripts used to build and boot the various VM's: > > https://wiki.freebsd.org/Jenkins#Repositories > > It's all on github, so if you think you have new scripts to add, or fixes > to existing scripts, > you can feel free to do a github pull request to contribute. > > -- > Craig -- Mateusz Guzik From owner-freebsd-testing@FreeBSD.ORG Wed Oct 29 05:49:43 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D76B770; Wed, 29 Oct 2014 05:49:43 +0000 (UTC) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 07EC01C6; Wed, 29 Oct 2014 05:49:43 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id l13so2717845iga.2 for ; Tue, 28 Oct 2014 22:49:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ir3Q46OOKAlwXJ2n3Bugq5nAv+kFjDdrDo2MxU/ex0g=; b=zxJb+cjJ59EmtqgoOahQfJgzcvBE8KxH5zuBYBY5t1fCSO6C34iH50LQlDlgG53GBZ keyWxCsr20zt2HRlVn4DNgfYttXNTV35DdEzyeRWxgVHEQk3PYoRay3lsdPUJp6C2m5j mOhHnBlDItehkLEvKmOnGm45EJpYVCMO2m+dBYUYJ1z9fipuphz2vsW6euu5xFruMuNm nSJ9QFnRPoPgu+yARwEfg4cQDu9J1wKckGQLFwbH4Pt4Y9d839O7aFq8tAA+1fMqXNj0 dg+espyw/gFr3f6Gyv55Z4nmum1iqjkmpmds4df2m8I+xD+MR7fCdvZcrNWeWKGnDPvS 0x4A== MIME-Version: 1.0 X-Received: by 10.50.78.195 with SMTP id d3mr35194340igx.26.1414561782250; Tue, 28 Oct 2014 22:49:42 -0700 (PDT) Received: by 10.50.193.135 with HTTP; Tue, 28 Oct 2014 22:49:42 -0700 (PDT) In-Reply-To: <20141029054401.GC12014@dft-labs.eu> References: <20141024053636.GH11222@dft-labs.eu> <20141029054401.GC12014@dft-labs.eu> Date: Tue, 28 Oct 2014 22:49:42 -0700 Message-ID: Subject: Re: Automatically running /usr/tests on stable/10 branch under Jenkins From: NGie Cooper To: Mateusz Guzik Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , FreeBSD stable , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Oct 2014 05:49:43 -0000 On Tue, Oct 28, 2014 at 10:44 PM, Mateusz Guzik wrote: ... > Are you aware of e.g. stress2 by pho? > > https://people.freebsd.org/~pho/stress/index.html +1 for this, but I don't have cycles to implement it in Jenkins. All I'll say is that using stress2 won't work with your existing preconditions based on how it gathers crash dumps, how long some of the tests may take, etc. But you could effectively script pho's magic to some degree. > Plugging this in should not be hard for someone who knows jenkins. Plugging it in should be relatively easy. Babysitting it will be a lot more entertaining. There's a reason why we don't run it in our default test runs at Isilon ;)... From owner-freebsd-testing@FreeBSD.ORG Thu Oct 30 04:21:03 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6D1D1710; Thu, 30 Oct 2014 04:21:03 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F53FB3A; Thu, 30 Oct 2014 04:21:02 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id gq15so3724207lab.29 for ; Wed, 29 Oct 2014 21:21:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Xb+reV2cXXpFJcNCEEM6/jQa/9gUXVaNXlvyJNNFLtA=; b=R+v95IQ7lJX6QtjEIkU5OlTzqOGYdNd4a/BnFURKSmMhOJ6jj/bhSa1w0EXpWqZJDn o6P9TiyiNvdn9+BTk+39+SCcgMxhkpRcUQdD22YxBNntjYDbEQvamj+eowH5wjXCYGZ9 rx5SNoG5k6mvefvcVBF7j6pDI/1KZmI30GxHug5OXmdwsAY0ioVaogiSql3Eb0jBf6qp DwRO3yvkSBAp2SiJkM+bV7HsbFXShulpVa9Vz5ucVXdFHVeuQoKOMuUtTXc/uQHL882S sk6F+u1KKrRMvnQnDkLpyslArHT/7Su/0l14vgOor7LVJYgGpxjYKsnOOJ6EUBznmSrh SO6A== MIME-Version: 1.0 X-Received: by 10.152.27.134 with SMTP id t6mr15751207lag.17.1414642860351; Wed, 29 Oct 2014 21:21:00 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.84.197 with HTTP; Wed, 29 Oct 2014 21:21:00 -0700 (PDT) In-Reply-To: <20141029054401.GC12014@dft-labs.eu> References: <20141024053636.GH11222@dft-labs.eu> <20141029054401.GC12014@dft-labs.eu> Date: Wed, 29 Oct 2014 21:21:00 -0700 X-Google-Sender-Auth: 6xfKa9OZrLvSrURca6Nj-oKjoGM Message-ID: Subject: Re: Automatically running /usr/tests on stable/10 branch under Jenkins From: Craig Rodrigues To: Mateusz Guzik Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-testing@freebsd.org" , FreeBSD stable , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Oct 2014 04:21:03 -0000 On Tue, Oct 28, 2014 at 10:44 PM, Mateusz Guzik wrote: > On Fri, Oct 24, 2014 at 09:09:48PM -0700, Craig Rodrigues wrote: > > Are you aware of e.g. stress2 by pho? > > https://people.freebsd.org/~pho/stress/index.html > > Plugging this in should not be hard for someone who knows jenkins. > > That's not me though. :) > Since you are making some good suggestions, would you be interested in learning enough about Jenkins on your own machine first, and and see if you can get a successful integration of stress2? Setting up Jenkins is quite easy under FreeBSD. There are a lot of howto's on the web for how to do it, and I have given multiple presentations: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups I don't mind when people make good suggestions, but when the good suggestions are not backed up by concrete offers to help out, then that doesn't look so good. We need more people contributing actual scripts and tests. There are plenty of people contributing suggestions. :) -- Craig From owner-freebsd-testing@FreeBSD.ORG Thu Oct 30 07:35:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FAB5B3C; Thu, 30 Oct 2014 07:35:14 +0000 (UTC) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49A467D; Thu, 30 Oct 2014 07:35:13 +0000 (UTC) Received: by mail-wi0-f178.google.com with SMTP id q5so6534438wiv.17 for ; Thu, 30 Oct 2014 00:35:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=1OVcV3o3amhGah0EZA93W3O2AFxGgWrNdyjbg0qzaBk=; b=sXHgV8hohzbaQmfFx7jnVaaBFSqUUN14v4Ks4EiIg1K3D81l/fP4S8iDJ4glXx/EKi saCaEPqT18iRl95tSD/pyMBYlbkd27QPKQqj3ONVd2Dj6g0SdOSnPaUYfCEW1EKbvqCL Bot3nCsA9t6z584l239WaKCAvGafd5FPm8V0infEKgGSOl6OimsfNABkBaYUYJU43rOz N+h4y8EZFX+vrkJDgbtbUkVFfP+YTjyYqrIOGXG/ItREirQo15plW2Zn2qd18o33Vpap sJOMmm7KvBcX15PMHCtc4uf4RUkUZgmQh/JNT8ZZaS4E2PRZ6fsAKvcyn1eQCu7GLUDK kNtw== X-Received: by 10.180.77.170 with SMTP id t10mr8645341wiw.57.1414654511050; Thu, 30 Oct 2014 00:35:11 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id q5sm21223109wiy.16.2014.10.30.00.35.09 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Thu, 30 Oct 2014 00:35:10 -0700 (PDT) Date: Thu, 30 Oct 2014 08:35:07 +0100 From: Mateusz Guzik To: Craig Rodrigues Subject: Re: Automatically running /usr/tests on stable/10 branch under Jenkins Message-ID: <20141030073507.GA24415@dft-labs.eu> References: <20141024053636.GH11222@dft-labs.eu> <20141029054401.GC12014@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" , FreeBSD stable , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Oct 2014 07:35:14 -0000 On Wed, Oct 29, 2014 at 09:21:00PM -0700, Craig Rodrigues wrote: > On Tue, Oct 28, 2014 at 10:44 PM, Mateusz Guzik wrote: > > > On Fri, Oct 24, 2014 at 09:09:48PM -0700, Craig Rodrigues wrote: > > > > Are you aware of e.g. stress2 by pho? > > > > https://people.freebsd.org/~pho/stress/index.html > > > > Plugging this in should not be hard for someone who knows jenkins. > > > > That's not me though. :) > > > > > Since you are making some good suggestions, would you be interested > in learning enough about Jenkins on your own machine first, > and and see if you can get a successful integration of stress2? > Setting up Jenkins is quite easy under FreeBSD. There are a lot of howto's > on the web for > how to do it, and I have given multiple presentations: > https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups > > I don't mind when people make good suggestions, but > when the good suggestions are not backed up by concrete offers to > help out, then that doesn't look so good. We need more people contributing > actual scripts > and tests. There are plenty of people contributing suggestions. :) > I'm not really interested in learning jenkins, got enough backlog as it is. I'm just putting stuff out in case it sounds ok and someone wants to do it. -- Mateusz Guzik From owner-freebsd-testing@FreeBSD.ORG Thu Oct 30 20:32:28 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B158DF4D for ; Thu, 30 Oct 2014 20:32:28 +0000 (UTC) Received: from zhost.uxsglobal.com (zhost.xinuos.com [144.202.236.63]) by mx1.freebsd.org (Postfix) with ESMTP id 83D6CEC6 for ; Thu, 30 Oct 2014 20:32:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zhost.uxsglobal.com (Postfix) with ESMTP id A44328C0394 for ; Thu, 30 Oct 2014 16:32:21 -0400 (EDT) Received: from zhost.uxsglobal.com ([127.0.0.1]) by localhost (zhost.uxsglobalhost.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BtQ8qRjKNUMh for ; Thu, 30 Oct 2014 16:32:21 -0400 (EDT) Received: from zhost.uxsglobal.com (localhost [127.0.0.1]) by zhost.uxsglobal.com (Postfix) with ESMTP id 851C78C03A3 for ; Thu, 30 Oct 2014 16:32:21 -0400 (EDT) Received: from zhost.uxsglobalhost.com (localhost [127.0.0.1]) by zhost.uxsglobal.com (Postfix) with ESMTP id 818988C0394 for ; Thu, 30 Oct 2014 16:32:21 -0400 (EDT) Date: Thu, 30 Oct 2014 16:32:21 -0400 (EDT) From: Susan Stanziano To: freebsd-testing@freebsd.org Message-ID: <926516146.1185082.1414701141476.JavaMail.zimbra@xinuos.com> In-Reply-To: <1332647925.1184837.1414700255905.JavaMail.zimbra@xinuos.com> Subject: Help with Bhyve VM install MIME-Version: 1.0 X-Originating-IP: [96.234.37.238] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - GC38 (Win)/8.0.6_GA_5922) Thread-Topic: Help with Bhyve VM install Thread-Index: Q3MUDx+EyxQ0AufIW5ce3hcZzloP6w== Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Oct 2014 20:32:28 -0000 I have a system installed with releng10.1 (with tests configured) and am trying to create a VM under Bhyve. I want to know which image to use for the VM install, specifically will the : FreeBSD-10.1-RC3-amd64-bootonly.iso install with the /usr/tests directory created? Or, should I use FreeBSD-10.1-RC3-amd64-disc1.iso ? Thanks, and any further advice would be appreciated also, Susan From owner-freebsd-testing@FreeBSD.ORG Thu Oct 30 22:03:56 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCDC0E07 for ; Thu, 30 Oct 2014 22:03:56 +0000 (UTC) Received: from mail-ie0-x22e.google.com (mail-ie0-x22e.google.com [IPv6:2607:f8b0:4001:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 942959CB for ; Thu, 30 Oct 2014 22:03:56 +0000 (UTC) Received: by mail-ie0-f174.google.com with SMTP id x19so88044ier.19 for ; Thu, 30 Oct 2014 15:03:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=KRF4zgW32enCUk9Z1oFl+tNGe0Wzkf1C0e1CRio5+CU=; b=wi+boudXaf5ssLZna+7KQOhxrJckArAlnvpI6gxDeyO6c9WGEJC52d4hwC+aGN740f pyFFym8JJjeRaJV4v8PM/4lHCwPQv7xzSgOGuiqlsV0lfjIChbZ2qhIELdziD54qrrxa lWNtvKuhhD2uyrKp0K8GaCLEVTZfiDM0W0aORWUvTlJ/kxhxDkJmnCT/NonNxKa3fkJv lIIAVoBZ7qeb+uWLl/NScKLt1mrE3nJIpaisOZMBOVi0seWOv6VQ3c4V+PJZ8QeiZ29H YYyhWmQrGMNKgKiqpR/Oc/J8Aw066breTpGMNB+Qd508Oy+hngNFt95npXDX4Cc63bUk HZkA== MIME-Version: 1.0 X-Received: by 10.50.93.6 with SMTP id cq6mr192100igb.7.1414706636050; Thu, 30 Oct 2014 15:03:56 -0700 (PDT) Received: by 10.50.193.135 with HTTP; Thu, 30 Oct 2014 15:03:56 -0700 (PDT) In-Reply-To: <926516146.1185082.1414701141476.JavaMail.zimbra@xinuos.com> References: <1332647925.1184837.1414700255905.JavaMail.zimbra@xinuos.com> <926516146.1185082.1414701141476.JavaMail.zimbra@xinuos.com> Date: Thu, 30 Oct 2014 15:03:56 -0700 Message-ID: Subject: Re: Help with Bhyve VM install From: NGie Cooper To: Susan Stanziano Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Oct 2014 22:03:56 -0000 On Thu, Oct 30, 2014 at 1:32 PM, Susan Stanziano wrote: > > > I have a system installed with releng10.1 (with tests configured) and am trying to create a VM under Bhyve. > > I want to know which image to use for the VM install, specifically will the : > > FreeBSD-10.1-RC3-amd64-bootonly.iso > > install with the /usr/tests directory created? Or, should I use FreeBSD-10.1-RC3-amd64-disc1.iso ? > > Thanks, and any further advice would be appreciated also, Hi Susan, Please try asking this question on the freebsd-virtualization@ list instead. Thank you! From owner-freebsd-testing@FreeBSD.ORG Fri Oct 31 02:39:56 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DDBB54C8; Fri, 31 Oct 2014 02:39:56 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39CB76D6; Fri, 31 Oct 2014 02:39:56 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id w7so5420497lbi.12 for ; Thu, 30 Oct 2014 19:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=sfuOR7CL115vhmX3U5MBpHBSDVvFBGXZtpJyTM9yrv8=; b=yV5nGk5uYZzYLAlZL5jC6JnnIf2elo8p0FjMjGfjFPEofqeFJkbXxcnw6giDCI1CwB 0hraasRxgLHM0BcqU12DMojVp9b2KV9zrTpesBeOWT1BOCJVHGzXOUkAdS8iefCp7Gqo Ro8fHEdUrdaHhL18G/OWPAPv+ug6rhmB2n+9xqHUGkDfKNx5k21vm0y172P6H4KlEDKu fw+X98dYedftHcdvLoxcMzI8o+8zsIjFwLiJK0YrbRL6tnvoZ1/RFsJtl8/T4B1Oyq9q VrJSGFKeoxt6S3DI1Rn904TASruwWfPq8PJZFMXpNG7gDLnenYOPdskaxUpftQMatBV1 f4Ag== MIME-Version: 1.0 X-Received: by 10.112.54.229 with SMTP id m5mr22939283lbp.11.1414723194074; Thu, 30 Oct 2014 19:39:54 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.84.197 with HTTP; Thu, 30 Oct 2014 19:39:54 -0700 (PDT) In-Reply-To: <926516146.1185082.1414701141476.JavaMail.zimbra@xinuos.com> References: <1332647925.1184837.1414700255905.JavaMail.zimbra@xinuos.com> <926516146.1185082.1414701141476.JavaMail.zimbra@xinuos.com> Date: Thu, 30 Oct 2014 19:39:54 -0700 X-Google-Sender-Auth: DX3fZZy1Oo8Hls1QpIh6qTMpS9E Message-ID: Subject: Re: Help with Bhyve VM install From: Craig Rodrigues To: Susan Stanziano Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-testing@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Oct 2014 02:39:57 -0000 On Thu, Oct 30, 2014 at 1:32 PM, Susan Stanziano wrote: > > I have a system installed with releng10.1 (with tests configured) and am > trying to create a VM under Bhyve. > > I want to know which image to use for the VM install, specifically will > the : > > FreeBSD-10.1-RC3-amd64-bootonly.iso > > install with the /usr/tests directory created? Or, should I use > FreeBSD-10.1-RC3-amd64-disc1.iso ? > Hi Susan, I'm not sure why Garrett felt it was necessary to redirect you to the freebsd-virtualization list. Although you are using bhyve, there is nothing specific to bhyve in your questions. No harm done, though, since I'm on both lists, and will answer your questions. FreeBSD 10.1 does not have /usr/tests populated by default. You have a few options OPTION 1: FreeBSD 10.1 ISO ======================= (1) Install FreeBSD-10.1-RC3-amd64-disc1.iso into a bhyve VM (2) Make sure that in the install menus you specify to install sources, which will be under /usr/src (3) Boot the VM (4) Create a file /etc/make.conf with this line: WITH_TESTS="yes" (5) Follow these steps 1-11 for rebuilding the binaries on your system from the source code in /usr/src: https://svnweb.freebsd.org/base/head/Makefile?revision=271168&view=markup#l70 (6) reboot the VM Tests will be available under /usr/tests OPTION 2: FreeBSD-CURRENT ISO =========================== (1) Download ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/11.0/FreeBSD-11.0-CURRENT-amd64-20141025-r273635-bootonly.iso (2) Install in a bhyve VM. (3) Make sure that in the installer you specify to install the tests (4) Reboot the VM (5) Tests will be available in /usr/tests OPTION 3: FreeBSD-CURRENT VM image ================================ (1) Download the pre-built "raw" VM disk image from: ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/11.0-CURRENT/amd64/Latest/FreeBSD-11.0-CURRENT-amd64.raw.xz (2) Use "unxz" to decompress it. (3) Boot the FreeBSD-11.0-CURRENT-amd64.raw image in bhyve (4) Tests will be available in /usr/tests -- Craig From owner-freebsd-testing@FreeBSD.ORG Fri Oct 31 21:52:43 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0008C758 for ; Fri, 31 Oct 2014 21:52:42 +0000 (UTC) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0FEB994 for ; Fri, 31 Oct 2014 21:52:42 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id eu11so8435459pac.23 for ; Fri, 31 Oct 2014 14:52:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=K46Mwbrn42d4s8wfWoTU+jybRcnPFtufjo/w8/fIjGg=; b=qTOFWPdXdUs+aSWa341isrzDNSwGe2KXRN+b3GKkphML+n1XmBKODUR6SI+/9SZXEu I0osEhjY5HzZCUU6SYSptbdQtgp8HQPqC3BkSfK3+UbqqSFLXG/dJV8lho/447zA198G rIRf6aBmVCyRZ5rLnCcwNLNgeXxPFHJoX0XONdRVnMhdxSeuYJGjRcy1EveRRdWB5oTx cb99YpQa/r/aHEzmklajFjrBOw8UgMJQ3t69glDy2YpzAlBAOqi7gseqHBvfXrVA0Yft V8zx2wlx/9ejpTm9iGI0wHiPFGzRR+vx0PdZrHWvhxkZEc/j5RZ8IbAuNb+qgH4HEhlB c1TA== X-Received: by 10.66.255.7 with SMTP id am7mr26661269pad.134.1414792361995; Fri, 31 Oct 2014 14:52:41 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:34d4:4abd:64bb:590a? ([2601:8:ab80:7d6:34d4:4abd:64bb:590a]) by mx.google.com with ESMTPSA id cc2sm10750776pbd.62.2014.10.31.14.52.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 31 Oct 2014 14:52:41 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_54A72C55-9A73-49FB-A2D5-9906B19B6EDE"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: help on testing for FreeBSD... From: Garrett Cooper In-Reply-To: <20141024230806.GO82214@funkthat.com> Date: Fri, 31 Oct 2014 14:52:40 -0700 Message-Id: <35585EB9-21D0-4AFC-8BA5-F2DD4968590F@gmail.com> References: <20141016001543.GD1852@funkthat.com> <69C71414-AE05-4AC2-B07D-23F5569C9FCA@gmail.com> <20141016003822.GE1852@funkthat.com> <20141016005248.GF1852@funkthat.com> <20141024230806.GO82214@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.1878.6) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Oct 2014 21:52:43 -0000 --Apple-Mail=_54A72C55-9A73-49FB-A2D5-9906B19B6EDE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Oct 24, 2014, at 16:08, John-Mark Gurney wrote: > John-Mark Gurney wrote this message on Wed, Oct 15, 2014 at 17:52 = -0700: >> NGie Cooper wrote this message on Wed, Oct 15, 2014 at 17:46 -0700: >>> On Wed, Oct 15, 2014 at 5:38 PM, John-Mark Gurney = wrote: >>>> Garrett Cooper wrote this message on Wed, Oct 15, 2014 at 17:25 = -0700: >>>>> Hey jmg@ (-testing CCed)! >>>>=20 >>>> Sure, np... >>>>=20 >>>>> On Oct 15, 2014, at 17:15, John-Mark Gurney = wrote: >>>>>=20 >>>>>> Hello, >>>>>>=20 >>>>>> I'm working on testing parts of OpenCrypto. I have committed = some work >>>>>> to p4 at: >>>>>> = https://p4db.freebsd.org/depotTreeBrowser.cgi?FSPC=3D//depot/projects/open= crypto/tests/sys/opencrypto&HIDEDEL=3DNO >>>>>>=20 >>>>>> But it requires a few files to be installed... I'm not sure = where the >>>>>> best location to install these are and now best to install them? >>>>>=20 >>>>> I have a couple questions: >>>>> - Where would the source live in the tree? >>>>=20 >>>> Which source are you talking about? The python source? or the = source >>>> for the input data files? If you mean the later, it's where they = will >>>> be committed... They are taken from NIST's CAVP program = unchanged... >>>> I COULD possibly write a complicated rules to fetch and extract the >>>> files, but decided not to... >>>=20 >>> The bulk majority of the driver source (.c, .h, etc). >>=20 >> The code that I'm testing is in src/sys/opencrypto (and there are = other >> drivers that can get tested too, but it still goes through the = opencrypto >> code)... Hence why I put them in tests/sys/opencrypto... If that's = the >> wrong location, let me know and I can move them... >>=20 >>>>> - Why are they written in python? >>>>=20 >>>> Because I'm too lazy to write code in C... I had already written >>>> framework code in Python, and C doesn't have a standard function to >>>> decode hex.. :) And error handling it much easier in python... >>>=20 >>> I ask because we don't have a means for doing unittest level = execution >>> right now and not being written in something that's in base or >>> installed automatically (perl for instance) would mean that your = test >>> code wouldn't be run by default on jenkins runs. >>=20 >> Yeh... Though as a kld will have to be loaded, and a sysctl set as >> root, there's other issues to having it automaticly run as part of >> jenkins... >=20 > Hello, any help w/ this? I haven't received a response on how to > integrate my code.. If I don't get help w/ this soon, I will add it > to src/tools/regression, and call it a day... Sorry for not responding sooner. I=92m off today, so I=92ll be around = the usual channels on #efnet if you need to track me down. Also, I spotted this in passing while trying to see if we had any = arc4random testcases=85 http://svn.freebsd.org/base/head/contrib/netbsd-tests/crypto/opencrypto/ This might be more worthwhile to integrate than pursing a custom test = infrastructure/tests route as it=92s a part of NetBSD=92s open crypto = testsuite (there=92s probably going to be some API differences, but they = should be relatively small). Thank you! --Apple-Mail=_54A72C55-9A73-49FB-A2D5-9906B19B6EDE Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUVASoAAoJEMZr5QU6S73eKWAIAIp95prBH6yfPaLIqbGUmedR LHn7Jdk9gtCADf+YEp82z8bVC/NpAQ2ml0+myFXpSusM19vxm0pyiEJggtDj6E88 fcqdulcufrltQxavu3uiQuxT4R+W//KMFpV+r1BpVeCelKOdw6TDhA+fa71seBGE ffkwK3BQSlYX/MyAos6NdfFRRTGTPAbwMycnWKzn+4zd/T7MUp5JiwvueMAtfgev 01Ctl1Ptp67s7A14t7yDWn/JGHUvvYcpBsvSbzox+6fgPf9Jt/Rj21n7G9hbeaAu Jt5wYRFPpv6dKA2PLzpzFrAZD8Gs3Iki0slrda2JSmv+4wfGNGUU2huxwCkdj9g= =elBq -----END PGP SIGNATURE----- --Apple-Mail=_54A72C55-9A73-49FB-A2D5-9906B19B6EDE-- From owner-freebsd-testing@FreeBSD.ORG Fri Oct 31 22:25:51 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E3C43E4 for ; Fri, 31 Oct 2014 22:25:51 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E6B6CC93 for ; Fri, 31 Oct 2014 22:25:50 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s9VMPm4E045109 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 31 Oct 2014 15:25:49 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s9VMPmhL045108; Fri, 31 Oct 2014 15:25:48 -0700 (PDT) (envelope-from jmg) Date: Fri, 31 Oct 2014 15:25:48 -0700 From: John-Mark Gurney To: Garrett Cooper Subject: Re: help on testing for FreeBSD... Message-ID: <20141031222548.GR8852@funkthat.com> References: <20141016001543.GD1852@funkthat.com> <69C71414-AE05-4AC2-B07D-23F5569C9FCA@gmail.com> <20141016003822.GE1852@funkthat.com> <20141016005248.GF1852@funkthat.com> <20141024230806.GO82214@funkthat.com> <35585EB9-21D0-4AFC-8BA5-F2DD4968590F@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <35585EB9-21D0-4AFC-8BA5-F2DD4968590F@gmail.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 31 Oct 2014 15:25:49 -0700 (PDT) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Oct 2014 22:25:51 -0000 Garrett Cooper wrote this message on Fri, Oct 31, 2014 at 14:52 -0700: > On Oct 24, 2014, at 16:08, John-Mark Gurney wrote: > > > John-Mark Gurney wrote this message on Wed, Oct 15, 2014 at 17:52 -0700: > >> NGie Cooper wrote this message on Wed, Oct 15, 2014 at 17:46 -0700: > >>> On Wed, Oct 15, 2014 at 5:38 PM, John-Mark Gurney wrote: > >>>> Garrett Cooper wrote this message on Wed, Oct 15, 2014 at 17:25 -0700: > >>>>> Hey jmg@ (-testing CCed)! > >>>> > >>>> Sure, np... > >>>> > >>>>> On Oct 15, 2014, at 17:15, John-Mark Gurney wrote: > >>>>> > >>>>>> Hello, > >>>>>> > >>>>>> I'm working on testing parts of OpenCrypto. I have committed some work > >>>>>> to p4 at: > >>>>>> https://p4db.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects/opencrypto/tests/sys/opencrypto&HIDEDEL=NO > >>>>>> > >>>>>> But it requires a few files to be installed... I'm not sure where the > >>>>>> best location to install these are and now best to install them? > >>>>> > >>>>> I have a couple questions: > >>>>> - Where would the source live in the tree? > >>>> > >>>> Which source are you talking about? The python source? or the source > >>>> for the input data files? If you mean the later, it's where they will > >>>> be committed... They are taken from NIST's CAVP program unchanged... > >>>> I COULD possibly write a complicated rules to fetch and extract the > >>>> files, but decided not to... > >>> > >>> The bulk majority of the driver source (.c, .h, etc). > >> > >> The code that I'm testing is in src/sys/opencrypto (and there are other > >> drivers that can get tested too, but it still goes through the opencrypto > >> code)... Hence why I put them in tests/sys/opencrypto... If that's the > >> wrong location, let me know and I can move them... > >> > >>>>> - Why are they written in python? > >>>> > >>>> Because I'm too lazy to write code in C... I had already written > >>>> framework code in Python, and C doesn't have a standard function to > >>>> decode hex.. :) And error handling it much easier in python... > >>> > >>> I ask because we don't have a means for doing unittest level execution > >>> right now and not being written in something that's in base or > >>> installed automatically (perl for instance) would mean that your test > >>> code wouldn't be run by default on jenkins runs. > >> > >> Yeh... Though as a kld will have to be loaded, and a sysctl set as > >> root, there's other issues to having it automaticly run as part of > >> jenkins... > > > > Hello, any help w/ this? I haven't received a response on how to > > integrate my code.. If I don't get help w/ this soon, I will add it > > to src/tools/regression, and call it a day... > > Sorry for not responding sooner. I?m off today, so I?ll be around the usual channels on #efnet if you need to track me down. Basicly, need help on where to install files that aren't executables, but are necessary for the tests to run... stimulus files and python module files.. If you want to take a look at what I have so far, go ahead and look at the link above to the p4 depot.. Is anyone going to be at MeetBSD that could help me with this?? Then we could sit down and knock this out... > Also, I spotted this in passing while trying to see if we had any arc4random testcases? > > http://svn.freebsd.org/base/head/contrib/netbsd-tests/crypto/opencrypto/ > > This might be more worthwhile to integrate than pursing a custom test infrastructure/tests route as it?s a part of NetBSD?s open crypto testsuite (there?s probably going to be some API differences, but they should be relatively small). Oh, didn't see NetBSD's, but their tests aren't very expansive.. only a single test vector for GCM, no tests for XTS, no tests for 3DES, and 7 tests for SHA1-HMAC... Good thing I looked at this.. our SHA1-HMAC has a bug in that it only allows 20 byte keys instead of arbitrary length keys.. The test vectors for NIST doesn't have any 20 byte keys... Opps... If we fix that, we'd have 1575 tests... So, even though the NetBSD tests would be useful to integrate, it's clear that they don't overlap much... Also, OpenBSD has some tests too, but they mostly compile code directly from the kernel instead of going through /dev/crypto... Which makes it harder for us to use the tests... Thanks for the help! -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."