From owner-freebsd-testing@FreeBSD.ORG Sun Feb 23 00:40:29 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1AEDC7FF; Sun, 23 Feb 2014 00:40:29 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 69DC71DAC; Sun, 23 Feb 2014 00:40:28 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id pv20so4016353lab.30 for ; Sat, 22 Feb 2014 16:40:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=FxXLICl+f4RblzthLDxTC9M/uxwXiiec9W/Mlui6yd8=; b=XljnSBN19mkhJ/Zr4qZ0P15GOg3A136ZtouTO2hBDw8j2wLez+eiG4RwEXBYZ0A/7o 1mMifmPE09e2izu21CyHaGh8e1WyKv/8yj/ecB/8gYuTCTlvYKXIBOFwMW1ttYFawk6g +kORDycORCQ52SqYCAJ9AZLeeBx6F+/TN4jgzIebC6EYkPEE0ii/aUeKpwIyw6NXYv9x jggzYTx6HLzZyso/4kXIRT+1sg/+JNyPKANrAoOvbuGXezB0WV8lsjtRqHQBaumGMBHG u6ykteNjYrgOaNKcXH9FLYuPwPF+DcvWr9bAyQXThUtv6RhIPVrWEKRCifrIICZeXKt6 t1gA== MIME-Version: 1.0 X-Received: by 10.152.27.193 with SMTP id v1mr8213679lag.4.1393116026459; Sat, 22 Feb 2014 16:40:26 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Sat, 22 Feb 2014 16:40:26 -0800 (PST) Date: Sat, 22 Feb 2014 16:40:26 -0800 X-Google-Sender-Auth: VwU4OEj0ndHbk-B__wztLSCY5_c Message-ID: Subject: Jenkins plus Kyua integration discussion From: Craig Rodrigues To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 00:40:29 -0000 Dear freebsd-testing, I previously sent a HEADSUP mail to this list, announcing that Jenkins has been initially deployed in the FreeBSD cluster by the jenkins-admin team. We have a status page here: http://wiki.freebsd.org/Jenkins One of the goals listed on that page is to have some sort of integration between Jenkins and Kyua. On May 15, I will be leading a working group on Jenkins continuous integration for FreeBSD at the Devsummit in Ottawa: https://wiki.freebsd.org/201405DevSummit/Jenkins Is there anyone on the freebsd-testing list who would be able to attend and participate? I've exchanged a few e-mails with Julian about this, but would like to open up discussion to the wider freebsd-testing community. Thanks. -- Craig From owner-freebsd-testing@FreeBSD.ORG Sun Feb 23 01:06:04 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3CC4E5A; Sun, 23 Feb 2014 01:06:04 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AAB9D107E; Sun, 23 Feb 2014 01:06:04 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id fa1so5064772pad.0 for ; Sat, 22 Feb 2014 17:06:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=LzqEWe4spnzut/+GWeD3kk1/layr2Mn0sUYPiIyzuP8=; b=UYMb0Dr3E3BY598on8d76IWp1Zd/CuAzQV5YAF5cicGDJgtspgmgDwjo6MVsxrOv7d iOlMi5Qpf6xmAnDgq8FpI261E07LGtHW86yYFVefcRnJhziiW1s8Zbl+f3R11X34ezcp VbHP1iUsStV3ExK0up8twuV/sh5WsA3+nTeATjRDAaNtauxHgMHEFsWe2HwWecBr3t9U RQc8d27oO2ZSyfOpbp2CvYn1HLY7tjcAMHAsZYgGfVmsSf9EQflEwfpp2NaECt5srJ3w SSK1RIBHyuOX5qgAc04Cm13zM2ejbDbtVsKXKt3nTBfIBz7XJP7Rl2ZqMn002ZgpFKSW 1aXA== X-Received: by 10.66.51.137 with SMTP id k9mr5820937pao.11.1393117564078; Sat, 22 Feb 2014 17:06:04 -0800 (PST) Received: from [192.168.20.5] (c-50-135-127-159.hsd1.wa.comcast.net. [50.135.127.159]) by mx.google.com with ESMTPSA id n6sm35304642pbj.22.2014.02.22.17.06.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 22 Feb 2014 17:06:02 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Jenkins plus Kyua integration discussion From: Garrett Cooper In-Reply-To: Date: Sat, 22 Feb 2014 17:06:00 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <579FE5F4-FAD7-4657-9BEC-FE3D07E4F112@gmail.com> References: To: Craig Rodrigues X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@freebsd.org, jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 01:06:05 -0000 Hi Craig! On Feb 22, 2014, at 4:40 PM, Craig Rodrigues = wrote: > Dear freebsd-testing, =85 > One of the goals listed on that page is to have some sort of = integration > between Jenkins and Kyua. Cool! > Is there anyone on the freebsd-testing list who would be able > to attend and participate? I=92m not going to be going to BSDCan this year, but there=92s a strong = chance that someone from EMC Isilon who=92s interested might be, and = Julio might be as well. Have members from the FreeBSD Foundation (Glen Barber, George = Nevill-Neil, etc) been engaged with this work? > I've exchanged a few e-mails with Julian about this, but > would like to open up discussion to the wider freebsd-testing = community. Is =93Julian=94 Julian Elischer perhaps? Thank you! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Sun Feb 23 01:33:54 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85DC04CA for ; Sun, 23 Feb 2014 01:33:54 +0000 (UTC) Received: from mail-qa0-f49.google.com (mail-qa0-f49.google.com [209.85.216.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3A449127A for ; Sun, 23 Feb 2014 01:33:53 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id w8so4862307qac.22 for ; Sat, 22 Feb 2014 17:33:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=nl+C1VSlLjrE2NupETVV8OPn+Glu3kF7jk3c6yQ43aE=; b=Kby+4l93KfM18RSmUz2xG/wW9TXroMrAOVA/ICJw1tJG8q9E0RBHaDBl5Pq4bHs0J2 VaRH3zIwgh1MZYX3BWZgOxZgqaV9fBFtg5916gr+ZQp90NjeMyUiKA9oa9RefuXF/EnR ilIN2aLH2zrtZnt/KFlwjwhEXRcjD97Wfms565uYnBqKYGMkwOQhgho1HhvUH4f0vYPt 3qgIpbygj6UBIM1fW2ZZnD4ogocoytyWWytF+6YuqZW3yrjD2mcQ8S4KkGrS7DB5Psjy oJy/2GbifuFxsjqGRKMZJOMaYYm/LS76fjSDoI+TTkkNc7HH4wk6deSZiAM1ZSUpuqT8 LsDw== X-Gm-Message-State: ALoCoQmHl87PORaQQ3N7TO/N0U1w7bQYEFhYmyVxWrltyFl82GX6+X4tdhwcU9X7OUCx/jmrkAJO X-Received: by 10.140.88.112 with SMTP id s103mr19817600qgd.47.1393118908348; Sat, 22 Feb 2014 17:28:28 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sat, 22 Feb 2014 17:28:08 -0800 (PST) X-Originating-IP: [172.16.49.156] In-Reply-To: References: From: Julio Merino Date: Sat, 22 Feb 2014 20:28:08 -0500 X-Google-Sender-Auth: ysq11QxoOzNgHsGYw0w8cj9FO_8 Message-ID: Subject: Re: Jenkins plus Kyua integration discussion To: Craig Rodrigues Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-testing@freebsd.org, jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 01:33:54 -0000 On Sat, Feb 22, 2014 at 7:40 PM, Craig Rodrigues wrote: > Dear freebsd-testing, > [...] > On May 15, I will be leading a working group on Jenkins continuous > integration for FreeBSD at the Devsummit in Ottawa: > > https://wiki.freebsd.org/201405DevSummit/Jenkins > > Is there anyone on the freebsd-testing list who would be able > to attend and participate? > Having just gotten notification of the acceptance of my talk, yes, I will be there! (FYI: I'll be talking about http://wiki.freebsd.org/TestSuite on Sunday morning.) From owner-freebsd-testing@FreeBSD.ORG Sun Feb 23 01:35:25 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A054E73A; Sun, 23 Feb 2014 01:35:25 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EB21C128E; Sun, 23 Feb 2014 01:35:24 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id hr17so2122810lab.29 for ; Sat, 22 Feb 2014 17:35:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=N3Lb7Ge2M9yl7kkWqURbOCK0rCN/4LqQH98qxK8tsPc=; b=PCbnlRh7HGiFXn7bqa4JpUZi1lUaCB2dNa18xhpPt3HKb1qqT3UOrdmzaSXK0uO/qR Y++rgAzfnNu33IBMWB+F1hbBsp3M3/srDtODByx5ukmIYMoXPUNw2iyROUSha4yrmuiw P/WjjxO6IqjYGL/lQs5AGtdM7/OJ/zR/KAguvASXqQCPPxh3CuQqkIsqUvPXPIH6WD4E 18hrr3p3C6tnh7rdJEKLDMMRu8OBYX99CBqrmSvCC6KvU8rNV4smhBASTTVIaNXVb2bB 1ouzXweJZCxVOYyZqATfxdQx8PC6t32c1YYwoFLPs0giYl8x1tmdw+LieCU/DAI8sXL+ l+tQ== MIME-Version: 1.0 X-Received: by 10.152.143.231 with SMTP id sh7mr8254913lab.26.1393119322906; Sat, 22 Feb 2014 17:35:22 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Sat, 22 Feb 2014 17:35:22 -0800 (PST) In-Reply-To: <579FE5F4-FAD7-4657-9BEC-FE3D07E4F112@gmail.com> References: <579FE5F4-FAD7-4657-9BEC-FE3D07E4F112@gmail.com> Date: Sat, 22 Feb 2014 17:35:22 -0800 X-Google-Sender-Auth: QudcIfWWfnD7SpoV_pPnSkjYYoA Message-ID: Subject: Re: Jenkins plus Kyua integration discussion From: Craig Rodrigues To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-testing@freebsd.org, jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 01:35:25 -0000 On Sat, Feb 22, 2014 at 5:06 PM, Garrett Cooper wrote: > I'm not going to be going to BSDCan this year, but there's a strong chance > that someone from EMC Isilon who's interested might be, and Julio might be > as well. > Please forward the info along to anyone you know at EMC Isilon (or wherever). The more people contributing to this effort, the better off FreeBSD will be in the long term. I want to keep the discussion out in the open on the freebsd-testing mailing list. > Have members from the FreeBSD Foundation (Glen Barber, George Nevill-Neil, > etc) been engaged with this work? > No. At the FreeBSD Vendor Summit in Nov. 2013, Ed Maste saw some stuff I had done with Jenkins, and had expressed some interest in getting this running in the FreeBSD cluster, but I haven't talked to him since then. Glen Barber and Sean Bruno have helped a bit from the Cluster Administration perspective in terms of allocating hardware for the effort. > > I've exchanged a few e-mails with Julian about this, but > > would like to open up discussion to the wider freebsd-testing community. > > Is "Julian" Julian Elischer perhaps? > No, sorry, I spelled his name wrong, I meant to say that I exchanged a few mails with Julio Merino, not Julian. -- Craig From owner-freebsd-testing@FreeBSD.ORG Sun Feb 23 18:34:42 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6E5D5526; Sun, 23 Feb 2014 18:34:42 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2FCA0188B; Sun, 23 Feb 2014 18:34:42 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kp14so5559682pab.34 for ; Sun, 23 Feb 2014 10:34:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=EqdorvmeWpqUinKegZnP+BS/6VjnV06oowqBHhT7u0w=; b=x3TnPS8jtefWLrIJ0Jg0LGC50z9w4r+K6bf0XvbQajsyzCH/jLiaHdnJLvhq4ZP28W ymrCZz429SsxKdjjToL+hZ5WgWvCEW5x/ylS2/kPa4L5QfAYSnos9fXwiR+/CG/Rl+Sg 8IlUrBNtHkW5C32qOLoSHcg0kWSsOR6ynPpnQFPlp7jiPk82UwKNNk/O3gd7fwSJFW7i tWKnu00dUkvHhzFIcVaVBNAB8jyxgNZPwDZ6nrXd6YZ12hyvBz9+faW9u12eOaMgMYg+ b1AWTMJYt9HzGo9UQdZXJamGDgky08g3CEADoatLTtjezNHJk40wnk4rjGaR2ReI2/sY GGTA== X-Received: by 10.66.149.231 with SMTP id ud7mr20794641pab.8.1393180481863; Sun, 23 Feb 2014 10:34:41 -0800 (PST) Received: from [192.168.20.5] (c-50-135-127-159.hsd1.wa.comcast.net. [50.135.127.159]) by mx.google.com with ESMTPSA id om6sm42277172pbc.43.2014.02.23.10.34.39 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 23 Feb 2014 10:34:40 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Jenkins plus Kyua integration discussion From: Garrett Cooper In-Reply-To: Date: Sun, 23 Feb 2014 10:34:39 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <8C23A652-943A-4B38-AFB2-66EF293FAE3E@gmail.com> References: <579FE5F4-FAD7-4657-9BEC-FE3D07E4F112@gmail.com> To: Craig Rodrigues X-Mailer: Apple Mail (2.1827) Cc: freebsd-testing@freebsd.org, jenkins-admin@freebsd.org, Benno Rice X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 18:34:42 -0000 On Feb 22, 2014, at 5:35 PM, Craig Rodrigues = wrote: > On Sat, Feb 22, 2014 at 5:06 PM, Garrett Cooper = wrote: >=20 >> I'm not going to be going to BSDCan this year, but there's a strong = chance >> that someone from EMC Isilon who's interested might be, and Julio = might be >> as well. >>=20 >=20 > Please forward the info along to anyone you know > at EMC Isilon (or wherever). The more people contributing to this = effort, > the better off FreeBSD will be in the long term. > I want to keep the discussion out in the open on > the freebsd-testing mailing list. Benno Rice is a good public point of contact for FreeBSD at EMC/Isilon. = He wouldn=92t be doing the work to integrate/use ATF/Kyua, but he might = be able to put you in touch with someone who could. >> Have members from the FreeBSD Foundation (Glen Barber, George = Nevill-Neil, >> etc) been engaged with this work? >>=20 >=20 > No. At the FreeBSD Vendor Summit in Nov. 2013, Ed Maste > saw some stuff I had done with Jenkins, and had expressed some = interest > in getting this running in the FreeBSD cluster, but I haven't talked = to > him since then. Glen Barber and Sean Bruno have helped a bit > from the Cluster Administration perspective in terms of allocating = hardware > for the effort. Hmmm=85 they need to be engaged eventually because the build system = doesn=92t build tests by default, and it would be nice if tests were = installable via a tarball in bsdinstall (it=92s not really today), but = this is something that probably should be in a side-thread. >>> I've exchanged a few e-mails with Julian about this, but >>> would like to open up discussion to the wider freebsd-testing = community. >>=20 >> Is "Julian" Julian Elischer perhaps? >>=20 >=20 > No, sorry, I spelled his name wrong, I meant to say that I exchanged > a few mails with Julio Merino, not Julian. Ah, ok! Thank you :)! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Mon Feb 24 09:05:00 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B840A51F; Mon, 24 Feb 2014 09:05:00 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD0031A13; Mon, 24 Feb 2014 09:04:59 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id gf5so2225137lab.7 for ; Mon, 24 Feb 2014 01:04:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=KRkLcxHc+/+Arz6ir99e9Fby4VLjMu9etyg9WlPIevY=; b=zh6qL+Iop7VIoTXqo4uwTfoz1f/qKPVy1b5QEyXbl2mcv5wAXRORpjtVM0nRf2pebn ksvgWQYnVNIjLA3jRsZWQjJbIiEXW8Wo5PNaHCQG6tR2lt8bbMFs80xutHxaA/ATTnDR hU38/etvqEob6nVJKWs9EW9RtMQ4JaCbOCPpIWV8CRUsTTbuhXdBLRoBfeb/Jm0knHSf oRR2AQDTbTLHDE7ytKh073j7d7sAJtqrTraFZIORtWPGzxsYFh2kPVSGkAlBz14XpHNA Oc8ZIfpP/U8WdBoieGR2SUt34hADCdhwlSOmoPwkXnK3amgh7N0/HyqMIg0z9htWyIM5 ePgg== MIME-Version: 1.0 X-Received: by 10.112.38.4 with SMTP id c4mr6022535lbk.84.1393232697995; Mon, 24 Feb 2014 01:04:57 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 24 Feb 2014 01:04:57 -0800 (PST) Date: Mon, 24 Feb 2014 01:04:57 -0800 X-Google-Sender-Auth: GJ-RLk0zanzAiFwB-3MMsyh7h-o Message-ID: Subject: March 13: Jenkins and BHyve presentation From: Craig Rodrigues To: freebsd-current Current , "freebsd-virtualization@freebsd.org" , freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 09:05:00 -0000 Hi, I will be giving a talk discussing what I have done settings up Jenkins ( https://jenkins.freebsd.org ) in the FreeBSD cluster, using BHyve VM's: The presentation will be on March 13 in Mountain View, California, U.S.A.: http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ If you think you can come, please RSVP on the Meetup site, so that the organizers can get a big enough room. I am interested in collaborating with FreeBSD hackers who can contribute to the effort and help expand testing of FreeBSD!! -- Craig From owner-freebsd-testing@FreeBSD.ORG Mon Feb 24 14:06:34 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05A4AE62; Mon, 24 Feb 2014 14:06:34 +0000 (UTC) Received: from mail-oa0-x234.google.com (mail-oa0-x234.google.com [IPv6:2607:f8b0:4003:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A5E5419D8; Mon, 24 Feb 2014 14:06:33 +0000 (UTC) Received: by mail-oa0-f52.google.com with SMTP id i4so7037542oah.39 for ; Mon, 24 Feb 2014 06:06:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=mPOxXr2bUSVkh6YxyOVwqm/RHKypXGGVsMk653bS8mY=; b=rmuboKV7bD6mwP6gdrmHsbUmR5hQ0ixBTbfHU8hKlBcSAC25y7CkTV3wRVReEf3k+q d1DDbp048QafLZUYkrnVLPBaGPyqvEICUjtFuKC8jHe5s6oBQktw8IMQZi9Rutjfqaa5 l76Pt7reH1nxjEmfxvHM2n2PwFSdUie7um8hwzH67/2jzMIOG7GWueL0b1EV8N08+m5S 1dUg69lvfOr9FXGB9pgDTJwc27LSYkuGz85wN0KK27MIGo8U9fuWOuJ4xM6DHcuLP9pg Pt1dljA6IB4t/X/0gAXYJKp5Dq4nqvwpN9pCg33SR2P+lV1Z7rRcAExfuJ80tJZqhvwm sYBA== X-Received: by 10.60.94.231 with SMTP id df7mr21469674oeb.69.1393250792949; Mon, 24 Feb 2014 06:06:32 -0800 (PST) MIME-Version: 1.0 Sender: jonathan.robert.anderson@gmail.com Received: by 10.182.59.18 with HTTP; Mon, 24 Feb 2014 06:06:12 -0800 (PST) In-Reply-To: References: From: Jonathan Anderson Date: Mon, 24 Feb 2014 14:06:12 +0000 X-Google-Sender-Auth: SERDqq8r3-GPntv7Qc7uH3u79ko Message-ID: Subject: Re: [HEADSUP] Jenkins running in FreeBSD cluster To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-testing@freebsd.org, freebsd-current Current X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 14:06:34 -0000 This is great stuff, thanks! Can you say what the relationship with tinderbox is intended to be? Is this supposed to subsume tinderbox, or will they have different niches? Is Jenkins going to start e-mailing likely culprits (with e.g. https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin)? Also, can we see more architectures built by Jenkins? Thanks, Jon On 22 February 2014 19:54, Craig Rodrigues wrote: > Hi, > > I just wanted to let the FreeBSD community know that > with the help of some FreeBSD hackers, > we have set up an initial Jenkins Continuous Integration > server in the FreeBSD cluster. We are the jenkins-admin team > and you can contact us at jenkins-admin@freebsd.org. > > We have a few initial builds going and you can see things here: > > https://jenkins.freebsd.org > > We are still working on a few problems, and have some > ambitious plans moving forward, which you can read about > on our status page: > > http://wiki.freebsd.org/Jenkins > > Lastly, if you are able to attend the FreeBSD DevSummit > in Ottawa later this year, we will have a working group discussion > on Jenkins and Continuous Integration testing for FreeBSD on May 15, 2014: > > https://wiki.freebsd.org/201405DevSummit/Jenkins > > We'd love to get more FreeBSD hackers involved to > get this going and improve continuous integration and testing > on FreeBSD. We would like to use the freebsd-testing@freebsd.org > mailing list for followup discussions. > > I'd like to thank my fellow members of the jenkins-admin team > for helping to get things going: > > Steve Kreuzer > Li-Wen Hsu > Steve Wills > R. Tyler Croy > > If we can integrate more automated testing of FreeBSD with this, > that would be really great! > -- > Craig > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Jonathan Anderson jonathan@FreeBSD.org http://freebsd.org/~jonathan/ From owner-freebsd-testing@FreeBSD.ORG Mon Feb 24 17:03:53 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CAA6BD45; Mon, 24 Feb 2014 17:03:53 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E6BC21D1F; Mon, 24 Feb 2014 17:03:52 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id gf5so3102293lab.35 for ; Mon, 24 Feb 2014 09:03:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=cIaoetxse3a42vdELD+YzomGsh5RFOyURS2nXUeAh28=; b=X7xL5MmQQbqZsTzhN+ms21HG+brm5EtppP7SGfKGpW4IePVzDniYgqKhc0x/ZcheAR 5ONVHcWHMrtmm9HmMwJoVyMb/bH8SQrKBp3Lbj0WC0qW+VxQZFwV3PQ0CmLjNUr60kVy Kf+9cLhmhYXjBjBdTBvBWtEi+Rnzz/L5r0E2k0nKAxvkJa0/WDTFQAbptSR0dRvu77Py JH8iWdVx9rKtgLjsAGi91csZXt8/ZSZpSpwvWC59hHN6Pxp+qe1q15oArm1QpdAwrWwk +09WZRaOCZdVuGfFIi0pZqWWzOP+b0iT0T2lc8OJnuFKRHhsjRjtXHW0D1LFeVtpbCOy k1jA== MIME-Version: 1.0 X-Received: by 10.152.25.165 with SMTP id d5mr75704lag.89.1393261430914; Mon, 24 Feb 2014 09:03:50 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 24 Feb 2014 09:03:50 -0800 (PST) Received: by 10.112.30.211 with HTTP; Mon, 24 Feb 2014 09:03:50 -0800 (PST) In-Reply-To: References: Date: Mon, 24 Feb 2014 09:03:50 -0800 X-Google-Sender-Auth: SlI1kZt7W4hrRIbXwnyLqFp_Nio Message-ID: Subject: Re: [HEADSUP] Jenkins running in FreeBSD cluster From: Craig Rodrigues To: Jonathan Anderson Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-testing@freebsd.org, freebsd-current Current X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 17:03:54 -0000 On Feb 24, 2014 6:06 AM, "Jonathan Anderson" wrote: > > This is great stuff, thanks! > > Can you say what the relationship with tinderbox is intended to be? Is this supposed to subsume tinderbox, or will they have different niches? Is Jenkins going to start e-mailing likely culprits (with e.g. https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin)? Also, can we see more architectures built by Jenkins? > > Thanks, > > > Jon Although Jenkins has a lot of tinderbox-like functionality, the effort to set up Jenkins in the FreeBSD cluster is independent of the current FreeBSD Tinderbox system. The FreeBSD Tinderbox is maintained by a separate set of volunteers, and we wish to peacefully coexist with all FreeBSD volunteer projects. However, if you have any ideas for expanded use of Jenkins in the FreeBSD cluster, such as email notifications, take a look at the contact information on our web page and let us know: http://wiki.freebsd.org/Jenkins We are open to feedback, and are just getting started! -- Craig From owner-freebsd-testing@FreeBSD.ORG Mon Feb 24 21:11:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09B26144; Mon, 24 Feb 2014 21:11:09 +0000 (UTC) Received: from mail-lb0-x22e.google.com (mail-lb0-x22e.google.com [IPv6:2a00:1450:4010:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E8145177E; Mon, 24 Feb 2014 21:11:07 +0000 (UTC) Received: by mail-lb0-f174.google.com with SMTP id l4so2675462lbv.33 for ; Mon, 24 Feb 2014 13:11:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=4qh0gfaLj7Nwd364cB+I7L5KeZesRFKqlhhXQmGc1No=; b=KDZ4m2REiY9zqxkM9xpiXj1J5O+TgnmFwGgDeiB4aJDFIxR3hmdtQUF7scg1DxzpU6 kX4Ca6xvzHTt/Wkhf2D83y1s2ON/iF/efExOATVcw9iTufMdS7aYmAVJ7WNMyBTinBg5 N1Br8gGNQ1S7lkaea/HDNU+tmPpxfJO+DW3cIIBYs/pGb1OPpXeuCLu+9DMTEYmPTqH4 Fta4fWMG9omwPy9pJLkCMyFz5iaWjqTAFiykGZJ7IiMMPYObcUVHy5mAkfmWi+8r2/QV cX1BVoHyLrJGIajIAmPsA0CoPtSWNHgjhgGR+isd3NgSwULF/+Sqil/aKeXk9Fysmoqv LG+w== MIME-Version: 1.0 X-Received: by 10.152.19.166 with SMTP id g6mr13432956lae.21.1393276265935; Mon, 24 Feb 2014 13:11:05 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 24 Feb 2014 13:11:05 -0800 (PST) Date: Mon, 24 Feb 2014 13:11:05 -0800 X-Google-Sender-Auth: 8YG6OFKaOGonBp4r3yUtBdkRBPE Message-ID: Subject: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: freebsd-testing@freebsd.org, jenkins-admin@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 21:11:09 -0000 Hi, Some Continuous Integration systems like Jenkins and Atlassian Bamboo can natively parse a test result format called JUnit XML. How hard would it be to add some code to Kyua to optionally be able to generate test result output in JUnit XML? Here is some example Junit XML output: http://nose2.readthedocs.org/en/latest/plugins/junitxml.html It would be super cool if by BSDCan we could demo: * start Kyua tests in Jenkins * have Kyua run the tests and generate the JUnit XML test output * have the tests results visible in the Jenkins web UI, something like: http://zbyhoo.files.wordpress.com/2011/05/jenkins-tests.png How hard would this be to do? Is there anyone available who is interested in working on this? -- Craig From owner-freebsd-testing@FreeBSD.ORG Mon Feb 24 23:07:15 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9BCD755C; Mon, 24 Feb 2014 23:07:15 +0000 (UTC) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DEC91128E; Mon, 24 Feb 2014 23:07:14 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id e4so3737734wiv.5 for ; Mon, 24 Feb 2014 15:07:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=XtaNqonDs+BSoKIU/kWW3dOCkUB4FFp6HcNjUVu4kcU=; b=r7R6o7OdAWUIpa6dT49C79j5qc0YemY8VZAoq40xJRktU6QxSWjH7CGbJdVZxhovNJ 7XeffXQIkrKADiGGO/XNlT/8z/LF4ez4lPkfp6FZ/isj7m1+kmZD1MKDID35uomxi+Do F5HPKzCVQxPGyZYFHyyu+B2CMW+mKf2LsTpRxUYUEinbp5dXJkOG4DOJqVHBMdordYsn f7wN3zBv0zL95bELNL1Ti6G1W2ekhDUj4pyJFV/h5pte8HdHR6KK6GUKUP56UbOHiPRf 24iWIPxmngCAXIHk41gLII/7K45C+IAeM5dTxks2/8lOMokvu4Wxg6GcTkZ+9urN7l8a EQww== MIME-Version: 1.0 X-Received: by 10.180.187.237 with SMTP id fv13mr16504274wic.26.1393283233340; Mon, 24 Feb 2014 15:07:13 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Mon, 24 Feb 2014 15:07:13 -0800 (PST) In-Reply-To: References: Date: Mon, 24 Feb 2014 16:07:13 -0700 X-Google-Sender-Auth: oRDbMQMy59c9jPUDFfdlodStBxM Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Alan Somers To: Craig Rodrigues Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2014 23:07:15 -0000 On Mon, Feb 24, 2014 at 2:11 PM, Craig Rodrigues wrote: > Hi, > > Some Continuous Integration systems like Jenkins > and Atlassian Bamboo can natively parse > a test result format called JUnit XML. > > How hard would it be to add some code > to Kyua to optionally be able to generate test result output > in JUnit XML? > > Here is some example Junit XML output: > > http://nose2.readthedocs.org/en/latest/plugins/junitxml.html > > It would be super cool if by BSDCan we could demo: > > * start Kyua tests in Jenkins > * have Kyua run the tests and generate the JUnit XML test output > * have the tests results visible in the Jenkins web UI, something like: > http://zbyhoo.files.wordpress.com/2011/05/jenkins-tests.png > > > How hard would this be to do? > > Is there anyone available who is interested in working on this? This is a very awesome idea. Kyua's report-html command is only 286 SLOC long, so this probably won't be very hard. Do you have a link to the format specification? I'm afraid that I can't do the work, however. I have extremely limited free time ATM. -Alan From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 00:23:06 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4935BC68; Tue, 25 Feb 2014 00:23:06 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3975818D7; Tue, 25 Feb 2014 00:23:05 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id y1so2702137lam.9 for ; Mon, 24 Feb 2014 16:23:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=zF5xwKyUhnJUWBCaZIEMD8EDDkoPcaR6rFXKy9DjS+4=; b=BgimT1CPj3GL55pRqNudZQCgohXSPx2/5g/oJ9pZRig3i8ZVj7CsIZKX6RzLdqFJ7y nYbTsyYCz8DCtnAjftAQM2f9+CZlFzRuiEm60Z63pajEkrmU6lQaADus1QFGkaJDGI7B xH6fy3hS/SuqNJ0KNtOisZLu1td6qlbzXI2Yx8sEBoCOaSQ9E0nMR2Ovw2x7wV2t+d3S 03JY2DpK1BlSD1pfJdD2FXJeoZP634E1R0vDPsaohvEF8emD884lq19KoNmn9C94lL/H /sR+iIapOydjx88V3VcnR6cCd5uhVr/xGp8vwLp+kU76WEMfPFmoJ3anITf29i00fN+B UwlA== MIME-Version: 1.0 X-Received: by 10.112.132.131 with SMTP id ou3mr13073216lbb.29.1393287783296; Mon, 24 Feb 2014 16:23:03 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 24 Feb 2014 16:23:03 -0800 (PST) In-Reply-To: References: Date: Mon, 24 Feb 2014 16:23:03 -0800 X-Google-Sender-Auth: nA4hBqd4BzoJYgVvU9PqAqRTaUY Message-ID: Subject: Re: March 13: Jenkins and BHyve presentation From: Craig Rodrigues To: freebsd-current Current , "freebsd-virtualization@freebsd.org" , freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Dmitry Morozovsky , Emanuel Haupt X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 00:23:06 -0000 On Mon, Feb 24, 2014 at 1:04 AM, Craig Rodrigues wrote: > Hi, > > I will be giving a talk discussing what I have done > settings up Jenkins ( https://jenkins.freebsd.org ) > in the FreeBSD cluster, using BHyve VM's: > > The presentation will be on March 13 in Mountain View, California, U.S.A.: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ > > If you think you can come, please RSVP on the Meetup site, so that > the organizers can get a big enough room. > > I am interested in collaborating with FreeBSD hackers who can > contribute to the effort and help expand testing of FreeBSD!! > Hi, For those who cannot attend the meetup in Mountain View, California, U.S.A., I will post a link to the slides after the presentation. I am not sure if video/webcast will be available. I have given a similar presentation at the 2013 FreeBSD Vendor Summit describing the use of Jenkins in the FreeNAS project, so you can look at this to get an idea: http://www.ixsystems.com/whats-new/jenkins-bhyve-and-webdriver-continuous-integration-testing-on-freenas/ However, my presentation on March 13 will be focused on how I set up https://jenkins.freebsd.org If you cannot make the March 13 meetup, I will be giving a similar presentation on May 17 at BSDCan: http://www.bsdcan.org/2014/schedule/events/445.en.html and on May 15 there will be a working group where we will discuss how to expand Continuous Testing in the FreeBSD project: https://wiki.freebsd.org/201405DevSummit Hope to see folks at one of these events, so we can do some interesting collaboration! -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 00:58:21 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1ABB1470; Tue, 25 Feb 2014 00:58:21 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 377111B22; Tue, 25 Feb 2014 00:58:20 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id c11so138734lbj.40 for ; Mon, 24 Feb 2014 16:58:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=47DaFuOI45wpK9449/2M0O4k4O1R0v80qalAPYcwKWo=; b=BF0b75Hm/6mZr/HfWnCTklGYlkEZcp5x0lxkKVoYJa9FpVXWioZZVkYsqYVbNjYhnu 5kUNW8CL/sjF2ViBOccsFCSdwlddfxNunCapc+jVL7mVkA/1qUGV0i7EqLHH5dPv9oeT E89kC2po7IRWwvWeTNEHZAL3JWCRnh5MnpPMRXYMd7IoIFseIF/08pCF5jkvhj12Gr8F ACBop522ay38VW74AE0W9Zkij5q9hFOrFDhGp40uVsgIWFfVrsnZMq19aiZx4sYn91bw jMK9H4GsZF3FBmZEo+8SXrp39rJtxrnEojSLM3GTJIv0mfZbiAyD5/VSleqp4mlvzmlM xU1w== MIME-Version: 1.0 X-Received: by 10.112.14.1 with SMTP id l1mr13384254lbc.39.1393289898152; Mon, 24 Feb 2014 16:58:18 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 24 Feb 2014 16:58:18 -0800 (PST) In-Reply-To: References: Date: Mon, 24 Feb 2014 16:58:18 -0800 X-Google-Sender-Auth: QBGWCf7zHEs-wiDLv_3izZSQQcI Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" , jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 00:58:21 -0000 On Mon, Feb 24, 2014 at 3:07 PM, Alan Somers wrote: > On Mon, Feb 24, 2014 at 2:11 PM, Craig Rodrigues > wrote: > > Hi, > > > > Some Continuous Integration systems like Jenkins > > and Atlassian Bamboo can natively parse > > a test result format called JUnit XML. > > > > How hard would it be to add some code > > to Kyua to optionally be able to generate test result output > > in JUnit XML? > > > > Here is some example Junit XML output: > > > > http://nose2.readthedocs.org/en/latest/plugins/junitxml.html > > > > It would be super cool if by BSDCan we could demo: > > > > * start Kyua tests in Jenkins > > * have Kyua run the tests and generate the JUnit XML test output > > * have the tests results visible in the Jenkins web UI, something like: > > http://zbyhoo.files.wordpress.com/2011/05/jenkins-tests.png > > > > > > How hard would this be to do? > > > > Is there anyone available who is interested in working on this? > > This is a very awesome idea. Kyua's report-html command is only 286 > SLOC long, so this probably won't be very hard. Do you have a link to > the format specification? I'm afraid that I can't do the work, > however. I have extremely limited free time ATM. > The closest thing I have to a format specification are links to these XSD files: https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd (simple) http://windyroad.com.au/dl/Open%20Source/JUnit.xsd (more complicated) -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 03:06:03 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9317910; Tue, 25 Feb 2014 03:06:03 +0000 (UTC) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F40911728; Tue, 25 Feb 2014 03:06:02 +0000 (UTC) Received: by mail-wi0-f178.google.com with SMTP id cc10so113293wib.11 for ; Mon, 24 Feb 2014 19:06:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=CPdF84RlqPHErU4QQNyjmm8qjnZT42FweB9sAGyGEEQ=; b=g9Qq/g0j82EetMGmj/madGuP1goJ1hduapx0hPO9AttEjN8lp1wvc370qXCwgKutnp h5OeuA+P5Q3mSht5amXvgoWASYenwJ+jByo9eK8mD09LyA5joeCNxPRmR47g7h4q3aDr KiRnDjRne7f0rG+YwrCxiqXPP6A9/F8QI6/InDbhfzVwdXMpfCgAQev2hXqQF7YXlH6A z/qimn6UGoDWuq4p2C8KNMJNMrx94j2OG/zMx+eKfn9GDAlGhnE3WUJUpbIeSpwAE7Yh mWR4FQp4fvZu00xuhdvOT5GjSSyFrtc+gVJACIjMVp7VjEJukxVvp0T5vJ+b7WrjYlWv Qq1A== MIME-Version: 1.0 X-Received: by 10.194.87.195 with SMTP id ba3mr25954wjb.53.1393297561406; Mon, 24 Feb 2014 19:06:01 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Mon, 24 Feb 2014 19:06:01 -0800 (PST) In-Reply-To: References: Date: Mon, 24 Feb 2014 20:06:01 -0700 X-Google-Sender-Auth: RIJAB87bqpPERCLWIbqFlNClxE0 Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Alan Somers To: Craig Rodrigues Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 03:06:03 -0000 On Mon, Feb 24, 2014 at 5:58 PM, Craig Rodrigues wrote: > > > > On Mon, Feb 24, 2014 at 3:07 PM, Alan Somers wrote: >> >> On Mon, Feb 24, 2014 at 2:11 PM, Craig Rodrigues >> wrote: >> > Hi, >> > >> > Some Continuous Integration systems like Jenkins >> > and Atlassian Bamboo can natively parse >> > a test result format called JUnit XML. >> > >> > How hard would it be to add some code >> > to Kyua to optionally be able to generate test result output >> > in JUnit XML? >> > >> > Here is some example Junit XML output: >> > >> > http://nose2.readthedocs.org/en/latest/plugins/junitxml.html >> > >> > It would be super cool if by BSDCan we could demo: >> > >> > * start Kyua tests in Jenkins >> > * have Kyua run the tests and generate the JUnit XML test output >> > * have the tests results visible in the Jenkins web UI, something like: >> > http://zbyhoo.files.wordpress.com/2011/05/jenkins-tests.png >> > >> > >> > How hard would this be to do? >> > >> > Is there anyone available who is interested in working on this? >> >> This is a very awesome idea. Kyua's report-html command is only 286 >> SLOC long, so this probably won't be very hard. Do you have a link to >> the format specification? I'm afraid that I can't do the work, >> however. I have extremely limited free time ATM. > > > > > The closest thing I have to a format specification are links to these XSD > files: > > https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd > (simple) > http://windyroad.com.au/dl/Open%20Source/JUnit.xsd (more complicated) On second thought, I think that Junit-XML output is a good idea in the short term, but not in the long term. In the long term, displaying Kyua results in Jenkins is nonideal because: 1) A Kyua run could easily have thousands of test cases, which are impractical to display in a flat view like Jenkins does. Some sort of hierarchical view will be necessary. 2) I don't see any "expected failure" results in the links that you provided. That's an absolute requirement for Kyua. Treating expected failures like ordinary failures will greatly reduce the utility of Kyua, by reducing its ability to spot regressions. 3) The Kyua DB format is designed to support viewing a test case's success history over time, even though the tools don't do it yet. That's going to be a killer feature when its implemented, but I doubt that Jenkins has support for that kind of thing. I think that it will be difficult if not impossible to add it via a Jenkins plugin, but perhaps I just lack imagination. In the long term, I think we'll need to write a native webapp for viewing Kyua results. But in the short term, it will definitely be easier to get Jenkins integration, so I think that it's worthwhile to pursue both. It sounds like we'll probably have to read the Jenkins source to figure out exactly what it thinks JUnit-XML format is. -Alan From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 03:12:01 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41380D40 for ; Tue, 25 Feb 2014 03:12:01 +0000 (UTC) Received: from mail-qc0-f174.google.com (mail-qc0-f174.google.com [209.85.216.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F0D8517F0 for ; Tue, 25 Feb 2014 03:12:00 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id e16so3087819qcx.19 for ; Mon, 24 Feb 2014 19:12:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=OtuB9onctmS5tnQ3VDRoIiQfgjVqoH+SjosEcyNZfIg=; b=kEwl3MAxtc0aknKvwLBroPngfRThs8cfvkwC4i5MIvalw9oXcmeLblEdltc9WwKkrO Tv2ucuaV6VibSZ4tfmgpwSv+nA1PGdSXy6Jub2sx7/rqHP3bK/UEA+bFrMpTyYC4q1ee L7wwRpElzGxazbGanVoTTeDSO0vTjOnid3VrzKsmY2aMUQFjGkg2rOprkX1Zg9adJy7I 2YV7aQJmCii1iemWS2/u/X5mDb/zNrygUswlE+ClCIjgSdc2bqpRisRPuUMY86J3DXxa LlgXpQqnR6sYgfW4J7vuEy4/e4PHPylDUYtvu1e0HS1RA2y6C+eG2ddmmCmuzRXM9f8S gp9g== X-Gm-Message-State: ALoCoQkNpWCCnfjenNQKpoqP3rFBxNhzhhIcm5Ndo0KCW7XWqj9jWxMfrfaGYCkZZIL8wMx93YsV X-Received: by 10.224.157.7 with SMTP id z7mr34924222qaw.37.1393297460237; Mon, 24 Feb 2014 19:04:20 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Mon, 24 Feb 2014 19:04:00 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: References: From: Julio Merino Date: Mon, 24 Feb 2014 22:04:00 -0500 X-Google-Sender-Auth: IVAVoSrHWjC4PsjBOMP4_PwEB54 Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? To: Craig Rodrigues Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-testing@freebsd.org, jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 03:12:01 -0000 On Mon, Feb 24, 2014 at 4:11 PM, Craig Rodrigues wrote: > > Hi, > How hard would it be to add some code > to Kyua to optionally be able to generate test result output > in JUnit XML? (For changes to Kyua itself, please don't forget to include the kyua-discuss (at googlegroups.com) mailing list.) > It would be super cool if by BSDCan we could demo: Certainly. I should have some decent amount of time in a couple of weeks to work on this. Improving reports is the highest priority thing that I have to focus on to improve the face of the test suite. > How hard would this be to do? > > > Is there anyone available who is interested in working on this? It's not hard: the report generation code is self-contained as you can see in cmd_report.cpp and cmd_report_html.cpp. And I think adding JUnit XML output would be a nice addition. In the mid-to-long term, I have some ideas on how to redo the whole report generation code to make it easier to extend and customize. (Well, basically just some ideas I wanna toy with -- which involve rewriting this whole thing in Lua so that adding new backends can be done without rebuilding Kyua.) But these changes will take some time. A first strategical step separate from that and that would make your proposal simpler is to remove the special-cased report-html and instead make the "report" command be able to spit out various formats based on a flag (e.g. --output=junit:/my/file.xml, --output=html:/my/file.html). I have to do this anyway to resolve some "show-stopper" issues in NetBSD land, and with this done plugging a JUnit backend would be trivial and mostly a copy/paste of the HTML backend. From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 03:35:37 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48E8213A for ; Tue, 25 Feb 2014 03:35:37 +0000 (UTC) Received: from sanddollar.geekisp.com (sanddollar.geekisp.com [216.168.135.167]) by mx1.freebsd.org (Postfix) with SMTP id F25A619CA for ; Tue, 25 Feb 2014 03:35:36 +0000 (UTC) Received: (qmail 15705 invoked by uid 1003); 25 Feb 2014 03:35:30 -0000 Received: from unknown (HELO kiwi) (tyler@monkeypox.org@173.239.70.2) by mail.geekisp.com with (DHE-RSA-AES256-SHA encrypted) SMTP; 25 Feb 2014 03:35:29 -0000 Date: Mon, 24 Feb 2014 19:35:26 -0800 From: "R. Tyler Croy" To: Alan Somers Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? Message-ID: <20140225033526.GN85115@kiwi> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="zBPbvmIlJjvpbu6L" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-testing@freebsd.org" , jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 03:35:37 -0000 --zBPbvmIlJjvpbu6L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable snipped some context, reply still inline On Mon, 24 Feb 2014, Alan Somers wrote: > On Mon, Feb 24, 2014 at 5:58 PM, Craig Rodrigues wr= ote: > >> This is a very awesome idea. Kyua's report-html command is only 286 > >> SLOC long, so this probably won't be very hard. Do you have a link to > >> the format specification? I'm afraid that I can't do the work, > >> however. I have extremely limited free time ATM. > > > > > > The closest thing I have to a format specification are links to these X= SD > > files: > > > > https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-= model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd > > (simple) > > http://windyroad.com.au/dl/Open%20Source/JUnit.xsd (more complicated) >=20 > On second thought, I think that Junit-XML output is a good idea in the > short term, but not in the long term. In the long term, displaying > Kyua results in Jenkins is nonideal because: >=20 > 1) A Kyua run could easily have thousands of test cases, which are > impractical to display in a flat view like Jenkins does. Some sort of > hierarchical view will be necessary. Jenkins already solves for this, if the Kyua generated JUnit-XML uses dot-delimited (foo.bar) "class names" then Jenkins will organize things int= o a tree-like hierarchy. Here's an example of the display with Jenkins itself and its 4000+ test cas= es: > 2) I don't see any "expected failure" results in the links that you > provided. That's an absolute requirement for Kyua. Treating expected > failures like ordinary failures will greatly reduce the utility of > Kyua, by reducing its ability to spot regressions. JUnit allows for the concept of "skipped" tests, which Jenkins' parsing code respects. IIRC this is just a "skipped=3Dtrue" on a , I believe t= hat would help solve for this case. > 3) The Kyua DB format is designed to support viewing a test case's > success history over time, even though the tools don't do it yet. > That's going to be a killer feature when its implemented, but I doubt > that Jenkins has support for that kind of thing. I think that it will > be difficult if not impossible to add it via a Jenkins plugin, but > perhaps I just lack imagination. Boy howdy does Jenkins definitely support this! Unfortunately it's somewhat limited to a Job itself, but you can get pretty good test history down to a test-case level; > In the long term, I think we'll need to write a native webapp for > viewing Kyua results. But in the short term, it will definitely be > easier to get Jenkins integration, so I think that it's worthwhile to > pursue both. It sounds like we'll probably have to read the Jenkins > source to figure out exactly what it thinks JUnit-XML format is. Depending on your level of fear/loathing for Java, I actually disagree that Kyua should spend time building a custom web app for viewing results. The Jenkins project supports a *very* rich set of plugin APIs and extension poi= nts to where writing a custom Kyua plugin would be relatively easy. There's plenty of non-Jenkins examples of generating JUnit XML for what it's worth, in the Ruby community, the ci_reporter gem generates XML, Cucumber d= oes natively as well, Python's nosetest-xml package has plenty of examples as w= ell. I took a look at the C++ in Kyua and I'm afraid I'm not capable enough to w= rite the reporter, but if somebody wants to ping me on IRC (rtyler) I can probab= ly help them a bit if they have trouble with the format. - R. Tyler Croy ------------------------------------------------------ Code: Chatter: % gpg --keyserver keys.gnupg.net --recv-key 3F51E16F ------------------------------------------------------ --zBPbvmIlJjvpbu6L Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlMMD34ACgkQFCbH3D9R4W9HSQCfbDAZBPEv1ZYkYeaBNfnpvaPi Pl8AnjkGrAKg6jWQ+A1NgwkKh9tJp+YI =0k2Z -----END PGP SIGNATURE----- --zBPbvmIlJjvpbu6L-- From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 16:11:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76265D3B for ; Tue, 25 Feb 2014 16:11:38 +0000 (UTC) Received: from relay02.pair.com (relay02.pair.com [209.68.5.16]) by mx1.freebsd.org (Postfix) with SMTP id 153DD1A4D for ; Tue, 25 Feb 2014 16:11:37 +0000 (UTC) Received: (qmail 54644 invoked from network); 25 Feb 2014 16:11:30 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay02.pair.com with SMTP; 25 Feb 2014 16:11:30 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s1PGBTaF066192 for ; Tue, 25 Feb 2014 17:11:29 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s1PGBTBJ066191 for freebsd-testing@freebsd.org; Tue, 25 Feb 2014 17:11:29 +0100 (CET) (envelope-from pho) Date: Tue, 25 Feb 2014 17:11:29 +0100 From: Peter Holm To: freebsd-testing@freebsd.org Subject: My first ATF test Message-ID: <20140225161129.GA59741@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 16:11:38 -0000 In order to understand how ATF works I wrote a small test so I had something to work with: http://people.freebsd.org/~pho/kern_descrip_test.diff Did I get it right? -- Peter From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 16:47:55 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C6FEABE for ; Tue, 25 Feb 2014 16:47:55 +0000 (UTC) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CB0421D59 for ; Tue, 25 Feb 2014 16:47:54 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id e4so4761339wiv.11 for ; Tue, 25 Feb 2014 08:47:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=akVBzOMOXAlFxsYGziAqYJoXM/fs6pRkDH/tIGFTgbA=; b=aDDYjcDIVHI7nH8rC6qb8FgBfmRNG5zf8CvAoH6Ip9EBd2Qzcn7wWd++1Me8/0TZsQ nZdgGGgijoO+MvlnLyl4BoQXTt0PLKpkMGJmpPEv+Vu8QvxOqfuG3lpUmot+xHXYweq7 uMb/iebAhd33Pf3aP/j6TVUw0pYqKX8igrodpLwWYxpHq+sibmrEPADaqXtPLJ9ApaYD /5sDULAvzM0JMyv7HEq6QfJz6IYATwcUv6yPP/UoZQjhuMrcsEX2gwa+U3G2tyt962kN y98c9yYm7Tt5JqEdc8z0OA5fcU7t0QvOGonbvZT1De29xC9j0SY3om23eXt+L3mp688+ AjFw== MIME-Version: 1.0 X-Received: by 10.180.97.37 with SMTP id dx5mr3898185wib.53.1393346873005; Tue, 25 Feb 2014 08:47:53 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Tue, 25 Feb 2014 08:47:52 -0800 (PST) In-Reply-To: <20140225161129.GA59741@x2.osted.lan> References: <20140225161129.GA59741@x2.osted.lan> Date: Tue, 25 Feb 2014 09:47:52 -0700 X-Google-Sender-Auth: CpCYweCU5BKrkAoRT3fdnS8R6Nc Message-ID: Subject: Re: My first ATF test From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 16:47:55 -0000 On Tue, Feb 25, 2014 at 9:11 AM, Peter Holm wrote: > In order to understand how ATF works I wrote a small test so I had > something to work with: > http://people.freebsd.org/~pho/kern_descrip_test.diff > Did I get it right? ATF-wise, it looks good. However, it's a bad idea to use random numbers in test code, except in stress tests. Random numbers result in irreproducible tests. How about replacing the body of dup2_r234131 with something like this? int fd1, fd2, ret; fd1 = open("/etc/passwd", O_RDONLY); fd2 = INT_MAX; ret = dup2(fd1, fd2); ATF_CHECK_EQ(-1, ret); ATF_CHECK_EQ(EBADF, errno); On a side note, perhaps WARNS should be set in atf.test.mk, so we won't have to set it in every other Makefile. -Alan From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 17:12:39 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EF4EF224; Tue, 25 Feb 2014 17:12:39 +0000 (UTC) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9408D10D9; Tue, 25 Feb 2014 17:12:39 +0000 (UTC) Received: by mail-vc0-f172.google.com with SMTP id lf12so7633918vcb.17 for ; Tue, 25 Feb 2014 09:12:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pkcj1ysaewpfHxLWw83e5kavEZ3b7dr3s69VFMNLP4o=; b=QGD4cgdHUVU90YlJEijfY0r6w5mALsuW+YW/ighFn3bbuMBZbPpvj914p4cOGRzc7y rFWs5l7pJQoMbmCnkIQymKVDHC5FNU63kOnfE2edJH5IvTV5CyVpv1CzOniL5htNeIBg T+QhHykaOOzws9WD3fncoJ546OZW+bKuuMKO9KnWZyMhn3QevFrpWpAJpCo53Db6gRgP 0VcMVV6UfH/bmi0OdkyIaENTR56cotOGJ4/Ve7NelOlGA41enoT7ketvf+61fwjY5IF2 vUN1stB0XlAba+vQLfEHsl16hmcx/BNhegMklNmriuS2r8zU4pKYeGvZXYaLZfCTokZ2 Xs7A== MIME-Version: 1.0 X-Received: by 10.58.170.69 with SMTP id ak5mr1890783vec.28.1393348358682; Tue, 25 Feb 2014 09:12:38 -0800 (PST) Received: by 10.58.235.166 with HTTP; Tue, 25 Feb 2014 09:12:38 -0800 (PST) In-Reply-To: References: <20140225161129.GA59741@x2.osted.lan> Date: Tue, 25 Feb 2014 12:12:38 -0500 Message-ID: Subject: Re: My first ATF test From: Mehmet Erol Sanliturk To: Alan Somers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 17:12:40 -0000 On Tue, Feb 25, 2014 at 11:47 AM, Alan Somers wrote: > On Tue, Feb 25, 2014 at 9:11 AM, Peter Holm wrote: > > In order to understand how ATF works I wrote a small test so I had > > something to work with: > > http://people.freebsd.org/~pho/kern_descrip_test.diff > > Did I get it right? > > ATF-wise, it looks good. However, it's a bad idea to use random > numbers in test code, except in stress tests. Random numbers result > in irreproducible tests. How about replacing the body of dup2_r234131 > with something like this? > > int fd1, fd2, ret; > fd1 = open("/etc/passwd", O_RDONLY); > fd2 = INT_MAX; > ret = dup2(fd1, fd2); > ATF_CHECK_EQ(-1, ret); > ATF_CHECK_EQ(EBADF, errno); > > On a side note, perhaps WARNS should be set in atf.test.mk, so we > won't have to set it in every other Makefile. > > -Alan > _______________________________________________ > > When random numbers are used , it is possible to make the runs reproducible in the following way : Generate a specified number of random numbers and store them into a file . During usage , for random numbers , traverse that file . This may be repeated any number of times for different other parameters . All of the runs will use the same random numbers . Then the results ( which they are generated from the same distribution ) may be compared with suitable statistical tests . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 18:33:13 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B07658A for ; Tue, 25 Feb 2014 18:33:13 +0000 (UTC) Received: from relay02.pair.com (relay02.pair.com [209.68.5.16]) by mx1.freebsd.org (Postfix) with SMTP id E83BD19B3 for ; Tue, 25 Feb 2014 18:33:12 +0000 (UTC) Received: (qmail 7106 invoked from network); 25 Feb 2014 18:33:11 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay02.pair.com with SMTP; 25 Feb 2014 18:33:11 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s1PIX6WU070767; Tue, 25 Feb 2014 19:33:07 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s1PIX6e7070766; Tue, 25 Feb 2014 19:33:06 +0100 (CET) (envelope-from pho) Date: Tue, 25 Feb 2014 19:33:06 +0100 From: Peter Holm To: Alan Somers Subject: Re: My first ATF test Message-ID: <20140225183306.GA70295@x2.osted.lan> References: <20140225161129.GA59741@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 18:33:13 -0000 On Tue, Feb 25, 2014 at 09:47:52AM -0700, Alan Somers wrote: > On Tue, Feb 25, 2014 at 9:11 AM, Peter Holm wrote: > > In order to understand how ATF works I wrote a small test so I had > > something to work with: > > http://people.freebsd.org/~pho/kern_descrip_test.diff > > Did I get it right? > > ATF-wise, it looks good. However, it's a bad idea to use random > numbers in test code, except in stress tests. Random numbers result > in irreproducible tests. How about replacing the body of dup2_r234131 > with something like this? > Thank you for looking at this. > int fd1, fd2, ret; > fd1 = open("/etc/passwd", O_RDONLY); > fd2 = INT_MAX; > ret = dup2(fd1, fd2); > ATF_CHECK_EQ(-1, ret); > ATF_CHECK_EQ(EBADF, errno); > OK. > On a side note, perhaps WARNS should be set in atf.test.mk, so we > won't have to set it in every other Makefile. > I agree. http://people.freebsd.org/~pho/kern_descrip_test-v2.diff I have a style question: You have the constant as first argument to ATF_CHECK_EQ. It seems that NetBSD has chosen to have it last, in most cases. Is your choice the FreeBSD standard? > -Alan -- Peter From owner-freebsd-testing@FreeBSD.ORG Tue Feb 25 18:42:53 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE4AA7B4 for ; Tue, 25 Feb 2014 18:42:53 +0000 (UTC) Received: from relay03.pair.com (relay03.pair.com [209.68.5.17]) by mx1.freebsd.org (Postfix) with SMTP id 912CD1ABA for ; Tue, 25 Feb 2014 18:42:53 +0000 (UTC) Received: (qmail 59729 invoked from network); 25 Feb 2014 18:36:12 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay03.pair.com with SMTP; 25 Feb 2014 18:36:12 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s1PIaBYu070871; Tue, 25 Feb 2014 19:36:11 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s1PIaB0Q070870; Tue, 25 Feb 2014 19:36:11 +0100 (CET) (envelope-from pho) Date: Tue, 25 Feb 2014 19:36:11 +0100 From: Peter Holm To: Mehmet Erol Sanliturk Subject: Re: My first ATF test Message-ID: <20140225183611.GA70774@x2.osted.lan> References: <20140225161129.GA59741@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 18:42:53 -0000 On Tue, Feb 25, 2014 at 12:12:38PM -0500, Mehmet Erol Sanliturk wrote: > On Tue, Feb 25, 2014 at 11:47 AM, Alan Somers <[1]asomers@freebsd.org> > wrote: > > On Tue, Feb 25, 2014 at 9:11 AM, Peter Holm wrote: > > In order to understand how ATF works I wrote a small test so I had > > something to work with: > > [2]http://people.freebsd.org/~pho/kern_descrip_test.diff > > Did I get it right? > ATF-wise, it looks good.  However, it's a bad idea to use random > numbers in test code, except in stress tests.  Random numbers > result > in irreproducible tests.  How about replacing the body of > dup2_r234131 > with something like this? >  int fd1, fd2, ret; >  fd1 =  open("/etc/passwd", O_RDONLY); >  fd2 = INT_MAX; >  ret = dup2(fd1, fd2); >  ATF_CHECK_EQ(-1, ret); >  ATF_CHECK_EQ(EBADF, errno); > On a side note, perhaps WARNS should be set in [3]atf.test.mk, so we > won't have to set it in every other Makefile. > -Alan > _______________________________________________ > > When random numbers are used , it is possible to make the runs > reproducible in the following way : > Generate a specified number of random numbers and store them into a > file . > During usage , for random numbers , traverse that file . > This may be repeated any number of times for different other parameters > . > All of the runs will use the same random numbers . > Then the results ( which they are generated from the same distribution > ) may be compared with suitable statistical tests . > Thank you very much . > Mehmet Erol Sanliturk > I guess I'm too used to using random values in tests :) Using random(3) with an initial seed of "1" could have been an alternative, but ... - Peter From owner-freebsd-testing@FreeBSD.ORG Wed Feb 26 03:59:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A89CB817; Wed, 26 Feb 2014 03:59:14 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 10CB815BA; Wed, 26 Feb 2014 03:59:13 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id l18so1178835wgh.33 for ; Tue, 25 Feb 2014 19:59:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=7xvP0PHRe/Jkr9C5XscgG+qiU/scRISu/GaVM3+5ynA=; b=UeJu3eJTgyKzihaxhlsscdkXSNleDfVQoMlsGd9/f0A1SD4G2FS2R/Wo1vuR9zuaMg Ekjg5pbKdRwkEha/PlsiUo4r0seJYojcrl7bwDCTjoOS2N79xZV7f3zP9+K458/ub4CK t/OQEGAyQZG24PawldBBrqOhoBbBPKYrXYN/n/bTQwiMNeKA2abfUmAEbTCUFHE4G7WF A7TZw4oYt60QT9QhINztDdiXQxhruTUAZG/AQi9JbjVESfMHMgbxSPAvccg0UEgcqL8m EhB44TtrsZ5bQSOEj046ZDghGmFYdZ/DIDE5O2BJvWxxaOF9z1q65K9fGC5nMn8G/oT+ ddnA== MIME-Version: 1.0 X-Received: by 10.180.187.16 with SMTP id fo16mr2905066wic.26.1393387152289; Tue, 25 Feb 2014 19:59:12 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Tue, 25 Feb 2014 19:59:12 -0800 (PST) In-Reply-To: <20140225183306.GA70295@x2.osted.lan> References: <20140225161129.GA59741@x2.osted.lan> <20140225183306.GA70295@x2.osted.lan> Date: Tue, 25 Feb 2014 20:59:12 -0700 X-Google-Sender-Auth: BfwdG0NHSVrmILHPE-Wf8ENsDIs Message-ID: Subject: Re: My first ATF test From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 03:59:14 -0000 On Tue, Feb 25, 2014 at 11:33 AM, Peter Holm wrote: > On Tue, Feb 25, 2014 at 09:47:52AM -0700, Alan Somers wrote: >> On Tue, Feb 25, 2014 at 9:11 AM, Peter Holm wrote: >> > In order to understand how ATF works I wrote a small test so I had >> > something to work with: >> > http://people.freebsd.org/~pho/kern_descrip_test.diff >> > Did I get it right? >> >> ATF-wise, it looks good. However, it's a bad idea to use random >> numbers in test code, except in stress tests. Random numbers result >> in irreproducible tests. How about replacing the body of dup2_r234131 >> with something like this? >> > > Thank you for looking at this. > >> int fd1, fd2, ret; >> fd1 = open("/etc/passwd", O_RDONLY); >> fd2 = INT_MAX; >> ret = dup2(fd1, fd2); >> ATF_CHECK_EQ(-1, ret); >> ATF_CHECK_EQ(EBADF, errno); >> > > OK. > >> On a side note, perhaps WARNS should be set in atf.test.mk, so we >> won't have to set it in every other Makefile. >> > > I agree. > http://people.freebsd.org/~pho/kern_descrip_test-v2.diff > > I have a style question: > You have the constant as first argument to ATF_CHECK_EQ. It seems > that NetBSD has chosen to have it last, in most cases. Is your > choice the FreeBSD standard? > The atf-c-api(3) man page doesn't recommend one or the other, and its examples show it both ways. My personal habit is to put the constant first. It's been ingrained by both googletest and Ruby's test/unit, which both have a clear policy. Personally, I would say that if you're editing an existing file, try to keep with its preexisting style. Otherwise, I don't think it matters. From owner-freebsd-testing@FreeBSD.ORG Wed Feb 26 04:02:40 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7FBA987 for ; Wed, 26 Feb 2014 04:02:40 +0000 (UTC) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6750F1633 for ; Wed, 26 Feb 2014 04:02:40 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id hm4so1627393wib.1 for ; Tue, 25 Feb 2014 20:02:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=/0NE6hhJvnYKqnWARun13bUGDcdKlZQ6ifGPdJv16/Y=; b=lz1zwHQeNh1xsQ6pe8eX/j2mq3RCqvYFCaTTOU7DVIuPht87yZVe2B8FxxHvesD1mV s643+KKpVhQ3jPui2ZlyDmcFPvYO79aXkbS93yVos9V+f4N0ZxJIru57abKcC9hv/XH3 EoDkaS9iDhAnljCNyycmEdD3KcNAXLalYQAdOInj1PkHiUSloRkHONHGq0cSuYt3vrbC 8Rl+HE09Xg4O+7s71zrR/R7SJRZ8SbBAL5Wb+MdguRIlXpeBF6kIzutOt77vdCq9ym5g DAf+JlZ92L3dwKzTUpklwIpV+6k0Fv0P1C6OvKq4WmeEjz1VSskNpwu5/cbRUQPlr2w2 pY3g== MIME-Version: 1.0 X-Received: by 10.180.187.237 with SMTP id fv13mr5962363wic.26.1393387358448; Tue, 25 Feb 2014 20:02:38 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Tue, 25 Feb 2014 20:02:38 -0800 (PST) In-Reply-To: <20140225183611.GA70774@x2.osted.lan> References: <20140225161129.GA59741@x2.osted.lan> <20140225183611.GA70774@x2.osted.lan> Date: Tue, 25 Feb 2014 21:02:38 -0700 X-Google-Sender-Auth: BrVLojkdbPaL90DNqcwTPdf5QCs Message-ID: Subject: Re: My first ATF test From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 04:02:41 -0000 On Tue, Feb 25, 2014 at 11:36 AM, Peter Holm wrote: > On Tue, Feb 25, 2014 at 12:12:38PM -0500, Mehmet Erol Sanliturk wrote: >> On Tue, Feb 25, 2014 at 11:47 AM, Alan Somers <[1]asomers@freebsd.org= > >> wrote: >> >> On Tue, Feb 25, 2014 at 9:11 AM, Peter Holm wrote: >> > In order to understand how ATF works I wrote a small test so I ha= d >> > something to work with: >> > [2]http://people.freebsd.org/~pho/kern_descrip_test.diff >> > Did I get it right? >> ATF-wise, it looks good. =C2 However, it's a bad idea to use random >> numbers in test code, except in stress tests. =C2 Random numbers >> result >> in irreproducible tests. =C2 How about replacing the body of >> dup2_r234131 >> with something like this? >> =C2 int fd1, fd2, ret; >> =C2 fd1 =3D =C2 open("/etc/passwd", O_RDONLY); >> =C2 fd2 =3D INT_MAX; >> =C2 ret =3D dup2(fd1, fd2); >> =C2 ATF_CHECK_EQ(-1, ret); >> =C2 ATF_CHECK_EQ(EBADF, errno); >> On a side note, perhaps WARNS should be set in [3]atf.test.mk, so w= e >> won't have to set it in every other Makefile. >> -Alan >> _______________________________________________ >> >> When random numbers are used , it is possible to make the runs >> reproducible in the following way : >> Generate a specified number of random numbers and store them into a >> file . >> During usage , for random numbers , traverse that file . >> This may be repeated any number of times for different other paramete= rs >> . >> All of the runs will use the same random numbers . >> Then the results ( which they are generated from the same distributio= n >> ) may be compared with suitable statistical tests . >> Thank you very much . >> Mehmet Erol Sanliturk >> > > I guess I'm too used to using random values in tests :) > Using random(3) with an initial seed of "1" could have been an > alternative, but ... But in this case it's not necessary. This isn't a stress test; it's a basic functional test. If Peter had been attempting to trigger a race condition or something like that, then random(3) with a fixed seed would indeed be appropriate. -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Feb 27 08:18:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 256FDD91 for ; Thu, 27 Feb 2014 08:18:09 +0000 (UTC) Received: from relay03.pair.com (relay03.pair.com [209.68.5.17]) by mx1.freebsd.org (Postfix) with SMTP id B7DAA1778 for ; Thu, 27 Feb 2014 08:18:08 +0000 (UTC) Received: (qmail 53715 invoked from network); 27 Feb 2014 08:18:01 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay03.pair.com with SMTP; 27 Feb 2014 08:18:01 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s1R8I03r030166 for ; Thu, 27 Feb 2014 09:18:00 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s1R8I0Lc030163 for freebsd-testing@freebsd.org; Thu, 27 Feb 2014 09:18:00 +0100 (CET) (envelope-from pho) Date: Thu, 27 Feb 2014 09:17:59 +0100 From: Peter Holm To: freebsd-testing@freebsd.org Subject: 100% repeatability? Message-ID: <20140227081759.GA29517@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 08:18:09 -0000 I looked at creating a test scenario for the following piece of code from src/sys/kern/kern_fork.c: 881 if ((nprocs >= maxproc - 10 && priv_check_cred(td->td_ucred, 882 PRIV_MAXPROC, 0) != 0) || nprocs >= maxproc) { 883 error = EAGAIN; 884 goto fail; 885 } I came up with a test scenario for this: http://people.freebsd.org/~pho/kern_fork_test.c But it is not 100% repeatable, as I would expect many other kernel tests will be. The test case runs as expected, until for example root decides to do a parallel buildworld. Is this a show stopper? -- Peter From owner-freebsd-testing@FreeBSD.ORG Thu Feb 27 12:55:54 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AD83268 for ; Thu, 27 Feb 2014 12:55:54 +0000 (UTC) Received: from relay00.pair.com (relay00.pair.com [209.68.5.9]) by mx1.freebsd.org (Postfix) with SMTP id CD97C14D9 for ; Thu, 27 Feb 2014 12:55:53 +0000 (UTC) Received: (qmail 75609 invoked from network); 27 Feb 2014 12:49:12 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay00.pair.com with SMTP; 27 Feb 2014 12:49:12 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s1RCnB1g038206 for ; Thu, 27 Feb 2014 13:49:11 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s1RCnB5B038205 for freebsd-testing@freebsd.org; Thu, 27 Feb 2014 13:49:11 +0100 (CET) (envelope-from pho) Date: Thu, 27 Feb 2014 13:49:11 +0100 From: Peter Holm To: freebsd-testing@freebsd.org Subject: The ATF cleanup routine Message-ID: <20140227124911.GA37892@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 12:55:54 -0000 It seems as thou the cleanup routine runs in a different process from the body. The NetBSD tests I found assume that they share address space. http://people.freebsd.org/~pho/cleanup_test.c $ kyua debug cleanup_test:maxfiles /usr/src/tests/sys/kern/cleanup_test.c:14: pid 74893 global = 0. /usr/src/tests/sys/kern/cleanup_test.c:20: pid 74894 cleanup_test:maxfiles -> passed $ -- Peter From owner-freebsd-testing@FreeBSD.ORG Thu Feb 27 17:20:32 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EF9A3F52 for ; Thu, 27 Feb 2014 17:20:31 +0000 (UTC) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B1F61DC5 for ; Thu, 27 Feb 2014 17:20:31 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hi5so915845wib.1 for ; Thu, 27 Feb 2014 09:20:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/SXg85GMVfob3u14MLPkgcJPEI4NoCfmgrLjHZDfs34=; b=ZaxEfnG3FCu6IRC79/qacKeOkKVw6Fcaifa3Y99Bpka643GNNQ9f21+UyGxBbVB5sS 3ofuN6I+KiU9JtN4ADS82ilkdOTfHuvLngskgKgHndnMIjUdTcU1Bc9Hb1zreFcB6rwB CA4Tf9rDz13P+AcvVbsY92uN0Shn4L3gp511unX5HcQ2sidmSn62RV4UpcXY/Y4LdZ+D f+CPN/ljKCgAgx8n7AoitP0XQSZZ6rzM5CKYoN60I3xM5wvhsFJPxgnMUbatNrDbHlYe 4mphTfJvzpyvGfBddcKd2S7ExZPo1GqEFr4zI2zm4n+vp8DJ81jiuoH/zjhb3SflcoBf 99sQ== MIME-Version: 1.0 X-Received: by 10.194.10.162 with SMTP id j2mr2347109wjb.92.1393521629890; Thu, 27 Feb 2014 09:20:29 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Thu, 27 Feb 2014 09:20:29 -0800 (PST) In-Reply-To: <20140227081759.GA29517@x2.osted.lan> References: <20140227081759.GA29517@x2.osted.lan> Date: Thu, 27 Feb 2014 10:20:29 -0700 X-Google-Sender-Auth: pQjtxY1H7xqhbzBX6OcVvyVTIBk Message-ID: Subject: Re: 100% repeatability? From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 17:20:32 -0000 On Thu, Feb 27, 2014 at 1:17 AM, Peter Holm wrote: > I looked at creating a test scenario for the following piece of code > from src/sys/kern/kern_fork.c: > > 881 if ((nprocs >= maxproc - 10 && priv_check_cred(td->td_ucred, > 882 PRIV_MAXPROC, 0) != 0) || nprocs >= maxproc) { > 883 error = EAGAIN; > 884 goto fail; > 885 } > > I came up with a test scenario for this: > > http://people.freebsd.org/~pho/kern_fork_test.c > > But it is not 100% repeatable, as I would expect many other kernel > tests will be. > The test case runs as expected, until for example root decides to do a > parallel buildworld. > Is this a show stopper? First of all, let me say thank you for writing ATF tests. It's great that the community is starting to use it. As for the test: 1) nlistf and memf should be explicitly initialized, and may as well be private variables in getprocs(). 2) global variables are unfortunate. Would it be possible for getprocs() to return maxprocs and nprocs as return arguments instead of global variables? Alternatively, it could return (maxprocs - nprocs), since that's the only way they're used. 3) It's confusing to have a testcase with the same name as a global variable. It doesn't confuse the compiler, since ATF_TC() and friends mangle the name. But it does confuse cscope and humans. 4) Regarding repeatability: non-repeatability is very very bad. It leads to spurious failures appearing in kyua's results, which cause casual testers to ignore the entire suite. If they come to expect a few spurious failures, then they'll ignore real failures. Serious testers learn to ignore the intermittently failing tests, which isn't good either. It simply takes too much effort to debug each and every failure; people won't do it. I would not commit a test that isn't repeatable. If nprocs and maxproc were jail-specific variables, then you could simply run this test inside of a jail. However, they look global to me. I think that the only good way to test the feature would be to run the test inside of a private bhyve instance with an extremely minimal set of services. That way you could ensure that no unexpected processes would be started. It would be much more work, though. You would also need a way to communicate to the guest OS. Perhaps an emulated serial port? -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Feb 27 20:31:22 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A608C120; Thu, 27 Feb 2014 20:31:22 +0000 (UTC) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0306C1279; Thu, 27 Feb 2014 20:31:21 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id 10so2051272lbg.35 for ; Thu, 27 Feb 2014 12:31:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=OvCf8lr0ONDGAEEWLmNjpWuqomsfTnnX75XamvLDthQ=; b=QOZbNvPt/jO94HD/83x+zDdqYSe020PpRkck8/jU+tTiGTo0fpf9xYGAc5lf40fDGp OOovY8eh1Rrfv3e0Ma3I+l+F6REYVRBlIYEYaFshEDXQrwBX1WWFH0FHp5YHDuwOjG3K g4k8kji2P8nu+Hbn0CoJ8n1lq4BNjjFXyAFQT7zuhv2VaB0bq8DYMyBn13S67iRGmFvn i4hjIFPNxnxwwnE9WJJvv+tnAHw6UlQNyXeQYD/uXhFoSgIPiG1eaxz5/S1eXuOEBlL8 uEPDu7QPhWgwQa/djOE9xVZp8aeHQl/f/nnHV3rL+LGpW7HM0rMVNPhSeQAqoRMrZhfb +rnA== MIME-Version: 1.0 X-Received: by 10.152.27.193 with SMTP id v1mr8108920lag.4.1393533079296; Thu, 27 Feb 2014 12:31:19 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Thu, 27 Feb 2014 12:31:19 -0800 (PST) Date: Thu, 27 Feb 2014 12:31:19 -0800 X-Google-Sender-Auth: F5i9Z7AeiydiW8l7KwIleOdLrBE Message-ID: Subject: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: kyua-discuss@googlegroups.com, "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 20:31:22 -0000 Hi, In this thread on the freebsd-testing mailing list I asked if it would be a good idea to extend Kyua so that it could generate test result output in JUnit XML test result format: http://lists.freebsd.org/pipermail/freebsd-testing/2014-February/000199.html JUnit XML is parsed natively by frameworks like Jenkins and Atlassian Bamboo. The response on that thread seemed favorable. Tyler Croy mentioned that he could help integrate and test things, but needs help on the C++ side of things. I'd like to get going on this and see if we can get a proof of concept to demo at the May 15 Continuous Testing working group that I am leading at the BSD Devsummit in Ottawa: https://wiki.freebsd.org/201405DevSummit/Jenkins Tyler, are you in the San Francisco Bay Area? I am near San Jose, so maybe we can get together and try hacking on this stuff some time between now and May. I know C++ if that helps. Thanks. -- Craig From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 00:03:46 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99E64494 for ; Sun, 2 Mar 2014 00:03:46 +0000 (UTC) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 360B61F60 for ; Sun, 2 Mar 2014 00:03:46 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hi5so3127922wib.1 for ; Sat, 01 Mar 2014 16:03:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GiSBWTDCtvWxRxa6wQYZrhOU7pX9R1xPvm0s34x0+y4=; b=R3Iyy5EqgMXy8R96Th5fyxdxjY2wmv8YwMcGe/eFQGJAxsFXDB0tZLjeYUpGeGZZmK KcIeISvTh6FGvrKHVZ7lBXKI4O5lpN/cwTUQ1/E1w212QyxEYvdgQH3Zg3twzTwzRb9u JTrbnb63m2Xkg/vleP031sfqkQ/Tz0piX+E9Cxh9eJga5olzgH5XEt2ccubpWqfDn77F 7GISgvDq2qKUiOBadAJAuwXct7APeyE2CTfzx1i2MdOH+2bbfTqkhvY9UsI3PDgwcO7K syWGYGCW248bj4aLW4CoOpVHTqokEAmLwCGHr/o7otR8CHL/Fek+SoUEU3IK8zMidweS +NDQ== MIME-Version: 1.0 X-Received: by 10.180.189.139 with SMTP id gi11mr9039591wic.53.1393718624007; Sat, 01 Mar 2014 16:03:44 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Sat, 1 Mar 2014 16:03:43 -0800 (PST) In-Reply-To: <20140227124911.GA37892@x2.osted.lan> References: <20140227124911.GA37892@x2.osted.lan> Date: Sat, 1 Mar 2014 17:03:43 -0700 X-Google-Sender-Auth: vznjxKPGzXYvJHFqMYajI3Eg-cM Message-ID: Subject: Re: The ATF cleanup routine From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 00:03:46 -0000 On Thu, Feb 27, 2014 at 5:49 AM, Peter Holm wrote: > It seems as thou the cleanup routine runs in a different process from > the body. The NetBSD tests I found assume that they share address > space. > > http://people.freebsd.org/~pho/cleanup_test.c > > $ kyua debug cleanup_test:maxfiles > /usr/src/tests/sys/kern/cleanup_test.c:14: pid 74893 > global = 0. /usr/src/tests/sys/kern/cleanup_test.c:20: pid 74894 > cleanup_test:maxfiles -> passed > $ > Since you aren't getting any help here, you should try posting this question to kyua-discuss@googlegroups.com. It's not FreeBSD-specific. -Alan From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 11:33:02 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01A77832; Sun, 2 Mar 2014 11:33:02 +0000 (UTC) Received: from mail-pa0-x233.google.com (mail-pa0-x233.google.com [IPv6:2607:f8b0:400e:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C39CC160C; Sun, 2 Mar 2014 11:33:01 +0000 (UTC) Received: by mail-pa0-f51.google.com with SMTP id kq14so2664112pab.10 for ; Sun, 02 Mar 2014 03:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3taY84HenFTBzvVQJUPgugI+DNCILiDUIwnYkuF+6OE=; b=MboSyjiKQSPCfP4BMKkWxpIDIioMEM1EuXV4hDI7rcpPZVcb0YDAxaie74eRiA4t1n wgvxMO4WmItpqquM0qgQD+4ZhxH9axbHWSlhHbtTmY7l/ghWsI0YBGpxQ228Fp+DAHVU icvEgpQ6glxoih2BKaCBnqgQ25NUqaqcvX34QU+dzDzE4/n53atsCH0szgq71IFKbFot 3QetuhK4OE80OiIIVvA7QR7w94CGc2/v8G6nLe7QRVsE4Bhigfeu77zQ1AfKGe+pEEhm TPVLO2baHM7Dg15BX4kuhBFyyCnP6Bho+M9yrtvlb7TM1nNEY7DIGfjCa1C9t3pz/ukz z+Rg== X-Received: by 10.66.189.226 with SMTP id gl2mr13863377pac.65.1393759981324; Sun, 02 Mar 2014 03:33:01 -0800 (PST) Received: from [192.168.20.5] (c-24-17-226-153.hsd1.wa.comcast.net. [24.17.226.153]) by mx.google.com with ESMTPSA id tu3sm59733630pab.1.2014.03.02.03.32.59 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 02 Mar 2014 03:33:00 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: The ATF cleanup routine From: Garrett Cooper In-Reply-To: Date: Sun, 2 Mar 2014 03:32:57 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <7C6A91CA-3C40-48BF-B230-4F388F05B8CA@gmail.com> References: <20140227124911.GA37892@x2.osted.lan> To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 11:33:02 -0000 On Mar 1, 2014, at 4:03 PM, Alan Somers wrote: > On Thu, Feb 27, 2014 at 5:49 AM, Peter Holm wrote: >> It seems as thou the cleanup routine runs in a different process from >> the body. The NetBSD tests I found assume that they share address >> space. >>=20 >> http://people.freebsd.org/~pho/cleanup_test.c >>=20 >> $ kyua debug cleanup_test:maxfiles >> /usr/src/tests/sys/kern/cleanup_test.c:14: pid 74893 >> global =3D 0. /usr/src/tests/sys/kern/cleanup_test.c:20: pid 74894 >> cleanup_test:maxfiles -> passed >> $ >>=20 >=20 > Since you aren't getting any help here, you should try posting this > question to kyua-discuss@googlegroups.com. It's not FreeBSD-specific. FWIW, this is a design caveat I discovered when trying to = implement global (test case wide) setup/cleanup routines, and port over = other testcases from NetBSD=92s libc test suite. Cheers, -Garrett= From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 13:56:26 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 13A87DCD for ; Sun, 2 Mar 2014 13:56:26 +0000 (UTC) Received: from mail-qa0-f41.google.com (mail-qa0-f41.google.com [209.85.216.41]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C5A2112D5 for ; Sun, 2 Mar 2014 13:56:25 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id j5so2551939qaq.0 for ; Sun, 02 Mar 2014 05:56:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=+aRjIJFrgI8OIc/vwSE0yEgIfIgtC5L7Xb1CKRNhkU4=; b=TcE5hHGAfOISccN4WmlZk+JSTdtY7XHct0WzE7MWvgclUAqB1fDm8c1SdWm5v9OEtC Xkw2EXKoUxsaknlsYQ4r4hnVJQF4R2e2Pm6dbwR2p0cPgEVoEwoFHvEBCPbrh7wQN0yE a9HcYKe6+PDMxZli8EuL/HqGjRuIdBN6OL1e64wtQACz/UpUajDLJj5LPqIXqY0JfYt1 KZ3AZbRoCE/M5JfJhNFV9JIiYRYXrONcbFrOAxzwSSb66jSj9uKjOw9iYazu+r5RSPLC gALd3pJOFbIZD/2YSWFhAQ2dBHaV0iEL6XzMXLGMqOFCFg/DEE5fOMdpZHIW627Nojqj Fqtw== X-Gm-Message-State: ALoCoQnPhZU3r1qabUr0tNXUQ9r0QyvT2xFyTixa4l/HQJm4NyjzNACfYMaDsM7nhXsHz1WEGnqP X-Received: by 10.224.114.141 with SMTP id e13mr16930088qaq.65.1393766907803; Sun, 02 Mar 2014 05:28:27 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 05:28:07 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: References: <20140227124911.GA37892@x2.osted.lan> From: Julio Merino Date: Sun, 2 Mar 2014 08:28:07 -0500 X-Google-Sender-Auth: _RX97OoknJC54ZKhghPEQ_p3Q34 Message-ID: Subject: Re: The ATF cleanup routine To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 13:56:26 -0000 On Sat, Mar 1, 2014 at 7:03 PM, Alan Somers wrote: > On Thu, Feb 27, 2014 at 5:49 AM, Peter Holm wrote: >> It seems as thou the cleanup routine runs in a different process from >> the body. The NetBSD tests I found assume that they share address >> space. >> >> http://people.freebsd.org/~pho/cleanup_test.c >> >> $ kyua debug cleanup_test:maxfiles >> /usr/src/tests/sys/kern/cleanup_test.c:14: pid 74893 >> global = 0. /usr/src/tests/sys/kern/cleanup_test.c:20: pid 74894 >> cleanup_test:maxfiles -> passed >> $ >> > > Since you aren't getting any help here, you should try posting this > question to kyua-discuss@googlegroups.com. It's not FreeBSD-specific. Yes, that's a good suggestion in general. But apologies for the delay. I was meaning to reply to this but forgot, and this update to the thread poked me to do it now. Peter did repost this to kyua-discuss and I have replied there; see: https://groups.google.com/forum/#!topic/kyua-discuss/w-_3v3hLPoc From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 16:24:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9154F41C for ; Sun, 2 Mar 2014 16:24:59 +0000 (UTC) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.freebsd.org (Postfix) with SMTP id 3814710C7 for ; Sun, 2 Mar 2014 16:24:58 +0000 (UTC) Received: (qmail 28359 invoked from network); 2 Mar 2014 16:18:17 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay01.pair.com with SMTP; 2 Mar 2014 16:18:17 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s22GIF1x042930; Sun, 2 Mar 2014 17:18:15 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s22GIFOv042929; Sun, 2 Mar 2014 17:18:15 +0100 (CET) (envelope-from pho) Date: Sun, 2 Mar 2014 17:18:15 +0100 From: Peter Holm To: Garrett Cooper Subject: Re: The ATF cleanup routine Message-ID: <20140302161815.GA42639@x2.osted.lan> References: <20140227124911.GA37892@x2.osted.lan> <7C6A91CA-3C40-48BF-B230-4F388F05B8CA@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7C6A91CA-3C40-48BF-B230-4F388F05B8CA@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 16:24:59 -0000 On Sun, Mar 02, 2014 at 03:32:57AM -0800, Garrett Cooper wrote: > On Mar 1, 2014, at 4:03 PM, Alan Somers wrote: > > > On Thu, Feb 27, 2014 at 5:49 AM, Peter Holm wrote: > >> It seems as thou the cleanup routine runs in a different process from > >> the body. The NetBSD tests I found assume that they share address > >> space. > >> > >> http://people.freebsd.org/~pho/cleanup_test.c > >> > >> $ kyua debug cleanup_test:maxfiles > >> /usr/src/tests/sys/kern/cleanup_test.c:14: pid 74893 > >> global = 0. /usr/src/tests/sys/kern/cleanup_test.c:20: pid 74894 > >> cleanup_test:maxfiles -> passed > >> $ > >> > > > > Since you aren't getting any help here, you should try posting this > > question to kyua-discuss@googlegroups.com. It's not FreeBSD-specific. > > FWIW, this is a design caveat I discovered when trying to implement global (test case wide) setup/cleanup routines, and port over other testcases from NetBSD?s libc test suite. > Cheers, > -Garrett Hello Garrett, Thank you for your reply. This is of cause not a problem, but just came as a surprise as the two first examples from NetBSD I looked at, assumes this. - Peter From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 20:34:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43424DCF for ; Sun, 2 Mar 2014 20:34:59 +0000 (UTC) Received: from mail-qa0-f52.google.com (mail-qa0-f52.google.com [209.85.216.52]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0123D162F for ; Sun, 2 Mar 2014 20:34:58 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id m5so2710511qaj.39 for ; Sun, 02 Mar 2014 12:34:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=5D2J0qXNQaKtjQi91gXO8jym55Nj09NnGJdE4r0qfGg=; b=lzc6ezjeEnv0Wqj2jf162BTE5Dw79FIljYpJOkV61xHuoEX3I/mlLkRvFjA7HPsB7J TAYHHmyhJw29RC2KVeLsbXcbAsTGDQJcrslb6r/mtoPRxRluhy5s9MZ9/oCbR3uZ97n5 8jSQXBLat1jJ18XKJGV7aBvxYwLVmBiBfBC4R3jEm1cChtbxY87upbCkUtp3w5+eSHys sOMAD7pNfomY+j5dQ+2m+POoUHEcqz26lsy4f8dbtm7+nuSiw4R4MvE9VNI8zdi2RyFf FAkA/eAwJ57ppGwDCGanm3EX3xB+kSdAgZ3Mpe0wTMEy0BnXGUqVWWf7pMpsB8mbzzbw 8lhQ== X-Gm-Message-State: ALoCoQnT0gJywF4WXPl+KHc/3SrmP4L34fRwtvT+zETuOuZvumifuP0nqpw4+HXcB/gsTwHxnA/w X-Received: by 10.140.97.137 with SMTP id m9mr3660517qge.95.1393792013553; Sun, 02 Mar 2014 12:26:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 12:26:33 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: References: From: Julio Merino Date: Sun, 2 Mar 2014 15:26:33 -0500 Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? To: kyua-discuss@googlegroups.com Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 20:34:59 -0000 On Thu, Feb 27, 2014 at 3:31 PM, Craig Rodrigues wrote: > Hi, > > In this thread on the freebsd-testing mailing list > I asked if it would be a good idea to extend > Kyua so that it could generate test result output > in JUnit XML test result format: > > http://lists.freebsd.org/pipermail/freebsd-testing/2014-February/000199.html > > JUnit XML is parsed natively by frameworks like Jenkins and > Atlassian Bamboo. > > The response on that thread seemed favorable. > > Tyler Croy mentioned that he could help integrate and test things, > but needs help on the C++ side of things. > > I'd like to get going on this and see if we can get a proof of concept > to demo at the May 15 Continuous Testing working group that I am > leading at the BSD Devsummit in Ottawa: (As I mentioned in the other thread) this is relatively easy to do and we have enough time to get it going by May 15th. The path of least resistance to get this out for now is: 1. Reintroduce support for --output=format:path to 'report' so that 'report' can output more than one format. The existence of 'report-html' as a separate command is a pretty ugly hack. 2. Split out the console-specific formatting from cmd_report.cpp into a separate module to clearly expose what is part of the formatting and what is not. Adding JUnit XML support should not require changes to other code. 3. Add a new module with the JUnit XML formatting code. I have finished with 1 (locally, not pushed out yet) and started working on 2, which is slightly trickier. I can also do 3 but not until after AsiaBSDCon 2014 (that is, mid-March). I suppose that with this done, integrating the output into Jenkins should be trivial. (Longer-term, I am pondering the option of converting all the report-generation code to Lua with extensive support for hooks. This would allow you to tweak the output format without having to change Kyua internals.) From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 20:40:20 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE6E9E66 for ; Sun, 2 Mar 2014 20:40:20 +0000 (UTC) Received: from mail-qg0-f48.google.com (mail-qg0-f48.google.com [209.85.192.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A7B3166F for ; Sun, 2 Mar 2014 20:40:20 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id a108so9011414qge.7 for ; Sun, 02 Mar 2014 12:40:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=UbIx1189pCQPTESEIrysqW93/gVVYEyOquaQTPh3vgs=; b=R2urmHVbFVSl9NR5kJt43IsDywtxTokRxPGYbpMXzih41+2MpTYCF/dvCehQNKAf5V 7RZZDqFb48rP1tmrQBEbkUDhZBYgHsmpFDC3mnriiKBuePOWSKu9z0n2UUypRRrJqvpQ BHlyUw10MNGbXaBZ6DGEGsVq8c7HNW9uK/Lwi89qXgpwL9DsXsGdZUHmZTm5+0uEq7um PgszPx2gInDpQ8ejWcPBob1m1aqLrMBWWj90SSZPwefif4rwPUtn9iJeyj5w/gLWnYFs tHLesXDek+/iSQf42O+SEwyq4LrzwGGeP3oWjKAbpJIjPJYc+8wKDTZ1Z62VN5KHYXqc FArw== X-Gm-Message-State: ALoCoQmIxAQVpYHDq5s/s8nm9AR99R10omYYL4p30GE8KzQ60iEc6Uy+nYiH0nJey/dDsz3WADv+ X-Received: by 10.224.7.10 with SMTP id b10mr19249323qab.50.1393792819555; Sun, 02 Mar 2014 12:40:19 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 12:39:59 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: References: <20140225161129.GA59741@x2.osted.lan> <20140225183306.GA70295@x2.osted.lan> From: Julio Merino Date: Sun, 2 Mar 2014 15:39:59 -0500 X-Google-Sender-Auth: R-odbSfe005wJj2wX6u09fdeg3Y Message-ID: Subject: Re: My first ATF test To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 20:40:20 -0000 On Tue, Feb 25, 2014 at 10:59 PM, Alan Somers wrote: > > The atf-c-api(3) man page doesn't recommend one or the other, and its > examples show it both ways. My personal habit is to put the constant > first. It's been ingrained by both googletest and Ruby's test/unit, > which both have a clear policy. Personally, I would say that if > you're editing an existing file, try to keep with its preexisting > style. Otherwise, I don't think it matters. It doesn't matter as you say and keeping consistency is good. However, I've also come to prefer putting the constant first nowadays given that this what pretty much all other testing libraries do. The inconsistencies come from the past. They should be fixed to prefer check(expected,actual). From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 20:44:37 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 607CFEBE for ; Sun, 2 Mar 2014 20:44:37 +0000 (UTC) Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1DF6316FE for ; Sun, 2 Mar 2014 20:44:36 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id f11so2746574qae.3 for ; Sun, 02 Mar 2014 12:44:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=1BQza+Q51AN6WF0ZAoPLNcN35a1ZPTAjdLJOwm7fQrM=; b=JTf8cf6JEB5lHxpa+VBZxlmKNggTP1SfHQHkgXjuq9EMbUrA+mI8Q5CQM1bGZgC/9W 7iBrX84tCJkPVgWIlOrhXOqvNGyPLtOU9KQeM7ppeMrHLEJhVJ/eT8lXv4oamucehbEg QB2bPY4l3/ggMZKeXBuX4hOY0LKYuGUFODI1JDHAE68OE/SQEBY2wdb1DSYCyx8bwcDn CdR4F/s0I+FB5yWELLJKJAk/bOOziBjegYYa1sd2DhQGEqkSu3IGagOOFCugymXhod8y zJGndZrhWS7CgLhPl0DSFRG/Uj3FSynaOX2KJj5q4tmouZ+OWaf0MBVhPEZ+vyLkC1g6 RSvQ== X-Gm-Message-State: ALoCoQnb5aH8t602jCVHARFyne8S/2vanrGiP9U15jxkoQZ+XKDOm5Z/87AI/Lf9L/yFgmYdgg8s X-Received: by 10.140.84.40 with SMTP id k37mr3239692qgd.98.1393793074844; Sun, 02 Mar 2014 12:44:34 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 12:44:14 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: <20140225161129.GA59741@x2.osted.lan> References: <20140225161129.GA59741@x2.osted.lan> From: Julio Merino Date: Sun, 2 Mar 2014 15:44:14 -0500 X-Google-Sender-Auth: LRcPMqFm0xUa5d80mlpY-aKEh9k Message-ID: Subject: Re: My first ATF test To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 20:44:37 -0000 On Tue, Feb 25, 2014 at 11:11 AM, Peter Holm wrote: > In order to understand how ATF works I wrote a small test so I had > something to work with: > http://people.freebsd.org/~pho/kern_descrip_test.diff > Did I get it right? Hello Peter, Looks good to me. One suggestion: that comment with the explanation for dup2_r234131 would be much better represented either as 1) a more descriptive test case name or 2) a test case description ("descr" metadata property). The revision number means nothing to readers. People running the test suite won't see the comment at all, but they will see the test case name all the time and they may see the description depending on the reporting format. For example, dup2__ebadf_when_2nd_arg_out_of_range would clearly explain what this is. You could later put the revision number in the "descr" property with a textual explanation if so desired. Cheers From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 20:47:02 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E501EF12 for ; Sun, 2 Mar 2014 20:47:02 +0000 (UTC) Received: from mail-qa0-f46.google.com (mail-qa0-f46.google.com [209.85.216.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9F27A1720 for ; Sun, 2 Mar 2014 20:47:01 +0000 (UTC) Received: by mail-qa0-f46.google.com with SMTP id i13so2768143qae.33 for ; Sun, 02 Mar 2014 12:47:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=pby86351a54jMXmAsyHgmFC6ZJuXNCvMGlRAZq0TwDA=; b=Pt477uRRMlWfYnz8rPwVyBTAZ1gbrlgGiZtpMGCPhYImyIb6r4Eb28geN2+1WaL0K6 QFUPbzTRne9qBQ/CUTzqkCS5ti4qLhAPK2HBZAgt7/Our6bUncl2CdOK7p5iiJO/Pc1b Ctcs7kDv+8S2pJWZ2Os8BJuJuyDuSJVIEt1aEqxw8Odj9E44r48uUfAYdWVYW6Yfv7Lw 3AnuJ0KPuI861tJVgdHz9jjeIKqA3tvHHhMX8YUAh6h1OQtcMBkQxG7O9dluTVx93YVm MOkhmDiGHGeuX1rFTOWreSHGlop3ByeDiq97pIyPvEajF5GmCiLd5pYh9XLNhUTpWVif kbgQ== X-Gm-Message-State: ALoCoQkW9ZeC0kDBN2Txvr5m1VmGhfn0k5g6Tmp/y6XXG/9OHdvm4J3aIz9u5jFlDNQtAPh/t8IR X-Received: by 10.140.39.50 with SMTP id u47mr18337408qgu.46.1393793221272; Sun, 02 Mar 2014 12:47:01 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 12:46:41 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: <20140225033526.GN85115@kiwi> References: <20140225033526.GN85115@kiwi> From: Julio Merino Date: Sun, 2 Mar 2014 15:46:41 -0500 X-Google-Sender-Auth: azjH0LJ4fveJkeQNScYjQN4gwio Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? To: "R. Tyler Croy" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 20:47:03 -0000 On Mon, Feb 24, 2014 at 10:35 PM, R. Tyler Croy wrote: >> 2) I don't see any "expected failure" results in the links that you >> provided. That's an absolute requirement for Kyua. Treating expected >> failures like ordinary failures will greatly reduce the utility of >> Kyua, by reducing its ability to spot regressions. > > > JUnit allows for the concept of "skipped" tests, which Jenkins' parsing code > respects. IIRC this is just a "skipped=true" on a , I believe that > would help solve for this case. No, it's not. An expected failure is not the same as skipped. Possibly the best approximation is to consider an expected failure as a pass. From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 21:14:51 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4F3A920 for ; Sun, 2 Mar 2014 21:14:51 +0000 (UTC) Received: from mail-qa0-f49.google.com (mail-qa0-f49.google.com [209.85.216.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9146419AC for ; Sun, 2 Mar 2014 21:14:51 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id cm18so519114qab.36 for ; Sun, 02 Mar 2014 13:14:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=Y9E8bP61uPJu+5w+BjTu09mNDJ0gg2oHR5exZqyAMc8=; b=BI5Ur2dsbLimkIIwf/Q7GdhqUtg/fjDVAswAJcpfiHOaK0z/JYAp6Nryw66rKCSRYc JirH8MyGzRnYsiSZzuiKu9mt0f8Dlcju/FEDdd+ZA0VL/V1bZEjC7v3sy4L6rKehl8ok QowNRpPZHOc0uGV88NU2BVspPZFLmWVx0i7+j4KhYFdxxMnjlfc7583lbC7XPXZ5r1d4 biLA3Lrtclf1Je0SPzu9KDi96lCf9acvVf1gPAy9v4se2p0Htn36C7MqzhYpv5fCFu8Q 9PdATY/Z6LGTmEKSTJ/wQoRO3pMwgCJBGKRDuebzkCWM9cZbjMLHRf/raxSd7ThCyQyO UoPw== X-Gm-Message-State: ALoCoQkEr0MlFoznwzKMN83XfZBmxVgKQtxKJJAT3EvdL57JMMXp5tPFsaHcACamO72XuY5wk/tX X-Received: by 10.224.157.7 with SMTP id z7mr19057801qaw.37.1393794890471; Sun, 02 Mar 2014 13:14:50 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 13:14:30 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: <96373EF8-77EA-4AA6-8422-63881C32E455@gmail.com> References: <20140124162759.GB90996@stack.nl> <0A68C9FE-6A3A-4DCF-92A9-A506D7581D69@gmail.com> <96373EF8-77EA-4AA6-8422-63881C32E455@gmail.com> From: Julio Merino Date: Sun, 2 Mar 2014 16:14:30 -0500 X-Google-Sender-Auth: bOJUH1iPhqq4WoYggs4CoKijLfo Message-ID: Subject: Re: [PATCH v2] convert /bin/sh tests over to ATF To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Jilles Tjoelker X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 21:14:51 -0000 On Mon, Jan 27, 2014 at 5:02 AM, Garrett Cooper wrote: > Hi all, > Here's the next patch for converting bin/sh/tests over to ATF. I've tried to incorporate all input from folks about simplifying the integration scripts and removing valid_osreldate functionality. Apologies for the really long delay... I suppose nothing has changed given that there were no replies to this. Soooo, some more comments: * Could functional_test be created with ATF_TESTS_SH ? * Rename the script in subversion to functional_test.sh (including the extension). * Do NOT cd out of the current work directory in functional_test, anywhere. This breaks all the assumptions in tests regarding the ability to write to the current work directory without caring about cleaning up. * Use atf_get_srcdir to locate where the data files are, not dirname $0. * Continue to use SUBDIR, not ATF_TESTS_SUBDIRS. The latter does not exist. * Reintroduce $FreeBSD$ to functional_test.sh. * The empty Makefiles in all the subdirectories are quite confusing. At the very least, explicitly define TESTSDIR as is customary everywhere else. * I'm still afraid of all this "auto-discovery" of test cases based on files, because it is quite confusing... but at least it's now a bit more self-contained and easier to strip out later. So we could keep it. * The FILES must continue to be installed. Does this actually work when running your test programs after a "make install"? Cheers From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 21:23:25 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 25E0CA01 for ; Sun, 2 Mar 2014 21:23:25 +0000 (UTC) Received: from mail-qc0-f178.google.com (mail-qc0-f178.google.com [209.85.216.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D3EA91A7B for ; Sun, 2 Mar 2014 21:23:24 +0000 (UTC) Received: by mail-qc0-f178.google.com with SMTP id i8so2975795qcq.37 for ; Sun, 02 Mar 2014 13:23:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=0zSrlUGQ/pW3WcO50tiGjP1ik3f5pzPMijXMT7QhB7E=; b=Wgr9CxcQi+3KhMTn7DUpDJCkuVYNvwWyU/1vUYWJcEnMwQiEcfdnU4FcFjzy0nAJfb PhJKVc9u9KMFd5cQVY98yPwoLr7grOIi7qsGqdHXNVmoq14cII/10x2jcDpXN4pp7VyT e9Fu4OIoZ1dt6nHv6djIipdrLt/MDDzWfjEB6+hZ1dWVlZw2Is5zBiqBwr5FRGp+ASWn QrE4fG0DIy/woTSPZN74mZVGOd2o65dIEzrjaGsJCyvIBt0Twp4RxFc2hYgOOje5V+i+ DaTdt07v/nkZ9mT1QrWUxGh3Lz0OE9a44MS8iKTFk9cj3tsZ4iASCQfGgnGuUcCcAav7 aVZQ== X-Gm-Message-State: ALoCoQmmSmDdko3oOb39IPiphcWBCwTsnLyPPBADFCHaHYZyXc2FIiiaPBj9+uwozdzpAsn6orZp X-Received: by 10.229.139.199 with SMTP id f7mr19219732qcu.2.1393795398261; Sun, 02 Mar 2014 13:23:18 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 13:22:58 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: <4A25ADF9-3714-4006-ACB6-F8C745CB38AE@gmail.com> References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> <4A25ADF9-3714-4006-ACB6-F8C745CB38AE@gmail.com> From: Julio Merino Date: Sun, 2 Mar 2014 16:22:58 -0500 X-Google-Sender-Auth: VkifBH0jK1PzQLIhyEBT2jgewno Message-ID: Subject: Re: [PATCH v2] convert bin/date over to ATF To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Giorgos Keramidas X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 21:23:25 -0000 On Mon, Jan 27, 2014 at 5:03 AM, Garrett Cooper wrote: > The attached patch incorporates the input from Julio for converting bin/date/tests over to ATF. Apologies for the delay here as well... Pretty much all style nits: * This file is full of boilerplate and it's really hard to actually see the tests. I think this would benefit from a simple helper function that calls atf_test_case and evals the tiny body() with the call to check. * I'd get rid of the descr definitions; they provide no value. * I'd also get rid of all the horizontal dividing lines. If you make the test case definitions compact, they just serve to increase the file length. * Remove the TZ and LC_ALL overrides. This is a given within ATF. * Why "format_string_functional_test"? I'd just call it "format_string_test". * I don't think the # vim trailing line belongs in the tree. From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 21:37:28 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 493F1E68 for ; Sun, 2 Mar 2014 21:37:28 +0000 (UTC) Received: from mail-qa0-f41.google.com (mail-qa0-f41.google.com [209.85.216.41]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 077CE1B5A for ; Sun, 2 Mar 2014 21:37:27 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id j5so2780440qaq.14 for ; Sun, 02 Mar 2014 13:37:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=f2qRBSdpEAa8Uh2nxLQNVsiHjb7plYpTbJZkSwJpA0E=; b=aKdQ5vOu7XwaUfE9KDUI2jtJQ99u3j4EhGJCk4kQlx11/Sk+qFFYuKA84Hd1QEHz5T gBsBltRR/cJHk5UOtBM7myITYZhvsN3kHOJ8M8ewxZBlJ1iZFTg8vGq/UMtg0blchA4F WsRr1YN0T0xcwxAvUSNC4TLDIzwo2KIw8AKpxSQkxXXcgIZQVwnyLVXL88rPy/iZXnLQ tlJ8QRT0V6msL4RHDZtKKxw/KGS9RmuSou9XOx1CwWnc1N5uSU+D+4JYZovnOGOmmj6J nuupG2dufj1nqRn00SUow33y6Xw5KFNYbgEAZJsX4iMwxb8IL9qVxNO0k8ragrVVbend riKw== X-Gm-Message-State: ALoCoQmdhyp4TLmnWoTOCFcGyQV2PvdrkIuR+28z4qKG/UjzEKT3cvFh5Rp39UlFFl7r/ZXOf5Lp X-Received: by 10.140.25.112 with SMTP id 103mr18081643qgs.39.1393795939904; Sun, 02 Mar 2014 13:32:19 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 13:31:59 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: <94D3F60A-1E12-418B-8313-E43C93878B95@gmail.com> References: <94D3F60A-1E12-418B-8313-E43C93878B95@gmail.com> From: Julio Merino Date: Sun, 2 Mar 2014 16:31:59 -0500 X-Google-Sender-Auth: 1_JijG4vquIvOCZHBjT0GKvYaWY Message-ID: Subject: Re: [PATCH] Integrate pjdfstest into the build To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 21:37:28 -0000 On Mon, Jan 27, 2014 at 5:41 AM, Garrett Cooper wrote: > Hi all, > The attached patch integrates pjdfstest into the build. It also simplifies some logic in tests/sys/Makefile when copying Kyuafile. Aaand one more time, apologies for the delay. I think I'm caught up now! * What's the point of PJDFSTEST_I_KNOW_WHAT_I_AM_DOING? *Why* is it a very bad idea to do so? * The 'tests' path component in pjdfstest/tests/ should go away; it is redundant. * Add $FreeBSD$ tags everywhere where missing. * The test files should move from tools/regression/pjdfstest/ into the new location under tests/sys/ and be renamed to follow the conventions under src/test (e.g. xx.t -> xx_test). Cheers From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 21:41:07 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2467A140 for ; Sun, 2 Mar 2014 21:41:07 +0000 (UTC) Received: from mail-qc0-f174.google.com (mail-qc0-f174.google.com [209.85.216.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D72141C05 for ; Sun, 2 Mar 2014 21:41:06 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id x13so2954586qcv.19 for ; Sun, 02 Mar 2014 13:41:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=z15Bd3qBOTkKaXCF9l81M/BoNuViyBHlPoQFSCkeuJs=; b=Bh8914l8x4HNVKp5Du8IcCyA6f0RGd9eMgDDsfBMLLO5LhkacdRGZ/c3N7FHwqso+k VYg2l1bBeSv8s911lqz7do5H4AkHnj76Em3zQwiiaME9u6p2juTavczAUtUGd/Do+wg6 ULkGZXQOUEJbAXSRf3rqJvpR+xc8CmEAfwfPxqHjnKhuyDG9YMbyTCm/+30Zke+RDaB/ eC5lo//7tyjKlXhY+r90XTXnr2knxdTHyn2qbS0GmALOlpfCiuNqarqfFuIGdnxHzCx7 MBHGxiORS07VFn82ZOPh3Hlrr+xO91Bn83WbLV6PaBU6lscyc1gzdAdq7AD1zsOPUxCe dKNw== X-Gm-Message-State: ALoCoQnEDTjcXJXELatF+OJh4GiiPweHtrdFuu29X0dJziigC+DDBAGGur3QW6jebnnYvz+XKJig X-Received: by 10.224.104.2 with SMTP id m2mr5967632qao.3.1393796460597; Sun, 02 Mar 2014 13:41:00 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 13:40:40 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: References: <20140227081759.GA29517@x2.osted.lan> From: Julio Merino Date: Sun, 2 Mar 2014 16:40:40 -0500 X-Google-Sender-Auth: mp_ZaXKVI5rEpJ87e8I3xGSCOKk Message-ID: Subject: Re: 100% repeatability? To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 21:41:07 -0000 [ Thread hijack coming! ] On Thu, Feb 27, 2014 at 12:20 PM, Alan Somers wrote: > > If nprocs and maxproc were jail-specific variables, then you could > simply run this test inside of a jail. However, they look global to > me. I think that the only good way to test the feature would be to > run the test inside of a private bhyve instance with an extremely > minimal set of services. That way you could ensure that no unexpected > processes would be started. It would be much more work, though. You > would also need a way to communicate to the guest OS. Perhaps an > emulated serial port? Wow... that sounds like a lot of overhead for an apparently-simple kernel test! I think we should discuss, and possibly document, a recommended way of running kernel-level tests for FreeBSD. We have nothing in this regard and reinventing the wheel for every test without thinking through the various alternatives upfront will be... "ugly" to say the least. Now, I do not have an answer to this. >From previous experience in NetBSD, I think that the "anykernel" design implemented there works very well for these kinds of tests. I do think FreeBSD would benefit from rump immensely, but the problem is that it's a complex thing to do implementation-wise. See http://en.wikipedia.org/wiki/Anykernel and http://wiki.netbsd.org/rumpkernel/ to get started. Without rump, all that is left are jails, bhyve instances and natively-run tests (and maybe something else?). Each has its own problems but are easier to use today than reimplementing rump. Maybe we should attempt to guide developers to one of these three options depending on what they are trying to test and provide helper code to make the setup easy? Or maybe we should investigate whether "porting" rump to FreeBSD would make sense? From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 22:54:57 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94FE99F3; Sun, 2 Mar 2014 22:54:57 +0000 (UTC) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5C9F912AF; Sun, 2 Mar 2014 22:54:57 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id w10so2923871pde.24 for ; Sun, 02 Mar 2014 14:54:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=XzADZYrhnA3Do9wgv++rt3aMnseh0DwcfqiGpOAYXSE=; b=RR0W4l5bUj7CgEfodEeikwAlmZiTDENKhOW+vgZSbsYz0ZfK1S8H+qILSAKY42tRZR Smef1CZ+sXaeZN99wANEOHvPQWFgyfBhZpT2X0tRM32LHHhdM12JTjHDIDCehcOH2Fp6 09SvDWpnVa+8+Dx29BufQyqGoeKQZrLsV7lECPz9yfcnrL60DubY/XQEQxpuwVgVuPYM hzFSEtcuR5eVApKJA6BHnFsgsU18v8MgYej5zItuFaVWWJFD1HIPl0hjE0kHCOL2MDYz ZGdh76sJdJhBFW/p9JeKHSkCSq+7f+OxshpaHvgOx7wAwgyfatVVa/uv9J4eF/sLkb3p k1eQ== X-Received: by 10.68.106.130 with SMTP id gu2mr15943381pbb.59.1393800897035; Sun, 02 Mar 2014 14:54:57 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:713a:b2d6:e030:b06f? ([2601:8:ab80:7d6:713a:b2d6:e030:b06f]) by mx.google.com with ESMTPSA id os1sm70682492pac.20.2014.03.02.14.54.54 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 02 Mar 2014 14:54:55 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: [PATCH] Integrate pjdfstest into the build From: Garrett Cooper In-Reply-To: Date: Sun, 2 Mar 2014 14:54:52 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <56D809A7-5387-475A-8C89-794E328F6935@gmail.com> References: <94D3F60A-1E12-418B-8313-E43C93878B95@gmail.com> To: Julio Merino X-Mailer: Apple Mail (2.1874) Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 22:54:57 -0000 On Mar 2, 2014, at 1:31 PM, Julio Merino wrote: > On Mon, Jan 27, 2014 at 5:41 AM, Garrett Cooper = wrote: >> Hi all, >> The attached patch integrates pjdfstest into the build. It = also simplifies some logic in tests/sys/Makefile when copying Kyuafile. >=20 > Aaand one more time, apologies for the delay. I think I'm caught up = now! >=20 > * What's the point of PJDFSTEST_I_KNOW_WHAT_I_AM_DOING? *Why* is it a > very bad idea to do so? It=92s a safety belt. I=92ve run into cases where interrupted test runs fill up filesystems = with data (either free space or inodes). Depending on what method you = use to clean up the contents of directories, it can take ages to delete = all of the files (rm -rf for instance is a very linear operation, but = parallel rm -rf=92s are considerably quicker). I=92ll add a comment to = this effect. > * The 'tests' path component in pjdfstest/tests/ should go away; it is > redundant. It=92s not redundant; it=92s needed for the conf file: # make all =3D=3D=3D> pjdfstest (all) =3D=3D=3D> tests (all) make[1]: don't know how to make conf. Stop make[1]: stopped in /usr/src/tests/sys/pjdfstest/tests *** Error code 2 Stop. make: stopped in /usr/src/tests/sys/pjdfstest # git checkout -- tests/Makefile=20 =85 # echo $? 0 > * Add $FreeBSD$ tags everywhere where missing. I=92ll add those. > * The test files should move from tools/regression/pjdfstest/ into the > new location under tests/sys/ and be renamed to follow the conventions > under src/test (e.g. xx.t -> xx_test). Please talk to Pawel about this. These tests have been developed and are = maintained (to some degree) outside FreeBSD: = https://github.com/pjd/pjdfstest . I=92d rather treat this as an = integration of the 3rd party package, not a refactor. If you want me to = install the tests like _test though, I=92d be more than happy to do = that. Thank you :)! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Sun Mar 2 23:00:28 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B774C8B for ; Sun, 2 Mar 2014 23:00:28 +0000 (UTC) Received: from mail-qa0-f66.google.com (mail-qa0-f66.google.com [209.85.216.66]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 149D212ED for ; Sun, 2 Mar 2014 23:00:27 +0000 (UTC) Received: by mail-qa0-f66.google.com with SMTP id m5so1018054qaj.1 for ; Sun, 02 Mar 2014 15:00:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=sR0rzFdYw7Jl1Jrd7kaSnzVRMs5HUv+JaAFl1kyO+Os=; b=Ryi8tmDIoWVJeFXRDemV92IPEAS2Zu8PP4x+JpgJOxXxgrpBJnzWUfUq8w6whpihfv bEL7CWGyyhHQlfUH8hlRmIC4QumPtUBO20ApargnddFWzXH7ZbAb8mpSH7HX9gTl7Koq nYk9OzU/HhL1lBg0SQjBQF9AvboJqIJ1RnFn8kbBh9wWo/bkt+C+pcj9LP/uEf/Hw3iB qzYNYgNW4kxTHpAEBvyH+vFhG6rjJ12lF4JDWM+LnX+DOdhSIceUkRyPER8X7Ulb2Wk6 jAsY9u7pUjVDZDR7gKfwDfHecM1FIMIKW/KoCJ7iVDrzTVSHqPUCbBx7dgCubpkmJ/NM lyuw== X-Gm-Message-State: ALoCoQluH2geRweQ/qV6bpepnQsgQqYAAroicisOqCdhQoM6UAx7tvBSaBDvWVf7RypXK5M2WuHz X-Received: by 10.140.40.5 with SMTP id w5mr18679727qgw.65.1393801221165; Sun, 02 Mar 2014 15:00:21 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 2 Mar 2014 15:00:01 -0800 (PST) X-Originating-IP: [108.176.158.82] In-Reply-To: <56D809A7-5387-475A-8C89-794E328F6935@gmail.com> References: <94D3F60A-1E12-418B-8313-E43C93878B95@gmail.com> <56D809A7-5387-475A-8C89-794E328F6935@gmail.com> From: Julio Merino Date: Sun, 2 Mar 2014 18:00:01 -0500 X-Google-Sender-Auth: A58BaNNo39RNqvGh4MiXPDXvXyA Message-ID: Subject: Re: [PATCH] Integrate pjdfstest into the build To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 23:00:28 -0000 On Sun, Mar 2, 2014 at 5:54 PM, Garrett Cooper wrote: >> * The test files should move from tools/regression/pjdfstest/ into the >> new location under tests/sys/ and be renamed to follow the conventions >> under src/test (e.g. xx.t -> xx_test). > > Please talk to Pawel about this. These tests have been developed and are = maintained (to some degree) outside FreeBSD: https://github.com/pjd/pjdfste= st . I'd rather treat this as an integration of the 3rd party package, not = a refactor. If you want me to install the tests like _test though, I'd be m= ore than happy to do that. Ah great; that's what I suspected given the mentions of other OSes in the c= ode! Then I believe this should really live under contrib/pjdfstest. Any reason it's not there? If it's external, then yes, we should keep the layout of the upstream sources without performing any renames (unless Pawel wants to do them). Thanks From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 06:30:49 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 067C0D9D for ; Mon, 3 Mar 2014 06:30:49 +0000 (UTC) Received: from relay03.pair.com (relay03.pair.com [209.68.5.17]) by mx1.freebsd.org (Postfix) with SMTP id 9A0BBDF0 for ; Mon, 3 Mar 2014 06:30:48 +0000 (UTC) Received: (qmail 9768 invoked from network); 3 Mar 2014 06:30:47 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay03.pair.com with SMTP; 3 Mar 2014 06:30:47 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s236UkR3070941; Mon, 3 Mar 2014 07:30:46 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s236Uk7V070940; Mon, 3 Mar 2014 07:30:46 +0100 (CET) (envelope-from pho) Date: Mon, 3 Mar 2014 07:30:46 +0100 From: Peter Holm To: Julio Merino Subject: Re: My first ATF test Message-ID: <20140303063046.GB70172@x2.osted.lan> References: <20140225161129.GA59741@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 06:30:49 -0000 On Sun, Mar 02, 2014 at 03:44:14PM -0500, Julio Merino wrote: > On Tue, Feb 25, 2014 at 11:11 AM, Peter Holm wrote: > > In order to understand how ATF works I wrote a small test so I had > > something to work with: > > http://people.freebsd.org/~pho/kern_descrip_test.diff > > Did I get it right? > > Hello Peter, > > Looks good to me. > > One suggestion: that comment with the explanation for dup2_r234131 > would be much better represented either as 1) a more descriptive test > case name or 2) a test case description ("descr" metadata property). > The revision number means nothing to readers. People running the test > suite won't see the comment at all, but they will see the test case > name all the time and they may see the description depending on the > reporting format. > > For example, dup2__ebadf_when_2nd_arg_out_of_range would clearly > explain what this is. You could later put the revision number in the > "descr" property with a textual explanation if so desired. > > Cheers Yes, I see your point. I'll fix this. -- Peter From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 16:52:36 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3BA4C06 for ; Mon, 3 Mar 2014 16:52:36 +0000 (UTC) Received: from sanddollar.geekisp.com (sanddollar.geekisp.com [216.168.135.167]) by mx1.freebsd.org (Postfix) with SMTP id 551C7E33 for ; Mon, 3 Mar 2014 16:52:35 +0000 (UTC) Received: (qmail 31275 invoked by uid 1003); 3 Mar 2014 16:52:29 -0000 Received: from unknown (HELO kiwi) (tyler@monkeypox.org@64.125.69.200) by mail.geekisp.com with (DHE-RSA-AES256-SHA encrypted) SMTP; 3 Mar 2014 16:52:29 -0000 Date: Mon, 3 Mar 2014 08:52:26 -0800 From: "R. Tyler Croy" To: freebsd-testing@freebsd.org Subject: atf-c for kernel function unit testing Message-ID: <20140303165226.GK85115@kiwi> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8CLqiYUo6qiluZPB" Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 16:52:36 -0000 --8CLqiYUo6qiluZPB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I'm wondering if anybody's explored using atf-c for adding unit tests to kernel level functions or driver code. I'd like to have a bit more confidence in the new driver code I'm pulling down on a regular basis for testing -CURRENT, and unit tests seem like a good way to accomplish that If this is something nobody's tinkered with yet, i'll try to find some tinkering time :) - R. Tyler Croy ------------------------------------------------------ Code: Chatter: % gpg --keyserver keys.gnupg.net --recv-key 3F51E16F ------------------------------------------------------ --8CLqiYUo6qiluZPB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlMUs0oACgkQFCbH3D9R4W8HOwCgkMAOdAeQp57sarV693Q8eumy U50An0DpLoI18ASrDIPU4X873++uKZ/a =dI1G -----END PGP SIGNATURE----- --8CLqiYUo6qiluZPB-- From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 17:33:02 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 598B5EE9 for ; Mon, 3 Mar 2014 17:33:02 +0000 (UTC) Received: from mail-qc0-f181.google.com (mail-qc0-f181.google.com [209.85.216.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 19C60283 for ; Mon, 3 Mar 2014 17:33:01 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id c9so4001251qcz.40 for ; Mon, 03 Mar 2014 09:33:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=xYEoNP6CS7YVCKWosuYu0m0hbnMK+cXpbWWg52QdZ+s=; b=fk9Vsw9Jr7YMakgIldqaCVlKYMBrUbjIeyUftuprf21AOoJRa78EEp6fpznuKYNpSv MBuj7Oe3ETX/vALZnmUqDsNWogOPf8i2b4J1McZ4RkrhdBf9OMJeSvCgp6e6qPdRBDgj ODGANr2ORZuYn8YuCaN3csODpYbcxzMYszwtYSwwe/T7TEOfbXjkpVnYrlSREy9pCJLv 0+9Jw6kv6csUx60TUrFL0AJbZZcbcBYv6TRZswbd6yVwK29YevUgIjqU1X7u86agQNL2 4InLGapr+Zp14Z6EUHOCEbEuis8R0DTClH63oD/BmRyE3JFlvYf5O5qth7mpqr64707Q FM+w== X-Gm-Message-State: ALoCoQmJMiKoWyNg0yqq5jTf3lEXTD9J0IvigAaFGKSbPJ9CTvbvWgEa3wN+up7mje5QdWih/piZ X-Received: by 10.224.157.7 with SMTP id z7mr24972956qaw.37.1393867980741; Mon, 03 Mar 2014 09:33:00 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Mon, 3 Mar 2014 09:32:40 -0800 (PST) X-Originating-IP: [2620:0:1003:1007:69c9:8c4b:609d:3fea] In-Reply-To: <20140303165226.GK85115@kiwi> References: <20140303165226.GK85115@kiwi> From: Julio Merino Date: Mon, 3 Mar 2014 12:32:40 -0500 X-Google-Sender-Auth: nN26zLCXerK9bcPKMwBbn629KuM Message-ID: Subject: Re: atf-c for kernel function unit testing To: "R. Tyler Croy" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 17:33:02 -0000 On Mon, Mar 3, 2014 at 11:52 AM, R. Tyler Croy wrote: > > I'm wondering if anybody's explored using atf-c for adding unit tests to kernel > level functions or driver code. We do this extensively in NetBSD: we have a mechanism (rump) to build unmodified kernel-level code as user-space binaries and we later use atf to unit-test the code in controlled tiny environments. (You'd think about this as super-lightweight virtualization of the kernel.) Achieving this for any non-trivial kernel-level code is hard however, but I believe that this approach has huge benefits. For example: we are able to unit-test all file system and networking code in user-space without any chances of crashing the system; there is also the benefit of being able to control the test environment fully without relying on any state of the running kernel. For more details see: http://en.wikipedia.org/wiki/Anykernel http://wiki.netbsd.org/rumpkernel/ If the code you are interested in testing can easily be built without ties to the kernel, you can relatively easily do the above. (In fact, we used to do this in NetBSD for some tiny areas of the code before we got rump.) At some point I also pondered the idea to write an interface and a kernel module to allow writing unit tests within the kernel. This might be interesting for some special cases (like testing live hardware maybe) but it's a very fragile setup that should not be pursued if other safer solutions suffice. From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 18:22:46 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5B83889; Mon, 3 Mar 2014 18:22:46 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 577499C8; Mon, 3 Mar 2014 18:22:46 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id im17so3961296vcb.33 for ; Mon, 03 Mar 2014 10:22:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=GwfS+NdjMBsLNFGJtBFvNbS9eZ6I4Fy3HtoBlaj2Hn4=; b=S4VLPez428NRYoTcR5zqnLu0kPPlaXYs/K2awqT/O8u6BLnFHUlkIrqmrDuSSJh0Jn MhsTU35rhFN4339kbnAJtarT0DbK1x+osXFzHsCAz0/F6o6KDkytg7+ZiEetzqurm+Bq 3PJW9uk9moiTNcOv5UZ1fc+qqb5DhWTvQYTPQlyan8pLdj+yDvDZoZ/str/zuAd+DJ3r 8Fgxsk+a+stLbqlutGmfcEU/bnypj/n3S8CAULp8x78nCxrMng/C7R6xLar791rs4iix vWv9f/piC7L8ZW5CCkKWzylBy9sDzZRji/Q7KUL8LxMxzDthcMeTsbUfvWHlT8TVSKJa sC7g== MIME-Version: 1.0 X-Received: by 10.221.34.211 with SMTP id st19mr18344966vcb.5.1393870965400; Mon, 03 Mar 2014 10:22:45 -0800 (PST) Received: by 10.221.67.136 with HTTP; Mon, 3 Mar 2014 10:22:45 -0800 (PST) In-Reply-To: References: <94D3F60A-1E12-418B-8313-E43C93878B95@gmail.com> <56D809A7-5387-475A-8C89-794E328F6935@gmail.com> Date: Mon, 3 Mar 2014 10:22:45 -0800 Message-ID: Subject: Re: [PATCH] Integrate pjdfstest into the build From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 18:22:46 -0000 On Sun, Mar 2, 2014 at 3:00 PM, Julio Merino wrote: > On Sun, Mar 2, 2014 at 5:54 PM, Garrett Cooper wrote= : >>> * The test files should move from tools/regression/pjdfstest/ into the >>> new location under tests/sys/ and be renamed to follow the conventions >>> under src/test (e.g. xx.t -> xx_test). >> >> Please talk to Pawel about this. These tests have been developed and are= maintained (to some degree) outside FreeBSD: https://github.com/pjd/pjdfst= est . I'd rather treat this as an integration of the 3rd party package, not= a refactor. If you want me to install the tests like _test though, I'd be = more than happy to do that. > > Ah great; that's what I suspected given the mentions of other OSes in the= code! > > Then I believe this should really live under contrib/pjdfstest. Any > reason it's not there? If it's external, then yes, we should keep the > layout of the upstream sources without performing any renames (unless > Pawel wants to do them). I think it should be, but I don't own pjdfstest ;).. (largely for jmmv's benefit) Ultimately if pjd went this route, pjdfstest needs to be put in the vendor repo, reimported into FreeBSD under contrib, and then linked into the tree appropriately. Thank you :), -Garrett From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 18:31:36 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F5EBC6B; Mon, 3 Mar 2014 18:31:36 +0000 (UTC) Received: from mail-vc0-x22a.google.com (mail-vc0-x22a.google.com [IPv6:2607:f8b0:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 26D4EAF7; Mon, 3 Mar 2014 18:31:36 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id hu8so4008097vcb.29 for ; Mon, 03 Mar 2014 10:31:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=k9uRFjPMUPIM0wlV7pvnOD6uOEPnGafbZ+YjTV9pjAA=; b=kz9D0o6WEZdNV/9I9HWJYYvocVLLTdBk4Jv/y+fSOItvTx0jGdXaAk1FqGj7Hwo6D4 5lNQztgDejf9nmygwxKv7J33lhPUD1YpfPtB0VFJqEZsUyp/TuouVQdT10ydgSn3MffB 3hCrXKFM1VULFC2SZE5YEiXSC9zAFCVESxHw8EWvTOUMybVaeky9tPip5W05EtJsLnn+ 55Vq1vJzELYDR9zVYRrwPfN58bhf7Lhpj/GyHaHgy01AL0FO7wZvckqp2cjgffrTrZ2n 9Lr5BzHzrVTUsu7b5gu6MBXDU5C9Hn+n0S69acbGjHIhK460Yb3BeSOU06bq747L7ZK7 2Gcg== MIME-Version: 1.0 X-Received: by 10.58.128.72 with SMTP id nm8mr18749187veb.18.1393871495358; Mon, 03 Mar 2014 10:31:35 -0800 (PST) Received: by 10.221.67.136 with HTTP; Mon, 3 Mar 2014 10:31:35 -0800 (PST) In-Reply-To: References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> <4A25ADF9-3714-4006-ACB6-F8C745CB38AE@gmail.com> Date: Mon, 3 Mar 2014 10:31:35 -0800 Message-ID: Subject: Re: [PATCH v2] convert bin/date over to ATF From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Giorgos Keramidas X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 18:31:36 -0000 On Sun, Mar 2, 2014 at 1:22 PM, Julio Merino wrote: > On Mon, Jan 27, 2014 at 5:03 AM, Garrett Cooper wrote: >> The attached patch incorporates the input from Julio for converting bin/date/tests over to ATF. > > Apologies for the delay here as well... > > Pretty much all style nits: > > * This file is full of boilerplate and it's really hard to actually > see the tests. I think this would benefit from a simple helper > function that calls atf_test_case and evals the tiny body() with the > call to check. Ok. > * I'd get rid of the descr definitions; they provide no value. Ok. > * I'd also get rid of all the horizontal dividing lines. If you make > the test case definitions compact, they just serve to increase the > file length. Ok. > * Remove the TZ and LC_ALL overrides. This is a given within ATF. Ok. > * Why "format_string_functional_test"? I'd just call it "format_string_test". I guess I could... > * I don't think the # vim trailing line belongs in the tree. Ok. -Garrett From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 18:40:25 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B6AAEDD; Mon, 3 Mar 2014 18:40:25 +0000 (UTC) Received: from mail-ve0-x236.google.com (mail-ve0-x236.google.com [IPv6:2607:f8b0:400c:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 445F1C11; Mon, 3 Mar 2014 18:40:25 +0000 (UTC) Received: by mail-ve0-f182.google.com with SMTP id jw12so1263563veb.41 for ; Mon, 03 Mar 2014 10:40:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=370eYkIMX5flgZniDrJn7LeMi8cw+sbFAKM5mMNRv0M=; b=lPsJOXbIIqRjZgCpLCsP9oZpAEd3KzTiCjDZCtTueg30/wixp0VlkpXK7KRBJnwAcQ uyxsP1TKDFBqKWYUFhldwlOgiul7TogksodV0CK6RXzRW9nF6W/CmSaYfusll2FRRbR+ ihW3A1sKxa0QItUZ8Q7ZTmhlhhuFxn+1K08BXLE9JQB8hkVGbVRNMIFuvuwOMwdDP0qd /V/IlkCROXfuLaIOje347VdZY8n6/iUnP+zMGCfTW1QkZNTx866xCJKqkmkDdAkI0v2Q A0Ke4ApTCVfKQp0J0DC/2vbhjeQbiTikXV2B7/cz9XVj0Wv1WJMbMeRmEWOn6+N6+j7K qeUA== MIME-Version: 1.0 X-Received: by 10.58.161.46 with SMTP id xp14mr127137veb.37.1393872024407; Mon, 03 Mar 2014 10:40:24 -0800 (PST) Received: by 10.221.67.136 with HTTP; Mon, 3 Mar 2014 10:40:24 -0800 (PST) In-Reply-To: References: <20140124162759.GB90996@stack.nl> <0A68C9FE-6A3A-4DCF-92A9-A506D7581D69@gmail.com> <96373EF8-77EA-4AA6-8422-63881C32E455@gmail.com> Date: Mon, 3 Mar 2014 10:40:24 -0800 Message-ID: Subject: Re: [PATCH v2] convert /bin/sh tests over to ATF From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Jilles Tjoelker X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 18:40:25 -0000 On Sun, Mar 2, 2014 at 1:14 PM, Julio Merino wrote: > On Mon, Jan 27, 2014 at 5:02 AM, Garrett Cooper wrote: >> Hi all, >> Here's the next patch for converting bin/sh/tests over to ATF. I've tried to incorporate all input from folks about simplifying the integration scripts and removing valid_osreldate functionality. > > Apologies for the really long delay... I suppose nothing has changed > given that there were no replies to this. Soooo, some more comments: > > * Could functional_test be created with ATF_TESTS_SH ? Yes. > * Rename the script in subversion to functional_test.sh (including the > extension). It's just in my git fork; I'll rename it (good catch). > * Do NOT cd out of the current work directory in functional_test, > anywhere. This breaks all the assumptions in tests regarding the > ability to write to the current work directory without caring about > cleaning up. > > * Use atf_get_srcdir to locate where the data files are, not dirname $0. Ok. > * Continue to use SUBDIR, not ATF_TESTS_SUBDIRS. The latter does not exist. Ok. > * Reintroduce $FreeBSD$ to functional_test.sh. Ok. > * The empty Makefiles in all the subdirectories are quite confusing. > At the very least, explicitly define TESTSDIR as is customary > everywhere else. I guess I could. It just seems redundant, but there aren't a ton of Makefiles. > * I'm still afraid of all this "auto-discovery" of test cases based on > files, because it is quite confusing... but at least it's now a bit > more self-contained and easier to strip out later. So we could keep > it. Well, it would be better if this auto-discovery of testcases was a generic driver that mimicked prove, but I understand your concern. That being said, it seems like it would fit better in a different location. > * The FILES must continue to be installed. Does this actually work > when running your test programs after a "make install"? Could you please clarify your concern here? I'm not sure I understood what you meant. > Cheers Thank you :)! -Garrett From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 20:32:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE7ED988; Mon, 3 Mar 2014 20:32:13 +0000 (UTC) Received: from mail-we0-x231.google.com (mail-we0-x231.google.com [IPv6:2a00:1450:400c:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 54175A37; Mon, 3 Mar 2014 20:32:13 +0000 (UTC) Received: by mail-we0-f177.google.com with SMTP id u57so1948625wes.8 for ; Mon, 03 Mar 2014 12:32:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=20cQWSOhMXTtT65Sy3UQWjRCLpEmMzu7hXJG3XRgzYQ=; b=otIbAhw2h56OY2/xmzKZCSNhcO0aWgmHQP71zByj95No9bUo0kr0EvNg1cXwkAcIYA jFmil2zti4DAH8SIRwWso4bMYeFeBJua65FFhh+jn7dxpA3p3vZ+9v7yUQwTeLeCiOMh YuAdhfwhnLFCN2cpCSWIqg7mZlu0sF97D+5yK1qCKf4tUjuGo/n+yR3EaXnVJ7LXt6Ru 4/GkZZejEUPCwPfI3BWrNhM4dlpCAj3dLEzBN1OTJHwYId9VfXsCUYQskMLlp0Eg9Tnh Fu1A15Lg8oXI2+ZbnCDT1S8y+PbdyFGL11qTE3GCHG35KIdlWP+bBGd7OuyNLKzB/JFS b1mA== MIME-Version: 1.0 X-Received: by 10.194.202.230 with SMTP id kl6mr21721600wjc.9.1393878717515; Mon, 03 Mar 2014 12:31:57 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Mon, 3 Mar 2014 12:31:57 -0800 (PST) In-Reply-To: References: <20140303165226.GK85115@kiwi> Date: Mon, 3 Mar 2014 13:31:57 -0700 X-Google-Sender-Auth: 3z0Qm8WkXaE0SQMXE6AkuL0m6sA Message-ID: Subject: Re: atf-c for kernel function unit testing From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 20:32:14 -0000 On Mon, Mar 3, 2014 at 10:32 AM, Julio Merino wrote: > On Mon, Mar 3, 2014 at 11:52 AM, R. Tyler Croy wrote: >> >> I'm wondering if anybody's explored using atf-c for adding unit tests to kernel >> level functions or driver code. > > We do this extensively in NetBSD: we have a mechanism (rump) to build > unmodified kernel-level code as user-space binaries and we later use > atf to unit-test the code in controlled tiny environments. (You'd > think about this as super-lightweight virtualization of the kernel.) > > Achieving this for any non-trivial kernel-level code is hard however, > but I believe that this approach has huge benefits. For example: we > are able to unit-test all file system and networking code in > user-space without any chances of crashing the system; there is also > the benefit of being able to control the test environment fully > without relying on any state of the running kernel. > > For more details see: > > http://en.wikipedia.org/wiki/Anykernel > http://wiki.netbsd.org/rumpkernel/ > > If the code you are interested in testing can easily be built without > ties to the kernel, you can relatively easily do the above. (In fact, > we used to do this in NetBSD for some tiny areas of the code before we > got rump.) > > At some point I also pondered the idea to write an interface and a > kernel module to allow writing unit tests within the kernel. This > might be interesting for some special cases (like testing live > hardware maybe) but it's a very fragile setup that should not be > pursued if other safer solutions suffice. Rump is definitely the ideal method. Lacking that, FreeBSD's options are to build kernel code in userspace, or to run unit tests in the kernel. Doing the latter is not currently possible with ATF. There are some places where it's currently done ad-hoc, such as in xnb(4), but it's not pretty. Compiling kernel code in userspace isn't always possible. When I wrote xnb(4), almost all of its unit tests depended on mbuf(9), and I found that there was no way to compile mbuf.h in userspace. ZFS is a rare exception. Almost all of the ZFS code is compiled in userspace as part of libzpool. That's how ztest works. My suggestion is to write new drivers to be compilable in userspace, rely on functional tests for existing drivers, and find somebody willing to port rump from NetBSD. -Alan From owner-freebsd-testing@FreeBSD.ORG Mon Mar 3 21:27:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C267795A; Mon, 3 Mar 2014 21:27:18 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1B45F2AA; Mon, 3 Mar 2014 21:27:17 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id mc6so4922488lab.8 for ; Mon, 03 Mar 2014 13:27:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=BTn9QwkE+rpe9AJw6i7OqdWnfoW3OYNADi2xjiDe690=; b=HU+4XdpQQU16P9WFcx/wPHkuWgZcZsW9vDkcld4iPQY+ixCD/ec2+/hsMiT1IrTEoU mdcGd3bP5+PXbqpqpSB/w1MqAv/j/o/s0baVxOjykDgYqP0PBzxwq8yzPEuYXDBQhY5a cYm56g7nbWFXxci+utC29prHQ/nuOMdJ2d/PUPxPhL/CEfxAhbwQI25N2yb8WY3e/jRw f6xoTNMNBE/7u+CXEePza0vMdDbjzS4wSdmV3d2df+h1nS+KuqsOs0cmBEMpzRRR9u3G 51eFUNWb3PINi1fEGd23eo6hiX0zHtbj9LxfPvCrieFRwSQPA3sCndy0ZkWcIZeFPlU3 Kt3Q== MIME-Version: 1.0 X-Received: by 10.112.51.65 with SMTP id i1mr442815lbo.80.1393882036143; Mon, 03 Mar 2014 13:27:16 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.30.211 with HTTP; Mon, 3 Mar 2014 13:27:16 -0800 (PST) In-Reply-To: References: Date: Mon, 3 Mar 2014 13:27:16 -0800 X-Google-Sender-Auth: yBfuIp6DgOBsRPtLSTVBD9kP12g Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: kyua-discuss@googlegroups.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 21:27:18 -0000 On Sun, Mar 2, 2014 at 12:26 PM, Julio Merino wrote: > > (As I mentioned in the other thread) this is relatively easy to do and > we have enough time to get it going by May 15th. > > The path of least resistance to get this out for now is: > > 1. Reintroduce support for --output=format:path to 'report' so that > 'report' can output more than one format. The existence of > 'report-html' as a separate command is a pretty ugly hack. > > 2. Split out the console-specific formatting from cmd_report.cpp into > a separate module to clearly expose what is part of the formatting and > what is not. Adding JUnit XML support should not require changes to > other code. > > 3. Add a new module with the JUnit XML formatting code. > I think that these are all fine goals. I might be able to help with 3. > > (Longer-term, I am pondering the option of converting all the > report-generation code to Lua with extensive support for hooks. This > would allow you to tweak the output format without having to change > Kyua internals.) > That's something to think about. After a few days of research I found two other test result output formats: (1) TestNG, which is a successor to JUnit: http://testng.org/doc/index.html , TestNG DTD: http://testng.org/testng-1.0.dtd.php (2) Test Anything Protocol, http://testanything.org/ I don't know how widely these formats are used, but people *are* using them. The easier we make it to customize Kyua's output formats, the easier it will be to get more people to use Kyua. -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Mar 5 00:35:44 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 74D7948D for ; Wed, 5 Mar 2014 00:35:44 +0000 (UTC) Received: from mail-qa0-f46.google.com (mail-qa0-f46.google.com [209.85.216.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2CE6A24F for ; Wed, 5 Mar 2014 00:35:43 +0000 (UTC) Received: by mail-qa0-f46.google.com with SMTP id i13so322411qae.33 for ; Tue, 04 Mar 2014 16:35:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:subject:message-id :mime-version:content-type:content-disposition:user-agent; bh=xhRFBoJN4RtZDQcP5udulIjL4MIj0ktr/W5QjAWFJL4=; b=MDqOLVtgSKdDZ0SM4p4l59/2jILNZUnLTGz0IjhuS437Tt9e4r7Niog48zOAeKXJeo 171PTmjVW8LeiM/m+atSzQVcTv1LfpvS8ac1Oo0Bn/zOvmySuq8NWxbZl3dwIbXBOjzL No8SfaQb8VaziGKFOw07hsMqAJtDkYdTaaaNp0u7xwe5xBHhfgLKOjunREMdQRPwyFn6 UEAlwre6daStNttojoKKR1UAh61aw4MjM8zlbspvZQGWBviVIOdCtzsXP8hn0QFzalXs OkFVfLWkX3VHm39MKf7WNNRl1LIyYYtaAEoOeBKob/Ouk+88hUo/oq9TeeYoPM6VZVoD 56Qw== X-Gm-Message-State: ALoCoQlf9Ellwf57RBybH2tYCYnEpa6FH9j5cfQ2SY5IlfXE5ppeEpat6IGmiBIdZV6zqcvnbsYg X-Received: by 10.140.85.35 with SMTP id m32mr3198020qgd.40.1393979243558; Tue, 04 Mar 2014 16:27:23 -0800 (PST) Received: from fbair.virtual.network (dhcp-172-26-12-18.nyc.corp.google.com [172.26.12.18]) by mx.google.com with ESMTPSA id a5sm2060362qae.2.2014.03.04.16.27.22 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Mar 2014 16:27:22 -0800 (PST) Sender: Julio Merino Date: Tue, 4 Mar 2014 19:27:21 -0500 From: Julio Merino To: freebsd-testing@freebsd.org Subject: Merging *.test.mk into bsd.test.mk Message-ID: <20140305002721.GA1156@fbair.virtual.network> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="yrj/dFKFPuw6o+aM" Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 00:35:44 -0000 --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello all, Having {atf,plain,tap}.test.mk is nice for modularity, but, in retrospect, it is not a good idea for these files to be user-facing. In a sense, these files are all internal implementation details of the glue code in bsd.test.mk. I am going to make bsd.test.mk the only publicly-visible mk fragment to control the build of tests. The behavior of this file should be determined by the declarative variables in the Makefile, not by what is being included. The main reason is that I'm now finding a situation where I'd like to build ATF and TAP test programs from a single directory. To do so, I'd need to include atf.test.mk and tap.test.mk simultaneously, both of which indirectly include bsd.test.mk. This, of course, causes trouble. Please see the attached patch (being tested now) for an idea of what this means. Anyone thinks this is a bad idea? Thanks! PS: This is a "prerequisite" to easily make all the Kyua-related code optional as some people have requested. But I'll provide more details on that in a followup thread. --yrj/dFKFPuw6o+aM Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="merge.diff" Index: bin/date/tests/Makefile =================================================================== --- bin/date/tests/Makefile (revision 262758) +++ bin/date/tests/Makefile (working copy) @@ -6,4 +6,4 @@ TAP_TESTS_SH= legacy_test -.include +.include Index: bin/mv/tests/Makefile =================================================================== --- bin/mv/tests/Makefile (revision 262758) +++ bin/mv/tests/Makefile (working copy) @@ -6,4 +6,4 @@ TAP_TESTS_SH= legacy_test -.include +.include Index: bin/pax/tests/Makefile =================================================================== --- bin/pax/tests/Makefile (revision 262758) +++ bin/pax/tests/Makefile (working copy) @@ -6,4 +6,4 @@ TAP_TESTS_PERL= legacy_test -.include +.include Index: bin/sh/tests/Makefile =================================================================== --- bin/sh/tests/Makefile (revision 262758) +++ bin/sh/tests/Makefile (working copy) @@ -15,4 +15,4 @@ SUBDIR+= builtins errors execution expansion parameters parser set-e -.include +.include Index: bin/test/tests/Makefile =================================================================== --- bin/test/tests/Makefile (revision 262758) +++ bin/test/tests/Makefile (working copy) @@ -12,4 +12,4 @@ # requested. See https://code.google.com/p/kyua/issues/detail?id=6 TEST_METADATA.legacy_test+= required_user="unprivileged" -.include +.include Index: lib/atf/libatf-c/tests/Makefile =================================================================== --- lib/atf/libatf-c/tests/Makefile (revision 262758) +++ lib/atf/libatf-c/tests/Makefile (working copy) @@ -35,4 +35,4 @@ ATF_TESTS_SH= pkg_config_test -.include +.include Index: lib/atf/libatf-c/tests/detail/Makefile =================================================================== --- lib/atf/libatf-c/tests/detail/Makefile (revision 262758) +++ lib/atf/libatf-c/tests/detail/Makefile (working copy) @@ -31,4 +31,4 @@ version_helper.o: atf-version .include "../../../common.mk" -.include +.include Index: lib/atf/libatf-c++/tests/Makefile =================================================================== --- lib/atf/libatf-c++/tests/Makefile (revision 262758) +++ lib/atf/libatf-c++/tests/Makefile (working copy) @@ -28,4 +28,4 @@ ATF_TESTS_SH= pkg_config_test -.include +.include Index: lib/atf/libatf-c++/tests/detail/Makefile =================================================================== --- lib/atf/libatf-c++/tests/detail/Makefile (revision 262758) +++ lib/atf/libatf-c++/tests/detail/Makefile (working copy) @@ -29,4 +29,4 @@ version_helper.o: atf-version .include "../../../common.mk" -.include +.include Index: lib/atf/tests/test-programs/Makefile =================================================================== --- lib/atf/tests/test-programs/Makefile (revision 262758) +++ lib/atf/tests/test-programs/Makefile (working copy) @@ -21,4 +21,4 @@ ATF_TESTS_SH_SRC_${_T}= common.sh ${_T}.sh .endfor -.include +.include Index: lib/libcrypt/tests/Makefile =================================================================== --- lib/libcrypt/tests/Makefile (revision 262758) +++ lib/libcrypt/tests/Makefile (working copy) @@ -9,4 +9,4 @@ CFLAGS+= -I${.CURDIR:H} LDADD+= -L${.OBJDIR:H} -lcrypt -.include +.include Index: libexec/atf/atf-check/tests/Makefile =================================================================== --- libexec/atf/atf-check/tests/Makefile (revision 262758) +++ libexec/atf/atf-check/tests/Makefile (working copy) @@ -9,4 +9,4 @@ ATF_TESTS_SH= atf-check_test -.include +.include Index: share/examples/tests/tests/atf/Makefile =================================================================== --- share/examples/tests/tests/atf/Makefile (revision 262758) +++ share/examples/tests/tests/atf/Makefile (working copy) @@ -27,4 +27,4 @@ # definitions from above. KYUAFILE= yes -.include +.include Index: share/examples/tests/tests/plain/Makefile =================================================================== --- share/examples/tests/tests/plain/Makefile (revision 262758) +++ share/examples/tests/tests/plain/Makefile (working copy) @@ -27,4 +27,4 @@ # definitions from above. KYUAFILE= yes -.include +.include Index: share/mk/atf.test.mk =================================================================== --- share/mk/atf.test.mk (revision 262758) +++ share/mk/atf.test.mk (working copy) @@ -1,9 +1,13 @@ # $FreeBSD$ # +# You must include bsd.test.mk instead of this file from your Makefile. +# # Logic to build and install ATF test programs; i.e. test programs linked # against the ATF libraries. -.include +.if !target(____) +.error atf.test.mk cannot be included directly. +.endif # List of C, C++ and shell test programs to build. # @@ -164,5 +168,3 @@ .endif .endif - -.include Index: share/mk/bsd.test.mk =================================================================== --- share/mk/bsd.test.mk (revision 262758) +++ share/mk/bsd.test.mk (working copy) @@ -2,13 +2,9 @@ # # Generic build infrastructure for test programs. # -# The code in this file is independent of the implementation of the test -# programs being built; this file just provides generic infrastructure for the -# build and the definition of various helper variables and targets. -# -# Makefiles should never include this file directly. Instead, they should -# include one of the various *.test.mk depending on the specific test programs -# being built. +# This is the only public file that should be included by Makefiles when +# tests are to be built. All other *.test.mk files are internal and not +# to be included directly. .include @@ -67,11 +63,6 @@ TESTS_LD_LIBRARY_PATH+= ${DESTDIR}/lib ${DESTDIR}/usr/lib TESTS_ENV+= LD_LIBRARY_PATH=${TESTS_LD_LIBRARY_PATH:tW:C/ +/:/g} -# List of all tests being built. This variable is internal should not be -# defined by the Makefile. The various *.test.mk modules extend this variable -# as needed. -_TESTS?= - # Path to the prefix of the installed Kyua CLI, if any. # # If kyua is installed from ports, we automatically define a realtest target @@ -79,6 +70,15 @@ # hierarchy specified by this variable. KYUA_PREFIX?= /usr/local +# List of all tests being built. The various *.test.mk modules extend this +# variable as needed. +_TESTS= + +# Pull in the definitions of all supported test interfaces. +.include +.include +.include + .if !empty(TESTS_SUBDIRS) SUBDIR+= ${TESTS_SUBDIRS} .endif Index: share/mk/plain.test.mk =================================================================== --- share/mk/plain.test.mk (revision 262758) +++ share/mk/plain.test.mk (working copy) @@ -1,10 +1,14 @@ # $FreeBSD$ # +# You must include bsd.test.mk instead of this file from your Makefile. +# # Logic to build and install plain test programs. A plain test programs it not # supposed to use any specific testing framework: all it does is run some code # and report the test's pass or fail status via a 0 or 1 exit code. -.include +.if !target(____) +.error plain.test.mk cannot be included directly. +.endif # List of C, C++ and shell test programs to build. # @@ -58,5 +62,3 @@ mv ${.TARGET}.tmp ${.TARGET} .endfor .endif - -.include Index: share/mk/tap.test.mk =================================================================== --- share/mk/tap.test.mk (revision 262758) +++ share/mk/tap.test.mk (working copy) @@ -1,5 +1,7 @@ # $FreeBSD$ # +# You must include bsd.test.mk instead of this file from your Makefile. +# # Logic to build and install TAP-compliant test programs. # # This is provided to support existing tests in the FreeBSD source tree @@ -6,7 +8,9 @@ # (particularly those coming from tools/regression/) that comply with the # Test Anything Protocol. It should not be used for new tests. -.include +.if !target(____) +.error tap.test.mk cannot be included directly. +.endif # List of C, C++ and shell test programs to build. # @@ -87,5 +91,3 @@ mv ${.TARGET}.tmp ${.TARGET} .endfor .endif - -.include Index: tests/sys/kern/Makefile =================================================================== --- tests/sys/kern/Makefile (revision 262758) +++ tests/sys/kern/Makefile (working copy) @@ -8,4 +8,4 @@ LDADD+= -lpthread -.include +.include Index: usr.bin/atf/atf-sh/tests/Makefile =================================================================== --- usr.bin/atf/atf-sh/tests/Makefile (revision 262758) +++ usr.bin/atf/atf-sh/tests/Makefile (working copy) @@ -23,4 +23,4 @@ chmod +x ${.TARGET}.tmp mv ${.TARGET}.tmp ${.TARGET} -.include +.include --yrj/dFKFPuw6o+aM-- From owner-freebsd-testing@FreeBSD.ORG Wed Mar 5 08:58:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A9D4995 for ; Wed, 5 Mar 2014 08:58:09 +0000 (UTC) Received: from relay03.pair.com (relay03.pair.com [209.68.5.17]) by mx1.freebsd.org (Postfix) with SMTP id 2B9A4CA6 for ; Wed, 5 Mar 2014 08:58:08 +0000 (UTC) Received: (qmail 42477 invoked from network); 5 Mar 2014 08:58:07 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay03.pair.com with SMTP; 5 Mar 2014 08:58:07 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s258w6Kb070645 for ; Wed, 5 Mar 2014 09:58:06 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s258w6Ad070644 for freebsd-testing@freebsd.org; Wed, 5 Mar 2014 09:58:06 +0100 (CET) (envelope-from pho) Date: Wed, 5 Mar 2014 09:58:06 +0100 From: Peter Holm To: freebsd-testing@freebsd.org Subject: Test scenario for sysctl kern.maxfiles Message-ID: <20140305085806.GA70478@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 08:58:09 -0000 Here's an attempt to verify that increasing kern.maxfiles works as expected. http://people.freebsd.org/~pho/kern_descrip_test-v3.diff -- Peter From owner-freebsd-testing@FreeBSD.ORG Wed Mar 5 12:33:10 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3D3346A for ; Wed, 5 Mar 2014 12:33:10 +0000 (UTC) Received: from relay00.pair.com (relay00.pair.com [209.68.5.9]) by mx1.freebsd.org (Postfix) with SMTP id 7DE6D25B for ; Wed, 5 Mar 2014 12:33:09 +0000 (UTC) Received: (qmail 93415 invoked from network); 5 Mar 2014 12:33:08 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay00.pair.com with SMTP; 5 Mar 2014 12:33:08 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s25CX7rV077902 for ; Wed, 5 Mar 2014 13:33:07 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s25CX7hb077901 for freebsd-testing@freebsd.org; Wed, 5 Mar 2014 13:33:07 +0100 (CET) (envelope-from pho) Date: Wed, 5 Mar 2014 13:33:07 +0100 From: Peter Holm To: freebsd-testing@freebsd.org Subject: A shell script test case Message-ID: <20140305123307.GA77355@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 12:33:10 -0000 I tried my hand with a test scenario for a current FreeBSD VM problem: At first I did not understand what I did wrong when I got: $ kyua test vm_pageout_test vm_pageout_test:__test_cases_list__ -> broken: Tester did not exit cleanly: kyua-atf-tester: Test program list did not return success [0.000s] 0/1 passed (1 failed) Committed action 15 $ but I figured it out. Now to the real question: Are regression tests like this one really ATF material? I would think that the long runtime could be prohibitive? $ kyua debug vm_pageout_test:basic FAIL UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 662 1 0 52 0 10268 0 pause IWs - 0:00.00 nfsuserd: master (nfsuserd) 25 845 1 0 20 0 12796 0 pause IWs - 0:00.00 sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue (sendmail) 0 850 1 0 20 0 10024 0 nanslp SWs - 0:00.00 /usr/sbin/cron -s 1001 4069 4068 0 20 0 11808 0 wait IWs 0 0:00.00 -bash (bash) 0 4115 4069 0 20 0 10496 0 wait IW 0 0:00.00 su 0 4116 4115 0 23 0 10772 0 pause IW 0 0:00.00 _su (csh) 1001 4354 4353 0 20 0 11808 0 wait IWs 1 0:00.00 -bash (bash) 0 4416 4354 0 20 0 10496 0 wait IW 1 0:00.00 su 0 4417 4416 0 23 0 10772 0 pause IW 1 0:00.00 _su (csh) 0 4419 4417 0 20 0 11808 0 wait IW 1 0:00.00 bash 0 6838 4419 0 52 0 13412 0 wait IW+ 1 0:00.00 kyua debug vm_pageout_test:basic 0 6857 6838 0 52 0 9900 0 wait IW 1 0:00.00 /usr/local/libexec/kyua-atf-tester -t1200 test -vhas.cleanup=false -vunprivileged-user=tests /usr/tests/sys/vm/vm_pageout_test basi vm_pageout_test:basic -> failed: Failed $ kyua test vm_pageout_test vm_pageout_test:basic -> failed: Failed [367.941s] 0/1 passed (1 failed) Committed action 14 $ -- Peter From owner-freebsd-testing@FreeBSD.ORG Wed Mar 5 17:08:51 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4170774 for ; Wed, 5 Mar 2014 17:08:51 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7E9B0213 for ; Wed, 5 Mar 2014 17:08:51 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id k14so1613279wgh.35 for ; Wed, 05 Mar 2014 09:08:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=7cOYvuLcfObZxd4eyE0heX94iEAzV0lVBbH7ajA8nVg=; b=AhiA3IUBqjyDXO6dYb98G1BDRE3a4RA/mwOTnQGISlrMv7RrzAKxWteR6wJWNLA2hs I1iIJCYhHvoKbKsRjr+D8LGzs9gQALlEZl4SiSI8VjFR59PzaAYYoo3s9Jdy7OOfO6VI hlcon5JKJTNn34JzM6Ugnx0zrYEh4u0AiSCvGGdZW3OK3WIq4WnG7PR6QBCCShAnfGI0 ylX5lKUqElKjNB3x6vO17Fg/yIcPo8d1cXeEyFbNioWaVnSWSyBTeWEpU9YWbDl9on6E ZQ5/yIpoISEQketTJJoAdY8rqvlWJhY2mqGK9cugFecygzL56J+7Yj8/tSzGI8rCIobJ YqSA== MIME-Version: 1.0 X-Received: by 10.194.75.225 with SMTP id f1mr2464962wjw.87.1394039329917; Wed, 05 Mar 2014 09:08:49 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Wed, 5 Mar 2014 09:08:49 -0800 (PST) In-Reply-To: <20140305085806.GA70478@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> Date: Wed, 5 Mar 2014 10:08:49 -0700 X-Google-Sender-Auth: J4LZUrv0g50Qz89efc2zMcb4OPQ Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 17:08:52 -0000 On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: > Here's an attempt to verify that increasing kern.maxfiles works as > expected. > > http://people.freebsd.org/~pho/kern_descrip_test-v3.diff > -- > Peter 1) done should be of type "static volatile sig_atomic_t", not int, because it's set by signal handlers. 2) using atexit() to register a cleanup routing is a hack. No doubt you already noticed that it's difficult to use Kyua's builtin cleanup capabilities because of the need to pass the value of oldmaxfiles. I too have experienced that frustration. Is there any way to pass values from the body of a testcase to its cleanup? Using atf_tc_set_md_var() would be one way, but the man page suggests that that function cannot be called from the body. Julio, is there a better way to do this? 3) Why do you openfiles(oldmaxfiles + 50, 0) instead of just openfiles(oldmaxfiles) ? It seems that the latter would also verify the assertion. 4) openfiles(oldmaxfiles + 50, 0) will fail if there are already 950 open files. A quick check on freefall showed that kern.openfiles was 935. Perhaps you should try opening openfiles(oldmaxfiles - kern.openfiles + 50, 0). That wouldn't be perfect, due to races, but it would be better. -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 11:23:31 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CC917E77 for ; Thu, 6 Mar 2014 11:23:31 +0000 (UTC) Received: from relay02.pair.com (relay02.pair.com [209.68.5.16]) by mx1.freebsd.org (Postfix) with SMTP id 82D8339E for ; Thu, 6 Mar 2014 11:23:31 +0000 (UTC) Received: (qmail 55040 invoked from network); 6 Mar 2014 11:23:23 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay02.pair.com with SMTP; 6 Mar 2014 11:23:23 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s26BNMU3015482; Thu, 6 Mar 2014 12:23:23 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s26BNMw9015481; Thu, 6 Mar 2014 12:23:22 +0100 (CET) (envelope-from pho) Date: Thu, 6 Mar 2014 12:23:22 +0100 From: Peter Holm To: Alan Somers Subject: Re: Test scenario for sysctl kern.maxfiles Message-ID: <20140306112322.GA10664@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 11:23:31 -0000 On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: > On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: > > Here's an attempt to verify that increasing kern.maxfiles works as > > expected. > > > > http://people.freebsd.org/~pho/kern_descrip_test-v3.diff > > -- > > Peter > > 1) done should be of type "static volatile sig_atomic_t", not int, > because it's set by signal handlers. > Yes, that is nicer (I learned something new today :-). But the use here works because there is a call to usleep(3) after each test, forcing the compiler to reload the "done" variable. > 2) using atexit() to register a cleanup routing is a hack. No doubt Why do you say that using atexit(3) is a hack? > you already noticed that it's difficult to use Kyua's builtin cleanup > capabilities because of the need to pass the value of oldmaxfiles. I > too have experienced that frustration. Is there any way to pass > values from the body of a testcase to its cleanup? Using > atf_tc_set_md_var() would be one way, but the man page suggests that > that function cannot be called from the body. Julio, is there a > better way to do this? > > 3) Why do you openfiles(oldmaxfiles + 50, 0) instead of just > openfiles(oldmaxfiles) ? It seems that the latter would also verify > the assertion. > The idea is to test "expansion" -> expand maxfiles to maxfiles + 1000 and test "more than was possible before". But that of cause only works on a moderately loaded host. > 4) openfiles(oldmaxfiles + 50, 0) will fail if there are already 950 > open files. A quick check on freefall showed that kern.openfiles was > 935. Perhaps you should try opening openfiles(oldmaxfiles - > kern.openfiles + 50, 0). That wouldn't be perfect, due to races, but > it would be better. > Yes I agree, that would be better. Now this raises an interesting question for me: Which environment do you guys expect ATF to run in? If it is hosts like freefall, any resource hogging tests are out of the question I would think. -- Peter From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 14:16:35 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24F48E49 for ; Thu, 6 Mar 2014 14:16:35 +0000 (UTC) Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D24F7A0F for ; Thu, 6 Mar 2014 14:16:34 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id r5so2871797qcx.18 for ; Thu, 06 Mar 2014 06:16:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Y3jQbOccKpEgXy5qsMKMFjgCpqVxG3ljNpn4XLEx8rM=; b=A0vClPz2wxhQEXFd3UP7ISJV/tadCdgdohGj2CCLsNHC4bt7u9JnHLCbsS0F8MOYXx D5HSkHKxzDP0MT9LvZtqgdkFe5SrkHKeVF0lCe4h3v4XRyzrYdW/os3D4AIIiV0/YsG1 99RDpp8sNtcG1C+1AwWxqH2FPPb9dxH6eWmfM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=Y3jQbOccKpEgXy5qsMKMFjgCpqVxG3ljNpn4XLEx8rM=; b=We+pkBdbLVAPBoXJGd9Li24k3/CiEzeRn1qv3rZAGGJ8Cvh0+0JCY1sjBEAxTPsJIw mOFmxR8Snr//S0axk7jdlEX7YLplswQY68UoRLP2o+aZ31fPR4g3YmVRokeI5opqtqE6 vDtBcHcgheDzdewZXs0YybiMh6v9CJh1q2mM1ZFjyLQZiBdlv34xN3nXjPytMEfsEOj0 ay7Tsrf/4+jnT3IZVmXMdW81xb1cjtz7fEqZl+f+ynQS3zVuLvi8eloSiiP8PCf0djDH MNTqQhpt/ek2vuCxToDD5meBOxpPHtM6JC+g975GMY2CPTbA1loB436Cw88eAYUf/C3e LAHw== X-Gm-Message-State: ALoCoQmvaYIxpiTU2WeX/7BC8/9u/VLaXwPpUKl12wQB+UUxvI1OdkuyxWqqKpfmO9STfmD9KWUy X-Received: by 10.224.80.70 with SMTP id s6mr9677261qak.44.1394115388673; Thu, 06 Mar 2014 06:16:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.147.225 with HTTP; Thu, 6 Mar 2014 06:15:58 -0800 (PST) In-Reply-To: <20140306112322.GA10664@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> From: Eitan Adler Date: Thu, 6 Mar 2014 09:15:58 -0500 Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles To: Peter Holm Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 14:16:35 -0000 On 6 March 2014 06:23, Peter Holm wrote: > On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: >> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: >> > Here's an attempt to verify that increasing kern.maxfiles works as >> > expected. >> > >> > http://people.freebsd.org/~pho/kern_descrip_test-v3.diff >> > -- >> > Peter >> >> 1) done should be of type "static volatile sig_atomic_t", not int, >> because it's set by signal handlers. >> > > Yes, that is nicer (I learned something new today :-). But the use > here works because there is a call to usleep(3) after each test, > forcing the compiler to reload the "done" variable. That isn't what sig_atomic_t is trying to prevent. It is an ""integer type of an object that can be accessed as an atomic entity, even in the presence of asynchronous interrupts."". In particular, on some machines it would be possible for the signal handler to observe a half-updated "int" type variable. On i386 sig_atomic_t happens to be an "int". On amd64 it happens to be a long. This is not contractual. -- Eitan Adler From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 14:32:32 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67B3D1FA for ; Thu, 6 Mar 2014 14:32:32 +0000 (UTC) Received: from mail-qg0-f46.google.com (mail-qg0-f46.google.com [209.85.192.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 269C6C25 for ; Thu, 6 Mar 2014 14:32:31 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id e89so7528425qgf.5 for ; Thu, 06 Mar 2014 06:32:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to:content-type; bh=xQcvRgoPe2JdJCeYAai9wY7SXknivm1qgdo+mzVYRSM=; b=BPU8Xc5nPRzvJXvcHKNkph9G6v7GXxYl0cW8E+z1BNeWsxuUG8K/9EfqWIkZuAsusO 8cT69NKlgBzxXea7d2IzVexuiLUWk+kfQ3/bhu1g1i/nrw6bwDtnHmpFC7v2HwIcl3qu t/Nx3S2+Vvj7nUseHXwxB+uknBnDxwpkH4tDUBkIUXVY7YGN5TZnxvLVndyxvoDfNdLq p5/4cPsAWtyamryCsr3g3eG/6XbhXF7yD9LaIt+vQv4mbvIuntffUgKnQXWmB6Anf3SU s/3xl2SEH7+Lxv2Tc148d6vdJ274AJjUXix4slyRp5wg37D3uvXlJnNBApL8W6qny1Nr u9Yg== X-Gm-Message-State: ALoCoQl3lnEZ5w4cHuJHOKtVEym1XvFgXAVvsZl+pbGeQ/EubMNUiwsA0z0Tu3fSa5pM5Bi3Bnsw X-Received: by 10.140.40.5 with SMTP id w5mr13455289qgw.65.1394115997098; Thu, 06 Mar 2014 06:26:37 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 6 Mar 2014 06:26:17 -0800 (PST) X-Originating-IP: [2620:0:1003:1021:5c49:13a2:f7c6:9920] From: Julio Merino Date: Thu, 6 Mar 2014 09:26:17 -0500 X-Google-Sender-Auth: fLo_gcJNme7AE5f3ft5C8qDCu2s Message-ID: Subject: Should we MFC tests into stable/10? To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 14:32:32 -0000 Hello all, Keeping the testing infrastructure in sync between current and stable/10 is, in my opinion, a worthy goal. For that reason, I have (finally) pulled up a bunch of related pending changes into the branch today and will continue to do so for upcoming improvements. However, I would like to gather your opinion on what to do about the tests themselves. Do you think it's worth keeping the tests between current and stable/10 in sync wherever possible? Because we have barely just started adding tests, this will certainly involve quite a bit of churn in MFCs -- but that's probably not a big deal. The tricky^Winteresting cases will come when tests start failing in only one of the two branches :-P My opinion is now leaning towards merging everything where it makes sense. What's yours? Thanks, -- Julio Merino From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 14:53:27 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D77AD20 for ; Thu, 6 Mar 2014 14:53:27 +0000 (UTC) Received: from mail-qc0-f169.google.com (mail-qc0-f169.google.com [209.85.216.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0C470E58 for ; Thu, 6 Mar 2014 14:53:26 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id i17so2982728qcy.14 for ; Thu, 06 Mar 2014 06:53:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=Mg9wXbKs+BJOU6Zjm1Cvsts6hxFGUmAvA87OqIIK2+s=; b=CxYogG6zPdC396bFScTPD6U+Lzerj6Pp3cU5sc4SN71u7odIB5WA5k2MuQf+E6lKbx Ewb6rw75sJ/fpiAMlxmcHmF106NyHwm32oDZJ/3Fb3ZvibW5qsviWqesyWCWdnCHsgkr aSe14M9r2NvFjzgrUD8+02xf62sxwgK0LbTCdqlaYXhv8bsnvqUiWQHtqiCAOK4lHiQl 50bHk0V6VyYsgLt0jRXcGFTt5uEg1Yg4XaH0Kojlt9M9Bm2EQ7bPRf+0iwa5/VarD3HT uuQqpfFnhrbWzVE3eXl52JhNmkYVOkGwbvBpbiixKPM04EsUl81QBkapnp7KwpTvewJO +eVw== X-Gm-Message-State: ALoCoQmaJvx+yaPsl2gpKAJNFMJnt5v1Pa1N3n9MXApQf1Wr3aLERZ6R4eT6dUC+XaUWDt502FOy X-Received: by 10.224.46.73 with SMTP id i9mr11232198qaf.65.1394117598168; Thu, 06 Mar 2014 06:53:18 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 6 Mar 2014 06:52:58 -0800 (PST) X-Originating-IP: [2620:0:1003:1021:5c49:13a2:f7c6:9920] In-Reply-To: <20140306112322.GA10664@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> From: Julio Merino Date: Thu, 6 Mar 2014 09:52:58 -0500 X-Google-Sender-Auth: rk1-NnBp-BbQtwqoquSJ6bSmp3w Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 14:53:27 -0000 On Thu, Mar 6, 2014 at 6:23 AM, Peter Holm wrote: > On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: >> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: >> > Here's an attempt to verify that increasing kern.maxfiles works as >> > expected. >> > >> > http://people.freebsd.org/~pho/kern_descrip_test-v3.diff >> > -- >> > Peter A couple of general comments: * In openfiles2(), it seems to me that 'i' should be size_t. That cast in the for loop looks like a hack. * Style detail: I'd change the code to say something like the following, which clearly separates the action from the handling of the result. r = open("/etc/passwd", O_RDONLY); if (ignore) { if (r == -1) break; } else { ATF_REQUIRE(r != -1); } * Why does this test rely on /etc/passwd? Just create a file in the current directory and open it. See the atf_utils_* in the atf-c-api(3) manpage for various helper functions to deal with files. If you want to depend on system files, the test should be declaring that explicitly with atf_set_md_var("require.files", "/etc/passwd"). * Does the period in the test case name kern.maxfiles__increase actually work? I'm surprised. * What's the rationale behind the TEST macro? More below. >> 1) done should be of type "static volatile sig_atomic_t", not int, >> because it's set by signal handlers. >> > > Yes, that is nicer (I learned something new today :-). But the use > here works because there is a call to usleep(3) after each test, > forcing the compiler to reload the "done" variable. > >> 2) using atexit() to register a cleanup routing is a hack. No doubt > > Why do you say that using atexit(3) is a hack? Because that's not how cleanup should be implemented in atf tests: you should be using the cleanup routines. The reason is that atexit(3) is not guaranteed to work: if your test crashes or is killed by Kyua because it overruns its deadline, your cleanup code won't work but a cleanup routine will. >> you already noticed that it's difficult to use Kyua's builtin cleanup >> capabilities because of the need to pass the value of oldmaxfiles. I >> too have experienced that frustration. Is there any way to pass >> values from the body of a testcase to its cleanup? Using >> atf_tc_set_md_var() would be one way, but the man page suggests that >> that function cannot be called from the body. Julio, is there a >> better way to do this? The "problem" is that the body and the cleanup run in different processes really, so there is no way you can pass state other than writing to a local file. This change: http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/kernel/t_sysv.c.diff?r1=1.3&r2=1.4&only_with_tag=MAIN shows a way to do it. Because these tests are dealing with scary global system state (which is a bad thing), I think that the hassle of having to do this cleanup dance is "kinda justified": not that many tests should need it and, for those that do, it becomes clear that maybe the test should be done in a different way. We could do better, yes; I don't disagree with that. Suggestions? > Now this raises an interesting question for me: Which environment do > you guys expect ATF to run in? If it is hosts like freefall, any > resource hogging tests are out of the question I would think. That's the interesting question. This test will probably be flaky and/or cause side-effects in the system while it runs because it relies on state that is out of control of the test. For the setup at kyua1.nyi.freebsd.org, this is not a problem because the tests run in a VM. But we want to encourage users to run the tests themselves and this kind of tests may be problematic. Here goes a possibility: add a configuration variable "allow_kernel_side_effects" or similar to kyua.conf (test_suites.FreeBSD.allow_kernel_side_effects = true). Then make the test case require the variable with atf_set_md_var("require.config", "allow_kernel_side_effects") so that this test only runs when the user has told Kyua to allow these tests. We can later change the kyua1.nyi.freebsd.org setup to define this variable but we leave it off by default for end users. From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 15:01:51 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3897D2DE; Thu, 6 Mar 2014 15:01:51 +0000 (UTC) Received: from mail-pd0-x22c.google.com (mail-pd0-x22c.google.com [IPv6:2607:f8b0:400e:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E9BB2F3A; Thu, 6 Mar 2014 15:01:50 +0000 (UTC) Received: by mail-pd0-f172.google.com with SMTP id p10so2641407pdj.17 for ; Thu, 06 Mar 2014 07:01:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=xvfpRUgTcmLmRljcdJLbNJXxIzTucL30UAN7NvTRDNo=; b=dMxGDKifg9bsk9aUAlvzxO8riIkchLnM0zHuASfmALxR7r9GnoOsWGXT4l/T+Pvj0L t4YkD0Bg8hCq+EnQQtIM07BlqIuUftbKlcp1XedaoHRgFSirin7cOpUOn8xG9fvV3Bd1 Fj24z1jN88Secfhh2NS2u2ptwYvgQNimirP54INe65qU3MwDJMxHYh//vEv1vVVI7V3C DXiBumSOKn3KfnPPL+xPOQTvt9F6YpvA9F36JudyOmUTS+GejXdrLMA6yV+f4vUWPlXG SvlaQL3Yi0wtVBXV5pz34XSlkcWsBhBVVHL8opU99yEI0t59UFm2DDpnS/jIC8sPMyBu pzBA== X-Received: by 10.68.172.37 with SMTP id az5mr14755697pbc.139.1394118110158; Thu, 06 Mar 2014 07:01:50 -0800 (PST) Received: from [192.168.20.11] (c-24-17-226-153.hsd1.wa.comcast.net. [24.17.226.153]) by mx.google.com with ESMTPSA id vf7sm20746536pbc.5.2014.03.06.07.01.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 06 Mar 2014 07:01:49 -0800 (PST) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <43135FE5-5A9D-4AD5-B65F-E6FC20B9E366@gmail.com> X-Mailer: iPhone Mail (11B651) From: Garrett Cooper Subject: Re: Should we MFC tests into stable/10? Date: Thu, 6 Mar 2014 07:01:47 -0800 To: Julio Merino Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 15:01:51 -0000 > On Mar 6, 2014, at 6:26, Julio Merino wrote: >=20 > Hello all, >=20 > Keeping the testing infrastructure in sync between current and > stable/10 is, in my opinion, a worthy goal. For that reason, I have > (finally) pulled up a bunch of related pending changes into the branch > today and will continue to do so for upcoming improvements. >=20 > However, I would like to gather your opinion on what to do about the > tests themselves. >=20 > Do you think it's worth keeping the tests between current and > stable/10 in sync wherever possible? Because we have barely just > started adding tests, this will certainly involve quite a bit of churn > in MFCs -- but that's probably not a big deal. The tricky^Winteresting > cases will come when tests start failing in only one of the two > branches :-P >=20 > My opinion is now leaning towards merging everything where it makes > sense. What's yours? You're probably going to run into similar problems that devs run into when M= FCing code. It depends on whether or not the tests require a specific piece of build/tes= t infrastructure, and whether or not the tests exercise a new feature or fix= /change behavior. The former case is trickier as there will be breaking points for things in t= he future, but the latter case is pretty straightforward (devs should be mer= ging tests back with code changes for new features/behavior). I vote yes for stable/10 now, but I would be wary of future merges as head a= nd stable/(head-1) diverge, and similarly stable/(head-2).= From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 15:32:50 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 911ACE7C for ; Thu, 6 Mar 2014 15:32:50 +0000 (UTC) Received: from relay02.pair.com (relay02.pair.com [209.68.5.16]) by mx1.freebsd.org (Postfix) with SMTP id 4655D2FF for ; Thu, 6 Mar 2014 15:32:50 +0000 (UTC) Received: (qmail 19420 invoked from network); 6 Mar 2014 15:32:48 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay02.pair.com with SMTP; 6 Mar 2014 15:32:48 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s26FWlCg023995; Thu, 6 Mar 2014 16:32:47 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s26FWlRs023994; Thu, 6 Mar 2014 16:32:47 +0100 (CET) (envelope-from pho) Date: Thu, 6 Mar 2014 16:32:47 +0100 From: Peter Holm To: Eitan Adler Subject: Re: Test scenario for sysctl kern.maxfiles Message-ID: <20140306153247.GA22830@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 15:32:50 -0000 On Thu, Mar 06, 2014 at 09:15:58AM -0500, Eitan Adler wrote: > On 6 March 2014 06:23, Peter Holm wrote: > > On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: > >> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: > >> > Here's an attempt to verify that increasing kern.maxfiles works as > >> > expected. > >> > > >> > http://people.freebsd.org/~pho/kern_descrip_test-v3.diff > >> > -- > >> > Peter > >> > >> 1) done should be of type "static volatile sig_atomic_t", not int, > >> because it's set by signal handlers. > >> > > > > Yes, that is nicer (I learned something new today :-). But the use > > here works because there is a call to usleep(3) after each test, > > forcing the compiler to reload the "done" variable. > > That isn't what sig_atomic_t is trying to prevent. It is an ""integer > type of an object that can be accessed as an atomic entity, even in > the presence of asynchronous interrupts."". In particular, on some > machines it would be possible for the signal handler to observe a > half-updated "int" type variable. > > On i386 sig_atomic_t happens to be an "int". On amd64 it happens to > be a long. This is not contractual. > I only just realize that the ATF test programs are threaded. Anyway it seems to be a good practice to always use "static volatile sig_atomic_t" for signal handler variables. -- Peter From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 15:44:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5F8C42C for ; Thu, 6 Mar 2014 15:44:59 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A845D604 for ; Thu, 6 Mar 2014 15:44:59 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id fa1so2833017pad.0 for ; Thu, 06 Mar 2014 07:44:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=I5TG+SSSeJ8BIGcRO3Cffm352VSLmtQE7FEEdw0Xe5s=; b=dFfBUDG6qZYntCM5xGrY1ch4QtTakCFBBbybzr3t8U5XZ4D1rI6u01Y3GUEi+xghfE pYh7vuzHiUkCtfjYbJUQR/dnCbTZ5n1zrvcPAurbYWFC1PDxIFgWZvpOUzkhs8zlDNOb emcEV/dqssHIq5NQJN8gVnVdCaljp9GI4UYd5ma6AnbIkPofw0Gj+CwDQKIChd+omMbw mKnXxYGpujiVxYet8DOndzJ3wL4OvMFwaTlVLLP5IP5066S3AAX9CDOVnePlW9MDDBVS ufNeVjs6MBvvGastFnJpN8YVFdT6/Jfbw6VchxOESOalRzj0EusCBEerjCxp+tWMD1B/ l3dA== X-Received: by 10.66.121.231 with SMTP id ln7mr15301249pab.33.1394120699270; Thu, 06 Mar 2014 07:44:59 -0800 (PST) Received: from [192.168.20.5] (c-24-17-226-153.hsd1.wa.comcast.net. [24.17.226.153]) by mx.google.com with ESMTPSA id bc4sm21077754pbb.2.2014.03.06.07.44.56 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 06 Mar 2014 07:44:57 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Test scenario for sysctl kern.maxfiles From: Garrett Cooper In-Reply-To: <20140306153247.GA22830@x2.osted.lan> Date: Thu, 6 Mar 2014 07:44:58 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <6A23D2B5-4EAA-46EF-A582-8C55FE0ED46B@gmail.com> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> <20140306153247.GA22830@x2.osted.lan> To: Peter Holm X-Mailer: Apple Mail (2.1874) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 15:45:00 -0000 On Mar 6, 2014, at 7:32 AM, Peter Holm wrote: > On Thu, Mar 06, 2014 at 09:15:58AM -0500, Eitan Adler wrote: >> On 6 March 2014 06:23, Peter Holm wrote: >>> On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: >>>> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: >>>>> Here's an attempt to verify that increasing kern.maxfiles works as >>>>> expected. >>>>>=20 >>>>> http://people.freebsd.org/~pho/kern_descrip_test-v3.diff >>>>> -- >>>>> Peter >>>>=20 >>>> 1) done should be of type "static volatile sig_atomic_t", not int, >>>> because it's set by signal handlers. >>>>=20 >>>=20 >>> Yes, that is nicer (I learned something new today :-). But the use >>> here works because there is a call to usleep(3) after each test, >>> forcing the compiler to reload the "done" variable. >>=20 >> That isn't what sig_atomic_t is trying to prevent. It is an = ""integer >> type of an object that can be accessed as an atomic entity, even in >> the presence of asynchronous interrupts."". In particular, on some >> machines it would be possible for the signal handler to observe a >> half-updated "int" type variable. >>=20 >> On i386 sig_atomic_t happens to be an "int". On amd64 it happens to >> be a long. This is not contractual. >>=20 >=20 > I only just realize that the ATF test programs are threaded. > Anyway it seems to be a good practice to always use "static volatile > sig_atomic_t" for signal handler variables. ATF forks, doesn=92t spawns threads: # grep -r pthread contrib/atf/ || echo not threaded not threaded I think that the point that others were trying to make here is that it = sets a good standard/precedence to program with atomicity in mind = instead of it not being involved, because of how signal handlers are = designed. Cheers, -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 15:51:49 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3E18870 for ; Thu, 6 Mar 2014 15:51:48 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BA77765C for ; Thu, 6 Mar 2014 15:51:48 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id ld10so2815648pab.12 for ; Thu, 06 Mar 2014 07:51:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Y+w/J2m/yR2ScYEooKfxhnBNZ59hPkt7BrojqmONEVc=; b=D/ZrLWTx7n6Ujlq6AFizcSp52ZM5VFq4w33l01L3mC95rnXMkVkNrZ9sfmLOepIb+/ hbdsJlVObr/IEW9ta55KYzt8BL5qIEiVarGPC97/29dAA+VlEWVsM4zy2zq3FBr/ZT18 c6OAG5BpIJta69VpSGTrkAZYg93wfSPBOu5qDiEI2eorKh/1MFpEsLwIoqvcHxTYt792 sEm8XB5J4val5I992YbjHJUzE3XUh/tWlBOThsiTE3Kf7RHN7v6vuaLCaayu6rQxgtLN upjeX2z30DfhaiFc6Av83F5Ifs1Roe0w7L6ET1Uj+/HzQrPkvsyQIH+xdk+X75YDyBRC N5ow== X-Received: by 10.68.189.33 with SMTP id gf1mr15080523pbc.111.1394121108461; Thu, 06 Mar 2014 07:51:48 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:1dcf:f87a:25ca:10b2? ([2601:8:ab80:7d6:1dcf:f87a:25ca:10b2]) by mx.google.com with ESMTPSA id n6sm21162761pbj.22.2014.03.06.07.51.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 06 Mar 2014 07:51:46 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: A shell script test case From: Garrett Cooper In-Reply-To: <20140305123307.GA77355@x2.osted.lan> Date: Thu, 6 Mar 2014 07:51:46 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <92FD8D48-FF95-41E6-A08D-F123E80AD8C3@gmail.com> References: <20140305123307.GA77355@x2.osted.lan> To: Peter Holm X-Mailer: Apple Mail (2.1874) Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 15:51:49 -0000 On Mar 5, 2014, at 4:33 AM, Peter Holm wrote: > I tried my hand with a test scenario for a current FreeBSD VM problem: > At first I did not understand what I did wrong when I got: >=20 > $ kyua test vm_pageout_test > vm_pageout_test:__test_cases_list__ -> broken: Tester did not exit > cleanly: kyua-atf-tester: Test program list did not return success > [0.000s] >=20 > 0/1 passed (1 failed) > Committed action 15 > $=20 >=20 > but I figured it out. >=20 > Now to the real question: Are regression tests like this one really > ATF material? I would think that the long runtime could be = prohibitive? You should set the timeout variable to something other than 300 if it = will take longer to execute (from atf-test-case(4)): timeout Type: integral. Optional; defaults to `300'. Specifies the maximum amount of time the test = case can run. This is particularly useful because some = tests can stall either because they are incorrectly = coded or because they trigger an anomalous behavior of = the pro- gram. It is not acceptable for these tests to = stall the whole execution of the test program. Can optionally be set to zero, in which case the = test case has no run-time limit. This is = discouraged. > $ kyua debug vm_pageout_test:basic > FAIL > UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND > 0 662 1 0 52 0 10268 0 pause IWs - 0:00.00 nfsuserd: = master (nfsuserd) > 25 845 1 0 20 0 12796 0 pause IWs - 0:00.00 sendmail: = Queue runner@00:30:00 for /var/spool/clientmqueue (sendmail) > 0 850 1 0 20 0 10024 0 nanslp SWs - 0:00.00 = /usr/sbin/cron -s > 1001 4069 4068 0 20 0 11808 0 wait IWs 0 0:00.00 -bash = (bash) > 0 4115 4069 0 20 0 10496 0 wait IW 0 0:00.00 su > 0 4116 4115 0 23 0 10772 0 pause IW 0 0:00.00 _su (csh) > 1001 4354 4353 0 20 0 11808 0 wait IWs 1 0:00.00 -bash = (bash) > 0 4416 4354 0 20 0 10496 0 wait IW 1 0:00.00 su > 0 4417 4416 0 23 0 10772 0 pause IW 1 0:00.00 _su (csh) > 0 4419 4417 0 20 0 11808 0 wait IW 1 0:00.00 bash > 0 6838 4419 0 52 0 13412 0 wait IW+ 1 0:00.00 kyua debug = vm_pageout_test:basic > 0 6857 6838 0 52 0 9900 0 wait IW 1 0:00.00 = /usr/local/libexec/kyua-atf-tester -t1200 test -vhas.cleanup=3Dfalse = -vunprivileged-user=3Dtests /usr/tests/sys/vm/vm_pageout_test basi > vm_pageout_test:basic -> failed: Failed > $ kyua test vm_pageout_test =20 > vm_pageout_test:basic -> failed: Failed [367.941s] Why this took 67 seconds (367 total) to fail beyond the standard timeout = (300) is a bit puzzling/concerning. I expect there to be an additional = 1-5 seconds of needed cleanup, but 67 seems a bit high... Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 16:29:43 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3395C93B for ; Thu, 6 Mar 2014 16:29:43 +0000 (UTC) Received: from relay00.pair.com (relay00.pair.com [209.68.5.9]) by mx1.freebsd.org (Postfix) with SMTP id DC72EA3E for ; Thu, 6 Mar 2014 16:29:42 +0000 (UTC) Received: (qmail 31581 invoked from network); 6 Mar 2014 16:29:35 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay00.pair.com with SMTP; 6 Mar 2014 16:29:35 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s26GTYLH025975; Thu, 6 Mar 2014 17:29:34 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s26GTYRZ025974; Thu, 6 Mar 2014 17:29:34 +0100 (CET) (envelope-from pho) Date: Thu, 6 Mar 2014 17:29:34 +0100 From: Peter Holm To: Garrett Cooper Subject: Re: Test scenario for sysctl kern.maxfiles Message-ID: <20140306162934.GB25614@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> <20140306153247.GA22830@x2.osted.lan> <6A23D2B5-4EAA-46EF-A582-8C55FE0ED46B@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6A23D2B5-4EAA-46EF-A582-8C55FE0ED46B@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 16:29:43 -0000 On Thu, Mar 06, 2014 at 07:44:58AM -0800, Garrett Cooper wrote: > On Mar 6, 2014, at 7:32 AM, Peter Holm wrote: > > > On Thu, Mar 06, 2014 at 09:15:58AM -0500, Eitan Adler wrote: > >> On 6 March 2014 06:23, Peter Holm wrote: > >>> On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: > >>>> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: > >>>>> Here's an attempt to verify that increasing kern.maxfiles works as > >>>>> expected. > >>>>> > >>>>> http://people.freebsd.org/~pho/kern_descrip_test-v3.diff > >>>>> -- > >>>>> Peter > >>>> > >>>> 1) done should be of type "static volatile sig_atomic_t", not int, > >>>> because it's set by signal handlers. > >>>> > >>> > >>> Yes, that is nicer (I learned something new today :-). But the use > >>> here works because there is a call to usleep(3) after each test, > >>> forcing the compiler to reload the "done" variable. > >> > >> That isn't what sig_atomic_t is trying to prevent. It is an ""integer > >> type of an object that can be accessed as an atomic entity, even in > >> the presence of asynchronous interrupts."". In particular, on some > >> machines it would be possible for the signal handler to observe a > >> half-updated "int" type variable. > >> > >> On i386 sig_atomic_t happens to be an "int". On amd64 it happens to > >> be a long. This is not contractual. > >> > > > > I only just realize that the ATF test programs are threaded. > > Anyway it seems to be a good practice to always use "static volatile > > sig_atomic_t" for signal handler variables. > > ATF forks, doesn?t spawns threads: > > # grep -r pthread contrib/atf/ || echo not threaded > not threaded > Hmm ... OK. $ ldd /usr/tests/sys/kern/unix_seqpacket_test /usr/tests/sys/kern/unix_seqpacket_test: libthr.so.3 => /lib/libthr.so.3 (0x2806f000) libatf-c.so.1 => /usr/lib/libatf-c.so.1 (0x28091000) libc.so.7 => /lib/libc.so.7 (0x280a5000) $ > I think that the point that others were trying to make here is that it sets a good standard/precedence to program with atomicity in mind instead of it not being involved, because of how signal handlers are designed. > Oh, I absolutely agree. -- Peter From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 16:37:25 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17F532A0 for ; Thu, 6 Mar 2014 16:37:25 +0000 (UTC) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.freebsd.org (Postfix) with SMTP id ABBB3B1F for ; Thu, 6 Mar 2014 16:37:24 +0000 (UTC) Received: (qmail 53311 invoked from network); 6 Mar 2014 16:37:17 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay01.pair.com with SMTP; 6 Mar 2014 16:37:17 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s26GbGjW026243; Thu, 6 Mar 2014 17:37:16 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s26GbGTu026242; Thu, 6 Mar 2014 17:37:16 +0100 (CET) (envelope-from pho) Date: Thu, 6 Mar 2014 17:37:16 +0100 From: Peter Holm To: Garrett Cooper Subject: Re: A shell script test case Message-ID: <20140306163716.GC25614@x2.osted.lan> References: <20140305123307.GA77355@x2.osted.lan> <92FD8D48-FF95-41E6-A08D-F123E80AD8C3@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <92FD8D48-FF95-41E6-A08D-F123E80AD8C3@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-testing@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 16:37:25 -0000 On Thu, Mar 06, 2014 at 07:51:46AM -0800, Garrett Cooper wrote: > On Mar 5, 2014, at 4:33 AM, Peter Holm wrote: > > > I tried my hand with a test scenario for a current FreeBSD VM problem: > > At first I did not understand what I did wrong when I got: > > > > $ kyua test vm_pageout_test > > vm_pageout_test:__test_cases_list__ -> broken: Tester did not exit > > cleanly: kyua-atf-tester: Test program list did not return success > > [0.000s] > > > > 0/1 passed (1 failed) > > Committed action 15 > > $ > > > > but I figured it out. > > > > Now to the real question: Are regression tests like this one really > > ATF material? I would think that the long runtime could be prohibitive? > > You should set the timeout variable to something other than 300 if it will take longer to execute (from atf-test-case(4)): > I should have included the Makefile. I used: "TEST_METADATA.vm_pageout_test+= timeout="1200" > timeout Type: integral. Optional; defaults to `300'. > > Specifies the maximum amount of time the test case can > run. This is particularly useful because some tests > can stall either because they are incorrectly coded or > because they trigger an anomalous behavior of the pro- > gram. It is not acceptable for these tests to stall > the whole execution of the test program. > > Can optionally be set to zero, in which case the test > case has no run-time limit. This is discouraged. > > > $ kyua debug vm_pageout_test:basic > > FAIL > > UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND > > 0 662 1 0 52 0 10268 0 pause IWs - 0:00.00 nfsuserd: master (nfsuserd) > > 25 845 1 0 20 0 12796 0 pause IWs - 0:00.00 sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue (sendmail) > > 0 850 1 0 20 0 10024 0 nanslp SWs - 0:00.00 /usr/sbin/cron -s > > 1001 4069 4068 0 20 0 11808 0 wait IWs 0 0:00.00 -bash (bash) > > 0 4115 4069 0 20 0 10496 0 wait IW 0 0:00.00 su > > 0 4116 4115 0 23 0 10772 0 pause IW 0 0:00.00 _su (csh) > > 1001 4354 4353 0 20 0 11808 0 wait IWs 1 0:00.00 -bash (bash) > > 0 4416 4354 0 20 0 10496 0 wait IW 1 0:00.00 su > > 0 4417 4416 0 23 0 10772 0 pause IW 1 0:00.00 _su (csh) > > 0 4419 4417 0 20 0 11808 0 wait IW 1 0:00.00 bash > > 0 6838 4419 0 52 0 13412 0 wait IW+ 1 0:00.00 kyua debug vm_pageout_test:basic > > 0 6857 6838 0 52 0 9900 0 wait IW 1 0:00.00 /usr/local/libexec/kyua-atf-tester -t1200 test -vhas.cleanup=false -vunprivileged-user=tests /usr/tests/sys/vm/vm_pageout_test basi > > vm_pageout_test:basic -> failed: Failed > > $ kyua test vm_pageout_test > > vm_pageout_test:basic -> failed: Failed [367.941s] > > Why this took 67 seconds (367 total) to fail beyond the standard timeout (300) is a bit puzzling/concerning. I expect there to be an additional 1-5 seconds of needed cleanup, but 67 seems a bit high... > Sorry about the confusion. It's the "sort /dev/zero" that take up most of the time; Just a simple way of generate some VM usage. Anyway I think I have come to the conclusion that these kind of resource hogging tests probably do not belong in ATF. But as "my first shell script test" it was a success :) > Thanks! > -Garrett -- Peter From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 17:41:25 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E35E5EE for ; Thu, 6 Mar 2014 17:41:25 +0000 (UTC) Received: from mail-wg0-x22e.google.com (mail-wg0-x22e.google.com [IPv6:2a00:1450:400c:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A88E3252 for ; Thu, 6 Mar 2014 17:41:24 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id z12so3594292wgg.17 for ; Thu, 06 Mar 2014 09:41:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=t3XYcbR/chFhOYDNPDIVyUKoyrquXCLQbJHONiKBHOE=; b=eaumJbQPycEKRHKB4zydZ4BWBAURnaQ0vnlwcRLWLfiBo36PJ2h1fUtyYULkGL5hu5 0pXALHLq/xXnti2ifRMzVOVdB5U+c7WTZtJQ2hZG60M/MvjA/xQHl/0rVDTTLiVpPcb7 3zCIKXwqBjqh9nuPcdOs/EfxK26ifY4OTSWLwzQZ9Lat71yBenaETbzXgnMmVwA764p2 EqsqA3DgoRtWdnwzBcb50NnRGhQH/idR2J3NgVsmK/ol3eh17QXXzemBuVmHZ//gstZm A62xY22R3m+JHyNCFoIo1dKzgvznUafRbxqbZK0ugB3LIUPBTJ4rGpSMYsKo5A7V0Z45 NWUQ== MIME-Version: 1.0 X-Received: by 10.194.57.239 with SMTP id l15mr12399049wjq.40.1394127670120; Thu, 06 Mar 2014 09:41:10 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Thu, 6 Mar 2014 09:41:10 -0800 (PST) In-Reply-To: <20140306162934.GB25614@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> <20140306153247.GA22830@x2.osted.lan> <6A23D2B5-4EAA-46EF-A582-8C55FE0ED46B@gmail.com> <20140306162934.GB25614@x2.osted.lan> Date: Thu, 6 Mar 2014 10:41:10 -0700 X-Google-Sender-Auth: wT0WkvYch1DmctqfoTWwBCcpFR4 Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: Garrett Cooper , "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 17:41:25 -0000 On Thu, Mar 6, 2014 at 9:29 AM, Peter Holm wrote: > On Thu, Mar 06, 2014 at 07:44:58AM -0800, Garrett Cooper wrote: >> On Mar 6, 2014, at 7:32 AM, Peter Holm wrote: >> >> > On Thu, Mar 06, 2014 at 09:15:58AM -0500, Eitan Adler wrote: >> >> On 6 March 2014 06:23, Peter Holm wrote: >> >>> On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: >> >>>> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: >> >>>>> Here's an attempt to verify that increasing kern.maxfiles works as >> >>>>> expected. >> >>>>> >> >>>>> http://people.freebsd.org/~pho/kern_descrip_test-v3.diff >> >>>>> -- >> >>>>> Peter >> >>>> >> >>>> 1) done should be of type "static volatile sig_atomic_t", not int, >> >>>> because it's set by signal handlers. >> >>>> >> >>> >> >>> Yes, that is nicer (I learned something new today :-). But the use >> >>> here works because there is a call to usleep(3) after each test, >> >>> forcing the compiler to reload the "done" variable. >> >> >> >> That isn't what sig_atomic_t is trying to prevent. It is an ""integer >> >> type of an object that can be accessed as an atomic entity, even in >> >> the presence of asynchronous interrupts."". In particular, on some >> >> machines it would be possible for the signal handler to observe a >> >> half-updated "int" type variable. >> >> >> >> On i386 sig_atomic_t happens to be an "int". On amd64 it happens to >> >> be a long. This is not contractual. >> >> >> > >> > I only just realize that the ATF test programs are threaded. >> > Anyway it seems to be a good practice to always use "static volatile >> > sig_atomic_t" for signal handler variables. >> >> ATF forks, doesn?t spawns threads: >> >> # grep -r pthread contrib/atf/ || echo not threaded >> not threaded >> > > Hmm ... OK. > > $ ldd /usr/tests/sys/kern/unix_seqpacket_test > /usr/tests/sys/kern/unix_seqpacket_test: > libthr.so.3 => /lib/libthr.so.3 (0x2806f000) > libatf-c.so.1 => /usr/lib/libatf-c.so.1 (0x28091000) > libc.so.7 => /lib/libc.so.7 (0x280a5000) > $ That's my fault. Some of the test cases themselves require threads; not the ATF framework. I could've used fork() for those test cases, but in this case using threads was easier so they could share address space. > >> I think that the point that others were trying to make here is that it sets a good standard/precedence to program with atomicity in mind instead of it not being involved, because of how signal handlers are designed. >> > > Oh, I absolutely agree. > -- > Peter > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 17:43:13 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D552363B for ; Thu, 6 Mar 2014 17:43:13 +0000 (UTC) Received: from mail-qc0-f169.google.com (mail-qc0-f169.google.com [209.85.216.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 94EA1269 for ; Thu, 6 Mar 2014 17:43:13 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id i17so3321082qcy.14 for ; Thu, 06 Mar 2014 09:43:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=viWFKh4Rz9iwvbxAcONd7pAjQuI58BBsDmkrLcHKWOI=; b=MpKbVT7Ua/TMBRZpwSNfAJlkCJ8/C+Kz2gbfldZWvEa4kdN4F8bM8fOVjA4xaw426F QUm5NL3DkVxMeCwodz3WyjVJyF6zsxeczbTHC52+Ul9owlnTtwbru8ypvIklRnC/LxcY BcafOSPFMmI61SMnkhG0RyG8yCZ/9Ui8OYkyI1iAuhvAsWqKmd+66jdG5po6b4oNINyf EMoQHm/T8s2DNzKqyw+hv/oZFHKBGDSwxnmTStX/dOSt3urIMoNWvC90Zbf5Mcx9rU9Q 2C0gcrzaZc/Dw+PBFvnldU3B2gqZtwNjy6gPB8myYNvRx/YZcmntK0p6DlHHEmy74Gx6 dmJQ== X-Gm-Message-State: ALoCoQnn7MC7CR7aKtALoEi3uT04/CAeZPWYzihjZJqBmpGUTC8qBUZqsyAdrBoNgZA5hwZaWeug X-Received: by 10.229.84.198 with SMTP id k6mr9381530qcl.20.1394127456062; Thu, 06 Mar 2014 09:37:36 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 6 Mar 2014 09:37:15 -0800 (PST) X-Originating-IP: [2620:0:1003:1021:5c49:13a2:f7c6:9920] In-Reply-To: <43135FE5-5A9D-4AD5-B65F-E6FC20B9E366@gmail.com> References: <43135FE5-5A9D-4AD5-B65F-E6FC20B9E366@gmail.com> From: Julio Merino Date: Thu, 6 Mar 2014 12:37:15 -0500 X-Google-Sender-Auth: eBAtRA-z2T071p9jwVLqPQjiw2I Message-ID: Subject: Re: Should we MFC tests into stable/10? To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 17:43:13 -0000 On Thu, Mar 6, 2014 at 10:01 AM, Garrett Cooper wrote: > >> On Mar 6, 2014, at 6:26, Julio Merino wrote: >> >> Hello all, >> >> Keeping the testing infrastructure in sync between current and >> stable/10 is, in my opinion, a worthy goal. For that reason, I have >> (finally) pulled up a bunch of related pending changes into the branch >> today and will continue to do so for upcoming improvements. >> >> However, I would like to gather your opinion on what to do about the >> tests themselves. >> >> Do you think it's worth keeping the tests between current and >> stable/10 in sync wherever possible? Because we have barely just >> started adding tests, this will certainly involve quite a bit of churn >> in MFCs -- but that's probably not a big deal. The tricky^Winteresting >> cases will come when tests start failing in only one of the two >> branches :-P >> >> My opinion is now leaning towards merging everything where it makes >> sense. What's yours? > > You're probably going to run into similar problems that devs run into when MFCing code. > > It depends on whether or not the tests require a specific piece of build/test infrastructure, and whether or not the tests exercise a new feature or fix/change behavior. > > The former case is trickier as there will be breaking points for things in the future, but the latter case is pretty straightforward (devs should be merging tests back with code changes for new features/behavior). > > I vote yes for stable/10 now, but I would be wary of future merges as head and stable/(head-1) diverge, and similarly stable/(head-2). Yeah, that's a reasonable point. However, I'd expect the churn to slow down post-11. If that happens, then MFCing tests will be a case-by-case call. stable/10 is special in this regard. From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 17:45:54 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AAD2A8BA; Thu, 6 Mar 2014 17:45:54 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EF35E28C; Thu, 6 Mar 2014 17:45:53 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id w62so3578903wes.28 for ; Thu, 06 Mar 2014 09:45:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=A1+WRqxirxzaE+XY81JKyDaBIzauEDukOrY0zpwj6Ek=; b=QdGoTXH4ipZ0QGjp2KxccE4jKbBNOLfR9VpHCNiE7CivEYu6lVmte5r2H1b/ucGxiL BIBsvZAIctzggKFPSdzwgG8GMwJPuKsZmd94tK0yRQ+IoCcRo6iPQho7JRrgPdbBVocK mbEwHkqUBRtE+c0ex6WWK8RmiNLbWgrJ9mDOJWs45YYlF04ieyFfLR7cdyJn7hbEowxA 2sNJh0PNJYcc62PIG1hoCpibinb0WPdik3hvivl0VyI5W5qnVecZZEg/k58yYyesZ6iW pmXPADy5qj+zt31PM8jLdwcDdYjlssbrSxDzGwKzlErcoVPRWte88dhANa7NDEt3pdRU b8rg== MIME-Version: 1.0 X-Received: by 10.194.157.41 with SMTP id wj9mr12483931wjb.34.1394127952458; Thu, 06 Mar 2014 09:45:52 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Thu, 6 Mar 2014 09:45:52 -0800 (PST) In-Reply-To: References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> Date: Thu, 6 Mar 2014 10:45:52 -0700 X-Google-Sender-Auth: 7oNcGUHnOEl8uhxd2xZkxu1djTM Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 17:45:54 -0000 On Thu, Mar 6, 2014 at 7:52 AM, Julio Merino wrote: > On Thu, Mar 6, 2014 at 6:23 AM, Peter Holm wrote: >> On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: >>> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: >>> > Here's an attempt to verify that increasing kern.maxfiles works as >>> > expected. >>> > >>> > http://people.freebsd.org/~pho/kern_descrip_test-v3.diff >>> > -- >>> > Peter > > A couple of general comments: > > * In openfiles2(), it seems to me that 'i' should be size_t. That cast > in the for loop looks like a hack. > > * Style detail: I'd change the code to say something like the > following, which clearly separates the action from the handling of the > result. > > r = open("/etc/passwd", O_RDONLY); > if (ignore) { > if (r == -1) > break; > } else { > ATF_REQUIRE(r != -1); > } > > * Why does this test rely on /etc/passwd? Just create a file in the > current directory and open it. See the atf_utils_* in the > atf-c-api(3) manpage for various helper functions to deal with files. > If you want to depend on system files, the test should be declaring > that explicitly with atf_set_md_var("require.files", "/etc/passwd"). > > * Does the period in the test case name kern.maxfiles__increase > actually work? I'm surprised. > > * What's the rationale behind the TEST macro? > > More below. > >>> 1) done should be of type "static volatile sig_atomic_t", not int, >>> because it's set by signal handlers. >>> >> >> Yes, that is nicer (I learned something new today :-). But the use >> here works because there is a call to usleep(3) after each test, >> forcing the compiler to reload the "done" variable. >> >>> 2) using atexit() to register a cleanup routing is a hack. No doubt >> >> Why do you say that using atexit(3) is a hack? > > Because that's not how cleanup should be implemented in atf tests: you > should be using the cleanup routines. The reason is that atexit(3) is > not guaranteed to work: if your test crashes or is killed by Kyua > because it overruns its deadline, your cleanup code won't work but a > cleanup routine will. > >>> you already noticed that it's difficult to use Kyua's builtin cleanup >>> capabilities because of the need to pass the value of oldmaxfiles. I >>> too have experienced that frustration. Is there any way to pass >>> values from the body of a testcase to its cleanup? Using >>> atf_tc_set_md_var() would be one way, but the man page suggests that >>> that function cannot be called from the body. Julio, is there a >>> better way to do this? > > The "problem" is that the body and the cleanup run in different > processes really, so there is no way you can pass state other than > writing to a local file. > > This change: > > http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/kernel/t_sysv.c.diff?r1=1.3&r2=1.4&only_with_tag=MAIN > > shows a way to do it. > > Because these tests are dealing with scary global system state (which > is a bad thing), I think that the hassle of having to do this cleanup > dance is "kinda justified": not that many tests should need it and, > for those that do, it becomes clear that maybe the test should be done > in a different way. > > We could do better, yes; I don't disagree with that. Suggestions? > >> Now this raises an interesting question for me: Which environment do >> you guys expect ATF to run in? If it is hosts like freefall, any >> resource hogging tests are out of the question I would think. > > That's the interesting question. This test will probably be flaky > and/or cause side-effects in the system while it runs because it > relies on state that is out of control of the test. > > For the setup at kyua1.nyi.freebsd.org, this is not a problem because > the tests run in a VM. But we want to encourage users to run the > tests themselves and this kind of tests may be problematic. > > Here goes a possibility: add a configuration variable > "allow_kernel_side_effects" or similar to kyua.conf > (test_suites.FreeBSD.allow_kernel_side_effects = true). Then make the > test case require the variable with atf_set_md_var("require.config", > "allow_kernel_side_effects") so that this test only runs when the user > has told Kyua to allow these tests. We can later change the > kyua1.nyi.freebsd.org setup to define this variable but we leave it > off by default for end users. I suggest the more specific name "allow_sysctl_side_effects". As I continue to add tests, config variables like this will become more important. In particular, I have some tests that create and destroy zpools, some that can cause panics, some that load modules, etc. Before I can upstream all of them, we'll need to agree on a consistent set of config variables that control test case execution. I haven't put much thought into it yet. How does NetBSD handle tests that have potentially harmful side effects? -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:09:32 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA82064D for ; Thu, 6 Mar 2014 18:09:32 +0000 (UTC) Received: from mail-qa0-f41.google.com (mail-qa0-f41.google.com [209.85.216.41]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A30096C2 for ; Thu, 6 Mar 2014 18:09:32 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id j5so2936271qaq.0 for ; Thu, 06 Mar 2014 10:09:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=KmPTyr6adu02o9KoM6hgOCYc69InWjfcPq6QbHsDVHg=; b=O4aoCxTd6X0sv7s2oe2n9Dl/Mc6uKd2fJW6Tblb2WmTMw1sjHq53mXTVphnOauqgXZ dSK8S7TmAYsbWVeHsT8fYkdqnTIVyG2C6cJ9cnWLEvzPpcC7Db5J4wKhKn6kNR1tUk/G dWrUkuGdFX9IfUZZOL2TxP1N9gCBSkSRao0KBMi6EImQqo/5XLL+XnB09f0wr4JUeT1H uME+XqrCZWYJ6yE7kl9CYACbnoKukg1zWyzqD5qesiP1JbiWQ4lZCyim62jyQfWD+fDx nSAmj/60BwE8+3phvFHjusuR3EzyzCB003E2NJURoj1e7rBcE/QIFtHjmUF4AEEJZG5c tPxg== X-Gm-Message-State: ALoCoQlH1rvBZMwAAKxrynCgRm7g82smb25c/3UcFKeUK9Qo6sreuprFuUb6fLpc9dQWXJyf8Vt1 X-Received: by 10.224.21.129 with SMTP id j1mr15849120qab.13.1394128946640; Thu, 06 Mar 2014 10:02:26 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 6 Mar 2014 10:02:06 -0800 (PST) X-Originating-IP: [2620:0:1003:1021:5c49:13a2:f7c6:9920] In-Reply-To: References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> From: Julio Merino Date: Thu, 6 Mar 2014 13:02:06 -0500 X-Google-Sender-Auth: TOYQKy9TOCN8earyKHft3A3Llsc Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:09:33 -0000 On Thu, Mar 6, 2014 at 12:45 PM, Alan Somers wrote: > I suggest the more specific name "allow_sysctl_side_effects". As I > continue to add tests, config variables like this will become more > important. In particular, I have some tests that create and destroy > zpools, some that can cause panics, some that load modules, etc. > Before I can upstream all of them, we'll need to agree on a consistent > set of config variables that control test case execution. I haven't > put much thought into it yet. Sure; more granularity++. I haven't put any thought on this though so please propose something :) Whatever we decide, 1) these variables need to be documented in tests(7), 2) a ready-to-enable template in ports/devel/kyua-cli/files/kyua.conf.in about them would be beneficial and 3) src/projects/jmmv/autotest/ should switch these on. > How does NetBSD handle tests that have > potentially harmful side effects? They don't AFAIK. The thing is that not that many tests have side-effects because the majority of kernel-level tests use rump to avoid this problem and be more reliable/deterministic. The few that have side-effects have ad-hoc cleanup routines. From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:12:46 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A82D7743 for ; Thu, 6 Mar 2014 18:12:46 +0000 (UTC) Received: from mail-qc0-f177.google.com (mail-qc0-f177.google.com [209.85.216.177]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 64EFA7D5 for ; Thu, 6 Mar 2014 18:12:46 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id w7so3312739qcr.36 for ; Thu, 06 Mar 2014 10:12:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=yV01C9rBZ46FNIiluH+LtelcHT+zmLSGmsmxzr90H4E=; b=fHSFD7EF/Ypsc28mCtqXUK8tCU+f5zH/i8cM9J+FpHwsjA1MJWFvwqmoBVQPQcG4oI qWw4JriITspxG2JOvBPN7RGoq0pgg/qYJGdKYqKm75itqUlAO6c9my83CxJeQHm0199s ScdgPgqFJ1vhK1mAFS3GZ3M9TerrN0UUsG0wzipH+m1z5Ojkq2nAjuROc6CpaAGDov2w GijOvlMAWOZ1g5F7Eq0p0rUih4MT6+RxZNOmX+kdq0dbk2pEVDQ59wxzIFipGwK/xhLC Ji4WjsmRYjKBGwmU4RjX+D6Uf6fk+ESdPLP9xnLAp+dFIQQfGXH2QXwJmSwZrF8Q0lwG bWkA== X-Gm-Message-State: ALoCoQl99aDXWUlLsEEx2CBJJUTuHNl0pU/BmiJfouhhBlksN0Or2GqoJ9fLKcyTOoH57gNogrVH X-Received: by 10.224.21.129 with SMTP id j1mr15706109qab.13.1394127719024; Thu, 06 Mar 2014 09:41:59 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 6 Mar 2014 09:41:38 -0800 (PST) X-Originating-IP: [2620:0:1003:1021:5c49:13a2:f7c6:9920] In-Reply-To: <20140306162934.GB25614@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> <20140306153247.GA22830@x2.osted.lan> <6A23D2B5-4EAA-46EF-A582-8C55FE0ED46B@gmail.com> <20140306162934.GB25614@x2.osted.lan> From: Julio Merino Date: Thu, 6 Mar 2014 12:41:38 -0500 X-Google-Sender-Auth: zdizhKqUkdvt8ATTt4Xcw9VYuIk Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: Garrett Cooper , "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:12:46 -0000 On Thu, Mar 6, 2014 at 11:29 AM, Peter Holm wrote: > On Thu, Mar 06, 2014 at 07:44:58AM -0800, Garrett Cooper wrote: >> >> ATF forks, doesn?t spawns threads: >> >> # grep -r pthread contrib/atf/ || echo not threaded >> not threaded >> > > Hmm ... OK. > > $ ldd /usr/tests/sys/kern/unix_seqpacket_test > /usr/tests/sys/kern/unix_seqpacket_test: > libthr.so.3 => /lib/libthr.so.3 (0x2806f000) > libatf-c.so.1 => /usr/lib/libatf-c.so.1 (0x28091000) > libc.so.7 => /lib/libc.so.7 (0x280a5000) > $ That's a property of this test program, not of atf: $ grep -i thread tests/sys/kern/Makefile LDADD+= -lpthread From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:19:10 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0294BC3F for ; Thu, 6 Mar 2014 18:19:10 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A15F586A for ; Thu, 6 Mar 2014 18:19:09 +0000 (UTC) Received: from maildlpprd54.lss.emc.com (maildlpprd54.lss.emc.com [10.106.48.158]) by mailuogwprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s26I275Q006389 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 6 Mar 2014 13:02:08 -0500 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com s26I275Q006389 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1394128928; bh=jJ2ZJSRtToOXhMSylrwL0AiK4uA=; h=From:To:Subject:Date:Message-ID:Content-Type:MIME-Version; b=VM6Nrctwf7I/ZWwQi+PxQScKu388vmOuWokR4M2QBVawn5vOoFf+zoRqhUpjRtvQ8 Iw0EN/bLRkEmCl1LKaMZdHxX2g3mAKUgDSSp7Z3MypoK4MpkPRXc+YncQ4vhiTyjq8 TqbfC278XnPPvtCsfSoRhUyXdMX1o3/+G+yWOYRA= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com s26I275Q006389 Received: from mailusrhubprd03.lss.emc.com (mailusrhubprd03.lss.emc.com [10.253.24.21]) by maildlpprd54.lss.emc.com (RSA Interceptor) for ; Thu, 6 Mar 2014 13:01:55 -0500 Received: from mxhub39.corp.emc.com (mxhub39.corp.emc.com [128.222.70.106]) by mailusrhubprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s26I1tjM006712 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Thu, 6 Mar 2014 13:01:55 -0500 Received: from MXHUB105.corp.emc.com (10.253.50.22) by mxhub39.corp.emc.com (128.222.70.106) with Microsoft SMTP Server (TLS) id 8.3.327.1; Thu, 6 Mar 2014 13:01:39 -0500 Received: from MX103CL02.corp.emc.com ([169.254.6.73]) by MXHUB105.corp.emc.com ([10.253.50.22]) with mapi id 14.03.0158.001; Thu, 6 Mar 2014 13:01:54 -0500 From: "Peel, Casey" To: "freebsd-testing@freebsd.org" Subject: tests on kyua1.nyi.freebsd.org against HEAD failing Thread-Topic: tests on kyua1.nyi.freebsd.org against HEAD failing Thread-Index: Ac85ZetRKh4m9x2ES4Gl2LhBwLhTzg== Date: Thu, 6 Mar 2014 18:01:54 +0000 Message-ID: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd03.lss.emc.com X-RSA-Classifications: public Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:19:10 -0000 Greetings all. Benno pointed out that the tests against the HEAD branch on http://kyua1.ny= i.freebsd.org appear to be misconfigured - is this a known issue? from http://kyua1.nyi.freebsd.org/head/data/0-LATEST/output.log: ------------ iterate: I: Fetching release.sh from svn://svn.freebsd.org/base/head A /tmp/autotest.xkE99Xcm/release.sh Export complete. iterate: I: Running '/tmp/autotest.xkE99Xcm/release.sh -c /tmp/autotest.xkE= 99Xcm/release.conf' in /root/autotest/node svn: E670008: Unable to connect to a repository at URL 'svn://svn.freebsd.o= rgbase/head' <---------- whoops! svn: E670008: Unknown hostname 'svn.freebsd.orgbase' iterate: W: Command failed with code 1 iterate: E: release build failed ------------ Based on http://kyua1.nyi.freebsd.org/head/data/, it looks like it stopped = working at 20140226-004654 (Feb 26th). Regards, -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:20:56 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7BB40C85; Thu, 6 Mar 2014 18:20:56 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 23BF68E5; Thu, 6 Mar 2014 18:20:56 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id lc6so2478493vcb.7 for ; Thu, 06 Mar 2014 10:20:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=iKHMLGLAXq7i4/LVvStKaN4dZo0nJYGgg/KXXrNKeHM=; b=eaglvkZNT4Bj5OdATdaXhZrigJp04cprmbZ7eX/LKrLJsQkWNJgy9fpwFBFZkMRybt X61sRrmoCnPJb128rbOtrUTiwWspZYIHNSWFs4B4/nR015iBssk9MoRgN5wFnOJSVV9V 2cLq2rgl7z08LcMSPPCopC0jm2bL0Y5m+L95lyyJdGZAKgtDKiqBa/caRJVhQYlbVHKW ZA6vVl/923aRzEiEzf9wQptCAVaJJ0LyMV3KiDkBn9WRNNtJz7HT3LyAJG5aoqIEO7Rh pew3rwr9218kuvnFH8QBMboi12T6y29Stzq/GdPio6jyAoKoulTz3qfvNjqmle5MR47L Lrrw== MIME-Version: 1.0 X-Received: by 10.58.128.72 with SMTP id nm8mr6451058veb.18.1394130055257; Thu, 06 Mar 2014 10:20:55 -0800 (PST) Received: by 10.221.67.136 with HTTP; Thu, 6 Mar 2014 10:20:55 -0800 (PST) In-Reply-To: References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> <20140306153247.GA22830@x2.osted.lan> <6A23D2B5-4EAA-46EF-A582-8C55FE0ED46B@gmail.com> <20140306162934.GB25614@x2.osted.lan> Date: Thu, 6 Mar 2014 10:20:55 -0800 Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: Garrett Cooper , Peter Holm , "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:20:56 -0000 On Thu, Mar 6, 2014 at 9:41 AM, Julio Merino wrote: > On Thu, Mar 6, 2014 at 11:29 AM, Peter Holm wrote: >> On Thu, Mar 06, 2014 at 07:44:58AM -0800, Garrett Cooper wrote: >>> >>> ATF forks, doesn?t spawns threads: >>> >>> # grep -r pthread contrib/atf/ || echo not threaded >>> not threaded >>> >> >> Hmm ... OK. >> >> $ ldd /usr/tests/sys/kern/unix_seqpacket_test >> /usr/tests/sys/kern/unix_seqpacket_test: >> libthr.so.3 => /lib/libthr.so.3 (0x2806f000) >> libatf-c.so.1 => /usr/lib/libatf-c.so.1 (0x28091000) >> libc.so.7 => /lib/libc.so.7 (0x280a5000) >> $ > > That's a property of this test program, not of atf: > > $ grep -i thread tests/sys/kern/Makefile > LDADD+= -lpthread That should be LDADD.unix_seqpacket_test+= -lpthread I'd be sure to test this before committing though. It *should* work, but bsd.progs.mk [in its current incarnation] breaks POLA with some variables compared to bsd.prog.mk in NetBSD. Cheers, -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:28:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5CD1496 for ; Thu, 6 Mar 2014 18:28:14 +0000 (UTC) Received: from mail-ve0-x22b.google.com (mail-ve0-x22b.google.com [IPv6:2607:f8b0:400c:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1EB53935 for ; Thu, 6 Mar 2014 18:28:14 +0000 (UTC) Received: by mail-ve0-f171.google.com with SMTP id cz12so3101908veb.30 for ; Thu, 06 Mar 2014 10:28:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=CV+5nLZnuqed2f+3fW4lHsY/QU0+sWWOXnMRS/aJBwM=; b=KXmS7q/Syj4Y/fnnB9a0KfK7h9L8Phz9HRO3rWAeHpWujy1viXDtHdfhE4Lzh9G2Wo DZFZBv/lxBScQgYriO07yTPk6IFF1WrvW+YOhQJsXOMoKmBr8bPhJR55+1t6h1SHKOdx n0YuaWsV/6RIizxzPQfKY/ipqunk+xX+1nvkqfAHg1htC8ISZpy2w0oveRO7SmOIXUW8 D5amLvpSlA45OVpWxYx+f8xi3f0GU0myMl4dl5a+xD6p+JW48UCL2B7yaE+TAWvOUkn4 JwAB/tUG4Eo5KOJERTUS65QIigO5uqvjJ+IIG8HibjjSw27M1D/VVfiq3RDzi1C59D8V SpBw== MIME-Version: 1.0 X-Received: by 10.58.161.205 with SMTP id xu13mr6380781veb.4.1394130493231; Thu, 06 Mar 2014 10:28:13 -0800 (PST) Received: by 10.221.67.136 with HTTP; Thu, 6 Mar 2014 10:28:13 -0800 (PST) Date: Thu, 6 Mar 2014 10:28:13 -0800 Message-ID: Subject: Structure of ATF testcases (was "Test scenario for sysctl kern.maxfiles") From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , kyua-discuss@googlegroups.com X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:28:14 -0000 On Thu, Mar 6, 2014 at 10:02 AM, Julio Merino wrote: > On Thu, Mar 6, 2014 at 12:45 PM, Alan Somers wrote: >> I suggest the more specific name "allow_sysctl_side_effects". As I >> continue to add tests, config variables like this will become more >> important. In particular, I have some tests that create and destroy >> zpools, some that can cause panics, some that load modules, etc. >> Before I can upstream all of them, we'll need to agree on a consistent >> set of config variables that control test case execution. I haven't >> put much thought into it yet. > > Sure; more granularity++. I haven't put any thought on this though so > please propose something :) > > Whatever we decide, 1) these variables need to be documented in > tests(7), 2) a ready-to-enable template in > ports/devel/kyua-cli/files/kyua.conf.in about them would be beneficial > and 3) src/projects/jmmv/autotest/ should switch these on. > >> How does NetBSD handle tests that have >> potentially harmful side effects? > > They don't AFAIK. The thing is that not that many tests have > side-effects because the majority of kernel-level tests use rump to > avoid this problem and be more reliable/deterministic. The few that > have side-effects have ad-hoc cleanup routines. Seems like another standalone tool should be developed to cleanup variables after exit per test and per test suite. Setup/cleanup in ATF doesn't seem to follow the model that many other testing frameworks do when it comes to executing testcases and managing setup/cleanup. I really wish things were like the JUnit/unittest model, i.e. - Test suite setup - For each testcase: - Run the testcase setup - If the setup passed, continue - Run the testcase - Run the testcase cleanup - Test suite teardown Thanks :)! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:35:35 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BB6A0783; Thu, 6 Mar 2014 18:35:35 +0000 (UTC) Received: from mail-ve0-x22b.google.com (mail-ve0-x22b.google.com [IPv6:2607:f8b0:400c:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6666CA20; Thu, 6 Mar 2014 18:35:35 +0000 (UTC) Received: by mail-ve0-f171.google.com with SMTP id cz12so3030344veb.16 for ; Thu, 06 Mar 2014 10:35:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=OLI7UhCN3paaeb/roqjy1VfVSHWAiFUWZglVTd8qNa8=; b=wU7MHAvOvFU3RniE2SJQVXUEHWNSKfL9QmXGu1PcKHST7JpDgqi6qKI3OJzv5telG+ H8WrdCY6mPE/9Y355njqPoRRG9OJfNjKLxSjtmNgCArUuaXCycjCaEMnyCzd/1L2Zf55 3vA33rEtJ27ZEXEkirvnvPxrwTFejqGfbfqaLD4lBxWZGihWxFl4XweCvyqkBU4tYiZh 4MPKNJC94DL7MCVhx2Cz5dpr1ZDHEghkyOP+0Ap8nG5l1MQoEpWhwxNlw9etgGjsE+Ts ffYUZ/HKKaDcKDTr9M4VsdB6thPoo3Osvrwyqi0QhpS4ZK/B2UIVVhXMTttz/dkFVs5m sHGg== MIME-Version: 1.0 X-Received: by 10.221.46.65 with SMTP id un1mr123195vcb.67.1394130934502; Thu, 06 Mar 2014 10:35:34 -0800 (PST) Received: by 10.221.67.136 with HTTP; Thu, 6 Mar 2014 10:35:34 -0800 (PST) In-Reply-To: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> Date: Thu, 6 Mar 2014 10:35:34 -0800 Message-ID: Subject: Re: tests on kyua1.nyi.freebsd.org against HEAD failing From: Garrett Cooper To: "Peel, Casey" , Julio Merino , Glen Barber Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:35:35 -0000 On Thu, Mar 6, 2014 at 10:01 AM, Peel, Casey wrote: > Greetings all. > > > > Benno pointed out that the tests against the HEAD branch on http://kyua1.nyi.freebsd.org appear to be misconfigured - is this a known issue? > > > > from http://kyua1.nyi.freebsd.org/head/data/0-LATEST/output.log: > > ------------ > > iterate: I: Fetching release.sh from svn://svn.freebsd.org/base/head > > A /tmp/autotest.xkE99Xcm/release.sh > > Export complete. > > iterate: I: Running '/tmp/autotest.xkE99Xcm/release.sh -c /tmp/autotest.xkE99Xcm/release.conf' in /root/autotest/node > > svn: E670008: Unable to connect to a repository at URL 'svn://svn.freebsd.orgbase/head' <---------- whoops! > > svn: E670008: Unknown hostname 'svn.freebsd.orgbase' > > iterate: W: Command failed with code 1 > > iterate: E: release build failed > > ------------ > > > > Based on http://kyua1.nyi.freebsd.org/head/data/, it looks like it stopped working at 20140226-004654 (Feb 26th). Hello Mr. Peel :)! Yes, that's definitely a bug because gjb changed how variables are defined in release.sh (mind the trailing slash!): http://svnweb.freebsd.org/base/head/release/release.conf.sample?r1=259079&r2=262499 http://svnweb.freebsd.org/base/head/release/release.sh?r1=259955&r2=262499 Too bad the extra slash wasn't kept in release.sh -- it would have avoided this breakage (I'm sure Julio isn't the only one who will have to update their release.conf). Julio, Looks like the NYI server needs to be updated and an extra slash needs to be added to the .conf file. Cheers! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:40:35 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1300DB2A; Thu, 6 Mar 2014 18:40:35 +0000 (UTC) Received: from mail0.glenbarber.us (mail0.glenbarber.us [IPv6:2607:fc50:1:2300:1001:1001:1001:face]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C380DAE5; Thu, 6 Mar 2014 18:40:34 +0000 (UTC) Received: from glenbarber.us (nucleus.glenbarber.us [IPv6:2001:470:8:1205:2:2:ff:100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id B1514B8B4; Thu, 6 Mar 2014 18:40:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us B1514B8B4 Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Thu, 6 Mar 2014 13:40:30 -0500 From: Glen Barber To: Garrett Cooper Subject: Re: tests on kyua1.nyi.freebsd.org against HEAD failing Message-ID: <20140306184030.GV87036@glenbarber.us> References: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="1J4USyVh4Jt5OGCL" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event User-Agent: Mutt/1.5.22 (2013-10-16) Cc: Julio Merino , "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:40:35 -0000 --1J4USyVh4Jt5OGCL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 06, 2014 at 10:35:34AM -0800, Garrett Cooper wrote: > On Thu, Mar 6, 2014 at 10:01 AM, Peel, Casey wrot= e: > > Greetings all. > > > > > > > > Benno pointed out that the tests against the HEAD branch on http://kyua= 1.nyi.freebsd.org appear to be misconfigured - is this a known issue? > > > > > > > > from http://kyua1.nyi.freebsd.org/head/data/0-LATEST/output.log: > > > > ------------ > > > > iterate: I: Fetching release.sh from svn://svn.freebsd.org/base/head > > > > A /tmp/autotest.xkE99Xcm/release.sh > > > > Export complete. > > > > iterate: I: Running '/tmp/autotest.xkE99Xcm/release.sh -c /tmp/autotest= =2ExkE99Xcm/release.conf' in /root/autotest/node > > > > svn: E670008: Unable to connect to a repository at URL 'svn://svn.freeb= sd.orgbase/head' <---------- whoops! > > > > svn: E670008: Unknown hostname 'svn.freebsd.orgbase' > > > > iterate: W: Command failed with code 1 > > > > iterate: E: release build failed > > > > ------------ > > > > > > > > Based on http://kyua1.nyi.freebsd.org/head/data/, it looks like it stop= ped working at 20140226-004654 (Feb 26th). >=20 > Hello Mr. Peel :)! >=20 > Yes, that's definitely a bug because gjb changed how variables are > defined in release.sh (mind the trailing slash!): >=20 > http://svnweb.freebsd.org/base/head/release/release.conf.sample?r1=3D2590= 79&r2=3D262499 > http://svnweb.freebsd.org/base/head/release/release.sh?r1=3D259955&r2=3D2= 62499 >=20 > Too bad the extra slash wasn't kept in release.sh -- it would have > avoided this breakage (I'm sure Julio isn't the only one who will have > to update their release.conf). >=20 > Julio, >=20 > Looks like the NYI server needs to be updated and an extra slash needs > to be added to the .conf file. >=20 Yes, it was an unfortunate change, but sadly was the "best" way I could find to change things around to be able to replace VCSCMD. Sorry about the breakage it caused. Changes like that will not be a regular occurrence. Glen --1J4USyVh4Jt5OGCL Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCAAGBQJTGMEeAAoJELls3eqvi17QSgAQAJLvvWdSs2M6/EpQJikihp7w XIcSaAeb10a+HPC4lM8JqcP32WRrhK1uOoJ/u2o7LexLLkFJOfm2tpU2LlTsp7Y3 ex6uzx51o1vyj1vlDyM486uOx2zXav90kQeHMf1LpQj2UvtsLYNEsb6hkrUBLdlX ojp96qqx6B55t1+L6jPA9lGghbB+DhdUc5+EAhzTcnSwxCLRIVZVGkdjNu2WkOd/ x4XyJGavHqJcFwohFxn1CRUG35dOInzmCKCwwiq+2H8gyyNsKg/7zte5uJOeLwzQ Rlc53X2U/jriBZImS4tD+yCmTdov6kgoRm4jzacZWKuxSnTNfM7wN6kV2717FMzu UxyHI3AFTi3uHTv+aE9ybcTA8QQD5TeF01rGNTySdY7XLQC9uQ53XBAqt6u8JdsL SRZeZ3JfKy4sgpyxfGXFSXk8Z3AeAIz5UPcyzZoiG5QEV7hIHoTFohPjV+NQKkt+ FT5DV+F/J7EG2Qw3cBYMumj59vhQgSUQmJO2CQKKKzYM3NeB+fG87QrjxVYNrtvf vXgS+rXAXuPQh7cExTrCwsjWrkJUJXTqJ6qG5eFwWGBja9X6XtAlGexQ5ZQi4Tia Uap4mj7JvbbLE5vW9It0UDRLFf+0jnzzLLzXP39u/RmvX/Ud323giuFtXmzWzTU7 C83Mc3y+h9xqkDGa3a5H =V4oI -----END PGP SIGNATURE----- --1J4USyVh4Jt5OGCL-- From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:42:29 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 842AEC47; Thu, 6 Mar 2014 18:42:29 +0000 (UTC) Received: from mail-vc0-x22b.google.com (mail-vc0-x22b.google.com [IPv6:2607:f8b0:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2DFAFAFD; Thu, 6 Mar 2014 18:42:29 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id le5so3000027vcb.2 for ; Thu, 06 Mar 2014 10:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ufJDsZ9woHqWKY4FB7ufcpjsgjXqB8FDpnDIEHS5wmc=; b=P74PrK+QYDo2XuABuR/OR3SfjDT8uQcKgY1edaTI7zAnAE8X542SsJQ/53N63xF2H8 IjBxw6EjjRhTw95tfZWqbCjexE8JZS7BbUYeDH1AwBqBaNLOI2OmTHshAciEJBb2EmYV JRXXniA5nntWPUZ7RAawv8XAVoW7mCLk8LWipuB/dr3y13iHHOcvi8gjlmBJ+xp3HZxd u8onnNIxt1UBP0Z3Bv8kfwC6dzEXDfn2OISmnlmPj2ss32k1mEEOlLzF5+Atrfos4VzJ WZliScXMGRccPOkkOvlGDlUgInopXjohTCHOa0Y+D7X7fLGBLCI7YLDwwhniiTkQFnCY Ckww== MIME-Version: 1.0 X-Received: by 10.220.106.84 with SMTP id w20mr6262218vco.18.1394131348286; Thu, 06 Mar 2014 10:42:28 -0800 (PST) Received: by 10.221.67.136 with HTTP; Thu, 6 Mar 2014 10:42:28 -0800 (PST) In-Reply-To: <20140306184030.GV87036@glenbarber.us> References: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> <20140306184030.GV87036@glenbarber.us> Date: Thu, 6 Mar 2014 10:42:28 -0800 Message-ID: Subject: Re: tests on kyua1.nyi.freebsd.org against HEAD failing From: Garrett Cooper To: Glen Barber Content-Type: text/plain; charset=ISO-8859-1 Cc: Julio Merino , "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:42:29 -0000 On Thu, Mar 6, 2014 at 10:40 AM, Glen Barber wrote: > On Thu, Mar 06, 2014 at 10:35:34AM -0800, Garrett Cooper wrote: >> On Thu, Mar 6, 2014 at 10:01 AM, Peel, Casey wrote: >> > Greetings all. >> > >> > >> > >> > Benno pointed out that the tests against the HEAD branch on http://kyua1.nyi.freebsd.org appear to be misconfigured - is this a known issue? ... >> Hello Mr. Peel :)! >> >> Yes, that's definitely a bug because gjb changed how variables are >> defined in release.sh (mind the trailing slash!): >> >> http://svnweb.freebsd.org/base/head/release/release.conf.sample?r1=259079&r2=262499 >> http://svnweb.freebsd.org/base/head/release/release.sh?r1=259955&r2=262499 >> >> Too bad the extra slash wasn't kept in release.sh -- it would have >> avoided this breakage (I'm sure Julio isn't the only one who will have >> to update their release.conf). >> >> Julio, >> >> Looks like the NYI server needs to be updated and an extra slash needs >> to be added to the .conf file. >> > > Yes, it was an unfortunate change, but sadly was the "best" way I could > find to change things around to be able to replace VCSCMD. > > Sorry about the breakage it caused. Changes like that will not be > a regular occurrence. Hmm... shouldn't this be documented in UPDATING? Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 18:49:52 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DA771A2; Thu, 6 Mar 2014 18:49:52 +0000 (UTC) Received: from mail0.glenbarber.us (mail0.glenbarber.us [IPv6:2607:fc50:1:2300:1001:1001:1001:face]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5B774B54; Thu, 6 Mar 2014 18:49:52 +0000 (UTC) Received: from glenbarber.us (nucleus.glenbarber.us [IPv6:2001:470:8:1205:2:2:ff:100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 06269B9DE; Thu, 6 Mar 2014 18:49:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us 06269B9DE Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Thu, 6 Mar 2014 13:49:49 -0500 From: Glen Barber To: Garrett Cooper Subject: Re: tests on kyua1.nyi.freebsd.org against HEAD failing Message-ID: <20140306184949.GW87036@glenbarber.us> References: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> <20140306184030.GV87036@glenbarber.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="lGQpFNrcSq0Rb43w" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event User-Agent: Mutt/1.5.22 (2013-10-16) Cc: Julio Merino , "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 18:49:52 -0000 --lGQpFNrcSq0Rb43w Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 06, 2014 at 10:42:28AM -0800, Garrett Cooper wrote: > On Thu, Mar 6, 2014 at 10:40 AM, Glen Barber wrote: > > Yes, it was an unfortunate change, but sadly was the "best" way I could > > find to change things around to be able to replace VCSCMD. > > > > Sorry about the breakage it caused. Changes like that will not be > > a regular occurrence. >=20 > Hmm... shouldn't this be documented in UPDATING? No, see r262862. Glen --lGQpFNrcSq0Rb43w Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCAAGBQJTGMNNAAoJELls3eqvi17QwvgP/AsSQ4jN/MT5N7WAM4aIr8XR 1HFM3hWDeeJ2KwKN0HAj1bWUuiwdBUdv0tLai7+dPAK9vDOSgQyZMvo9fCViNGzM gJUa5JZ1fT5AtZDQJywTq+XkM0Qs5FgZ2Q8zAwcUC57UvtMv1i8m1KrWOPbShLrx hhiSMRWLAnU2Rv8GonkxQm8HeJCQCPSpJi+3+KVPKVU24U0TDmVC/jzcab3uQ2qn XtfdP37RWSdGIGgQCOU0OC3GUDhXnyXG/3c2xq1YsaNe4I75IffujU2XWTqn8Il+ ls7+2+XmAQ7HV1Wb0TXZk4MCHDpk/6AgCwFM0C59E1K4QwI39rvnd+Q2bW34MmtH FtDHfMOCT2nWsawdAx9FTwZ3GLN9eXU+oj99HmF+BpNqaC8on9EBRe9xKmrL3DJw s2gmYvZ9xK8EDsk+67F1v+sLghhimX1l08vf0j8bLJU0frogxIAmmzauz7465CN8 BEyLY4/qDxgZ/pRSUdOyBIGtPwgajIsVV3pprWj630sFvfBPzeZdxJirDtNHtfHh uH0Esmp5h+czPuyOTUEbJmoDi/Zocp1DaFSz4+aCt6B+J72QlGVreV1KTR6yHPzT fq5vIyAuzU2cPTEpStTvy0kDKMRmIkx+IQU74y8hX3bxy5RwWH8mO7g1oSTL3U6L fOhmZ1lafOAj0Q1P4Giw =bQSw -----END PGP SIGNATURE----- --lGQpFNrcSq0Rb43w-- From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 19:13:33 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33991D3A; Thu, 6 Mar 2014 19:13:33 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D0074DC0; Thu, 6 Mar 2014 19:13:32 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id ld13so3090173vcb.32 for ; Thu, 06 Mar 2014 11:13:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=g1tg3tGj1LkKfOwJvBk7Iw7PaPT4rUq25cLkYWFlN+c=; b=H4tQ6xX5+XyB/Sza14p36NhA1lcSd+yV0u0EZFmvdvUGNZa47dBVxjcLgm+lvjtr4P ne7w2LJgNHthLYKaARmFr8BIuXmDmbPoUv3NTfUr4fLOvZekOnlKM9rrSA80RyhL4TzP 0paCl+wbKI9V/XsdSNY5gvFNJ28SGnrtJ/b8rz2gYhP3lz++mFzcVTHxu2Q0KgqIXnz3 nH6DDIl/Pbh7iAPckTtaRoPzsUaksgPLNqIMbcUmfzNgeXwp73iIzmhqmw3EQvfrlsCS CaLax9x+ND0FfUb7qpirR9YLQFXOKVJGRoK1B5lw3pcb72dKUqQhA/K7CGyEYMioCkw0 NBew== MIME-Version: 1.0 X-Received: by 10.58.229.4 with SMTP id sm4mr6740157vec.10.1394133211862; Thu, 06 Mar 2014 11:13:31 -0800 (PST) Received: by 10.221.67.136 with HTTP; Thu, 6 Mar 2014 11:13:31 -0800 (PST) In-Reply-To: <20140306184949.GW87036@glenbarber.us> References: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> <20140306184030.GV87036@glenbarber.us> <20140306184949.GW87036@glenbarber.us> Date: Thu, 6 Mar 2014 11:13:31 -0800 Message-ID: Subject: Re: tests on kyua1.nyi.freebsd.org against HEAD failing From: Garrett Cooper To: Glen Barber Content-Type: text/plain; charset=ISO-8859-1 Cc: Julio Merino , "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 19:13:33 -0000 On Thu, Mar 6, 2014 at 10:49 AM, Glen Barber wrote: > On Thu, Mar 06, 2014 at 10:42:28AM -0800, Garrett Cooper wrote: >> On Thu, Mar 6, 2014 at 10:40 AM, Glen Barber wrote: >> > Yes, it was an unfortunate change, but sadly was the "best" way I could >> > find to change things around to be able to replace VCSCMD. >> > >> > Sorry about the breakage it caused. Changes like that will not be >> > a regular occurrence. >> >> Hmm... shouldn't this be documented in UPDATING? > > No, see r262862. Should that be keyed off of VCSCMD? Also, there's nothing wrong with using URIs for paths with git (if that was part of the concern), e.g. `git archive file://$PWD master`. I do that quite frequently actually to test local changes before committing them to repos... Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 19:19:33 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54710FF1; Thu, 6 Mar 2014 19:19:33 +0000 (UTC) Received: from mail0.glenbarber.us (mail0.glenbarber.us [208.86.227.67]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 22DC3E23; Thu, 6 Mar 2014 19:19:33 +0000 (UTC) Received: from glenbarber.us (nucleus.glenbarber.us [IPv6:2001:470:8:1205:2:2:ff:100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 8DD11BD05; Thu, 6 Mar 2014 19:19:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us 8DD11BD05 Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Thu, 6 Mar 2014 14:19:29 -0500 From: Glen Barber To: Garrett Cooper Subject: Re: tests on kyua1.nyi.freebsd.org against HEAD failing Message-ID: <20140306191929.GX87036@glenbarber.us> References: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> <20140306184030.GV87036@glenbarber.us> <20140306184949.GW87036@glenbarber.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="AAkN98o3X3ouhQaz" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event User-Agent: Mutt/1.5.22 (2013-10-16) Cc: Julio Merino , "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 19:19:33 -0000 --AAkN98o3X3ouhQaz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 06, 2014 at 11:13:31AM -0800, Garrett Cooper wrote: > On Thu, Mar 6, 2014 at 10:49 AM, Glen Barber wrote: > > On Thu, Mar 06, 2014 at 10:42:28AM -0800, Garrett Cooper wrote: > >> On Thu, Mar 6, 2014 at 10:40 AM, Glen Barber wrote: > >> > Yes, it was an unfortunate change, but sadly was the "best" way I co= uld > >> > find to change things around to be able to replace VCSCMD. > >> > > >> > Sorry about the breakage it caused. Changes like that will not be > >> > a regular occurrence. > >> > >> Hmm... shouldn't this be documented in UPDATING? > > > > No, see r262862. >=20 > Should that be keyed off of VCSCMD? >=20 No. > Also, there's nothing wrong with using URIs for paths with git (if > that was part of the concern), e.g. `git archive file://$PWD master`. >=20 The concern was not breaking SRCBRANCH by prefixing it with '/' unconditionally. The release.conf.sample shows an empty SVNROOT and SRCBRANCH set to the full URL to a non-svn repository, which would expand to '/https://'. I have no intention of doing per-VCS rules in release.sh further than this. Glen --AAkN98o3X3ouhQaz Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCAAGBQJTGMpBAAoJELls3eqvi17QKy4P/R9+IE2McbOwB8FjiR0gXX7R HBVcMuKLnL9GZNj+dWkpBFuCvpwyImBvVyRYWu1tQGed3FNpB0cYYWuH5OMwcw/A aeuryCWqV+8TZYKFr3UDWm4t6UduemJLdZLB3wMwBKx3nHbRL4rEzABIMZJYWiMp n075UmBw1awh2KKIc5GJqSOARqExF9guL1ZU/Efxi+GSUkleQjKs2+mUHi8xFcPv 11hTJZB/tVGEzQJG2yhdnHwSg99aGn7maR6SF8X0PL10gY9Jx420ovsU+kCNQSUD +jN32P7C7HBi0Gm0IldNc0kmRUjStBcAO1qXIZeYuofu2X3roSTmn3trqzno3IXE 48kwykWPv0/hZ4HsSjBcD/pmVGAQi71ZoLUWD4VWcRHPotSBeVNdtTpavbQh7Ql2 O5hl0J1moTFS88COwLOiQEwUpQC/hxOFyhaEBVYvgTuH1+nJRPFfBiimx+meeK5R G/vRjnxyFc10yaNhzWboy7VRj6mlduQKknXSIfL9rQPIpNts8ExrRdXSgo7MLLpP UEjL2oqpPyVcXgwlgbbzUClVnt2Z9aC+9uqQLvuJo3dTWUSpfITLfuSNQLEOenrd pJtNcYPimMjfOBEdZ6O+YNVK4M0L12MsJuQTbLPAf2DwrZC2DcGG8e4P5LnxhBwY UlSqgV1hPI1g3HSXMy/R =pHqh -----END PGP SIGNATURE----- --AAkN98o3X3ouhQaz-- From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 19:55:22 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E99CB1A for ; Thu, 6 Mar 2014 19:55:22 +0000 (UTC) Received: from mail-qc0-f181.google.com (mail-qc0-f181.google.com [209.85.216.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EE3C225D for ; Thu, 6 Mar 2014 19:55:21 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id e9so3477299qcy.26 for ; Thu, 06 Mar 2014 11:55:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=FHi3xQVSeUfJfphQBphcR9MzdLVex8PUDqfH5zvrmLY=; b=MnNBLE47/0A/jYZe0jBD6xSP0FTYCCNfT+WzO8xsWGBbW5RZmBToXtNgZ6uXR301xo F5EnCptfvbCkIPxGx2SxaJApR3InN2rmvw4UNVp/ZV+uWJSwhO4AMzsBYhlegFz8N9EA O2P1BVhRylvt4K1FA3Ld3Ei2vC7Fx9rRE/oTq70j2+qldvMhw2J7aBl7hlwDdRIHFjbH Q0BQwPu1PDx+StPyunIZhtxUHV6CcdYL+IpLXbomdmzGWNI5CW/Re703oxK5yCnPGgLG Xc8j9L7/JA8uZT/Cc/m9KNn8nPtzkqmoNW9mf3XHw70ip5swlKgCLGbH5b3GckA4Q7Ks qtig== X-Gm-Message-State: ALoCoQm2xcSsENWui1ZqGmXXGJYXsA79RaKDoZBlAKpuB7cYPHY5wNOIIRKpNHsoOeqNu8T+EcW6 X-Received: by 10.224.21.129 with SMTP id j1mr16552424qab.13.1394135715349; Thu, 06 Mar 2014 11:55:15 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 6 Mar 2014 11:54:55 -0800 (PST) X-Originating-IP: [2620:0:1003:1021:5c49:13a2:f7c6:9920] In-Reply-To: References: From: Julio Merino Date: Thu, 6 Mar 2014 14:54:55 -0500 X-Google-Sender-Auth: eqBEKVqle0wI9WYfprL4vJSgFlI Message-ID: Subject: Re: Structure of ATF testcases (was "Test scenario for sysctl kern.maxfiles") To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , kyua-discuss@googlegroups.com X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 19:55:22 -0000 On Thu, Mar 6, 2014 at 1:28 PM, Garrett Cooper wrote: > > Seems like another standalone tool should be developed to cleanup > variables after exit per test and per test suite. Which variables? I can see some benefit in providing helper functions to store and load arbitrary data structures to/from disk with one-liners, but is that what you mean? > Setup/cleanup in ATF doesn't seem to follow the model that many other > testing frameworks do when it comes to executing testcases and > managing setup/cleanup. I really wish things were like the > JUnit/unittest model, i.e. > > - Test suite setup > - For each testcase: > - Run the testcase setup > - If the setup passed, continue > - Run the testcase > - Run the testcase cleanup > - Test suite teardown Sure... but the systems that implement this setup assume that all the steps above run under one process. That's fine in the vast majority of unit tests but the model fails when tests modify system-wide state and crash before they get to the cleanup step. Keep in mind that one of the first design decisions for ATF was to support test cases like the above where system-wide state may be modified and you need to revert the changes. Because test cases can crash at any point, having an out-of-process cleanup routine was a must. This comes with its oddities though. All said, I'd also like a more JUnit-like model because it's easier to understand and satisfies 99% of the cases (assuming you have rump). From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 20:09:30 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6CA431A4 for ; Thu, 6 Mar 2014 20:09:30 +0000 (UTC) Received: from mail-qa0-f48.google.com (mail-qa0-f48.google.com [209.85.216.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 28C3C3D8 for ; Thu, 6 Mar 2014 20:09:29 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so2985869qaj.21 for ; Thu, 06 Mar 2014 12:09:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=/QOZ3GdwH8TmU0+wN4r2A8yFeQSgJ+9U/kVa/FvRXz8=; b=NEBQ30fFHijMXZlUtzV2Th6PCEgZL3WWxEKndaqmXszoX+izmu7XW1UOsDmqwo50ND 6AQHcyjN5L5tbBhmkY4sFqU70x7SOa4izTiL+ODNcw8jp+SrU1IC6oRgLbnmG7KJBSfz 2Iha7Jwa7fo0NFbpd5AmRYmkJCIVN9vfg8rVGEXfIheTLfpzA6bwrrW132/335l0Wnuy Q3cP7e5wDi2hqcZx7tgM/R3n+FjeKRC8ECoGsx3Wh4Gey2+Y4+pfxOO87hYW//5YvHpv 1LyVO+9FTLGzboBjFnC5nP/3cbnjyGP7fbGV9a3UsmJYzmwMcjoiBaTiVedf6yxdY+hK PH/w== X-Gm-Message-State: ALoCoQk1i7FwvD/oRTDGUR/s522z/Pgkp+rOzvY2W2isTWPrV993Rv/5gi2PKfHCfh9pK00a8lJo X-Received: by 10.224.46.73 with SMTP id i9mr13419487qaf.65.1394136562647; Thu, 06 Mar 2014 12:09:22 -0800 (PST) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 6 Mar 2014 12:09:02 -0800 (PST) X-Originating-IP: [2620:0:1003:1021:5c49:13a2:f7c6:9920] In-Reply-To: References: <16437CC5729B5345AF77F816513376E81296BC51@MX103CL02.corp.emc.com> From: Julio Merino Date: Thu, 6 Mar 2014 15:09:02 -0500 X-Google-Sender-Auth: s9BUJK7pzu5aHHLXIk1KVA18eiY Message-ID: Subject: Re: tests on kyua1.nyi.freebsd.org against HEAD failing To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Glen Barber X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 20:09:30 -0000 [ Removing my personal email address. If you could use the one at freebsd.org, I'd appreciate it as I'm trying to separate personal stuff from OSS work :-P ] On Thu, Mar 6, 2014 at 1:35 PM, Garrett Cooper wrote: > Julio, > > Looks like the NYI server needs to be updated and an extra slash needs > to be added to the .conf file. Thanks for diagnosing this Garret. Should be fixed now. From owner-freebsd-testing@FreeBSD.ORG Thu Mar 6 20:02:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF3ABDD9; Thu, 6 Mar 2014 20:02:58 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8A15B35B; Thu, 6 Mar 2014 20:02:58 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id lc6so2650560vcb.35 for ; Thu, 06 Mar 2014 12:02:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZC8jRYVrhwbgWowkfMc9Y1vNiVPrXjTeZw0MBznsT04=; b=CC3aE//5hP0qxBP1fdPK/ztVOW19as5DgmIY9DQZTYv8/6vEKKnUAQFAmAcKIk5TCX 2VZgUQ1i80MmQKgRlShlYT5iyCASTfkQjG96Zij61dkLl6Ods90qcUgEbQsPw4ePBPzc pWT9KAFUXLhOFusP+i5YV7XxUsjZMzo8BDz3FATunOZWbwTVHsnLavrooxWSfd6HanWZ AEAm2fVBzWhp937wJAa6znLqoIslMpkmYJLUqZyNY3qpqaeaNOswrDVExP8/4C/w4naP 3OtbwxrknpURUadslQDsn736SSMbFsvX6KUtgvDy1zD8XgStuWENrLguCBmq3sE3g5jM GL4A== MIME-Version: 1.0 X-Received: by 10.52.248.138 with SMTP id ym10mr9213531vdc.18.1394136177716; Thu, 06 Mar 2014 12:02:57 -0800 (PST) Received: by 10.221.67.136 with HTTP; Thu, 6 Mar 2014 12:02:57 -0800 (PST) In-Reply-To: References: Date: Thu, 6 Mar 2014 12:02:57 -0800 Message-ID: Subject: Re: Structure of ATF testcases (was "Test scenario for sysctl kern.maxfiles") From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , kyua-discuss@googlegroups.com X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 20:02:59 -0000 On Thu, Mar 6, 2014 at 11:54 AM, Julio Merino wrote: > On Thu, Mar 6, 2014 at 1:28 PM, Garrett Cooper wrote: >> >> Seems like another standalone tool should be developed to cleanup >> variables after exit per test and per test suite. > > Which variables? sysctls in this case. Isilon would probably love this and other folks would too no doubt. > I can see some benefit in providing helper functions to store and load > arbitrary data structures to/from disk with one-liners, but is that > what you mean? Sorta kinda. >> Setup/cleanup in ATF doesn't seem to follow the model that many other >> testing frameworks do when it comes to executing testcases and >> managing setup/cleanup. I really wish things were like the >> JUnit/unittest model, i.e. >> >> - Test suite setup >> - For each testcase: >> - Run the testcase setup >> - If the setup passed, continue >> - Run the testcase >> - Run the testcase cleanup >> - Test suite teardown > > Sure... but the systems that implement this setup assume that all the > steps above run under one process. That's fine in the vast majority of > unit tests but the model fails when tests modify system-wide state and > crash before they get to the cleanup step. Agreed. Hence the need for tools that restore system state and careful code reviews and code design. > Keep in mind that one of the first design decisions for ATF was to > support test cases like the above where system-wide state may be > modified and you need to revert the changes. Because test cases can > crash at any point, having an out-of-process cleanup routine was a > must. This comes with its oddities though. Yes, but it falls on the developer to handle global state outside of the atf test directory sandbox. ATF shouldn't worry about protecting someone from doing something like fiddling around with sysctls and files in /etc (for example) -- it's the test developer's job to not do things like this. > All said, I'd also like a more JUnit-like model because it's easier to > understand and satisfies 99% of the cases (assuming you have rump). Yup yup! -Garrett From owner-freebsd-testing@FreeBSD.ORG Sun Mar 9 11:40:22 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0245C844 for ; Sun, 9 Mar 2014 11:40:22 +0000 (UTC) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.freebsd.org (Postfix) with SMTP id BCF67EC1 for ; Sun, 9 Mar 2014 11:40:21 +0000 (UTC) Received: (qmail 35476 invoked from network); 9 Mar 2014 11:40:13 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay01.pair.com with SMTP; 9 Mar 2014 11:40:13 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s29BeDcx057086; Sun, 9 Mar 2014 12:40:13 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s29BeDJo057085; Sun, 9 Mar 2014 12:40:13 +0100 (CET) (envelope-from pho) Date: Sun, 9 Mar 2014 12:40:12 +0100 From: Peter Holm To: Julio Merino Subject: Re: Test scenario for sysctl kern.maxfiles Message-ID: <20140309114012.GA54149@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 11:40:22 -0000 On Thu, Mar 06, 2014 at 09:52:58AM -0500, Julio Merino wrote: > On Thu, Mar 6, 2014 at 6:23 AM, Peter Holm wrote: > > On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: > >> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: > >> > Here's an attempt to verify that increasing kern.maxfiles works as > >> > expected. > >> > > >> > http://people.freebsd.org/~pho/kern_descrip_test-v3.diff > >> > -- > >> > Peter > > A couple of general comments: > > * In openfiles2(), it seems to me that 'i' should be size_t. That cast > in the for loop looks like a hack. > Done. > * Style detail: I'd change the code to say something like the > following, which clearly separates the action from the handling of the > result. > > r = open("/etc/passwd", O_RDONLY); > if (ignore) { > if (r == -1) > break; > } else { > ATF_REQUIRE(r != -1); > } > Rewritten. > * Why does this test rely on /etc/passwd? Just create a file in the > current directory and open it. See the atf_utils_* in the > atf-c-api(3) manpage for various helper functions to deal with files. > If you want to depend on system files, the test should be declaring > that explicitly with atf_set_md_var("require.files", "/etc/passwd"). > I mistakenly relied on numerous NetBSD examples that uses /etc/passwd. > * Does the period in the test case name kern.maxfiles__increase > actually work? I'm surprised. > It doesn't. > * What's the rationale behind the TEST macro? > Removed test code. > More below. > > >> 1) done should be of type "static volatile sig_atomic_t", not int, > >> because it's set by signal handlers. > >> > > > > Yes, that is nicer (I learned something new today :-). But the use > > here works because there is a call to usleep(3) after each test, > > forcing the compiler to reload the "done" variable. > > > >> 2) using atexit() to register a cleanup routing is a hack. No doubt > > > > Why do you say that using atexit(3) is a hack? > > Because that's not how cleanup should be implemented in atf tests: you > should be using the cleanup routines. The reason is that atexit(3) is > not guaranteed to work: if your test crashes or is killed by Kyua > because it overruns its deadline, your cleanup code won't work but a > cleanup routine will. > Skipped using atexit(3). > >> you already noticed that it's difficult to use Kyua's builtin cleanup > >> capabilities because of the need to pass the value of oldmaxfiles. I > >> too have experienced that frustration. Is there any way to pass > >> values from the body of a testcase to its cleanup? Using > >> atf_tc_set_md_var() would be one way, but the man page suggests that > >> that function cannot be called from the body. Julio, is there a > >> better way to do this? > > The "problem" is that the body and the cleanup run in different > processes really, so there is no way you can pass state other than > writing to a local file. > > This change: > > http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/kernel/t_sysv.c.diff?r1=1.3&r2=1.4&only_with_tag=MAIN > > shows a way to do it. > > Because these tests are dealing with scary global system state (which > is a bad thing), I think that the hassle of having to do this cleanup > dance is "kinda justified": not that many tests should need it and, > for those that do, it becomes clear that maybe the test should be done > in a different way. > > We could do better, yes; I don't disagree with that. Suggestions? > > > Now this raises an interesting question for me: Which environment do > > you guys expect ATF to run in? If it is hosts like freefall, any > > resource hogging tests are out of the question I would think. > > That's the interesting question. This test will probably be flaky > and/or cause side-effects in the system while it runs because it > relies on state that is out of control of the test. > > For the setup at kyua1.nyi.freebsd.org, this is not a problem because > the tests run in a VM. But we want to encourage users to run the > tests themselves and this kind of tests may be problematic. > > Here goes a possibility: add a configuration variable > "allow_kernel_side_effects" or similar to kyua.conf > (test_suites.FreeBSD.allow_kernel_side_effects = true). Then make the > test case require the variable with atf_set_md_var("require.config", > "allow_kernel_side_effects") so that this test only runs when the user > has told Kyua to allow these tests. We can later change the > kyua1.nyi.freebsd.org setup to define this variable but we leave it > off by default for end users. Updated dif: http://people.freebsd.org/~pho/kern_descrip_test-v4.diff -- Peter From owner-freebsd-testing@FreeBSD.ORG Sun Mar 9 20:42:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 583F9F02 for ; Sun, 9 Mar 2014 20:42:59 +0000 (UTC) Received: from mail-qg0-f42.google.com (mail-qg0-f42.google.com [209.85.192.42]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1194127F for ; Sun, 9 Mar 2014 20:42:58 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so4927382qgd.1 for ; Sun, 09 Mar 2014 13:42:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=epqT1ocR2JGjNOqgTFH3bxcr6JnDRZ8hzY9UURout2U=; b=BYzzoaMxVEfelMCUmSbMwhJiniyEm0IVkDcAs3oUw2Hk8PkoFFu16IiFQ5htDXGZ9C cKrEjCHKKVfRgQuHKifIaYdSl+BNFi377sclL1b9v4sJ7+aMGQo3RAcsRMz53lt01i0Q jWgkejsU8In35vccSJEGMXAx+fVDuhtA2aH2NdZUaXE3IL2UtOT0nf6mWGeAjn1A/4lR 7eiTepyJLAMOoYDL3ItYl6Mm2kmx4MS8ldfUDzLvCTXfhwOjmd1+OuNqQJfnhB0T0Jr9 Oxs1S2pRll52tfnAcN+5dRGoGxFrjZnjV1ZNrbnso8L2bc4w0zIUmX661uPMYfsuiwTH 8NKA== X-Gm-Message-State: ALoCoQkH7luDvzQ4b/fhTkWjIGlcOOtQN94sQF50KhDTtiBBimB2nWc3JCESktpEw7B9GFYmW7t9 X-Received: by 10.224.161.140 with SMTP id r12mr36444384qax.24.1394397771805; Sun, 09 Mar 2014 13:42:51 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sun, 9 Mar 2014 13:42:31 -0700 (PDT) X-Originating-IP: [2620:0:1003:1021:a1b3:b33:a0fa:1de6] In-Reply-To: References: <20140124162759.GB90996@stack.nl> <0A68C9FE-6A3A-4DCF-92A9-A506D7581D69@gmail.com> <96373EF8-77EA-4AA6-8422-63881C32E455@gmail.com> From: Julio Merino Date: Sun, 9 Mar 2014 16:42:31 -0400 X-Google-Sender-Auth: 6ML2oNBC03qbPHgYBAy_DVfC9zk Message-ID: Subject: Re: [PATCH v2] convert /bin/sh tests over to ATF To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Jilles Tjoelker X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Mar 2014 20:42:59 -0000 On Mon, Mar 3, 2014 at 1:40 PM, Garrett Cooper wrote: > On Sun, Mar 2, 2014 at 1:14 PM, Julio Merino wrote: >> * The FILES must continue to be installed. Does this actually work >> when running your test programs after a "make install"? > > Could you please clarify your concern here? I'm not sure I understood > what you meant. The tests need to populate /usr/tests/bin/sh/ with the functional_test.sh driver and all its supporting data files. As I understand it, your driver code assumes that the data files are external (i.e. not bundled into the script itself), so those files must be available from the same location as where the driver lives. Hence you still need to use FILES to ensure the data files get installed into /usr/tests/. ... but maybe I'm missing something from your patch; haven't reviewed it again to double-check this. From owner-freebsd-testing@FreeBSD.ORG Mon Mar 10 10:16:28 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F076C17F for ; Mon, 10 Mar 2014 10:16:28 +0000 (UTC) Received: from relay02.pair.com (relay02.pair.com [209.68.5.16]) by mx1.freebsd.org (Postfix) with SMTP id 8F8E3112 for ; Mon, 10 Mar 2014 10:16:28 +0000 (UTC) Received: (qmail 75471 invoked from network); 10 Mar 2014 10:16:21 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay02.pair.com with SMTP; 10 Mar 2014 10:16:21 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s2AAGLD6092972 for ; Mon, 10 Mar 2014 11:16:21 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s2AAGLO4092971 for freebsd-testing@freebsd.org; Mon, 10 Mar 2014 11:16:21 +0100 (CET) (envelope-from pho) Date: Mon, 10 Mar 2014 11:16:21 +0100 From: Peter Holm To: freebsd-testing@freebsd.org Subject: "require.user: unprivileged" retains operator group Message-ID: <20140310101620.GA83688@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 10:16:29 -0000 $ kyua test user_test user_test:rootuser -> skipped: Requires root privileges [0.001s] user_test:nonrootuser -> broken: Caught unexpected exception: Tester failed with code 2; this is a bug [0.039s] 1/2 passed (1 failed) Committed action 1 $ su Password: root@x4:/usr/tests/sys/kern # kyua test user_test user_test:rootuser -> passed [0.031s] user_test:nonrootuser -> passed [0.022s] 2/2 passed (0 failed) Committed action 50 root@x4:/usr/tests/sys/kern # kyua debug user_test:rootuser uid=0(root) gid=0(wheel) groups=0(wheel),5(operator) user_test:rootuser -> passed root@x4:/usr/tests/sys/kern # kyua debug user_test:nonrootuser uid=977(tests) gid=65534(nobody) groups=65534(nobody),5(operator) user_test:nonrootuser -> passed root@x4:/usr/tests/sys/kern # http://people.freebsd.org/~pho/user_test.c -- Peter From owner-freebsd-testing@FreeBSD.ORG Mon Mar 10 15:00:42 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82A70D79 for ; Mon, 10 Mar 2014 15:00:42 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1E8552C2 for ; Mon, 10 Mar 2014 15:00:41 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id n12so8871487wgh.12 for ; Mon, 10 Mar 2014 08:00:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=C5qP+JPL15zdSCTk6R2onKy1O8ZmhDuwy7rAmZRN4Jk=; b=fcM47lBp1zNpQZ5l3130j0R459NuEPxgXVBPF+H0Ow2boO5Zf/PWVc49nzFG4sU3Yt o2ZOJT4e9x0I78u3tyTc1uGdTfJepVJsO0P7xF1szM/qIIBO4NcoqKtSJ3kYYI3lxwwW ci6HnqRVqh2A0A2QKzap7LJOl4JHc3SXZwK0uoEFHtNl2JPuN+BS3hKpR3whpeCs3TeJ oYf0ZellUoLSdSlrAPfxXbNKJo6AeiqJIxnug9R7cBUCyvJINUqk6o4BboFECtoNRZSS g/NzdlcQg2WILFDz3tUf1aUPvDLmfslwx7mstfyE1pMh/pUB6KN66ioBRVx1l4LTsktn rzAQ== MIME-Version: 1.0 X-Received: by 10.194.7.196 with SMTP id l4mr458918wja.92.1394463640614; Mon, 10 Mar 2014 08:00:40 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Mon, 10 Mar 2014 08:00:40 -0700 (PDT) In-Reply-To: <20140310101620.GA83688@x2.osted.lan> References: <20140310101620.GA83688@x2.osted.lan> Date: Mon, 10 Mar 2014 09:00:40 -0600 X-Google-Sender-Auth: P9r55g2oV71KKRitdhS7Y4DBwBE Message-ID: Subject: Re: "require.user: unprivileged" retains operator group From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 15:00:42 -0000 On Mon, Mar 10, 2014 at 4:16 AM, Peter Holm wrote: > $ kyua test user_test > user_test:rootuser -> skipped: Requires root privileges [0.001s] > user_test:nonrootuser -> broken: Caught unexpected exception: Tester failed with code 2; this is a bug [0.039s] > > 1/2 passed (1 failed) > Committed action 1 > $ su > Password: > root@x4:/usr/tests/sys/kern # kyua test user_test > user_test:rootuser -> passed [0.031s] > user_test:nonrootuser -> passed [0.022s] > > 2/2 passed (0 failed) > Committed action 50 > root@x4:/usr/tests/sys/kern # kyua debug user_test:rootuser > uid=0(root) gid=0(wheel) groups=0(wheel),5(operator) > user_test:rootuser -> passed > root@x4:/usr/tests/sys/kern # kyua debug user_test:nonrootuser > uid=977(tests) gid=65534(nobody) groups=65534(nobody),5(operator) > user_test:nonrootuser -> passed > root@x4:/usr/tests/sys/kern # > > http://people.freebsd.org/~pho/user_test.c > -- > Peter I can reproduce it. Looks like an upstream bug in Kyua. -Alan From owner-freebsd-testing@FreeBSD.ORG Mon Mar 10 15:14:03 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C5D443A; Mon, 10 Mar 2014 15:14:03 +0000 (UTC) Received: from mail-pb0-x236.google.com (mail-pb0-x236.google.com [IPv6:2607:f8b0:400e:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3B194665; Mon, 10 Mar 2014 15:14:03 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id ma3so7358028pbc.41 for ; Mon, 10 Mar 2014 08:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=iYYw5pMmUvvKS+tPO0FUd802Caq1ZAsSDsvnrBKjjSs=; b=M5BgddOjUR7mU79VOQE7BOqk7FXky2a1kPuuht411dAjEy/7WjhB2z2eCXSkx7g6bs Gk4lcKCbTAZgbUf6Q5zksZrAN+Qim21OWsjfeFDmyGsrg3acnx30l3YcHoBCcg693a0s IvAxILCTxJfpnHixWQ1/oRfIJnr9mcN+GgNRCm6rjYugYhMjaJ8OZ1z6p6arGGt7r0qP 1UNNcJJWxcEjC6NNocOChlEtVCftm5xAa9wObc8yNWxVO0MGDdf1Nqbt1fdu0OE3llv+ RvJNpmokHMs+Tx3PKQNUAGEFSNFehwH+ybZNBhZnLTk7EasD6IeN6k2K3HFNIrX8g57Y IJ4g== X-Received: by 10.68.249.100 with SMTP id yt4mr3984084pbc.165.1394464442812; Mon, 10 Mar 2014 08:14:02 -0700 (PDT) Received: from [192.168.20.11] (c-24-17-226-153.hsd1.wa.comcast.net. [24.17.226.153]) by mx.google.com with ESMTPSA id js7sm66332785pbc.35.2014.03.10.08.14.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 10 Mar 2014 08:14:01 -0700 (PDT) References: <20140310101620.GA83688@x2.osted.lan> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <27ACFD8F-51FD-4ED0-9325-992267964742@gmail.com> X-Mailer: iPhone Mail (11B651) From: Garrett Cooper Subject: Re: "require.user: unprivileged" retains operator group Date: Mon, 10 Mar 2014 08:14:01 -0700 To: Alan Somers Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 15:14:03 -0000 > On Mar 10, 2014, at 8:00, Alan Somers wrote: >=20 >> On Mon, Mar 10, 2014 at 4:16 AM, Peter Holm wrote: >> $ kyua test user_test >> user_test:rootuser -> skipped: Requires root privileges [0.001s] >> user_test:nonrootuser -> broken: Caught unexpected exception: Tester fa= iled with code 2; this is a bug [0.039s] >>=20 >> 1/2 passed (1 failed) >> Committed action 1 >> $ su >> Password: >> root@x4:/usr/tests/sys/kern # kyua test user_test >> user_test:rootuser -> passed [0.031s] >> user_test:nonrootuser -> passed [0.022s] >>=20 >> 2/2 passed (0 failed) >> Committed action 50 >> root@x4:/usr/tests/sys/kern # kyua debug user_test:rootuser >> uid=3D0(root) gid=3D0(wheel) groups=3D0(wheel),5(operator) >> user_test:rootuser -> passed >> root@x4:/usr/tests/sys/kern # kyua debug user_test:nonrootuser >> uid=3D977(tests) gid=3D65534(nobody) groups=3D65534(nobody),5(operator) >> user_test:nonrootuser -> passed >> root@x4:/usr/tests/sys/kern # >>=20 >> http://people.freebsd.org/~pho/user_test.c >> -- >> Peter >=20 > I can reproduce it. Looks like an upstream bug in Kyua. Yes. Unfortunately unless you get the appropriate mode for Kyua, it won't be= able to setuid to a lower privilege unless it's run as root.. Cheers! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Mon Mar 10 16:13:46 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16608E3A for ; Mon, 10 Mar 2014 16:13:46 +0000 (UTC) Received: from relay00.pair.com (relay00.pair.com [209.68.5.9]) by mx1.freebsd.org (Postfix) with SMTP id BDF30CA3 for ; Mon, 10 Mar 2014 16:13:45 +0000 (UTC) Received: (qmail 53948 invoked from network); 10 Mar 2014 16:13:37 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay00.pair.com with SMTP; 10 Mar 2014 16:13:37 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s2AGDbkb003003; Mon, 10 Mar 2014 17:13:37 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s2AGDbff003002; Mon, 10 Mar 2014 17:13:37 +0100 (CET) (envelope-from pho) Date: Mon, 10 Mar 2014 17:13:37 +0100 From: Peter Holm To: Garrett Cooper Subject: Re: "require.user: unprivileged" retains operator group Message-ID: <20140310161337.GA2811@x2.osted.lan> References: <20140310101620.GA83688@x2.osted.lan> <27ACFD8F-51FD-4ED0-9325-992267964742@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <27ACFD8F-51FD-4ED0-9325-992267964742@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 16:13:46 -0000 On Mon, Mar 10, 2014 at 08:14:01AM -0700, Garrett Cooper wrote: > > > On Mar 10, 2014, at 8:00, Alan Somers wrote: > > > >> On Mon, Mar 10, 2014 at 4:16 AM, Peter Holm wrote: > >> $ kyua test user_test > >> user_test:rootuser -> skipped: Requires root privileges [0.001s] > >> user_test:nonrootuser -> broken: Caught unexpected exception: Tester failed with code 2; this is a bug [0.039s] > >> > >> 1/2 passed (1 failed) > >> Committed action 1 > >> $ su > >> Password: > >> root@x4:/usr/tests/sys/kern # kyua test user_test > >> user_test:rootuser -> passed [0.031s] > >> user_test:nonrootuser -> passed [0.022s] > >> > >> 2/2 passed (0 failed) > >> Committed action 50 > >> root@x4:/usr/tests/sys/kern # kyua debug user_test:rootuser > >> uid=0(root) gid=0(wheel) groups=0(wheel),5(operator) > >> user_test:rootuser -> passed > >> root@x4:/usr/tests/sys/kern # kyua debug user_test:nonrootuser > >> uid=977(tests) gid=65534(nobody) groups=65534(nobody),5(operator) > >> user_test:nonrootuser -> passed > >> root@x4:/usr/tests/sys/kern # > >> > >> http://people.freebsd.org/~pho/user_test.c > >> -- > >> Peter > > > > I can reproduce it. Looks like an upstream bug in Kyua. > > Yes. Unfortunately unless you get the appropriate mode for Kyua, it won't be able to setuid to a lower privilege unless it's run as root.. > Cheers! > -Garrett Guess you both talk about the "Caught unexpected exception" issue? What about the fact that the unprivileged user is member of group "operator"? -- Peter From owner-freebsd-testing@FreeBSD.ORG Mon Mar 10 17:15:08 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94ECA4EA for ; Mon, 10 Mar 2014 17:15:08 +0000 (UTC) Received: from mail-qc0-f177.google.com (mail-qc0-f177.google.com [209.85.216.177]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5525634A for ; Mon, 10 Mar 2014 17:15:08 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id w7so7971605qcr.22 for ; Mon, 10 Mar 2014 10:15:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=OAl8wh7hu5Z2Ie04asYa91u1Tnu7JYbb9fypzqYfGXk=; b=Z2JndvhcYrsfKCQ8Df5bYY5nzXaesvSfwJge6F8p9JqBL5fCpWZr0JjuuIsv0Wz2qQ aTgamR7R4e0bj0Fgg8bHuqg5EHeSN1gkpzjrcN0Rj9woniYqzS75WRdzFGRozeJ+SMDg TMXZq/+/VpUcgcKP4FIieFn4l28HNMxZ+s4aIHp46vCMeoqWndGwEyaH5UOR7992PkA4 v/Dpx8wCg59aRmoev/nceKZOdC1DfNDvwYHAlAfpxh/tOXM4rcaWICxBtBMlibZpgKu9 CUwd48eGQkeQE5ACnoI6ApRTsv6CnOCFVJv7l5YuDTYUk+stg0lqx/GPGxR+0uy6b64H bp4Q== X-Gm-Message-State: ALoCoQkqyo2kH5oNXUF3Gr89GkiVymuWIQrkGtKm3ca9F/2gD6rKpw64CNZaHrK+7HV7rXjSN9FK X-Received: by 10.224.173.1 with SMTP id n1mr41989203qaz.48.1394471707109; Mon, 10 Mar 2014 10:15:07 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Mon, 10 Mar 2014 10:14:47 -0700 (PDT) X-Originating-IP: [2620:0:1003:1021:a46d:cb0b:a4bf:b6b9] In-Reply-To: <20140310101620.GA83688@x2.osted.lan> References: <20140310101620.GA83688@x2.osted.lan> From: Julio Merino Date: Mon, 10 Mar 2014 13:14:47 -0400 X-Google-Sender-Auth: l9LXzieERMUpVD53YGS3Z58JfHA Message-ID: Subject: Re: "require.user: unprivileged" retains operator group To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 17:15:08 -0000 On Mon, Mar 10, 2014 at 6:16 AM, Peter Holm wrote: > root@x4:/usr/tests/sys/kern # kyua debug user_test:nonrootuser > uid=977(tests) gid=65534(nobody) groups=65534(nobody),5(operator) > user_test:nonrootuser -> passed I guess that should be fixed, yes. Dropping secondary groups was not part of the feature, but I think it should be. Can you file a bug at https://github.com/jmmv/kyua/issues ? I'll hopefully get a chance to look at it during AsiaBSDCon :) From owner-freebsd-testing@FreeBSD.ORG Mon Mar 10 23:32:17 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32E2C292 for ; Mon, 10 Mar 2014 23:32:17 +0000 (UTC) Received: from mail-qa0-f46.google.com (mail-qa0-f46.google.com [209.85.216.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DF487F78 for ; Mon, 10 Mar 2014 23:32:16 +0000 (UTC) Received: by mail-qa0-f46.google.com with SMTP id i13so7578482qae.19 for ; Mon, 10 Mar 2014 16:32:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=LPGlqhXj0BeS2VjxGPhXuTwrN45RefmYF3GLAeoaxBk=; b=PXIfltNUksAreFikbJUnj/6vSZsjQbcvRrVxzO1yqtS0nxkWv8zYaUl4K0LlHfp6ZH Iajv8mzgGup5ZzFdBGZQ0amnPAiQngerpl7SlgWz+PU4xNnleYEi5zPJv8fgHZ/8pZ2X waWz2+13tBR8HwlEKxW4JHjASW50Qzn7kvu4j2lhh300Byd7IGwIyRbdqrwDIYespaax DCzuEbNC90dPeFgghgHhWRgeisfNyQOrwj2rik2+C7XOlGqx81xRc36B60+y4esgTmxR CSkhhAepZGLSSNddURWzOtClJzfsBBfMo4ykRtyoxu8PTzx+bPXgcQY2UVR59WcQ0iJ6 HUOg== X-Gm-Message-State: ALoCoQl806t7zq9HW5HJwdiz7IgWm15Jost2nujvvdIkmXgGgNFPT4iKBvcG8/To8Qv8U+VyeKgp X-Received: by 10.140.102.2 with SMTP id v2mr6732437qge.27.1394488452990; Mon, 10 Mar 2014 14:54:12 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Mon, 10 Mar 2014 14:53:51 -0700 (PDT) X-Originating-IP: [2620:0:1003:1021:95e6:6843:a3da:2d60] In-Reply-To: References: <94D3F60A-1E12-418B-8313-E43C93878B95@gmail.com> <56D809A7-5387-475A-8C89-794E328F6935@gmail.com> From: Julio Merino Date: Mon, 10 Mar 2014 17:53:51 -0400 X-Google-Sender-Auth: jGmz6fZShN7Aw36a16z7hUulYkc Message-ID: Subject: Re: [PATCH] Integrate pjdfstest into the build To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 23:32:17 -0000 On Mon, Mar 3, 2014 at 1:22 PM, Garrett Cooper wrote: > On Sun, Mar 2, 2014 at 3:00 PM, Julio Merino wrote: >> Then I believe this should really live under contrib/pjdfstest. Any >> reason it's not there? If it's external, then yes, we should keep the >> layout of the upstream sources without performing any renames (unless >> Pawel wants to do them). > > I think it should be, but I don't own pjdfstest ;).. > (largely for jmmv's benefit) Ultimately if pjd went this route, > pjdfstest needs to be put in the vendor repo, reimported into FreeBSD > under contrib, and then linked into the tree appropriately. pjd and I discussed this over IRC and he says that this is fine as long as the main structure of pjdfstest is maintained. Ensuring the structure is, arguably, the whole point of putting the code into contrib so this is not a problem. So, we should: 1. Import pjdfstest under a vendor repository. 2. Merge pjdfstest into contrib/pjdfstest/. 3. Add reachover Makefiles and the configuration file somewhere under tests/sys/ (?). 4. Remove all of tools/regression/pjdfstest/. 5. Plug into the test suite! I'll put this in my to-do queue so, eventually, I will get to it... but if any of you can do so instead the better :-) From owner-freebsd-testing@FreeBSD.ORG Wed Mar 12 22:10:34 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A710464 for ; Wed, 12 Mar 2014 22:10:34 +0000 (UTC) Received: from mail-pb0-x234.google.com (mail-pb0-x234.google.com [IPv6:2607:f8b0:400e:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 354A4B39 for ; Wed, 12 Mar 2014 22:10:34 +0000 (UTC) Received: by mail-pb0-f52.google.com with SMTP id rr13so147116pbb.25 for ; Wed, 12 Mar 2014 15:10:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5N69iZL1hxGjLaWyyzmPZRbkxgfan6BNT8fF1Fxlucw=; b=hQ8Ye0ORNlC9K+nFiJgT2Z41B5LR84WJY9hMyeiJgxbwV/U9x3yd0YrNjPG/8JlSeF QBSgUFK8XWsuhXXmxrFc6NNDIfapis0EPVt+1Qx63OPFswpk4y74b3vlvwCE0YoIzC0T ry+L3bgBQC7ffWUzz5yDtypJmCwK9rCa+0xKONG7ocX30R5Z+wY7pDs6ush0SAeHZmb3 MPY+6FNlYzjitHUWNVTUDVVY85kHrN8eixF/JDirftbpDgxoLSdWXzGK9JMxLOAzDJe0 OJpWAZ2RKZynnLEgF5q0oBhdKuW8sFJRvmZujkkQvy60SQnFaZ1f9bwOPRbDoo1IdslA Qlbg== MIME-Version: 1.0 X-Received: by 10.68.190.163 with SMTP id gr3mr7992515pbc.103.1394662233887; Wed, 12 Mar 2014 15:10:33 -0700 (PDT) Received: by 10.68.63.38 with HTTP; Wed, 12 Mar 2014 15:10:33 -0700 (PDT) Date: Wed, 12 Mar 2014 23:10:33 +0100 Message-ID: Subject: GSoC Idea: Working on modernizing the tests of ATF From: tuchalia To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Mar 2014 22:10:34 -0000 Hi there, I'm a student willing to do this work. I'm currently interested in a few projects, and I really like this one as well. Where should I start looking to write a strong proposal? -- Daniel From owner-freebsd-testing@FreeBSD.ORG Thu Mar 13 02:01:30 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96B915A5 for ; Thu, 13 Mar 2014 02:01:30 +0000 (UTC) Received: from mail-qg0-f45.google.com (mail-qg0-f45.google.com [209.85.192.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 54F291DA for ; Thu, 13 Mar 2014 02:01:29 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id j5so1126516qga.4 for ; Wed, 12 Mar 2014 19:01:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=Pa280MnDgn2Qk3834uXLediETs0H1gmHCt34icGA3DM=; b=YQfbaht52HMFYNMOKgp2wYZ3ZlDxlRf1pJQKCaKo5CsWUcFgjFg+BWzSNr/G07vxjg p6pdQS4C0Yk95rCY6D1SMBW0t278Jm4MKXvz3tXZz98U8/SIKeV+gDBqC1mHeiDbBfhF eMmjyg+tgsoBUoV2ByrtdMA8SkZTCpMFMUz+WiW8nMscxTLFzmq711SEZ9NvnmgOwi44 T+5FDRZf73MKSHzy5IFiH9Z39dE3Q7U7/9uY0NcABUeAo0zfMTa0mnn0xI8IFjUIappY SLsH/8f5LyOVi1JqnCWVeiOTjmrVWKolcxBJMPWzfN1+tcaWvcx9mlCtvrU8j7JTd1iJ p2wg== X-Gm-Message-State: ALoCoQmm1ldfHzZDKf8vtwN72PvbnWIcTpH6B7jQuH6E2yxffnuKipbfpSYAcNOLhleppbH6/e9H X-Received: by 10.224.147.77 with SMTP id k13mr880579qav.64.1394676082613; Wed, 12 Mar 2014 19:01:22 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Wed, 12 Mar 2014 19:01:02 -0700 (PDT) X-Originating-IP: [2401:fa00:4:1012:5cf9:3224:4bc7:7478] In-Reply-To: References: From: Julio Merino Date: Thu, 13 Mar 2014 11:01:02 +0900 X-Google-Sender-Auth: ojhrEhe2rCr3KQHjMWHBteHiUJE Message-ID: Subject: Re: GSoC Idea: Working on modernizing the tests of ATF To: tuchalia Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 02:01:30 -0000 On Thu, Mar 13, 2014 at 7:10 AM, tuchalia wrote: > Hi there, > > I'm a student willing to do this work. > I'm currently interested in a few projects, and I really like this one as > well. > > Where should I start looking to write a strong proposal? Hello, You should start by looking at the information in the TestSuite wiki page. Try to get familiar with the examples referenced from there: look for the code, read it, try to do some edits, etc. Then you can also dive into the code in tools/regression/ and tools/test/ to see what would need to be migrated. Here, you can use the prove(1) tool to run the tests that end in .t, so you could give that a try and see how that works as well. And, lastly, you can also look at the NetBSD source tree under the src/tests/ directory[1] or at Garret's GitHub repository[2]. There are many test programs there that can be easily made to run on FreeBSD. Getting one of those, hooking it into the build and preparing a patch for that would be a nice exercise. Hope this helps. 1: http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/?only_with_tag=MAIN 2: https://github.com/yaneurabeya/freebsd From owner-freebsd-testing@FreeBSD.ORG Thu Mar 13 02:10:17 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C868644 for ; Thu, 13 Mar 2014 02:10:17 +0000 (UTC) Received: from mail-qg0-f45.google.com (mail-qg0-f45.google.com [209.85.192.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DB34B277 for ; Thu, 13 Mar 2014 02:10:16 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id j5so1150919qga.4 for ; Wed, 12 Mar 2014 19:10:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=q/BtDfJxpqHbPvmhqBaUlMmTyo/JUBbCNUEkNv/IpZo=; b=Rtay8WfU3pRQYhi3vImm56NzR9OopU6EPXSuoaeEJ1NBAWhb2z3G41WaiHR0UFddVR CIxFUm3t3U664Q3L5Lfu1IkD3QKHMHOw9E6TE4vwxAv7ZR+8crl5cOuIJzQyPPGsOV5T HF9yMIoC+Xz+qMZ6xb+r3YO8RcJIRfiw3UfqXdFQePR34opdVPDfxYegTEuaSN//B0ah qmumcn7UP+j7ZwqtMZD61jrgVsulEfzSriM7ha0d1PtUUwyvxE3yoVlSyuLzp4YCcLFh UtUwRyDfW7UYsq5WWtbLtAg+tUNADvCjDDXv3SUC8azmEVeTdhEExMvzSvwpF/Fp5jaZ +UMg== X-Gm-Message-State: ALoCoQmVf65b6Z8xIHAakQ97Dn/sEmz4LzBAOAT15zOJbtGYBI8XEH9olY0XoVLYvm7luSa+WwsE X-Received: by 10.140.29.139 with SMTP id b11mr888510qgb.48.1394676615975; Wed, 12 Mar 2014 19:10:15 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Wed, 12 Mar 2014 19:09:55 -0700 (PDT) X-Originating-IP: [2401:fa00:4:1012:5cf9:3224:4bc7:7478] In-Reply-To: References: From: Julio Merino Date: Thu, 13 Mar 2014 11:09:55 +0900 X-Google-Sender-Auth: UWiAaa86iKh-9Ep5USoQwmlfRuc Message-ID: Subject: Re: Structure of ATF testcases (was "Test scenario for sysctl kern.maxfiles") To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , kyua-discuss@googlegroups.com X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 02:10:17 -0000 On Fri, Mar 7, 2014 at 5:02 AM, Garrett Cooper wrote: > On Thu, Mar 6, 2014 at 11:54 AM, Julio Merino wrote: >> On Thu, Mar 6, 2014 at 1:28 PM, Garrett Cooper wrote: >>> >>> Seems like another standalone tool should be developed to cleanup >>> variables after exit per test and per test suite. >> >> Which variables? > > sysctls in this case. Isilon would probably love this and other folks > would too no doubt. Writing such support shouldn't be too hard. I can imagine a sysctl(8) wrapper stored in the test's work directory, put in the PATH, so that it captured all variables written to from the test. All this could be set up by a "sysctl-test setup" script or similar. This script would record the old values in a separate file within the work directory. Later, in the cleanup phase, you'd invoke a "sysctl-test cleanup" script that read the list of overridden variables and restored their previous values. We can easily do this as a FreeBSD-specific thingy for now and put it in the source tree. I think it'd be a good way to test the idea this way. We could generalize it into ATF later on, but adding hooks for sysctl is nasty for portability reasons... From owner-freebsd-testing@FreeBSD.ORG Thu Mar 13 20:23:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 607538B1 for ; Thu, 13 Mar 2014 20:23:14 +0000 (UTC) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F3273B47 for ; Thu, 13 Mar 2014 20:23:13 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id hm4so4071134wib.0 for ; Thu, 13 Mar 2014 13:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Qb3h/ud3jC5tHZ3POwCjTBFyCUO54sF3WRPb5uMLEPU=; b=nj8r8PYxeYcJBCeb/MQMeH0fZ6lS6ob6orOnqhVtNpfr2z40eyNxFSSNuEZ60e/CU9 WQvkDzKwmz1wnnou/y7QxIch3rEO64QLfkVeIWuRhQakCwSuMiNjkwL21NUhfI4dv6Dk RygJAZhFqtTL0NNUjSIBFJda/4XIndKYAJB9BAHGPdtpLh0Djx0BgW4gtuB5n9jmoH1Z zK1avrrDhRYY+BqHVoEdcZXfYl6ybXyJ4QEmU5dEXlyvo3q3gV/FgkNp0oU85+6FK7N3 7dM85WZOXFGDp0S/YQTr00HqnGCux6v5DNOvEp+8d2u/iFtW/oFNLdugoih6wbtgUcnW sGVA== MIME-Version: 1.0 X-Received: by 10.180.97.37 with SMTP id dx5mr3082122wib.53.1394742192409; Thu, 13 Mar 2014 13:23:12 -0700 (PDT) Received: by 10.194.168.197 with HTTP; Thu, 13 Mar 2014 13:23:12 -0700 (PDT) Date: Thu, 13 Mar 2014 14:23:12 -0600 Message-ID: Subject: Where to put test cases for stuff in /etc? From: asomers@gmail.com To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 20:23:14 -0000 I occasionally make changes to /etc/rc.subr and /etc/rc.d/*. Where can I write ATF test cases for those files? Should I create etc/tests/rc.subr_test.sh and etc/rc.d/tests/routing_test.sh ? -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Mar 13 20:40:08 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F906B9F for ; Thu, 13 Mar 2014 20:40:08 +0000 (UTC) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2491FC4A for ; Thu, 13 Mar 2014 20:40:08 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id fa1so1635485pad.14 for ; Thu, 13 Mar 2014 13:40:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=0FxuPmIBMGNk9OawuKfZSaO2XMYKdza9ePRQwQnUuYU=; b=SflNA12ILYIn2tiHZoV7xVoTXW3ug1YISTd3TJrBgmdoG+BLALJeL23gPAjsS8lf/4 5XkFH91iUqPc5pMZPW0niEdWe49uBbMTfTzck+8W3WUalj4pHnleGotZ4F3j4I6q2UKB Q1eHfX+6sTwFjdxOZ7TKDURQlm0giDHlOYtlt8i56wdFTBRPmYXOM6H2zB0d6C0NzlRD +1G9cR4v5WfzsimMMazFZUeQghDhAWcYnjS7mdgfMT0+nnuBsALcFyuSgS7SN+4j3XKK WVJtzK8D3fMLfxOKtNBBJyZaYVaN4LUx3ED0hxDLUmpyLHZn13XHnkO0aCY/UksQO9yE BABQ== X-Received: by 10.68.36.41 with SMTP id n9mr4632079pbj.99.1394743207629; Thu, 13 Mar 2014 13:40:07 -0700 (PDT) Received: from [10.206.122.58] ([166.170.40.7]) by mx.google.com with ESMTPSA id kc9sm9809843pbc.25.2014.03.13.13.40.05 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 13 Mar 2014 13:40:06 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <423AB61A-6099-40F1-8931-B5339FFAEB65@gmail.com> X-Mailer: iPhone Mail (11B651) From: Garrett Cooper Subject: Re: Where to put test cases for stuff in /etc? Date: Thu, 13 Mar 2014 13:40:03 -0700 To: "asomers@gmail.com" Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 20:40:08 -0000 > On Mar 13, 2014, at 13:23, asomers@gmail.com wrote: >=20 > I occasionally make changes to /etc/rc.subr and /etc/rc.d/*. Where > can I write ATF test cases for those files? Should I create > etc/tests/rc.subr_test.sh and etc/rc.d/tests/routing_test.sh ? Putting it under tests/etc would be a better idea for now. The build sys= tem under etc/ is a bit of a mess... Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Fri Mar 14 06:09:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A331BC7; Fri, 14 Mar 2014 06:09:14 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2FBE75F0; Fri, 14 Mar 2014 06:09:13 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so1403082lan.40 for ; Thu, 13 Mar 2014 23:09:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=EJA3OnlupPDI2EsH0T1+qIW/MoSVYJem5Vf5P/ZKc8c=; b=Xa3kYCutRRrLwLVz7h2TEUAIJu//YfDkIiEPz0tXQUM8fdRubcOq3iTPa3YneAMgO2 7Oi2okaB83cmnFjVioNT5vHGGWuRcvxa03Pq6A2CHiOsftQEaBDoNc2cEo+Hd8OVR0u/ W8d8NjGrvr+zEFFGJaKPWItdROiMEm+yWq+PM73vqZdCyZgaQ++M7PScZZKlWWeIRO47 E0+z6H6CWSTkUm64fxOQgSq02SUEtjMHQhrpNcTLuO9IUqRWwI/Z/7xvMMeajkuXJ3aZ 5bsodq8w0Ogi8Cu1K4q59kJ5cynm0DGRgnK/rsPhx2jHut43GHN3iXp+m/Fia5WEnKuk RLUg== MIME-Version: 1.0 X-Received: by 10.153.8.135 with SMTP id dk7mr4208838lad.18.1394777351184; Thu, 13 Mar 2014 23:09:11 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Thu, 13 Mar 2014 23:09:11 -0700 (PDT) In-Reply-To: References: Date: Thu, 13 Mar 2014 23:09:11 -0700 X-Google-Sender-Auth: XdssAH0JaveZ3LE3JX0VwvtJGw8 Message-ID: Subject: Re: March 13: Jenkins and BHyve presentation From: Craig Rodrigues To: freebsd-current Current , "freebsd-virtualization@freebsd.org" , "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 06:09:14 -0000 On March 13, 2014, I gave a talk at the Bay Area FreeBSD User Group on the current state of Jenkins and BHyve work in the FreeBSD cluster. The slides are now online: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups -- Craig From owner-freebsd-testing@FreeBSD.ORG Fri Mar 14 07:53:56 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E2F96D0 for ; Fri, 14 Mar 2014 07:53:56 +0000 (UTC) Received: from mail-qa0-f53.google.com (mail-qa0-f53.google.com [209.85.216.53]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0B557F9C for ; Fri, 14 Mar 2014 07:53:55 +0000 (UTC) Received: by mail-qa0-f53.google.com with SMTP id w8so2137417qac.40 for ; Fri, 14 Mar 2014 00:53:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=zbS3YcMM3tx6dPPJVIr7yMnp4HeEZTpzA6XS/nvBBI4=; b=iGJ0hKGb1bvjPok5m7vW97e63TPi8lNgVIyQYkL4+iIJk02k5szd71FdT0O1xViOIQ g89kVUpaqFnNuInOGOX/FYFIS8XQb9Ax0VMGRDwBqaEGdDZcCM/puXnLxpgJQYVbt8tF GgxoHNULHXOTusrFQ/HVEqbYlTleqXjTultagWDmlUtgkLcN3gC8TrzCzpcb7bZd9aAs MrZDa3kotDUGtucioB9gxanY7au4qPfHvL0+vBaWbtTiQNqzPgy40lCVCbl17W6X4yLA WhBAgzXljWml8b8e7PrcqLo01qh4Bhe56sA/WuPJf3tYJvIh8G0wizLxQLIS0bpNTaEr wwXQ== X-Gm-Message-State: ALoCoQmP1jgEj2UOKItGP6lcjiOoNXrAPnsGU7qZJ6UWVRj9Xj5b6nF1iCHQ3ItK22A5RYfDPseE X-Received: by 10.140.102.2 with SMTP id v2mr7607581qge.27.1394783629782; Fri, 14 Mar 2014 00:53:49 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Fri, 14 Mar 2014 00:53:29 -0700 (PDT) X-Originating-IP: [180.42.49.96] In-Reply-To: <423AB61A-6099-40F1-8931-B5339FFAEB65@gmail.com> References: <423AB61A-6099-40F1-8931-B5339FFAEB65@gmail.com> From: Julio Merino Date: Fri, 14 Mar 2014 16:53:29 +0900 X-Google-Sender-Auth: PZ7z-HMMCAp6DmC5yeEpd1xnX0E Message-ID: Subject: Re: Where to put test cases for stuff in /etc? To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "asomers@gmail.com" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 07:53:56 -0000 On Fri, Mar 14, 2014 at 5:40 AM, Garrett Cooper wrote: > >> On Mar 13, 2014, at 13:23, asomers@gmail.com wrote: >> >> I occasionally make changes to /etc/rc.subr and /etc/rc.d/*. Where >> can I write ATF test cases for those files? Should I create >> etc/tests/rc.subr_test.sh and etc/rc.d/tests/routing_test.sh ? Tests for these would be awesome. > Putting it under tests/etc would be a better idea for now. The build system under etc/ is a bit of a mess... What's the problem with putting tests in etc/tests/ and what would it take to fix that? From owner-freebsd-testing@FreeBSD.ORG Fri Mar 14 13:11:55 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 531723AA; Fri, 14 Mar 2014 13:11:55 +0000 (UTC) Received: from mail-pa0-x22e.google.com (mail-pa0-x22e.google.com [IPv6:2607:f8b0:400e:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2087C8F6; Fri, 14 Mar 2014 13:11:55 +0000 (UTC) Received: by mail-pa0-f46.google.com with SMTP id kp14so2630042pab.33 for ; Fri, 14 Mar 2014 06:11:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=rwrbI+q5w9bJD4LTAkw3x9lHMYL6pYt7fUH2HpRESQA=; b=T9zinvQVBuPieLpPyla8AjP97Ii2ZvNruWA/vsspy5ytNAN1B5HgtOa7aXi7Qcsyfz BtsLBl+SjJplKmpznx1DXxk+ccVyivS0cgTJUZuJIFjoa3QOV8FUfoiXCeOF+U707DoX 8U8gFTfut9miBaOFfvbLT1nmKFmlre+XtkR8K3EgEortNsFaeUaVV+203vOJKHgSHdwX r2cntNP8u7AmcKv7syah3OB6rg897XwaJIp46ohyWSfkkTWAg0bNxf45EoAYvsz2qAzf gdzCVpjlj7Pehq2o6FrASkcwaO21ew1eSEUbgQQkMrvgjjW+fNUF6WFe0RA0JzReFti/ DIXA== X-Received: by 10.68.28.69 with SMTP id z5mr8974261pbg.121.1394802714769; Fri, 14 Mar 2014 06:11:54 -0700 (PDT) Received: from [192.168.20.11] (c-24-17-226-153.hsd1.wa.comcast.net. [24.17.226.153]) by mx.google.com with ESMTPSA id db3sm16624441pbb.10.2014.03.14.06.11.53 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 14 Mar 2014 06:11:53 -0700 (PDT) References: <423AB61A-6099-40F1-8931-B5339FFAEB65@gmail.com> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <769C5BD0-D803-46EB-AD18-A3031AC2036E@gmail.com> X-Mailer: iPhone Mail (11B651) From: Garrett Cooper Subject: Re: Where to put test cases for stuff in /etc? Date: Fri, 14 Mar 2014 06:11:52 -0700 To: Julio Merino Cc: "freebsd-testing@freebsd.org" , "asomers@gmail.com" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 13:11:55 -0000 > On Mar 14, 2014, at 0:53, Julio Merino wrote: >=20 >> On Fri, Mar 14, 2014 at 5:40 AM, Garrett Cooper w= rote: >>=20 >>> On Mar 13, 2014, at 13:23, asomers@gmail.com wrote: >>>=20 >>> I occasionally make changes to /etc/rc.subr and /etc/rc.d/*. Where >>> can I write ATF test cases for those files? Should I create >>> etc/tests/rc.subr_test.sh and etc/rc.d/tests/routing_test.sh ? >=20 > Tests for these would be awesome. >=20 >> Putting it under tests/etc would be a better idea for now. The build s= ystem under etc/ is a bit of a mess... >=20 > What's the problem with putting tests in etc/tests/ and what would it > take to fix that? You'd need to create ad hoc make recipes that handled WITH_TEST properly, or= enhance the Makefiles to use bsd.*.mk. Cheers, -Garrett= From owner-freebsd-testing@FreeBSD.ORG Fri Mar 14 16:30:17 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0BAFC76C; Fri, 14 Mar 2014 16:30:17 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3E615F0F; Fri, 14 Mar 2014 16:30:16 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id cc10so5629820wib.2 for ; Fri, 14 Mar 2014 09:30:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=RB7L6Z2Ajij+X+cocwLhfx9MZvGsKopvvmflno8crPM=; b=rZzuOnBR70qvCC2Zk9EnSrfYFd/KY+F4s0MoAscw6PXAVBkUFVJrG3tXoCvNVwkKF+ SqYEld4lHhLr+c0gzOI+I7bqC2LZgFY+Zzbk9FJdLzi81jo5f919BCQ5dLgt0dGXNu6Z H11/7Smya0tBZlTXVE+Y3zctfl9AasW/x7YDa2LEo1xeUanvR9xF/ecI7fPmZdPlW8Nm RksgTCnQRBZ0HaVyIURDldJLS5wHditCFigCgNpn+YP0A1mlPSVl60GoO30WtaMHixxk olYXrnOPDErXzkaGBFTQ1+EBZa0Kbm5K51tvDCZwRIpB2p6r6XFOHr5+kSjNEyh7fMUO 5I6g== MIME-Version: 1.0 X-Received: by 10.194.7.196 with SMTP id l4mr179479wja.92.1394814614551; Fri, 14 Mar 2014 09:30:14 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Fri, 14 Mar 2014 09:30:14 -0700 (PDT) In-Reply-To: <20140309114012.GA54149@x2.osted.lan> References: <20140305085806.GA70478@x2.osted.lan> <20140306112322.GA10664@x2.osted.lan> <20140309114012.GA54149@x2.osted.lan> Date: Fri, 14 Mar 2014 10:30:14 -0600 X-Google-Sender-Auth: WOFvL3nglMUV4f6U67Y0IwtD2JM Message-ID: Subject: Re: Test scenario for sysctl kern.maxfiles From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 16:30:17 -0000 On Sun, Mar 9, 2014 at 5:40 AM, Peter Holm wrote: > On Thu, Mar 06, 2014 at 09:52:58AM -0500, Julio Merino wrote: >> On Thu, Mar 6, 2014 at 6:23 AM, Peter Holm wrote: >> > On Wed, Mar 05, 2014 at 10:08:49AM -0700, Alan Somers wrote: >> >> On Wed, Mar 5, 2014 at 1:58 AM, Peter Holm wrote: >> >> > Here's an attempt to verify that increasing kern.maxfiles works as >> >> > expected. >> >> > >> >> > http://people.freebsd.org/~pho/kern_descrip_test-v3.diff >> >> > -- >> >> > Peter >> >> A couple of general comments: >> >> * In openfiles2(), it seems to me that 'i' should be size_t. That cast >> in the for loop looks like a hack. >> > > Done. > >> * Style detail: I'd change the code to say something like the >> following, which clearly separates the action from the handling of the >> result. >> >> r = open("/etc/passwd", O_RDONLY); >> if (ignore) { >> if (r == -1) >> break; >> } else { >> ATF_REQUIRE(r != -1); >> } >> > > Rewritten. > >> * Why does this test rely on /etc/passwd? Just create a file in the >> current directory and open it. See the atf_utils_* in the >> atf-c-api(3) manpage for various helper functions to deal with files. >> If you want to depend on system files, the test should be declaring >> that explicitly with atf_set_md_var("require.files", "/etc/passwd"). >> > > I mistakenly relied on numerous NetBSD examples that uses /etc/passwd. > >> * Does the period in the test case name kern.maxfiles__increase >> actually work? I'm surprised. >> > > It doesn't. > >> * What's the rationale behind the TEST macro? >> > > Removed test code. > >> More below. >> >> >> 1) done should be of type "static volatile sig_atomic_t", not int, >> >> because it's set by signal handlers. >> >> >> > >> > Yes, that is nicer (I learned something new today :-). But the use >> > here works because there is a call to usleep(3) after each test, >> > forcing the compiler to reload the "done" variable. >> > >> >> 2) using atexit() to register a cleanup routing is a hack. No doubt >> > >> > Why do you say that using atexit(3) is a hack? >> >> Because that's not how cleanup should be implemented in atf tests: you >> should be using the cleanup routines. The reason is that atexit(3) is >> not guaranteed to work: if your test crashes or is killed by Kyua >> because it overruns its deadline, your cleanup code won't work but a >> cleanup routine will. >> > > Skipped using atexit(3). > >> >> you already noticed that it's difficult to use Kyua's builtin cleanup >> >> capabilities because of the need to pass the value of oldmaxfiles. I >> >> too have experienced that frustration. Is there any way to pass >> >> values from the body of a testcase to its cleanup? Using >> >> atf_tc_set_md_var() would be one way, but the man page suggests that >> >> that function cannot be called from the body. Julio, is there a >> >> better way to do this? >> >> The "problem" is that the body and the cleanup run in different >> processes really, so there is no way you can pass state other than >> writing to a local file. >> >> This change: >> >> http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/kernel/t_sysv.c.diff?r1=1.3&r2=1.4&only_with_tag=MAIN >> >> shows a way to do it. >> >> Because these tests are dealing with scary global system state (which >> is a bad thing), I think that the hassle of having to do this cleanup >> dance is "kinda justified": not that many tests should need it and, >> for those that do, it becomes clear that maybe the test should be done >> in a different way. >> >> We could do better, yes; I don't disagree with that. Suggestions? >> >> > Now this raises an interesting question for me: Which environment do >> > you guys expect ATF to run in? If it is hosts like freefall, any >> > resource hogging tests are out of the question I would think. >> >> That's the interesting question. This test will probably be flaky >> and/or cause side-effects in the system while it runs because it >> relies on state that is out of control of the test. >> >> For the setup at kyua1.nyi.freebsd.org, this is not a problem because >> the tests run in a VM. But we want to encourage users to run the >> tests themselves and this kind of tests may be problematic. >> >> Here goes a possibility: add a configuration variable >> "allow_kernel_side_effects" or similar to kyua.conf >> (test_suites.FreeBSD.allow_kernel_side_effects = true). Then make the >> test case require the variable with atf_set_md_var("require.config", >> "allow_kernel_side_effects") so that this test only runs when the user >> has told Kyua to allow these tests. We can later change the >> kyua1.nyi.freebsd.org setup to define this variable but we leave it >> off by default for end users. > > Updated dif: http://people.freebsd.org/~pho/kern_descrip_test-v4.diff > > -- > Peter Be sure to add a 'atf_tc_set_md_var(tc, "require.config", "allow_sysctl_side_effects");' line to kern_maxfiles__increase's header. Using symlinks as a key-value store? Tricksy! I like it. And as tricky as this is, it probably deserves some explanatory comments. Restoring kern.maxfiles in the body of the test as well as the cleanup is redundant. I think it's ok to only do it in the cleanup. The "r != -1" check in openfiles2 at line 97 is tautological. There is no way for that test to fail, thanks to the preceeding ATF_REQUIRE. The RENDEZVOUS thing is a little hacky. You could replace it with named semaphores and that would eliminate 2 of the 3 the sleeps. I suppose you could also eliminate the 3rd sleep by replacing the SIGUSR1 handler and done variable with a different named semaphore. Alternatively, you could have openfiles2 sleep forever, and have openfiles send SIGTERM to all of its children when it's done. But RENDEZVOUS works ok, I guess. Check your whitespace. You've got leading spaces in kern_maxfiles__increase's head and in the ATF_TPADD_TCS() body. And trailing whitespace in "#define PARALLEL 4 ". Other style(9) violations include a magic number at line 162, a too-long line at line 180, and a return statement that doesn't enclose its argument in parens at line 193. Looks good otherwise, -Alan From owner-freebsd-testing@FreeBSD.ORG Fri Mar 14 22:04:32 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA21EA2B; Fri, 14 Mar 2014 22:04:32 +0000 (UTC) Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 320A6381; Fri, 14 Mar 2014 22:04:32 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id p61so2598257wes.41 for ; Fri, 14 Mar 2014 15:04:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=TJ7a2iTw/lIYsJfEVtth1815nHgtkZIs/vu89by6sgA=; b=b5KhTo8DesHmFen1gwywwuOncAOHP+nahEyvLSxB+FMLk7qzObmRyTEx6buEIN4pKc BqZjuU++R6GX+n7T+76WZMT1odecVWznrLJHS6ozqM8cGK5SjKJOt414rQ7blIdo9UDd A2BZLs4jg6epIg1+x6F4uefNzewtxKnxAowspLUaWGyi7Qlz1Iayd3hdF11JZmGl6KJO OS2Vw5dr92QS5BIMnEywxa6bwzIvY2CpTEYNTAOdVMHgCeRCrltrdoOnE1I7Xp7IhMJ1 HjERcRJYWIe0tu9RiwoY2NtJfZTBHbwNLxu3KSHbfZmFYlDYXwojNs2FGOaqGL7EyM5J hLZw== MIME-Version: 1.0 X-Received: by 10.194.63.236 with SMTP id j12mr8600875wjs.5.1394834669793; Fri, 14 Mar 2014 15:04:29 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Fri, 14 Mar 2014 15:04:29 -0700 (PDT) Date: Fri, 14 Mar 2014 16:04:29 -0600 X-Google-Sender-Auth: nbHoFTf03rWC7IIocXuXK5eEc6I Message-ID: Subject: ATF config variables for FreeBSD tests From: Alan Somers To: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 22:04:32 -0000 Another thread recently raised the issue of tests that cause potentially harmful changes to sysctl variables. We all agreed to conditionalize that test based on an ATF config variable. Now Ken and I both want to add tests with different harmful side effects. I propose the following list of config variables. If they sound good to everyone, I'll add them to tests(7) and to an example kyua.conf file in devel/kyua. test_suites.FreeBSD.allow_sysctl_side_effects If defined, enables tests that cause potentially harmful changes to globally relevant sysctl variables. Requested by Peter for kern_descrip_test:maxfiles__increase test_suites.FreeBSD.fibs Defined to a space delimited list of fib numbers. Tests that change routing tables must use these fibs instead of the default. They will be skipped if the variable is undefined. I'm working on such tests right now. test_suites.FreeBSD.disks Defined to a space delimited list of disk device nodes. Tests that format, mount, write to, disable, or have any other harmful effect on disks must use these devices. test_suites.FreeBSD.allow_devfs_side_effects. If defined, enables tests that can create or destroy semipermanent device nodes, like disk devices. Without this variable, tests may still create and destroy device nodes that are normally transient, like /dev/tap* and /dev/pts/*. This variable requires further explanation: Naively, one would think that test_suites.FreeBSD.disks would be sufficient protection for tests that create and destroy disk device nodes. However, Ken is working on a test that will cause hard drives to disappear and reappear in a different order, possibly resulting in their devnames changing. It could even result in "holes" appearing in the list of da unit numbers. If the sysadmin defines test_suites.FreeBSD.disks to "/dev/da1 /dev/da2 /dev/da3", it would be possible for those same devices to be named "/dev/da2 /dev/da1 /dev/da4" by the time Ken's test finishes. The permanent disappearance of /dev/da3 could confuse future tests that rely on test_suites.FreeBSD.disks. They would try to open /dev/da3 and fail. Thus, it may not be safe to run Ken's test in the same run as other tests that access disks. test_suites.FreeBSD.unit_tests test_suites.FreeBSD.functional_tests test_suites.FreeBSD.stress_tests These three variables would address a different concern of Peter's. He was concerned that his test might require too much run time, annoying other users of the rest of the test suite. By using these variables, a sysadmin could easily disable stress tests, which tend to take a very long time to run. Conversely, unit tests are normally very fast. A kyua run that only ran unit tests could easily be done after every buildworld, for example, while one that ran functional tests as well could be done perhaps several times per day. This is just an idea I have. I haven't tried it in practice. What do you think? -Alan From owner-freebsd-testing@FreeBSD.ORG Sat Mar 15 04:21:27 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B228BD8 for ; Sat, 15 Mar 2014 04:21:27 +0000 (UTC) Received: from mail-qc0-f169.google.com (mail-qc0-f169.google.com [209.85.216.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 17383A55 for ; Sat, 15 Mar 2014 04:21:27 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id i17so3905588qcy.14 for ; Fri, 14 Mar 2014 21:21:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=/oCT46l7wh8xQ2akpcfbmVH2T7mnQNOoh07WvtPT208=; b=XLyhcYBBdaTBBun4maHXYEWKev8KeebEJKZS6ig4uolYMLG2S11W8JP+ChyQoKjQpL n6EaxZ2Ic5W3bVwvKxN1pa+i6Y+WLqp754wY7wxd1paMgjc/dm+nvUHm9BK+/NHdlJoO tNt4pELbnSxVcNDMrFPL9vqKQNn2UgtVTKm9lcmXf6ug5JtpvlxWpVSxwRAT1gdqujwr +/zV6W/8tAtczmOZsTq7v3DjUHZAxczMoiqYL6GCYsWetevPWbkdD/+GZ1E+0Z3Er4zh sVQ3BSdCC3lxoD37X4PX5vKdNmzPOn90wlZZN7C9HnH1Ln1Y4Ig31datti+CxZsKj51k dJ2A== X-Gm-Message-State: ALoCoQlUT9I1AOjRbmfJp7vO64+Vi7p20+4hYlQ4wWutrIxgfO0nc88qDxYRhhBpFtdFpTq1V8dp X-Received: by 10.140.30.98 with SMTP id c89mr13498755qgc.13.1394857280098; Fri, 14 Mar 2014 21:21:20 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Fri, 14 Mar 2014 21:21:00 -0700 (PDT) X-Originating-IP: [180.42.49.96] In-Reply-To: References: From: Julio Merino Date: Sat, 15 Mar 2014 13:21:00 +0900 X-Google-Sender-Auth: 3BCJFPpG39kKPlf_c2UJw8Uwo_o Message-ID: Subject: Re: ATF config variables for FreeBSD tests To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 04:21:27 -0000 On Sat, Mar 15, 2014 at 7:04 AM, Alan Somers wrote: > Another thread recently raised the issue of tests that cause > potentially harmful changes to sysctl variables. We all agreed to > conditionalize that test based on an ATF config variable. Now Ken and > I both want to add tests with different harmful side effects. I > propose the following list of config variables. If they sound good to > everyone, I'll add them to tests(7) and to an example kyua.conf file > in devel/kyua. > > test_suites.FreeBSD.allow_sysctl_side_effects > If defined, enables tests that cause potentially harmful changes to > globally relevant sysctl variables. Requested by Peter for > kern_descrip_test:maxfiles__increase > > test_suites.FreeBSD.fibs > Defined to a space delimited list of fib numbers. Tests that change > routing tables must use these fibs instead of the default. They will > be skipped if the variable is undefined. I'm working on such tests > right now. > > test_suites.FreeBSD.disks > Defined to a space delimited list of disk device nodes. Tests that > format, mount, write to, disable, or have any other harmful effect on > disks must use these devices. > > test_suites.FreeBSD.allow_devfs_side_effects. > If defined, enables tests that can create or destroy semipermanent > device nodes, like disk devices. Without this variable, tests may > still create and destroy device nodes that are normally transient, > like /dev/tap* and /dev/pts/*. This variable requires further > explanation: [...] I think these all sound reasonable. Can these expected side-effects be reversed in the test cleanup? The descriptions above should clearly mention these details. For example, for sysctl-related tests I'd expect them to restore the original values just fine... but maybe that's not the case in the other examples you mention? > test_suites.FreeBSD.unit_tests > test_suites.FreeBSD.functional_tests > test_suites.FreeBSD.stress_tests > These three variables would address a different concern of Peter's. > He was concerned that his test might require too much run time, > annoying other users of the rest of the test suite. By using these > variables, a sysadmin could easily disable stress tests, which tend to > take a very long time to run. Conversely, unit tests are normally > very fast. A kyua run that only ran unit tests could easily be done > after every buildworld, for example, while one that ran functional > tests as well could be done perhaps several times per day. This is > just an idea I have. I haven't tried it in practice. > > What do you think? Doing this separation by sizes is important for the reasons you outline, but I have a different proposal that I have had in mind for a while and only recently started to work on. The idea is, simply, to introduce a "size" metadata variable to the test cases and then make every test declare if they are "small", "medium", "large", etc. With this, the user is allowed to customize, in kyua.conf, the timeouts for every test case size -- with some reasonable defaults for modern machines. (I've seen this implemented in other test suites and is an approach that works pretty well.) The reason to do it this way is to kill the "timeout" property, which is unreliable as it is not possible to make good predictions on run time when you are going to run the same test on multiple architectures. With this approach, we'd then say that qemu VMs have much longer timeouts for every class, for example. And lastly, we'd just need a simple "filtering" feature in the kyua cli to allow specifying which size of tests to run (or to filter by any other metadata property, for that matter). From owner-freebsd-testing@FreeBSD.ORG Sat Mar 15 16:32:42 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2F67A40; Sat, 15 Mar 2014 16:32:41 +0000 (UTC) Received: from mail-pd0-x230.google.com (mail-pd0-x230.google.com [IPv6:2607:f8b0:400e:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C31EAB27; Sat, 15 Mar 2014 16:32:41 +0000 (UTC) Received: by mail-pd0-f176.google.com with SMTP id r10so3810233pdi.7 for ; Sat, 15 Mar 2014 09:32:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0danhyGGgylQaB9OWfFKPLEpUIHkhB9ZBWinYh9j4lA=; b=V6OlriRnkg3cAGTAKMR0gytDMWr1iUFkfFB+7H1xcRvWqyscjPWDBzw+kqZNDDLZ1u 7Pv54+dK8hwE3B+m8xzHzwa6gOg+A62gltakOUMjHwm8+b9MZ9GX5oUXJcvTTndQhdXV rKgxDxwiSIH18WtPueftZdTgMZtaAjQh74bOGoq4wfByNK6E70Jaqrqc+etUMk+hXbQR XFIIXiZbQKZ4MOAPMMEAGQeUYC42TfwtaZC60QsHoOEediz4c18y5snflJl//zHYE9i/ 9rZwVt9ia/6f6CjaV+14wIJaI/sQysenPLEFSmb2IFVAnTlSfsjbTSd0jDGxclScDHgv aD5Q== MIME-Version: 1.0 X-Received: by 10.66.141.144 with SMTP id ro16mr15587808pab.131.1394901161147; Sat, 15 Mar 2014 09:32:41 -0700 (PDT) Received: by 10.68.63.38 with HTTP; Sat, 15 Mar 2014 09:32:41 -0700 (PDT) In-Reply-To: References: Date: Sat, 15 Mar 2014 17:32:41 +0100 Message-ID: Subject: Re: GSoC Idea: Working on modernizing the tests of ATF From: dpl To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 16:32:42 -0000 I'm posting here in case some student sees this and want to create a proposal for this. I'm not finally going to write a proposal for this project. So, please, go ahead. :) 2014-03-13 3:01 GMT+01:00 Julio Merino : > On Thu, Mar 13, 2014 at 7:10 AM, tuchalia wrote: > > Hi there, > > > > I'm a student willing to do this work. > > I'm currently interested in a few projects, and I really like this one as > > well. > > > > Where should I start looking to write a strong proposal? > > Hello, > > You should start by looking at the information in the TestSuite wiki > page. Try to get familiar with the examples referenced from there: > look for the code, read it, try to do some edits, etc. > > Then you can also dive into the code in tools/regression/ and > tools/test/ to see what would need to be migrated. Here, you can use > the prove(1) tool to run the tests that end in .t, so you could give > that a try and see how that works as well. > > And, lastly, you can also look at the NetBSD source tree under the > src/tests/ directory[1] or at Garret's GitHub repository[2]. There are > many test programs there that can be easily made to run on FreeBSD. > Getting one of those, hooking it into the build and preparing a patch > for that would be a nice exercise. > > Hope this helps. > > 1: http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/?only_with_tag=MAIN > 2: https://github.com/yaneurabeya/freebsd > -- Daniel From owner-freebsd-testing@FreeBSD.ORG Sat Mar 15 17:03:15 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9A80B42 for ; Sat, 15 Mar 2014 17:03:15 +0000 (UTC) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8E7C3DEB for ; Sat, 15 Mar 2014 17:03:15 +0000 (UTC) Received: by mail-ob0-f178.google.com with SMTP id wp18so3868595obc.23 for ; Sat, 15 Mar 2014 10:03:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=BVmPVBLDvCA/NkhFdYq7QdMr69E9wYm4moJ0eJpkqI4=; b=bRbDYu++H8gdwg4xBXv1uTlXejwGcUpWaQKxrV4gnIpCtRey5Dty5oByxlTE0AwN0G rhlSBLGo/BpuhU27uhdCZY2FnoEpuPywBKQrowGA3EWTBwDeEp4v1z9TOenWIg8ReBxq RVj4Rq3NXX3azccQuZRlZD2Pgtaaj9nmQUEVnBifBTCPnZHNvsiS8PAt/qroyxEmiLVQ S4JNGtEkxy9lK4rlfRwohSznhWPRVjrIz6DWgDJaYUTmbm2rFDccGD1RwubER4njYbh7 aGXRBaTistZVZz7je00cpqd1uHe7sGNdW4QI+0wOZFmm5qc6yjxlmmt0z8r8t6q32+vd u4VA== MIME-Version: 1.0 X-Received: by 10.60.15.131 with SMTP id x3mr12775946oec.15.1394902994921; Sat, 15 Mar 2014 10:03:14 -0700 (PDT) Received: by 10.182.118.6 with HTTP; Sat, 15 Mar 2014 10:03:14 -0700 (PDT) In-Reply-To: References: Date: Sat, 15 Mar 2014 19:03:14 +0200 Message-ID: Subject: Re: GSoC Idea: Working on modernizing the tests of ATF From: Anastasios Mag To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 17:03:15 -0000 Hi Julio, currently i have a problem installing kyua. In installation instructions it guides me to run "./configure" but configure files aren't executable. What to do ? On Sat, Mar 15, 2014 at 6:32 PM, dpl wrote: > I'm posting here in case some student sees this and want to create a > proposal for this. > I'm not finally going to write a proposal for this project. So, please, go > ahead. :) > > > 2014-03-13 3:01 GMT+01:00 Julio Merino : > > > On Thu, Mar 13, 2014 at 7:10 AM, tuchalia wrote: > > > Hi there, > > > > > > I'm a student willing to do this work. > > > I'm currently interested in a few projects, and I really like this one > as > > > well. > > > > > > Where should I start looking to write a strong proposal? > > > > Hello, > > > > You should start by looking at the information in the TestSuite wiki > > page. Try to get familiar with the examples referenced from there: > > look for the code, read it, try to do some edits, etc. > > > > Then you can also dive into the code in tools/regression/ and > > tools/test/ to see what would need to be migrated. Here, you can use > > the prove(1) tool to run the tests that end in .t, so you could give > > that a try and see how that works as well. > > > > And, lastly, you can also look at the NetBSD source tree under the > > src/tests/ directory[1] or at Garret's GitHub repository[2]. There are > > many test programs there that can be easily made to run on FreeBSD. > > Getting one of those, hooking it into the build and preparing a patch > > for that would be a nice exercise. > > > > Hope this helps. > > > > 1: http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/?only_with_tag=MAIN > > 2: https://github.com/yaneurabeya/freebsd > > > > > > -- > Daniel > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" > -- Building the kernel...! From owner-freebsd-testing@FreeBSD.ORG Sat Mar 15 20:33:37 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7701F3F6; Sat, 15 Mar 2014 20:33:37 +0000 (UTC) Received: from mail-we0-x232.google.com (mail-we0-x232.google.com [IPv6:2a00:1450:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8C7621D8; Sat, 15 Mar 2014 20:33:36 +0000 (UTC) Received: by mail-we0-f178.google.com with SMTP id u56so3254491wes.37 for ; Sat, 15 Mar 2014 13:33:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=PLEih5FhNMWiFasxt47/VMf68ZFC6mjnPCytZpzpzdQ=; b=Z+VF5DiKMcA25BTgp6CCquRhkvJCB9jmcdpPdPo/J1UKxZKUNtdvkHrZbeIjOK2pYs yMEvG4RHgOfBXteWx6urtmG+NTkZFzmu/JHkX7pvnyoZJ5Gap6mxZHpyK9xPaps/Quah j7C38h7JhJe9Ck+6ach+f0hPQ31lJZ9UYK3lSjlSG3u3hGpB4JBEz9z5UIbbbH+F5L2I naumGyoe+UsDNV3604Y8xDpa6wRnUM/8MD9bmguoNJk33C34wUSiuHrJIpr00E3KkWRV NJxnEkjgqbJxMj7uO1woRZxWieXUFfYt4ODVLl82mxxPP/OWm2LOQ+oxKpuFZ54cobxm Kn9g== MIME-Version: 1.0 X-Received: by 10.180.73.19 with SMTP id h19mr3474079wiv.40.1394915615059; Sat, 15 Mar 2014 13:33:35 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.197 with HTTP; Sat, 15 Mar 2014 13:33:34 -0700 (PDT) In-Reply-To: References: Date: Sat, 15 Mar 2014 14:33:35 -0600 X-Google-Sender-Auth: NNmlc0xG_1BzXD2c__tdDz0fMUo Message-ID: Subject: Re: ATF config variables for FreeBSD tests From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 20:33:37 -0000 On Fri, Mar 14, 2014 at 10:21 PM, Julio Merino wrote: > On Sat, Mar 15, 2014 at 7:04 AM, Alan Somers wrote: >> Another thread recently raised the issue of tests that cause >> potentially harmful changes to sysctl variables. We all agreed to >> conditionalize that test based on an ATF config variable. Now Ken and >> I both want to add tests with different harmful side effects. I >> propose the following list of config variables. If they sound good to >> everyone, I'll add them to tests(7) and to an example kyua.conf file >> in devel/kyua. >> >> test_suites.FreeBSD.allow_sysctl_side_effects >> If defined, enables tests that cause potentially harmful changes to >> globally relevant sysctl variables. Requested by Peter for >> kern_descrip_test:maxfiles__increase >> >> test_suites.FreeBSD.fibs >> Defined to a space delimited list of fib numbers. Tests that change >> routing tables must use these fibs instead of the default. They will >> be skipped if the variable is undefined. I'm working on such tests >> right now. >> >> test_suites.FreeBSD.disks >> Defined to a space delimited list of disk device nodes. Tests that >> format, mount, write to, disable, or have any other harmful effect on >> disks must use these devices. >> >> test_suites.FreeBSD.allow_devfs_side_effects. >> If defined, enables tests that can create or destroy semipermanent >> device nodes, like disk devices. Without this variable, tests may >> still create and destroy device nodes that are normally transient, >> like /dev/tap* and /dev/pts/*. This variable requires further >> explanation: > [...] > > I think these all sound reasonable. > > Can these expected side-effects be reversed in the test cleanup? For the first two, yes. I doubt that Kyua could do it automatically, but each test case certainly can. Perhaps even for the fourth, though it would be tricky. But there's no way to reverse the effects for the third. The ZFS tests, for example, create and destroy zpools. How would you reverse that? You can't. Whatever was previously on the disk is gone. > > The descriptions above should clearly mention these details. For > example, for sysctl-related tests I'd expect them to restore the > original values just fine... but maybe that's not the case in the > other examples you mention? > >> test_suites.FreeBSD.unit_tests >> test_suites.FreeBSD.functional_tests >> test_suites.FreeBSD.stress_tests >> These three variables would address a different concern of Peter's. >> He was concerned that his test might require too much run time, >> annoying other users of the rest of the test suite. By using these >> variables, a sysadmin could easily disable stress tests, which tend to >> take a very long time to run. Conversely, unit tests are normally >> very fast. A kyua run that only ran unit tests could easily be done >> after every buildworld, for example, while one that ran functional >> tests as well could be done perhaps several times per day. This is >> just an idea I have. I haven't tried it in practice. >> >> What do you think? > > Doing this separation by sizes is important for the reasons you > outline, but I have a different proposal that I have had in mind for a > while and only recently started to work on. > > The idea is, simply, to introduce a "size" metadata variable to the > test cases and then make every test declare if they are "small", > "medium", "large", etc. With this, the user is allowed to customize, > in kyua.conf, the timeouts for every test case size -- with some > reasonable defaults for modern machines. (I've seen this implemented > in other test suites and is an approach that works pretty well.) > > The reason to do it this way is to kill the "timeout" property, which > is unreliable as it is not possible to make good predictions on run > time when you are going to run the same test on multiple > architectures. With this approach, we'd then say that qemu VMs have > much longer timeouts for every class, for example. This sounds good, as long as test authors have good guidelines on how to appraise their tests' sizes. > > And lastly, we'd just need a simple "filtering" feature in the kyua > cli to allow specifying which size of tests to run (or to filter by > any other metadata property, for that matter). I don't like this idea. We tried it at work, and it didn't work out very well. Basically, I automatically assigned sizes (short, medium, long) to all of our tests based on their runtimes, so users could select to only run the short or medium tests on the bench. The problem is that the classification didn't make sense. For an expedited test run, you don't want the shortest tests; what you want are the tests with the most value per unit time. Unit tests have a high value and a very short runtime. Stress tests have a very long runtime and arguably low value since they don't always have consistent results. At my previous job there was a large department whose duties included curating test suites and deciding which tests would be included in the short runs. FreeBSD isn't going to have that, but we could still ask test authors to classify tests along the lines that I suggested. -Alan From owner-freebsd-testing@FreeBSD.ORG Sun Mar 16 00:49:39 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C25977F7 for ; Sun, 16 Mar 2014 00:49:39 +0000 (UTC) Received: from mail-qg0-f46.google.com (mail-qg0-f46.google.com [209.85.192.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7A7858BF for ; Sun, 16 Mar 2014 00:49:39 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id e89so12259232qgf.5 for ; Sat, 15 Mar 2014 17:49:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=gQwCcxawyts8g61fj+/emWXwFqLvyfjdKYiHTBvt5H0=; b=DR2SOc9hKFmzlRUZN0V3Ab+1jSrkTFRTTd/8AUxZMVWWQ2ZecqW3WlxRH5n07lnMzb nYtL9UEG4robQO1N8vh3Cgg3fwG7kw0IXIAKK9tgQ+v+hOStHFQYm98Ns5TDSUGSQSa2 oEBh3BOE1SX9Q/elbygClKKs/CdEprz1dOzxWNr3we4DT8T9K3Q9ij2dJ0EbuA13ibhb 2ECSnh/KTT5E8G74fjLhbAZfXtnh5ECKs9XiP/yAKbs4TqPSveJ1hn96TlagunRNlETZ eiqte7zZ8yBm8Ri9OVfcXO+tGQuKpbGwjhC61iQZ5DA6xR5BxosRHcsf1NPZikyG8VQf 26jQ== X-Gm-Message-State: ALoCoQl9EdEHfbxHtTGvWg+Sey5Ikhr8HuFGfoiCYD7+gLyr/DFKmeYqPuMLD+Ae1nviQIImasQh X-Received: by 10.224.137.202 with SMTP id x10mr19224qat.80.1394930972238; Sat, 15 Mar 2014 17:49:32 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sat, 15 Mar 2014 17:49:11 -0700 (PDT) X-Originating-IP: [180.42.49.96] In-Reply-To: References: From: Julio Merino Date: Sun, 16 Mar 2014 09:49:11 +0900 X-Google-Sender-Auth: jnvFkdmNPRsLnkY65Uzw4s1pbdI Message-ID: Subject: Re: ATF config variables for FreeBSD tests To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 00:49:39 -0000 On Sun, Mar 16, 2014 at 5:33 AM, Alan Somers wrote: > On Fri, Mar 14, 2014 at 10:21 PM, Julio Merino wrote: [...] >> I think these all sound reasonable. >> >> Can these expected side-effects be reversed in the test cleanup? > > For the first two, yes. I doubt that Kyua could do it automatically, > but each test case certainly can. Right. I do not think Kyua should get into these tricky details either, if only because they are too OS-specific. But we can certainly offer additional scripts/functions (where it makes sense) in the FreeBSD test suite to simplify the test cases that might need this. > Perhaps even for the fourth, though > it would be tricky. But there's no way to reverse the effects for the > third. The ZFS tests, for example, create and destroy zpools. How > would you reverse that? You can't. Whatever was previously on the > disk is gone. Ah, I guess I missed that detail. Couldn't those tests run on top of vnd devices though? [...] >> And lastly, we'd just need a simple "filtering" feature in the kyua >> cli to allow specifying which size of tests to run (or to filter by >> any other metadata property, for that matter). > > I don't like this idea. I hope you are objecting to the filtering by test sizes, not the filtering itself! See below. > We tried it at work, and it didn't work out > very well. Basically, I automatically assigned sizes (short, medium, > long) to all of our tests based on their runtimes, so users could > select to only run the short or medium tests on the bench. The > problem is that the classification didn't make sense. For an > expedited test run, you don't want the shortest tests; what you want > are the tests with the most value per unit time. Unit tests have a > high value and a very short runtime. Stress tests have a very long > runtime and arguably low value since they don't always have consistent > results. > > At my previous job there was a large department whose duties included > curating test suites and deciding which tests would be included in the > short runs. FreeBSD isn't going to have that, but we could still ask > test authors to classify tests along the lines that I suggested. That's a good point. If I understand you correctly, what is important is to manually curate the set of "smoke tests" that run very quickly and that provide a reasonably high assurance that major subsystems haven't broken. We can do this too pretty easily, but configuration variables are the wrong mechanism. We can already define this information in metadata properties as long as you prefix them with X-. So all we'd need is a way to tell Kyua to only run tests marked with such property (i.e. with the filtering feature described above). From owner-freebsd-testing@FreeBSD.ORG Sun Mar 16 01:07:52 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47CDBC5F for ; Sun, 16 Mar 2014 01:07:52 +0000 (UTC) Received: from mail-qc0-f170.google.com (mail-qc0-f170.google.com [209.85.216.170]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 01DDDA1D for ; Sun, 16 Mar 2014 01:07:51 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id e9so4653842qcy.1 for ; Sat, 15 Mar 2014 18:07:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=zyA++DgDWOOQM11Pgd/AXLDtLX1R0A/TaBQMSnbA/yI=; b=Z3FAflaMuTB5BLQSC54mIpUa4GS7ymmf1Qxo3hLSdKICjQgTqFFOAQPF3j78Cbdkfy bWN6rUajtLItrYvg2nKpU8T1lrtxZ00JPjK1EXufr36DyrgQ3FLBoEr9W9ir0V9wnvCO bAMEuEyZQ2q6L1f22tBn1KalCZDRYVSKYafby0CELuzcD0hz1VIYNKk40D28+Tl4ckao de1wPnRlS9vMUyBJhyRF9m56fiZ9BgrHUmK7OV+zZlVKwAPMROKrXsWuA1rGTe9xBLq3 ATOMmHKrauvQ2jqg7eGaiHKsbjTnA3K0f/SdUSCAVk9NcOa0LuRDN7IKnENXbeugOcXS fI+w== X-Gm-Message-State: ALoCoQnoDkNs41pQ9e3dG9FMUsgKZ96zYL4LxbSQatgUUse+6GKrkAkNgI8WiEAKyJIy8Wcw1JAW X-Received: by 10.224.57.81 with SMTP id b17mr19379522qah.44.1394930398137; Sat, 15 Mar 2014 17:39:58 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Sat, 15 Mar 2014 17:39:38 -0700 (PDT) X-Originating-IP: [180.42.49.96] In-Reply-To: References: From: Julio Merino Date: Sun, 16 Mar 2014 09:39:38 +0900 X-Google-Sender-Auth: QeUFVvwcHo1Ddbdula8ctHaiRAY Message-ID: Subject: Re: GSoC Idea: Working on modernizing the tests of ATF To: Anastasios Mag Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 01:07:52 -0000 On Sun, Mar 16, 2014 at 2:03 AM, Anastasios Mag wrote: > Hi Julio, currently i have a problem installing kyua. > In installation instructions it guides me to run "./configure" but > configure files aren't executable. > What to do ? What do you mean? Please paste the actual output of what you are seeing. Note that you can just use "pkg install kyua" to get kyua installed on your FreeBSD machine. No need to build from source if you don't want to! From owner-freebsd-testing@FreeBSD.ORG Sun Mar 16 04:39:06 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B154FF for ; Sun, 16 Mar 2014 04:39:06 +0000 (UTC) Received: from mail-qc0-f180.google.com (mail-qc0-f180.google.com [209.85.216.180]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 06E52C50 for ; Sun, 16 Mar 2014 04:39:05 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id x3so4676555qcv.11 for ; Sat, 15 Mar 2014 21:38:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=OfdKYyZ3NiQJ16VWAJesRcxbqZXvEhxoSIRiE7zbbQU=; b=Vsz5zXZ4eNrZn/7Sl4R3KX8mzbO9rxGpdyK7Wi1R/S9HVa2Hio7DuUR6ag0bhjZ9fF 5Lmzv71QLAelRGn3npMDanMWq7wxP0jrZsIRalVzBZ/WOSWYHIoh87Wggg67YLOMl3RW GTWUCeNGV3rCNLxC9SGaXmLxnxui/dE9lFLO0jkDhSkY+G4Q+flHpFs0dp6FQuhjcN2K n0d6gkEve2o9vvob0ZaageqBfBSN2qJ02g2x4CNnaAjqfbk2elNI6D/Eoszgn44jLM8w f82jaM8cKjZWrkEUPnAd8C4tAKtsYVeDmdmt/vjIQXoD1SspQT4Zuy/JGsVH1TArBFY0 4P7A== X-Gm-Message-State: ALoCoQnydVit2IMyShdAfYdi9lGLbAl4637cEHDI5v5ulm78IvO0ksruWaevgv9HmMUlcpB2s3jC X-Received: by 10.140.40.5 with SMTP id w5mr18913013qgw.65.1394944739330; Sat, 15 Mar 2014 21:38:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.83.102 with HTTP; Sat, 15 Mar 2014 21:38:39 -0700 (PDT) X-Originating-IP: [180.42.49.96] In-Reply-To: References: From: Julio Merino Date: Sun, 16 Mar 2014 13:38:39 +0900 Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? To: kyua-discuss@googlegroups.com Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 04:39:06 -0000 On Tue, Mar 4, 2014 at 6:27 AM, Craig Rodrigues wrote: > On Sun, Mar 2, 2014 at 12:26 PM, Julio Merino wrote: >> >> >> (As I mentioned in the other thread) this is relatively easy to do and >> we have enough time to get it going by May 15th. >> >> The path of least resistance to get this out for now is: >> >> 1. Reintroduce support for --output=format:path to 'report' so that >> 'report' can output more than one format. The existence of >> 'report-html' as a separate command is a pretty ugly hack. >> >> 2. Split out the console-specific formatting from cmd_report.cpp into >> a separate module to clearly expose what is part of the formatting and >> what is not. Adding JUnit XML support should not require changes to >> other code. Alright; these two are now done in github. The code is not very pretty (OK, not pretty at all) but it should now be possible to relatively easily bolt on JUnit XML formatting code. Still have to integrate HTML output generation into the new code structure to ensure that this is true. >> 3. Add a new module with the JUnit XML formatting code. > > I think that these are all fine goals. > I might be able to help with 3. If you want to try that's cool, but if you can provide me some sample outputs and a pointer to the schema I will do this sometime soon. > The easier we make it to customize Kyua's output formats, the easier it will > be > to get more people to use Kyua. Indeed. But let's focus on BSDCan for now ;-) From owner-freebsd-testing@FreeBSD.ORG Sun Mar 16 04:58:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01383229; Sun, 16 Mar 2014 04:58:14 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4ED9FD8C; Sun, 16 Mar 2014 04:58:13 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so2853138lan.12 for ; Sat, 15 Mar 2014 21:58:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=CQzoTyhLkNjPQVHnV5vnrfGzOHCqkZ2qtdSQy9nl4+M=; b=fqyFkKVKyCEqVm19gs6c7urOFX6/DK4g9FbROriK9yFcqQSRS6Gb81uSVAt9LcNR9W fQRilphNUr4ph6DwqGpTXLlbV4jyOar70l41dQ1fnTYJ0VDIW0etfYX/yG++g+CdFLgS ExjvSET91nbO2lwpYTVeuk3wqUzrFnZYa5VsvhMaOgeWPM3r44nYHTQjSNJGa4mB3VPI sLXkG+ZBrbYz+3DoFIupr/g20fEkBg7KpaVGlbgJcC26Tkk6bp+6N3X2UCO4B5XHLRwZ A+XL1/werWr8CfXD6Va13HYI8uSYAiBlhV2gBTUsrnnWxnvPzek6PYfAMoJaANgM13jF 5eHA== MIME-Version: 1.0 X-Received: by 10.152.170.137 with SMTP id am9mr11971748lac.15.1394945891255; Sat, 15 Mar 2014 21:58:11 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Sat, 15 Mar 2014 21:58:11 -0700 (PDT) In-Reply-To: References: Date: Sat, 15 Mar 2014 21:58:11 -0700 X-Google-Sender-Auth: GcXnIgEbJPrdttuUN4JH6g6R5L0 Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: kyua-discuss@googlegroups.com Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 04:58:14 -0000 On Sat, Mar 15, 2014 at 9:38 PM, Julio Merino wrote: >> >>> 3. Add a new module with the JUnit XML formatting code. >> >> I think that these are all fine goals. >> I might be able to help with 3. > > If you want to try that's cool, but if you can provide me some sample > outputs and a pointer to the schema I will do this sometime soon. Nice! Sounds like you are off to the races. Are these links enough to get you started? http://nose2.readthedocs.org/en/latest/plugins/junitxml.html https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd http://windyroad.com.au/dl/Open%20Source/JUnit.xsd Once you start generating JUnit XML output, it is quite easy to test this in Jenkins by doing (under FreeBSD): (1) pkg install devel/jenkins (2) add jenkins_enable="YES" to /etc/rc.conf (3) service jenkins start (4) import the XML via the web interface: http://blog.zuehlke.com/wp-content/uploads/2013/10/configure_unit_test_results-755x266.png If you don't want to spend time on setting up Jenkins, you can certainly send any sample XML outputs to jenkins-admin@freebsd.org and we can test those for you. -- Craig -- Craig From owner-freebsd-testing@FreeBSD.ORG Sun Mar 16 15:56:57 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1221DEE6; Sun, 16 Mar 2014 15:56:57 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 29BCF807; Sun, 16 Mar 2014 15:56:56 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id m15so3792641wgh.27 for ; Sun, 16 Mar 2014 08:56:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ytTKLYPhs8HOtoVzN6ggFErkPQrZbV0AN+/IY94BrhE=; b=rbEb0tGqZe1X/oUKgxSWByRWyl2/Q2J8qRd6VHJyEhgi9l6ACtiyYTzfaxO3+sTSz4 dnPNOqvT4gWFT7TlLHLnrAvc2vJzkkt2vIJWbwyNPTcBRpr4YO7zXTtVaxLEvHy3hcsH sJxDeMd2VwON+Z101jZctYbdrlsvex0FW1C37Llr61i+G0LMq17pF1Hsm2w7fNJfWcTo HM5eBSGx58Pj0WZbsZZ0QuHBJ7Crbag9V6Ij1I+7ul0IkU8CgUeBYGBeu2mO7q6BcDgh Msc2Sm03x1//HWxTH9gRhCx+pq2PxMBlfE58XLJUJbphFxyMBGgch1uU35X4YuYNZhuE yF1w== MIME-Version: 1.0 X-Received: by 10.180.106.167 with SMTP id gv7mr6255229wib.40.1394985414558; Sun, 16 Mar 2014 08:56:54 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Sun, 16 Mar 2014 08:56:54 -0700 (PDT) In-Reply-To: References: Date: Sun, 16 Mar 2014 09:56:54 -0600 X-Google-Sender-Auth: dhHCmejZiFk5mFKPmpUC0zLz9X0 Message-ID: Subject: Re: ATF config variables for FreeBSD tests From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 15:56:57 -0000 On Sat, Mar 15, 2014 at 6:49 PM, Julio Merino wrote: > On Sun, Mar 16, 2014 at 5:33 AM, Alan Somers wrote: >> On Fri, Mar 14, 2014 at 10:21 PM, Julio Merino wrote: > [...] >>> I think these all sound reasonable. >>> >>> Can these expected side-effects be reversed in the test cleanup? >> >> For the first two, yes. I doubt that Kyua could do it automatically, >> but each test case certainly can. > > Right. I do not think Kyua should get into these tricky details > either, if only because they are too OS-specific. But we can certainly > offer additional scripts/functions (where it makes sense) in the > FreeBSD test suite to simplify the test cases that might need this. Definitely. This is a good case for a library of reusable test code. For example, I have two separate test programs that would like to share setup and cleanup code for FIBs. Should I put it in /usr/tests/include/fibs.sh or /usr/tests/lib/fibs.sh ? I don't like those options, because it looks like fibs.sh is a test program designed to test stuff in /include or /lib. Perhaps /usr/tests/tests/sh/include/fibs.sh? I'm really not sure where the best place would be. What do you think? > >> Perhaps even for the fourth, though >> it would be tricky. But there's no way to reverse the effects for the >> third. The ZFS tests, for example, create and destroy zpools. How >> would you reverse that? You can't. Whatever was previously on the >> disk is gone. > > Ah, I guess I missed that detail. > > Couldn't those tests run on top of vnd devices though? FreeBSD doesn't have vnd(4), though it has md(4), which is similar. md(4) devices could be used for some ZFS tests. as it happens, ZFS can also use file-backed vdevs. But these workarounds don't work in all cases. Sometimes you need real disks: 1) Using file-backed vdevs doesn't exercise vdev_geom.c, where I've one a lot of work. 2) Neither file-backed vdevs nor md(4) devices have physical path information, which is needed to test some hotspare functionality. Only da(4) devices that are attached to ses(4) expanders have that. 3) There is no way to remove an active file-backed vdev, and I don't think that you can destroy an in-use md(4) device either. Therefore, to test how ZFS handles drive removals requires real drives. 4) gibbs has made many tweaks to ZFS to better support 4K and 8K sector drives. There is no way to emulate those with file-backed vdevs. Perhaps it could be done with md(4) and gnop(8), but I've never tried. 5) pjd and Steve Hartland have been working on TRIM support. That's not supported by file-backed vdevs, and I doubt that md(4) supports it either. 6) Copy-on-write on top of copy-on-write is very slow. If your system uses ZFS root, then both file-backed vdevs and file-backed md(4) devices are doing COW-on-COW. Using physical disks for the tests greatly speeds the tests' runtimes. Ideally, the ZFS tests would use test_suites.FreeBSD.disks if it is defined, and use file-backed vdevs otherwise. Tests that absolutely require physical disks would be skipped if test_suites.FreeBSD.disks isn't defined. It would take some work, but it's doable. > > [...] >>> And lastly, we'd just need a simple "filtering" feature in the kyua >>> cli to allow specifying which size of tests to run (or to filter by >>> any other metadata property, for that matter). >> >> I don't like this idea. > > I hope you are objecting to the filtering by test sizes, not the > filtering itself! See below. > >> We tried it at work, and it didn't work out >> very well. Basically, I automatically assigned sizes (short, medium, >> long) to all of our tests based on their runtimes, so users could >> select to only run the short or medium tests on the bench. The >> problem is that the classification didn't make sense. For an >> expedited test run, you don't want the shortest tests; what you want >> are the tests with the most value per unit time. Unit tests have a >> high value and a very short runtime. Stress tests have a very long >> runtime and arguably low value since they don't always have consistent >> results. >> >> At my previous job there was a large department whose duties included >> curating test suites and deciding which tests would be included in the >> short runs. FreeBSD isn't going to have that, but we could still ask >> test authors to classify tests along the lines that I suggested. > > That's a good point. If I understand you correctly, what is important > is to manually curate the set of "smoke tests" that run very quickly > and that provide a reasonably high assurance that major subsystems > haven't broken. > > We can do this too pretty easily, but configuration variables are the > wrong mechanism. We can already define this information in metadata > properties as long as you prefix them with X-. So all we'd need is a > way to tell Kyua to only run tests marked with such property (i.e. > with the filtering feature described above). I was assuming that a test case could put in its header 'require.config "stress_tests"', or something like that. Then runtime filtering would work by which variables the user defines. That way, no new work would be required in Kyua. But it sounds like you're talking about something different. IIUC, you're suggesting that a test case would say in its header 'atf_set "X-runtime" "stress_test"' and then some TBD filtering mechanism in Kyua selects on it. That would certainly be more intuitive. Is that what you had in mind? -Alan From owner-freebsd-testing@FreeBSD.ORG Sun Mar 16 16:53:44 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DF4AE22; Sun, 16 Mar 2014 16:53:44 +0000 (UTC) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [IPv6:2607:f8b0:400e:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4BA3EC87; Sun, 16 Mar 2014 16:53:44 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id bj1so4736877pad.2 for ; Sun, 16 Mar 2014 09:53:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=sI9rv95pwKNcCdPNNJmDLmDI4KduVGQH/cWAuZPmwhw=; b=KPFA9rsTTHo2jhHHoRW885o8Io0DNySr9qcfsNySJtDKsrXrhcE8HvsZ1mpnQCwomN efIjAs4GQPBK6gdSQJna7QzF6YmViPs71GuROQMrz4GUWJvgn87PupzrrIIz088p9+Uq J52K3zRBtJBD9RposJtbJFlsEjshZP+hwnmxh33uX5Zxa0rP07F1fFluntM3rX7HCIMe fWIdKZpziVMtHW5IQh9zXFZmnSWmZXS0aHyrI7XkzGs+ds7JHgFPCFxLPnZ3tMr2Ksej 18fQudU4g1b/UjiFhHyhWNUwEKP5fyJYD+OL5HDj+6jfEszdz8rw4Yk8J4DslQOv3sg/ TZ+w== X-Received: by 10.68.201.67 with SMTP id jy3mr21134006pbc.20.1394988823938; Sun, 16 Mar 2014 09:53:43 -0700 (PDT) Received: from [192.168.20.11] (c-24-17-226-153.hsd1.wa.comcast.net. [24.17.226.153]) by mx.google.com with ESMTPSA id dk1sm35711721pbc.46.2014.03.16.09.53.42 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 16 Mar 2014 09:53:42 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <6BADC2BD-3C99-44F7-B6FB-6139AB8752D9@gmail.com> X-Mailer: iPhone Mail (11B651) From: Garrett Cooper Subject: Re: ATF config variables for FreeBSD tests Date: Sun, 16 Mar 2014 09:53:42 -0700 To: Alan Somers Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 16:53:44 -0000 > On Mar 16, 2014, at 8:56, Alan Somers wrote: > >> On Sat, Mar 15, 2014 at 6:49 PM, Julio Merino wrote: >>> On Sun, Mar 16, 2014 at 5:33 AM, Alan Somers wrote: >>> On Fri, Mar 14, 2014 at 10:21 PM, Julio Merino wrote: >> [...] >>>> I think these all sound reasonable. >>>> >>>> Can these expected side-effects be reversed in the test cleanup? >>> >>> For the first two, yes. I doubt that Kyua could do it automatically, >>> but each test case certainly can. >> >> Right. I do not think Kyua should get into these tricky details >> either, if only because they are too OS-specific. But we can certainly >> offer additional scripts/functions (where it makes sense) in the >> FreeBSD test suite to simplify the test cases that might need this. > > Definitely. This is a good case for a library of reusable test code. > For example, I have two separate test programs that would like to > share setup and cleanup code for FIBs. Should I put it in > /usr/tests/include/fibs.sh or /usr/tests/lib/fibs.sh ? I don't like > those options, because it looks like fibs.sh is a test program > designed to test stuff in /include or /lib. Perhaps > /usr/tests/tests/sh/include/fibs.sh? I'm really not sure where the > best place would be. What do you think? /usr/tests/libexec seems logical.. >> >>> Perhaps even for the fourth, though >>> it would be tricky. But there's no way to reverse the effects for the >>> third. The ZFS tests, for example, create and destroy zpools. How >>> would you reverse that? You can't. Whatever was previously on the >>> disk is gone. >> >> Ah, I guess I missed that detail. >> >> Couldn't those tests run on top of vnd devices though? > > FreeBSD doesn't have vnd(4), though it has md(4), which is similar. > md(4) devices could be used for some ZFS tests. as it happens, ZFS > can also use file-backed vdevs. But these workarounds don't work in > all cases. Sometimes you need real disks: > 1) Using file-backed vdevs doesn't exercise vdev_geom.c, where I've > one a lot of work. > 2) Neither file-backed vdevs nor md(4) devices have physical path > information, which is needed to test some hotspare functionality. > Only da(4) devices that are attached to ses(4) expanders have that. > 3) There is no way to remove an active file-backed vdev, and I don't > think that you can destroy an in-use md(4) device either. Therefore, > to test how ZFS handles drive removals requires real drives. > 4) gibbs has made many tweaks to ZFS to better support 4K and 8K > sector drives. There is no way to emulate those with file-backed > vdevs. Perhaps it could be done with md(4) and gnop(8), but I've > never tried. > 5) pjd and Steve Hartland have been working on TRIM support. That's > not supported by file-backed vdevs, and I doubt that md(4) supports it > either. > 6) Copy-on-write on top of copy-on-write is very slow. If your system > uses ZFS root, then both file-backed vdevs and file-backed md(4) > devices are doing COW-on-COW. Using physical disks for the tests > greatly speeds the tests' runtimes. > > Ideally, the ZFS tests would use test_suites.FreeBSD.disks if it is > defined, and use file-backed vdevs otherwise. Tests that absolutely > require physical disks would be skipped if test_suites.FreeBSD.disks > isn't defined. It would take some work, but it's doable. > >> >> [...] >>>> And lastly, we'd just need a simple "filtering" feature in the kyua >>>> cli to allow specifying which size of tests to run (or to filter by >>>> any other metadata property, for that matter). >>> >>> I don't like this idea. >> >> I hope you are objecting to the filtering by test sizes, not the >> filtering itself! See below. >> >>> We tried it at work, and it didn't work out >>> very well. Basically, I automatically assigned sizes (short, medium, >>> long) to all of our tests based on their runtimes, so users could >>> select to only run the short or medium tests on the bench. The >>> problem is that the classification didn't make sense. For an >>> expedited test run, you don't want the shortest tests; what you want >>> are the tests with the most value per unit time. Unit tests have a >>> high value and a very short runtime. Stress tests have a very long >>> runtime and arguably low value since they don't always have consistent >>> results. >>> >>> At my previous job there was a large department whose duties included >>> curating test suites and deciding which tests would be included in the >>> short runs. FreeBSD isn't going to have that, but we could still ask >>> test authors to classify tests along the lines that I suggested. >> >> That's a good point. If I understand you correctly, what is important >> is to manually curate the set of "smoke tests" that run very quickly >> and that provide a reasonably high assurance that major subsystems >> haven't broken. >> >> We can do this too pretty easily, but configuration variables are the >> wrong mechanism. We can already define this information in metadata >> properties as long as you prefix them with X-. So all we'd need is a >> way to tell Kyua to only run tests marked with such property (i.e. >> with the filtering feature described above). > > I was assuming that a test case could put in its header > 'require.config "stress_tests"', or something like that. Then runtime > filtering would work by which variables the user defines. That way, > no new work would be required in Kyua. But it sounds like you're > talking about something different. IIUC, you're suggesting that a > test case would say in its header 'atf_set "X-runtime" "stress_test"' > and then some TBD filtering mechanism in Kyua selects on it. That > would certainly be more intuitive. Is that what you had in mind? > > -Alan > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" From owner-freebsd-testing@FreeBSD.ORG Sun Mar 16 21:53:00 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF2AAAFA; Sun, 16 Mar 2014 21:53:00 +0000 (UTC) Received: from mail-oa0-x236.google.com (mail-oa0-x236.google.com [IPv6:2607:f8b0:4003:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 88F75989; Sun, 16 Mar 2014 21:53:00 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n16so4818675oag.41 for ; Sun, 16 Mar 2014 14:52:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=730CJqz7/b1yF/IzgAAYUHA9F0941u+CPkQeIcq9wDA=; b=tRh1Go0JUmnjn4E6Ky7mshLI22nWiOu7zMo+iRKWqFoR5zW0uYvhgahJTOUzb49PN9 /kKWMtVdwbJPpHG7ql8VGlT2mNS94La19z4B3VUeN+Hpj+Lt5o/y45Few0B/JL17SGVs s5aSAHKLlcDRurMkViXX5cv1UY6CBVMYIYbFZg51FF8dp/ExpE6+Yh35zAOsdLVXfg4Y 8JLW2G/KK7G+mbPjEAd1oA0m9arMyYqBa7s2CR3vtxNqdOkCfcD1GBtnZKn4YswwC1RN K/fMhR0UQdqxILSjX1e4wfqbyklRUhqp8KWDxCBgou35OKZwkHdNxp1XX3+NulW0qzvh cXLw== MIME-Version: 1.0 X-Received: by 10.182.113.195 with SMTP id ja3mr7660069obb.46.1395006779488; Sun, 16 Mar 2014 14:52:59 -0700 (PDT) Received: by 10.182.118.6 with HTTP; Sun, 16 Mar 2014 14:52:59 -0700 (PDT) In-Reply-To: References: Date: Sun, 16 Mar 2014 23:52:59 +0200 Message-ID: Subject: Re: GSoC Idea: Working on modernizing the tests of ATF From: Anastasios Mag To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 21:53:00 -0000 over here "For the impatient: $ ./configure $ make $ make check Gain root privileges # make install Drop root privileges $ make installcheck " when i run "./configure" nothing happens. There is only one file related to configure named "configure.ac" but it isn't executable. On Sun, Mar 16, 2014 at 2:39 AM, Julio Merino wrote: > On Sun, Mar 16, 2014 at 2:03 AM, Anastasios Mag > wrote: > > Hi Julio, currently i have a problem installing kyua. > > In installation instructions it guides me to run "./configure" but > > configure files aren't executable. > > What to do ? > > What do you mean? Please paste the actual output of what you are seeing. > > Note that you can just use "pkg install kyua" to get kyua installed on > your FreeBSD machine. No need to build from source if you don't want > to! > -- Building the kernel...! From owner-freebsd-testing@FreeBSD.ORG Sun Mar 16 23:44:52 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2CB92FD1; Sun, 16 Mar 2014 23:44:52 +0000 (UTC) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 90F2E241; Sun, 16 Mar 2014 23:44:51 +0000 (UTC) Received: by mail-wi0-f176.google.com with SMTP id hr14so1464431wib.3 for ; Sun, 16 Mar 2014 16:44:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=0jGISwrg+zzuPGzkLgPhetsSFV6MYw9N+/1pwhkz8C4=; b=I/oVZfLc31IpkyTrT5dX+p3+TyngzTBXQhUpG6i8RVaI/ddiAkQo/HHU963dxOpFY3 0f3xW4mY7YDKwBhlmVZ8VO2LILfEHU9inqmgWR74LLKqnM8IJ/ca6FUIXJD56JIzEygg OwExIPDn23rbNO4vfYD2iHVzyc+OLsrIgwM2llz/BHJtiUml61gxq0u193BZSjPOQ3Cc wripB5f6Ph02wEbz1GxE8r3DM622f30zy8Tdo0KBWkJpSa9Bl9ECDrkH51J8UUt1F3TP 666DztfjCTT8zfIlmuf+/JCK95Slx95vOvTUdvsmaxDjyQDh0wkFYWB/nuM/TcHknUsI c1DA== MIME-Version: 1.0 X-Received: by 10.194.92.147 with SMTP id cm19mr3415769wjb.37.1395013489951; Sun, 16 Mar 2014 16:44:49 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Sun, 16 Mar 2014 16:44:49 -0700 (PDT) In-Reply-To: References: Date: Sun, 16 Mar 2014 17:44:49 -0600 X-Google-Sender-Auth: SZqxRTSLPGKKZtdi59P9J7pXUe8 Message-ID: Subject: Re: GSoC Idea: Working on modernizing the tests of ATF From: Alan Somers To: Anastasios Mag Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 23:44:52 -0000 On Sun, Mar 16, 2014 at 3:52 PM, Anastasios Mag wrote: > over here > > "For the impatient: > > > $ ./configure > $ make > $ make check > Gain root privileges > # make install > Drop root privileges > $ make installcheck > " > > when i run "./configure" nothing happens. There is only one file > related to configure named "configure.ac" but it isn't executable. Sounds like you checked out from git. You should only do that if you are developing on Kyua itself. If not, you can install Kyua from ports like Julio said. Failing that, you can install from one of the downloads on github. But if you really want to work on the head from git, read the INSTALL file. It will tell you to run "autoreconf -i -s" before "./configure". -Alan > > > > On Sun, Mar 16, 2014 at 2:39 AM, Julio Merino wrote: > >> On Sun, Mar 16, 2014 at 2:03 AM, Anastasios Mag >> wrote: >> > Hi Julio, currently i have a problem installing kyua. >> > In installation instructions it guides me to run "./configure" but >> > configure files aren't executable. >> > What to do ? >> >> What do you mean? Please paste the actual output of what you are seeing. >> >> Note that you can just use "pkg install kyua" to get kyua installed on >> your FreeBSD machine. No need to build from source if you don't want >> to! >> > > > > -- > Building the kernel...! > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" From owner-freebsd-testing@FreeBSD.ORG Mon Mar 17 15:14:12 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B3BEE3C8; Mon, 17 Mar 2014 15:14:12 +0000 (UTC) Received: from mail-wg0-x230.google.com (mail-wg0-x230.google.com [IPv6:2a00:1450:400c:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CA7B3FC3; Mon, 17 Mar 2014 15:14:11 +0000 (UTC) Received: by mail-wg0-f48.google.com with SMTP id l18so4655916wgh.31 for ; Mon, 17 Mar 2014 08:14:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=BbG9EdAm08dEthh2nYztiV1IZvNjN77Q9v8rNPd7mbQ=; b=Uqyh+e75K7X3KwdHdG8aGvdpZv71RV0VyFXOBtB5SSsLCDArev3bHudSLw0ApdZPK5 ZgoQcBP/0pUxfAleD5o+1U97Ofhcct/KRW/DO5EvW6dzsR6ONx2dPLJBI+NzEDjEoixs Z0qrO1AFr54ekIn+9ZtPbX0MWlz33Ap/S8CYXMyk0EW9V5Ha3QsFjwK10keXV0ePi17b QJKU5U+RqU89ukytd+y805XpcXgoVUxQCanu2EqqX235e9qdVK5hi87eOwjkL3ieFWSp pLnvcGYKvGshz470QFduxVOwGUhxeKNOfYzVF3dOslr+2duVgQzQud7h87j5KawItpSZ WA1w== MIME-Version: 1.0 X-Received: by 10.180.73.19 with SMTP id h19mr10127808wiv.40.1395069250307; Mon, 17 Mar 2014 08:14:10 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Mon, 17 Mar 2014 08:14:10 -0700 (PDT) In-Reply-To: <6BADC2BD-3C99-44F7-B6FB-6139AB8752D9@gmail.com> References: <6BADC2BD-3C99-44F7-B6FB-6139AB8752D9@gmail.com> Date: Mon, 17 Mar 2014 09:14:10 -0600 X-Google-Sender-Auth: LxYeJhPeBz2LDmGnRV5JjjP9n1o Message-ID: Subject: Re: ATF config variables for FreeBSD tests From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 15:14:12 -0000 On Sun, Mar 16, 2014 at 10:53 AM, Garrett Cooper wrote: > >> On Mar 16, 2014, at 8:56, Alan Somers wrote: >> >>> On Sat, Mar 15, 2014 at 6:49 PM, Julio Merino wrote: >>>> On Sun, Mar 16, 2014 at 5:33 AM, Alan Somers wrote: >>>> On Fri, Mar 14, 2014 at 10:21 PM, Julio Merino wrote: >>> [...] >>>>> I think these all sound reasonable. >>>>> >>>>> Can these expected side-effects be reversed in the test cleanup? >>>> >>>> For the first two, yes. I doubt that Kyua could do it automatically, >>>> but each test case certainly can. >>> >>> Right. I do not think Kyua should get into these tricky details >>> either, if only because they are too OS-specific. But we can certainly >>> offer additional scripts/functions (where it makes sense) in the >>> FreeBSD test suite to simplify the test cases that might need this. >> >> Definitely. This is a good case for a library of reusable test code. >> For example, I have two separate test programs that would like to >> share setup and cleanup code for FIBs. Should I put it in >> /usr/tests/include/fibs.sh or /usr/tests/lib/fibs.sh ? I don't like >> those options, because it looks like fibs.sh is a test program >> designed to test stuff in /include or /lib. Perhaps >> /usr/tests/tests/sh/include/fibs.sh? I'm really not sure where the >> best place would be. What do you think? > > /usr/tests/libexec seems logical.. But that's where tests for the stuff in /libexec go. If anything, I think /usr/libexec/tests would be more appropriate. Unfortunately, such files would share the same source location as the tests for the stuff in /libexec. But I think it's preferable to have a confusing source layout to a confusing installed layout. > >>> >>>> Perhaps even for the fourth, though >>>> it would be tricky. But there's no way to reverse the effects for the >>>> third. The ZFS tests, for example, create and destroy zpools. How >>>> would you reverse that? You can't. Whatever was previously on the >>>> disk is gone. >>> >>> Ah, I guess I missed that detail. >>> >>> Couldn't those tests run on top of vnd devices though? >> >> FreeBSD doesn't have vnd(4), though it has md(4), which is similar. >> md(4) devices could be used for some ZFS tests. as it happens, ZFS >> can also use file-backed vdevs. But these workarounds don't work in >> all cases. Sometimes you need real disks: >> 1) Using file-backed vdevs doesn't exercise vdev_geom.c, where I've >> one a lot of work. >> 2) Neither file-backed vdevs nor md(4) devices have physical path >> information, which is needed to test some hotspare functionality. >> Only da(4) devices that are attached to ses(4) expanders have that. >> 3) There is no way to remove an active file-backed vdev, and I don't >> think that you can destroy an in-use md(4) device either. Therefore, >> to test how ZFS handles drive removals requires real drives. >> 4) gibbs has made many tweaks to ZFS to better support 4K and 8K >> sector drives. There is no way to emulate those with file-backed >> vdevs. Perhaps it could be done with md(4) and gnop(8), but I've >> never tried. >> 5) pjd and Steve Hartland have been working on TRIM support. That's >> not supported by file-backed vdevs, and I doubt that md(4) supports it >> either. >> 6) Copy-on-write on top of copy-on-write is very slow. If your system >> uses ZFS root, then both file-backed vdevs and file-backed md(4) >> devices are doing COW-on-COW. Using physical disks for the tests >> greatly speeds the tests' runtimes. >> >> Ideally, the ZFS tests would use test_suites.FreeBSD.disks if it is >> defined, and use file-backed vdevs otherwise. Tests that absolutely >> require physical disks would be skipped if test_suites.FreeBSD.disks >> isn't defined. It would take some work, but it's doable. >> >>> >>> [...] >>>>> And lastly, we'd just need a simple "filtering" feature in the kyua >>>>> cli to allow specifying which size of tests to run (or to filter by >>>>> any other metadata property, for that matter). >>>> >>>> I don't like this idea. >>> >>> I hope you are objecting to the filtering by test sizes, not the >>> filtering itself! See below. >>> >>>> We tried it at work, and it didn't work out >>>> very well. Basically, I automatically assigned sizes (short, medium, >>>> long) to all of our tests based on their runtimes, so users could >>>> select to only run the short or medium tests on the bench. The >>>> problem is that the classification didn't make sense. For an >>>> expedited test run, you don't want the shortest tests; what you want >>>> are the tests with the most value per unit time. Unit tests have a >>>> high value and a very short runtime. Stress tests have a very long >>>> runtime and arguably low value since they don't always have consistent >>>> results. >>>> >>>> At my previous job there was a large department whose duties included >>>> curating test suites and deciding which tests would be included in the >>>> short runs. FreeBSD isn't going to have that, but we could still ask >>>> test authors to classify tests along the lines that I suggested. >>> >>> That's a good point. If I understand you correctly, what is important >>> is to manually curate the set of "smoke tests" that run very quickly >>> and that provide a reasonably high assurance that major subsystems >>> haven't broken. >>> >>> We can do this too pretty easily, but configuration variables are the >>> wrong mechanism. We can already define this information in metadata >>> properties as long as you prefix them with X-. So all we'd need is a >>> way to tell Kyua to only run tests marked with such property (i.e. >>> with the filtering feature described above). >> >> I was assuming that a test case could put in its header >> 'require.config "stress_tests"', or something like that. Then runtime >> filtering would work by which variables the user defines. That way, >> no new work would be required in Kyua. But it sounds like you're >> talking about something different. IIUC, you're suggesting that a >> test case would say in its header 'atf_set "X-runtime" "stress_test"' >> and then some TBD filtering mechanism in Kyua selects on it. That >> would certainly be more intuitive. Is that what you had in mind? >> >> -Alan >> _______________________________________________ >> freebsd-testing@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-testing >> To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" From owner-freebsd-testing@FreeBSD.ORG Tue Mar 18 02:30:04 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 42DD1381; Tue, 18 Mar 2014 02:30:04 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 696B3AEF; Tue, 18 Mar 2014 02:30:03 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id hr17so4310102lab.18 for ; Mon, 17 Mar 2014 19:30:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=0u1b5SOFQu2CPJKHU8W9k8UWgVfT7ZSwCEXj3ulgbTU=; b=p1SnQoFfYbl1X+CfHwRlo5Cdmeyf+2o9y/sGtG0kjWSH32fOCIQQR5XZx6fCWqsgp7 o5L6FxLBc80ZL0N1AeokixL3fbFE/PBKvIKs7cZ+JDQ5JRrqSfdw931cK2myWV1cDBQ+ Ddvd0Zv5agO7rKtdc4iXTiNzjwWzp5FSMI+ha9TJb1NngwZxVNFvNgduWYbmXaiFOsbi iUy6Xm/D6gaMRpFlOzSa83B6uWJJ98dVdZZGjhSuP5UyrkjKePVOhmJTe1ByB7I2Ep4F Z61ta6o0oZyZXbBN0rNo7npubVFvlh3weFM323+kE0n49Ok+tfNunaDwP9JX740o8rAV a0cg== MIME-Version: 1.0 X-Received: by 10.112.201.1 with SMTP id jw1mr16267lbc.47.1395109801492; Mon, 17 Mar 2014 19:30:01 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Mon, 17 Mar 2014 19:30:01 -0700 (PDT) Date: Mon, 17 Mar 2014 19:30:01 -0700 X-Google-Sender-Auth: L_m8vYR8Hh34716tXgRH18bCqVE Message-ID: Subject: Scripts for booting FreeBSD images from the install ISO for use in Jenkins? From: Craig Rodrigues To: freebsd-current Current , "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 Cc: "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 02:30:04 -0000 Hi, For the BSD DevSummit in May, one of the items on our agenda: https://wiki.freebsd.org/201405DevSummit/Jenkins is to talk about writing scripts which can take a FreeBSD ISO image, and then boot it and run it on a remote system or in a VM to install the OS. After the OS is up, we would like to run tests. All of this would be triggered from Jenkins. Does anyone have scripts which can do this? Can they be contributed to the Jenkins effort on FreeBSD? If you have scripts in Python, Ruby, Bourne shell, etc. are all fine, or even recipes in automation frameworks like Puppet, Ansible, Chef, SaltStack, etc., please let us know! :) -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Mar 19 18:44:27 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D16D2FE; Wed, 19 Mar 2014 18:44:27 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 70009D99; Wed, 19 Mar 2014 18:44:26 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so6273096lbj.31 for ; Wed, 19 Mar 2014 11:44:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=bsV2TpV77A89Mt5Q5VKNPhZR+XFhoynSkyV5nfUiMO4=; b=NImjUr3/UVNL+ybBvvhgJOo4oykODbgaAmph6mof+Kbk6SESoZqzj3tW9SLX4jaOGW t7WtWQuz7bQ2RPyGAhDcEee9rjgXsmtbJ5CB9DU+LxJlpOkwOdz3LC/jYeKECWk7unA3 71IqDpzrQ/bJUna1zXHokFZHtYvMGmv+Xbyoshe5XU6jj/Iigw9ozqaeJ83qsE05E+Oc mYGBHOqT+cDWo2bYYXm+NhfvJEYDu3UQ9gZPc2h9PAZHs89cK/f7ozKFCOEypVSdmlOR zPxSMqibCkLa+8rZY7hXMh06bRfrSbmwOoTEdJNsMOW07y1oI+mYz7ClluKQbh+Ha39X 1qJg== MIME-Version: 1.0 X-Received: by 10.152.22.37 with SMTP id a5mr26923345laf.4.1395254663500; Wed, 19 Mar 2014 11:44:23 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Wed, 19 Mar 2014 11:44:23 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 Mar 2014 11:44:23 -0700 X-Google-Sender-Auth: u1Ssv9mrZ-n0HRptuHoSPYZQGZc Message-ID: Subject: Re: March 13: Jenkins and BHyve presentation From: Craig Rodrigues To: freebsd-current Current , "freebsd-virtualization@freebsd.org" , "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 18:44:27 -0000 On Mon, Feb 24, 2014 at 1:04 AM, Craig Rodrigues wrote: > The presentation will be on March 13 in Mountain View, California, U.S.A.: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ Thanks to Annie Zhang and the rest of the iXsystems marketing department who recorded the video and did the editing, the video for this presentation is now online: Go to: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups and click on the link for the video. -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Mar 19 21:48:19 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3BD1965C for ; Wed, 19 Mar 2014 21:48:19 +0000 (UTC) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD6B91BA for ; Wed, 19 Mar 2014 21:48:18 +0000 (UTC) Received: from maildlpprd01.lss.emc.com (maildlpprd01.lss.emc.com [10.253.24.33]) by mailuogwprd01.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2JLmAaD004243 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 19 Mar 2014 17:48:10 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com s2JLmAaD004243 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1395265690; bh=YOFwgBLFbf/ZDRtf55Dh1zDD3iI=; h=From:To:Subject:Date:Message-ID:Content-Type:MIME-Version; b=uCkBakDcSbHJE19MiEvg5+6gX1o6WlG98ZuUNO5JDEsQrpeztxl+v5L5A3mNuiXvS riJ+K/H2mo6fDzV6ze7K3QmPnYUlP0C3uETBnPQrv9B18GvfM++Fnbv6pCpdAIzkM+ yPdUHU60WVcTELWL4x84w1e+J6dd91NUyjhQB0Ak= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com s2JLmAaD004243 Received: from mailusrhubprd01.lss.emc.com (mailusrhubprd01.lss.emc.com [10.253.24.19]) by maildlpprd01.lss.emc.com (RSA Interceptor) for ; Wed, 19 Mar 2014 17:47:58 -0400 Received: from mxhub27.corp.emc.com (mxhub27.corp.emc.com [10.254.110.183]) by mailusrhubprd01.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2JLlv3V009999 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Wed, 19 Mar 2014 17:47:57 -0400 Received: from MXHUB106.corp.emc.com (10.253.58.23) by mxhub27.corp.emc.com (10.254.110.183) with Microsoft SMTP Server (TLS) id 8.3.327.1; Wed, 19 Mar 2014 17:47:58 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB106.corp.emc.com ([10.253.58.23]) with mapi id 14.03.0158.001; Wed, 19 Mar 2014 17:47:57 -0400 From: "Peel, Casey" To: "freebsd-testing@freebsd.org" Subject: kyua requirement testing for tap tests Thread-Topic: kyua requirement testing for tap tests Thread-Index: Ac9Du+p7nnVb7NkxQDqgzWl+66f7vA== Date: Wed, 19 Mar 2014 21:47:56 +0000 Message-ID: <16437CC5729B5345AF77F816513376E812981CA4@MX103CL02.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd01.lss.emc.com X-RSA-Classifications: public Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 21:48:19 -0000 The src/usr.sbin/sed/tests/multi_test.sh test requires the /usr/share/dict/= words file but it doesn't explicitly state this. We don't ship that file an= d thus the test fails. What is the correct way to specify requirements for = tap tests such that kyua skips this test if it doesn't exist? The only opti= on I've come up with thus far is commenting out the multi_test suite altoge= ther from the Makefile. -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 From owner-freebsd-testing@FreeBSD.ORG Wed Mar 19 23:05:01 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0298FB9 for ; Wed, 19 Mar 2014 23:05:01 +0000 (UTC) Received: from mail-ve0-x236.google.com (mail-ve0-x236.google.com [IPv6:2607:f8b0:400c:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8F62EB26 for ; Wed, 19 Mar 2014 23:05:01 +0000 (UTC) Received: by mail-ve0-f182.google.com with SMTP id jw12so32052veb.41 for ; Wed, 19 Mar 2014 16:05:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=FRm4bF/Bt7YLeH52Wo+wWLdUBO/J8Kgz2UyxhWUQw3M=; b=kqUbHHX+IOv8tGiXi521oRcKz/ZogIgRb3/jazqPahFPwBqdxwjM/9Tk0MfHt4mRWT n7U0D15AFfUegrFMqDsTRXoduAcmCTtgMGUxoUMV6E8TYj0zJXN7N8WTu0DAYE047/5W jI+rN+w2ohBozFbzebbYVb0lBgUmfAIHESWXzZ7IHxs5sNVe12HC/3MDaknpBw+ycAac 8lOka9XnFM5vWXcqE514zO/joU9w9DXFzU1eRYEyc/GKNM3EBR637zgBrTBXSmIoBEBQ B9XW/vqTyFI9KYhyGJsL42rR2oAf5PrYSY5Ejval9rjy1SHJynit0d4gQxLwSGxAUdrE XXzQ== MIME-Version: 1.0 X-Received: by 10.52.141.105 with SMTP id rn9mr1613217vdb.44.1395270300719; Wed, 19 Mar 2014 16:05:00 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.58.231.105 with HTTP; Wed, 19 Mar 2014 16:05:00 -0700 (PDT) In-Reply-To: <16437CC5729B5345AF77F816513376E812981CA4@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E812981CA4@MX103CL02.corp.emc.com> Date: Wed, 19 Mar 2014 17:05:00 -0600 X-Google-Sender-Auth: qM_RZi10rYn6Ew9_kRoYZtkOZLs Message-ID: Subject: Re: kyua requirement testing for tap tests From: Alan Somers To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 23:05:01 -0000 On Wed, Mar 19, 2014 at 3:47 PM, Peel, Casey wrote: > The src/usr.sbin/sed/tests/multi_test.sh test requires the /usr/share/dic= t/words file but it doesn't explicitly state this. We don't ship that file = and thus the test fails. What is the correct way to specify requirements fo= r tap tests such that kyua skips this test if it doesn't exist? The only op= tion I've come up with thus far is commenting out the multi_test suite alto= gether from the Makefile. You need to set the required_files metadata variable for the test program. I don't think that the TAP API has a way to do that, but can you do it in the Kyuafile like this? tap_test_program(name=3D"multi_test", required_files=3D"/usr/share/dict/wor= ds") -Alan > > -- Casey > > -- > Performance Test Ninja > casey.peel@isilon.com / 206.777.7945 > > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org= " From owner-freebsd-testing@FreeBSD.ORG Wed Mar 19 23:29:54 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4EBB3A51 for ; Wed, 19 Mar 2014 23:29:54 +0000 (UTC) Received: from mail-qc0-f179.google.com (mail-qc0-f179.google.com [209.85.216.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0CF5AD21 for ; Wed, 19 Mar 2014 23:29:53 +0000 (UTC) Received: by mail-qc0-f179.google.com with SMTP id m20so64682qcx.24 for ; Wed, 19 Mar 2014 16:29:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=oEhoL1EzLE66LyPSKdtWeu2ioqCfR5Go6GOMIiiowL4=; b=MgHxDpPKJJFxMIBPNGeK44gML+0kB0N4EuPFDd+4Ax4t+8VR6byeyRy7yrmeBq50Iz BiGfbYuwoJ47AlqSvCw2fObo6e91gQ9CWjtSbDhjLXNd5NHB70eWxi30FMg6Vl0xSxOH rXGnJGIUNAhjJdbpTya2YzKosSFYhesG0tMya3xUMagvYPVRXGyl9ZlhetnZV3dFbKHp 97zjpmwV+uy1ORbx0dyuQ+pSl4KD8N6VfkeeRPUST8X67YRHXNutTL/eMVdeP9ffNVpC u1iLzhRc1JR59ZDaDy2q4qELyrPGOTkMpwTYbB0smU21IJ2pi2QrNOUqjXNeysUvqOdm 1d9Q== X-Gm-Message-State: ALoCoQm+QRo5hDMSyhonmv0sMji1pCYVIVxLXhh/lgHA2qZVl7dnCTNmqBo49evZkp3vb+NOC7gO X-Received: by 10.140.40.5 with SMTP id w5mr43697015qgw.65.1395271787650; Wed, 19 Mar 2014 16:29:47 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Wed, 19 Mar 2014 16:29:27 -0700 (PDT) X-Originating-IP: [1.242.225.82] In-Reply-To: References: <16437CC5729B5345AF77F816513376E812981CA4@MX103CL02.corp.emc.com> From: Julio Merino Date: Thu, 20 Mar 2014 08:29:27 +0900 X-Google-Sender-Auth: TQoMvQaYO_KOJGlu_XKGm3hFnEU Message-ID: Subject: Re: kyua requirement testing for tap tests To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 23:29:54 -0000 On Thu, Mar 20, 2014 at 8:05 AM, Alan Somers wrote: > On Wed, Mar 19, 2014 at 3:47 PM, Peel, Casey wrot= e: >> The src/usr.sbin/sed/tests/multi_test.sh test requires the /usr/share/di= ct/words file but it doesn't explicitly state this. We don't ship that file= and thus the test fails. What is the correct way to specify requirements f= or tap tests such that kyua skips this test if it doesn't exist? The only o= ption I've come up with thus far is commenting out the multi_test suite alt= ogether from the Makefile. > > You need to set the required_files metadata variable for the test > program. I don't think that the TAP API has a way to do that, but can > you do it in the Kyuafile like this? > > tap_test_program(name=3D"multi_test", required_files=3D"/usr/share/dict/w= ords") Yes you can. (Note that those parenthesis ought to be braces though.) And you could do: TEST_METADATA.multi_test+=3D required_files=3D"/usr/share/dict/words" in the Makefile to avoid having to write a Kyuafile by hand. Fixed in r263388; thanks! From owner-freebsd-testing@FreeBSD.ORG Thu Mar 20 16:10:13 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 07837667 for ; Thu, 20 Mar 2014 16:10:13 +0000 (UTC) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A987E9A5 for ; Thu, 20 Mar 2014 16:10:11 +0000 (UTC) Received: from maildlpprd04.lss.emc.com (maildlpprd04.lss.emc.com [10.253.24.36]) by mailuogwprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2KGA9BP009208 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 20 Mar 2014 12:10:10 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd03.lss.emc.com s2KGA9BP009208 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1395331810; bh=z7y9lQKp98D1YFGw2zI0PZDAg1w=; h=From:To:Subject:Date:Message-ID:Content-Type:MIME-Version; b=ZdEGBrTnu3gvrWZIqVyHjl4JuO/QEt2j0HHRkRy6ohf2vwT7pu0Dhc7VXMUeFjbY0 JqIbmvO7Ir485IIQKo4BuSSr8wXX3Q0L47qtwg1oPZrrCYBk55RSP4JHbPhohPjjv/ 7gkVMiHlHwJo8tYVH+STQuv+JKbjP2E6eP0zPpCE= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd03.lss.emc.com s2KGA9BP009208 Received: from mailusrhubprd03.lss.emc.com (mailusrhubprd03.lss.emc.com [10.253.24.21]) by maildlpprd04.lss.emc.com (RSA Interceptor) for ; Thu, 20 Mar 2014 09:09:52 -0700 Received: from mxhub35.corp.emc.com (mxhub35.corp.emc.com [10.254.93.83]) by mailusrhubprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2KG9qCC015847 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Thu, 20 Mar 2014 12:09:52 -0400 Received: from MXHUB106.corp.emc.com (10.253.58.23) by mxhub35.corp.emc.com (10.254.93.83) with Microsoft SMTP Server (TLS) id 8.3.327.1; Thu, 20 Mar 2014 12:09:52 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB106.corp.emc.com ([10.253.58.23]) with mapi id 14.03.0158.001; Thu, 20 Mar 2014 12:09:51 -0400 From: "Peel, Casey" To: "freebsd-testing@freebsd.org" Subject: fix for sbin/mdconfig/tests/legacy_test.sh Thread-Topic: fix for sbin/mdconfig/tests/legacy_test.sh Thread-Index: Ac9EVs4FVLMc2CgKTqGoWnT8Xk8kwA== Date: Thu, 20 Mar 2014 16:09:51 +0000 Message-ID: <16437CC5729B5345AF77F816513376E81298241F@MX103CL02.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd03.lss.emc.com X-RSA-Classifications: public Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 16:10:13 -0000 The sbin/mdconfig/tests/legacy_test.sh script fails because it is looking f= or the wrong runner (diff is against our internal code, but the change is t= rivial): Index: sbin/mdconfig/tests/legacy_test.sh =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sbin/mdconfig/tests/legacy_test.sh (revision 397720) +++ sbin/mdconfig/tests/legacy_test.sh (working copy) @@ -38,7 +38,7 @@ TESTDIR=3D$(dirname $(realpath $0)) -__PERL__ -w -U $TESTDIR/run $TESTDIR/mdconfig.test > /dev/null +__PERL__ -w -U $TESTDIR/run.pl $TESTDIR/mdconfig.test > /dev/null if [ $? -eq 0 ]; then echo "ok 1" -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 From owner-freebsd-testing@FreeBSD.ORG Thu Mar 20 17:16:21 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C304FA9; Thu, 20 Mar 2014 17:16:21 +0000 (UTC) Received: from mail-ve0-x232.google.com (mail-ve0-x232.google.com [IPv6:2607:f8b0:400c:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 91B6022E; Thu, 20 Mar 2014 17:16:20 +0000 (UTC) Received: by mail-ve0-f178.google.com with SMTP id jw12so1292803veb.37 for ; Thu, 20 Mar 2014 10:16:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=//kSXxTtoqSqclgvop1ADcZ8au5BgFRd2HbhiBf7aHk=; b=VDQGQnJnXg9bf2sgX4JraZV0cTBIEnxdRXXs0ce49I1z4nlst40i2rUYLm3VsrOOnr 0wA3TPDNGI6KihBzmJkpbnpG1Fsn5XiuZWpuB6yKdmjvVwwFA63jsP+Cp6KrgqjNypnL OdK3fZ5EBpRqNCzmGn20sjxk1G1VD8AsTgxoD2SI7VNScxZ+aHgpBUuPsdNBskoJ5xJh n/m8DqOAMkvNJlTUVzEShFe06KhFBHbRGfCZo31zaAxCaTKfNHiYQaLh8vg4H+MMSXuT maU2lmsWvtaDRkRSgVdI8LRqVp/HUsLIFAlgsF9SUND+1d9uv9fM20CFTXJMrHXNxlD9 xuSg== MIME-Version: 1.0 X-Received: by 10.221.29.196 with SMTP id rz4mr34665121vcb.8.1395335779688; Thu, 20 Mar 2014 10:16:19 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.58.231.105 with HTTP; Thu, 20 Mar 2014 10:16:19 -0700 (PDT) In-Reply-To: References: <6BADC2BD-3C99-44F7-B6FB-6139AB8752D9@gmail.com> Date: Thu, 20 Mar 2014 11:16:19 -0600 X-Google-Sender-Auth: 959S1sLS-u4gq87VvRV-GN9fv1c Message-ID: Subject: Re: ATF config variables for FreeBSD tests From: Alan Somers To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 17:16:21 -0000 On Mon, Mar 17, 2014 at 9:14 AM, Alan Somers wrote: > On Sun, Mar 16, 2014 at 10:53 AM, Garrett Cooper wrote: >> >>> On Mar 16, 2014, at 8:56, Alan Somers wrote: >>> >>>> On Sat, Mar 15, 2014 at 6:49 PM, Julio Merino wrote: >>>>> On Sun, Mar 16, 2014 at 5:33 AM, Alan Somers wrote: >>>>> On Fri, Mar 14, 2014 at 10:21 PM, Julio Merino wrote: >>>> [...] >>>>>> I think these all sound reasonable. >>>>>> >>>>>> Can these expected side-effects be reversed in the test cleanup? >>>>> >>>>> For the first two, yes. I doubt that Kyua could do it automatically, >>>>> but each test case certainly can. >>>> >>>> Right. I do not think Kyua should get into these tricky details >>>> either, if only because they are too OS-specific. But we can certainly >>>> offer additional scripts/functions (where it makes sense) in the >>>> FreeBSD test suite to simplify the test cases that might need this. >>> >>> Definitely. This is a good case for a library of reusable test code. >>> For example, I have two separate test programs that would like to >>> share setup and cleanup code for FIBs. Should I put it in >>> /usr/tests/include/fibs.sh or /usr/tests/lib/fibs.sh ? I don't like >>> those options, because it looks like fibs.sh is a test program >>> designed to test stuff in /include or /lib. Perhaps >>> /usr/tests/tests/sh/include/fibs.sh? I'm really not sure where the >>> best place would be. What do you think? >> >> /usr/tests/libexec seems logical.. > > But that's where tests for the stuff in /libexec go. If anything, I > think /usr/libexec/tests would be more appropriate. Unfortunately, > such files would share the same source location as the tests for the > stuff in /libexec. But I think it's preferable to have a confusing > source layout to a confusing installed layout. > >> >>>> >>>>> Perhaps even for the fourth, though >>>>> it would be tricky. But there's no way to reverse the effects for the >>>>> third. The ZFS tests, for example, create and destroy zpools. How >>>>> would you reverse that? You can't. Whatever was previously on the >>>>> disk is gone. >>>> >>>> Ah, I guess I missed that detail. >>>> >>>> Couldn't those tests run on top of vnd devices though? >>> >>> FreeBSD doesn't have vnd(4), though it has md(4), which is similar. >>> md(4) devices could be used for some ZFS tests. as it happens, ZFS >>> can also use file-backed vdevs. But these workarounds don't work in >>> all cases. Sometimes you need real disks: >>> 1) Using file-backed vdevs doesn't exercise vdev_geom.c, where I've >>> one a lot of work. >>> 2) Neither file-backed vdevs nor md(4) devices have physical path >>> information, which is needed to test some hotspare functionality. >>> Only da(4) devices that are attached to ses(4) expanders have that. >>> 3) There is no way to remove an active file-backed vdev, and I don't >>> think that you can destroy an in-use md(4) device either. Therefore, >>> to test how ZFS handles drive removals requires real drives. >>> 4) gibbs has made many tweaks to ZFS to better support 4K and 8K >>> sector drives. There is no way to emulate those with file-backed >>> vdevs. Perhaps it could be done with md(4) and gnop(8), but I've >>> never tried. >>> 5) pjd and Steve Hartland have been working on TRIM support. That's >>> not supported by file-backed vdevs, and I doubt that md(4) supports it >>> either. >>> 6) Copy-on-write on top of copy-on-write is very slow. If your system >>> uses ZFS root, then both file-backed vdevs and file-backed md(4) >>> devices are doing COW-on-COW. Using physical disks for the tests >>> greatly speeds the tests' runtimes. >>> >>> Ideally, the ZFS tests would use test_suites.FreeBSD.disks if it is >>> defined, and use file-backed vdevs otherwise. Tests that absolutely >>> require physical disks would be skipped if test_suites.FreeBSD.disks >>> isn't defined. It would take some work, but it's doable. >>> >>>> >>>> [...] >>>>>> And lastly, we'd just need a simple "filtering" feature in the kyua >>>>>> cli to allow specifying which size of tests to run (or to filter by >>>>>> any other metadata property, for that matter). >>>>> >>>>> I don't like this idea. >>>> >>>> I hope you are objecting to the filtering by test sizes, not the >>>> filtering itself! See below. >>>> >>>>> We tried it at work, and it didn't work out >>>>> very well. Basically, I automatically assigned sizes (short, medium, >>>>> long) to all of our tests based on their runtimes, so users could >>>>> select to only run the short or medium tests on the bench. The >>>>> problem is that the classification didn't make sense. For an >>>>> expedited test run, you don't want the shortest tests; what you want >>>>> are the tests with the most value per unit time. Unit tests have a >>>>> high value and a very short runtime. Stress tests have a very long >>>>> runtime and arguably low value since they don't always have consistent >>>>> results. >>>>> >>>>> At my previous job there was a large department whose duties included >>>>> curating test suites and deciding which tests would be included in the >>>>> short runs. FreeBSD isn't going to have that, but we could still ask >>>>> test authors to classify tests along the lines that I suggested. >>>> >>>> That's a good point. If I understand you correctly, what is important >>>> is to manually curate the set of "smoke tests" that run very quickly >>>> and that provide a reasonably high assurance that major subsystems >>>> haven't broken. >>>> >>>> We can do this too pretty easily, but configuration variables are the >>>> wrong mechanism. We can already define this information in metadata >>>> properties as long as you prefix them with X-. So all we'd need is a >>>> way to tell Kyua to only run tests marked with such property (i.e. >>>> with the filtering feature described above). >>> >>> I was assuming that a test case could put in its header >>> 'require.config "stress_tests"', or something like that. Then runtime >>> filtering would work by which variables the user defines. That way, >>> no new work would be required in Kyua. But it sounds like you're >>> talking about something different. IIUC, you're suggesting that a >>> test case would say in its header 'atf_set "X-runtime" "stress_test"' >>> and then some TBD filtering mechanism in Kyua selects on it. That >>> would certainly be more intuitive. Is that what you had in mind? >>> >>> -Alan I committed the man page change for the four variables we agreed on. But I didn't add the runtime selection variables. It sounds like Julio has bigger plans for those, and they'll require upstream work on Kyua. -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Mar 20 23:37:42 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 777A46FC for ; Thu, 20 Mar 2014 23:37:42 +0000 (UTC) Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3114165C for ; Thu, 20 Mar 2014 23:37:41 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id f11so1702898qae.3 for ; Thu, 20 Mar 2014 16:37:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=D36w4Otv8ywqMMwIRYYSpsoU1Wx10f9GakcDv6ofqyo=; b=KW0VLuF761NrqKclB5KhVUEhJa2jiCnMyh4Yl1nrLKntcEEPE9QqHU74zB5rB3v0O8 CwelCyp+JOIUGvx24Gt9UNLo+0T0YALlbcNP8ZZyEzHE/o9FDvVaHldNUcCByjNHzURR SFbIS5St//VLziWh1l+VqSAV2k+sIuKpMniE55F2412cJM5Z8mBueNg603mpony5WsxS yqgiiLYYN6Tx4FpB73y4kyjwpFoxBTouo9JmpYSIfGgM0TbdUaeSr3ypdDbMDrxQ5YF+ cZLWlJWaR5K/wvg7c7gHBgv93TzZYneGvyLd5LmaNFluS3HXARa8AReSYyjVO7QV/qd9 0Myw== X-Gm-Message-State: ALoCoQkFlpr4vbuTlvDCvaVZ+QIiLoDCgckRfFvX5/bZHVNH8rBDTddJcH1lKFXLW8ht7mV1oyin X-Received: by 10.229.139.199 with SMTP id f7mr53996544qcu.2.1395358655580; Thu, 20 Mar 2014 16:37:35 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 20 Mar 2014 16:37:15 -0700 (PDT) X-Originating-IP: [1.242.225.82] In-Reply-To: References: <6BADC2BD-3C99-44F7-B6FB-6139AB8752D9@gmail.com> From: Julio Merino Date: Fri, 21 Mar 2014 08:37:15 +0900 X-Google-Sender-Auth: m8d5iJr2gv3sX5sZDmUwJmBKXuI Message-ID: Subject: Re: ATF config variables for FreeBSD tests To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 23:37:42 -0000 On Fri, Mar 21, 2014 at 2:16 AM, Alan Somers wrote: > > I committed the man page change for the four variables we agreed on. > But I didn't add the runtime selection variables. It sounds like > Julio has bigger plans for those, and they'll require upstream work on > Kyua. Thanks. Will try to get to this soon; shouldn't be hard at all at this point! From owner-freebsd-testing@FreeBSD.ORG Thu Mar 20 23:44:54 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 385218A3 for ; Thu, 20 Mar 2014 23:44:54 +0000 (UTC) Received: from mail-qc0-f181.google.com (mail-qc0-f181.google.com [209.85.216.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E5CD4792 for ; Thu, 20 Mar 2014 23:44:53 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id e9so1947997qcy.40 for ; Thu, 20 Mar 2014 16:44:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=cHUgYbj0vPl9i0wkAbzjD8ZjzLlClc2ovfPr4QE71co=; b=A5t+mBZFIOyW75pPEG/yGy6yGPkED1gdddf6kFsf5YkjFIxq/CRS5kTzLt56+k6ISl 5Urkl741cIcxZOwkfwBuL8uLuwZkj4MPnLXJekaWhymVM5peuq9+wlP3O9duIk6O5J0n XFnI83J3rwi7Ap1+K1RQlP4vzKx0mhyrPgHaV4GVEcyOIhuyO+adK8j06JtbkbwTMBjI 7tVIwq+Hsx9yhy8viKnslDr3IPWCCHn3wLRCpg+B5UdjlkHLqDmOH5I85MfkIRiEsYZg jxg2ueCUwgH3q5zFFiUl1W9VdNeudwlPBQXCLHpnocqZH8/NmhFzEcoJ/8Fex5Kn2v1y khmA== X-Gm-Message-State: ALoCoQnbt03yGUDHM0Boale62x2tDQlRWiLFGab/091g6i1AIGJZsq0qP2SJvRvlByc7Qh/9sUBE X-Received: by 10.224.119.141 with SMTP id z13mr52453972qaq.48.1395358609664; Thu, 20 Mar 2014 16:36:49 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 20 Mar 2014 16:36:29 -0700 (PDT) X-Originating-IP: [1.242.225.82] In-Reply-To: References: <6BADC2BD-3C99-44F7-B6FB-6139AB8752D9@gmail.com> From: Julio Merino Date: Fri, 21 Mar 2014 08:36:29 +0900 X-Google-Sender-Auth: dq15IpX0plQjsDHy9erzqAq9vk8 Message-ID: Subject: Re: ATF config variables for FreeBSD tests To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kenneth D. Merry" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 23:44:54 -0000 On Tue, Mar 18, 2014 at 12:14 AM, Alan Somers wrote: > On Sun, Mar 16, 2014 at 10:53 AM, Garrett Cooper wrote: >> /usr/tests/libexec seems logical.. > > But that's where tests for the stuff in /libexec go. If anything, I > think /usr/libexec/tests would be more appropriate. Unfortunately, > such files would share the same source location as the tests for the > stuff in /libexec. But I think it's preferable to have a confusing > source layout to a confusing installed layout. Indeed. But if this is shared supporting code, there is no reason for it to live under /usr/tests. After all, the ATF libraries are not installed inside /usr/tests and do not live in src/tests/! For C, we'd just have a libfreebsd_tests.so or similar under /usr/lib/private and put the source under src/lib/ (and its tests --of course!-- under /usr/tests/lib/private/libfreebsd_tests/). For shell, it depends on how the "libraries" look like. One option is to have some installed .sh files that get sourced at runtime within the script; the other is to just concatenate the helper code to the test programs at build time and not install the library. In the former case, we'd just install this under /usr/libexec/something; in the latter, there is nothing to install. (There is also the option that these shell libraries are actually separate binaries, of course, but then the libexec layout also applies.) And if we are concerned about the code looking like actual tests, you'd always use .subr instead of .sh for supporting code as is done in some other places (e.g. rc.subr). From owner-freebsd-testing@FreeBSD.ORG Fri Mar 21 02:40:10 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2804C147; Fri, 21 Mar 2014 02:40:10 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 20F7A934; Fri, 21 Mar 2014 02:40:08 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id b8so1226777lan.26 for ; Thu, 20 Mar 2014 19:40:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=D0o61ra25zJw+RuT/2p57l9/QikuqC1mfgf/oCgsYZI=; b=GG/QVdsidnJNW1TasrVZuBLbr0KhLaL9RpzSsmV6B/1npOcQkDFDXvdivzp0PNOaHh 9uqC9HI8D7Pvn6cXwZlbbqEccjskUd439ktSR783coR3j3abNE7cfG46Djy+YB8s1gnK g8Q3LFVcxYgnom8GKwq/zWSbysHuNAKJqrvCuG4miID+DOtNQihgHileZIqsJuuy4uMG nuO8kmsrojIv5WYkNuwjeaEPLXMDwqNaQEawClZVawUA5S9WDYL24vWfUg3b6YjoTKyZ kupsjf5fub6gltNLNVgM0IZdGyQM2ixoNZVm3YhDqBUJVZmq9XJRHDBDBi+tkKqX4yni qISQ== MIME-Version: 1.0 X-Received: by 10.112.209.5 with SMTP id mi5mr22676854lbc.30.1395369606730; Thu, 20 Mar 2014 19:40:06 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Thu, 20 Mar 2014 19:40:06 -0700 (PDT) In-Reply-To: <20140318102615.21fa7fde@suse3.ewadmin.local> References: <20140318102615.21fa7fde@suse3.ewadmin.local> Date: Thu, 20 Mar 2014 19:40:06 -0700 X-Google-Sender-Auth: 7C8mgzzGL5O7id0sadDZPa4DAAw Message-ID: Subject: Re: Scripts for booting FreeBSD images from the install ISO for use in Jenkins? From: Craig Rodrigues To: freebsd-current Current Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 02:40:10 -0000 On Tue, Mar 18, 2014 at 2:26 AM, Rainer Duffner wrote: > Am Mon, 17 Mar 2014 19:30:01 -0700 > schrieb Craig Rodrigues : > >> Hi, >> >> For the BSD DevSummit in May, one of the items >> on our agenda: >> >> https://wiki.freebsd.org/201405DevSummit/Jenkins >> >> is to talk about writing scripts which can take a FreeBSD ISO image, >> and then boot it and run it on a remote system or in a VM >> to install the OS. After the OS is up, we would like to run tests. >> All of this would be triggered from Jenkins. >> >> Does anyone have scripts which can do this? >> Can they be contributed to the Jenkins effort on FreeBSD? >> >> If you have scripts in Python, Ruby, Bourne shell, etc. are all fine, >> or even recipes in automation frameworks like Puppet, Ansible, Chef, >> SaltStack, etc., >> please let us know! :) > > > > I would have loved to attend this talk: > > > http://2014.asiabsdcon.org/timetable.html.en#P7A > > > Hopefully, more documentation and/or the slides/the video for this talk > will become available. Rainer, Thanks for posting that link! It is highly relevant to my original posting. That looks like a really good presentation, and I also wish I could have attended the talk! There seem to be many automation frameworks for provisioning and booting VM's and real machines. I just need to learn one of them. :) -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Mar 26 22:26:37 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C71D796 for ; Wed, 26 Mar 2014 22:26:37 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd53.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF17D607 for ; Wed, 26 Mar 2014 22:26:36 +0000 (UTC) Received: from maildlpprd54.lss.emc.com (maildlpprd54.lss.emc.com [10.106.48.158]) by mailuogwprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2QMQSat015317 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 26 Mar 2014 18:26:28 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com s2QMQSat015317 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1395872788; bh=iOx3O+1hexg1bz7srKxeSoYxdNI=; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; b=SgCZWMT+VOQ0EpDry4R9TQb5kualbaT49n50BOSO4932FrLayrCr+0u3PwU3LK12F rHFI1H6L/KYCGDYlIalqtX74IjLbGXA2kcltIuSZ0tzbK5SD0JV2hxjsF4bBwIFjpL 0ush2pVfZRj8gwV+/UEFFaJ0vncOy5vjjXeP1+yk= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com s2QMQSat015317 Received: from mailusrhubprd03.lss.emc.com (mailusrhubprd03.lss.emc.com [10.253.24.21]) by maildlpprd54.lss.emc.com (RSA Interceptor) for ; Wed, 26 Mar 2014 18:26:24 -0400 Received: from mxhub13.corp.emc.com (mxhub13.corp.emc.com [128.222.70.234]) by mailusrhubprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2QMQNpK015176 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Wed, 26 Mar 2014 18:26:24 -0400 Received: from MX01A.corp.emc.com ([169.254.255.185]) by mxhub13.corp.emc.com ([128.222.70.234]) with mapi; Wed, 26 Mar 2014 18:26:23 -0400 From: "Kilner, Peter" To: "freebsd-testing@freebsd.org" Date: Wed, 26 Mar 2014 18:26:22 -0400 Subject: ATF Test Cases Thread-Topic: ATF Test Cases Thread-Index: Ac9JQmsUZwcUuhowRj2375aBEsvZcw== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd03.lss.emc.com X-RSA-Classifications: public Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 22:26:37 -0000 Hello, I hope I am posting this question to the correct list. Is there a way to have a single ATF test case report multiple pass/fail res= ults? Currently I have seen that one can include a number of test conditio= ns in a single test case (for example many aft_checks). However the test c= ase will only report one pass/fail result for that case. I would like to build a test case that will iterate though many configurati= ons and would like to see a pass/fail for each config. However this is dif= ficult to implement with multiply test cases because of the for loops that = I am using. Any suggestions would be appreciated. Thank you, Peter Kilner Software Engineer EMC, Isilon Seattle peter.kilner@isilon.com 206-777-7859 From owner-freebsd-testing@FreeBSD.ORG Wed Mar 26 22:54:40 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A09E34EE for ; Wed, 26 Mar 2014 22:54:40 +0000 (UTC) Received: from mail-ve0-x22a.google.com (mail-ve0-x22a.google.com [IPv6:2607:f8b0:400c:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5ECF3927 for ; Wed, 26 Mar 2014 22:54:40 +0000 (UTC) Received: by mail-ve0-f170.google.com with SMTP id pa12so3209495veb.29 for ; Wed, 26 Mar 2014 15:54:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=8LOffpPiMqoeHQaBpAJQudzfKO9yKUkV16mpLOOij/k=; b=TqJ+3fgGG3rZLyGWOrk/o5mDFW8BbnZjVlolliTmc7V+B7+gKr0Hbe+R/fekKmrI3H q7xfubmRxgQbXp83LRdyTQWzINUrH3JD6pLhI3hP+xCDC1hProsnwHt2MUkQyWVBboss e5IIQCiZFD5rHGapauahRswHQHgYKkpkSab7qxnlZDU2PjJnVWzZos41GR16BfCGZNRP YAoKtYHVIEl/BJVqPWdDT5Eqp3lohBb04i+5lKLVsJC4ezCaLGyXeYUNaLaAEk81avHj zJSfqmzQpLwgKi1A7YnusOZhWD5HUg/pU5ExnLuNlBhzp+NUnPF5wmIZX9vEDKx7Yopr 9aiw== MIME-Version: 1.0 X-Received: by 10.58.57.67 with SMTP id g3mr61649531veq.3.1395874478409; Wed, 26 Mar 2014 15:54:38 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.58.207.169 with HTTP; Wed, 26 Mar 2014 15:54:38 -0700 (PDT) In-Reply-To: References: Date: Wed, 26 Mar 2014 16:54:38 -0600 X-Google-Sender-Auth: uPRCnph6yCpL1GuaZu1qtP0hcQs Message-ID: Subject: Re: ATF Test Cases From: Alan Somers To: "Kilner, Peter" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 22:54:40 -0000 On Wed, Mar 26, 2014 at 4:26 PM, Kilner, Peter wrote= : > Hello, > > I hope I am posting this question to the correct list. > > Is there a way to have a single ATF test case report multiple pass/fail r= esults? Currently I have seen that one can include a number of test condit= ions in a single test case (for example many aft_checks). However the test= case will only report one pass/fail result for that case. Nope. > > I would like to build a test case that will iterate though many configura= tions and would like to see a pass/fail for each config. However this is d= ifficult to implement with multiply test cases because of the for loops tha= t I am using. > > Any suggestions would be appreciated. Is this in C/C++ or sh? With atf-c-api(3), you can call atf_expect() instead of atf_require(). That way the test will continue even after one of the expectations failed. If multiple expectations fail, then I think kyua will print an error message for each failed expectation. I could be wrong however. Also, I don't think that equivalent functionality exists in atf-sh-api(3). Apart from that, you pretty much have to write multiple test cases. Note that separate test cases run in separate processes. If your for loop is very long and this is a C test program, using multiple test cases will be much slower than using a single test case with multiple expectations. -Alan > > Thank you, > > Peter Kilner > Software Engineer > EMC, Isilon Seattle > peter.kilner@isilon.com > 206-777-7859 > > > > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org= " From owner-freebsd-testing@FreeBSD.ORG Thu Mar 27 01:48:24 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 456A9A99 for ; Thu, 27 Mar 2014 01:48:24 +0000 (UTC) Received: from mail-qa0-f43.google.com (mail-qa0-f43.google.com [209.85.216.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03A9AABE for ; Thu, 27 Mar 2014 01:48:23 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id j15so3200665qaq.30 for ; Wed, 26 Mar 2014 18:48:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=TdjVgEwHK8Ghtx76vh6UzWj85Dg7fANsAe54TQAsM5Q=; b=hIs8G5yD6iD+Aqs6Y+XH6Y58BUwlPksZnilAe5PIq0Vhsk855i8ffLgt1hBnmFAw8t 1bb8Y6bSVKudM0CghLPqp+jBWh9MUzdnk1Eo4zYDbtCUi7ATeAWOvcm9BQA1fwxq90o/ GG5IjFkCc80QfzmbiZe9iOXYzQFG0uZzd9JhXw/ACLb7nLrrlyUWhbr2u7KeaAk9JNTC tcDwe5xunPOANJOswG2fn3M8LMZPtaBFkuE0KjMvA5RZ8ljAeLS5Gnb5E8tjpdwv5T8r RrKeUx5P0ufg+d4Jr+DxuhBRlEu0k77d1LVxYObTSxDazKQQJsF/QvVK3FxHXpZSHYfl Mxmw== X-Gm-Message-State: ALoCoQmIuJFsiMZYqymwsMxCBK0PwXn26lp4IYhgPse0KmjBu+L7elYt8PHeLKVf6Y8HoTPjrLiE X-Received: by 10.140.97.137 with SMTP id m9mr6207084qge.95.1395884897748; Wed, 26 Mar 2014 18:48:17 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Wed, 26 Mar 2014 18:47:56 -0700 (PDT) X-Originating-IP: [2401:fa00:d:c:b03e:2e82:92b6:e21d] In-Reply-To: References: From: Julio Merino Date: Thu, 27 Mar 2014 10:47:56 +0900 X-Google-Sender-Auth: yDgbjXHPNWVSPYy6f73ouCQSMOk Message-ID: Subject: Re: ATF Test Cases To: "Kilner, Peter" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 01:48:24 -0000 On Thu, Mar 27, 2014 at 7:26 AM, Kilner, Peter wrote= : > Hello, Hi Peter, Alan already answered this pretty well, but let me throw in my 2 cents: > I hope I am posting this question to the correct list. > > Is there a way to have a single ATF test case report multiple pass/fail r= esults? Currently I have seen that one can include a number of test condit= ions in a single test case (for example many aft_checks). However the test= case will only report one pass/fail result for that case. That's by design: one test case should represent one (and only one) scenario. Therefore, there can only be one meaningful result. > I would like to build a test case that will iterate though many configura= tions and would like to see a pass/fail for each config. However this is d= ifficult to implement with multiply test cases because of the for loops tha= t I am using. It sounds like you actually should split your test case into smaller ones and name each test case accordingly. Your email doesn't provide a lot of details, but I guess that if you are testing more than one configuration file is because each configuration file is exposing a specific behavior of the program under test. Consider this: "test_configuration" sounds like a really broad test case name and provides no information about what is wrong when the test fails. But if you consider "test_configuration__missing_hostname", "test_configuration__invalid_syntax", "test_configuration__duplicate_host_stanzas" as separate test cases, then when one fails it becomes obvious what specific case is broken. Doing this needn't be hard. If you are using atf-sh, my suggestion is to put all of the test case body in a separate function. Then, with a loop, you dynamically define small test case bodies (using eval) that call the auxiliary function with the right parameters. If you are using atf-c, you can do something similar, maybe even abusing ma= cros. But basically: just move the code to an auxiliary function and have all the individual test cases call it as a one-liner with the right arguments. See the following for some more ideas: https://wiki.freebsd.org/TestSuite/Structure#Test_programs_vs._test_case= s Please let me know if anything is not clear or lacks detail. From owner-freebsd-testing@FreeBSD.ORG Thu Mar 27 04:00:29 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CFE05BB4; Thu, 27 Mar 2014 04:00:29 +0000 (UTC) Received: from mail-ve0-x229.google.com (mail-ve0-x229.google.com [IPv6:2607:f8b0:400c:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78A21863; Thu, 27 Mar 2014 04:00:29 +0000 (UTC) Received: by mail-ve0-f169.google.com with SMTP id pa12so3601306veb.28 for ; Wed, 26 Mar 2014 21:00:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=JY+3eUxf5eL6gZWTYFKK0G8KuqDR2hpEa65iN7u7C2M=; b=kGbOA/Z9ADNqSxPNJdpfPbSF9mBe43ztxpsFMA1clPSius5cpBn2bKYtQikIH78V0H NCrJCNXrx8JeYf0paBkSx4fdJ/Rc7OWAEvpj7q5HeUdERaEZsorfXlTosOpKDKdu489v mBKMX39jJDKeuGy6QjbpRvHOSnZlwax0bwsQmdPbwcwAUJ6AL963b5nVrOya27Y8Tc0G z2l8FmMH703CSXxJYym8RYwOc6/EB/phBcW2fNBNRK7+ZPuSDMOftNoXjaK2hKY5Drku q8oRBZVGggzAsQa25QbwbKCwRFXejmzC+zt4buYvwFdtkqhUrVodT6nk6IE1a8muSi5y 9OuA== MIME-Version: 1.0 X-Received: by 10.52.26.161 with SMTP id m1mr42851756vdg.24.1395892828533; Wed, 26 Mar 2014 21:00:28 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.58.207.169 with HTTP; Wed, 26 Mar 2014 21:00:28 -0700 (PDT) In-Reply-To: References: Date: Wed, 26 Mar 2014 22:00:28 -0600 X-Google-Sender-Auth: P-T9SfDqYQy0ZwjqN_lChAedaJU Message-ID: Subject: Re: ATF Test Cases From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , "Kilner, Peter" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 04:00:30 -0000 On Wed, Mar 26, 2014 at 7:47 PM, Julio Merino wrote: > On Thu, Mar 27, 2014 at 7:26 AM, Kilner, Peter wro= te: >> Hello, > > Hi Peter, > > Alan already answered this pretty well, but let me throw in my 2 cents: > >> I hope I am posting this question to the correct list. >> >> Is there a way to have a single ATF test case report multiple pass/fail = results? Currently I have seen that one can include a number of test condi= tions in a single test case (for example many aft_checks). However the tes= t case will only report one pass/fail result for that case. > > That's by design: one test case should represent one (and only one) > scenario. Therefore, there can only be one meaningful result. > >> I would like to build a test case that will iterate though many configur= ations and would like to see a pass/fail for each config. However this is = difficult to implement with multiply test cases because of the for loops th= at I am using. > > It sounds like you actually should split your test case into smaller > ones and name each test case accordingly. Your email doesn't provide > a lot of details, but I guess that if you are testing more than one > configuration file is because each configuration file is exposing a > specific behavior of the program under test. > > Consider this: "test_configuration" sounds like a really broad test > case name and provides no information about what is wrong when the > test fails. But if you consider > "test_configuration__missing_hostname", > "test_configuration__invalid_syntax", > "test_configuration__duplicate_host_stanzas" as separate test cases, > then when one fails it becomes obvious what specific case is broken. > > Doing this needn't be hard. > > If you are using atf-sh, my suggestion is to put all of the test case > body in a separate function. Then, with a loop, you dynamically > define small test case bodies (using eval) that call the auxiliary > function with the right parameters. I never thought about using eval to dynamically define testcases. Can you please give an example? > > If you are using atf-c, you can do something similar, maybe even abusing = macros. > > But basically: just move the code to an auxiliary function and have > all the individual test cases call it as a one-liner with the right > arguments. > > See the following for some more ideas: > > https://wiki.freebsd.org/TestSuite/Structure#Test_programs_vs._test_ca= ses > > Please let me know if anything is not clear or lacks detail. > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org= " From owner-freebsd-testing@FreeBSD.ORG Thu Mar 27 04:33:19 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82850FA4 for ; Thu, 27 Mar 2014 04:33:19 +0000 (UTC) Received: from mail-qg0-f47.google.com (mail-qg0-f47.google.com [209.85.192.47]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3CA96B0A for ; Thu, 27 Mar 2014 04:33:18 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id 63so2151258qgz.34 for ; Wed, 26 Mar 2014 21:33:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=hJkbBIiFvFea77kNYyrPfOauZyj0QZ1QxiZb+wVAJyw=; b=YaD78AFQNeZ9O7l1EswfQBgDYtgZPsbecqBuB1gsgAziyiuAPhTg5kvCjG9oF7bZyQ sKXkGz8DPWpnTYu+wR23MpdNhno67K8wv9IRWckWWAo1a3AaxAsWhUvsUPOnztp316ss nFpfXNDqueDpdSmpwGANcPw2T+w7DpREtZVxlwBleH1WdoVZg4XPJlHed7kxeMgbZxEj 07S/NJ5/03CZVkeeHgLsomInfBTc5MEcy351BIsCRknh7DfzkIGhuQX0w5WVaohSgFUP JWSPbyhg4ke5tiAf64G8Vh/pblVuGo68XOPmkWAr+kiu3oF0EKl3sy3w7TnDnxAeJUH6 aQzw== X-Gm-Message-State: ALoCoQnyI3RSCutojVGmNthEixw38Ku++vzvADX4PKzFHZM9GpsK1BenMTyPBGYNsQ/pXqeQvjaH X-Received: by 10.140.27.109 with SMTP id 100mr64124468qgw.14.1395894792059; Wed, 26 Mar 2014 21:33:12 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Wed, 26 Mar 2014 21:32:51 -0700 (PDT) X-Originating-IP: [2401:fa00:d:c:b03e:2e82:92b6:e21d] In-Reply-To: References: From: Julio Merino Date: Thu, 27 Mar 2014 13:32:51 +0900 X-Google-Sender-Auth: z0ElpGsKU0oQnrcyQ6NqFsxovQs Message-ID: Subject: Re: ATF Test Cases To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "Kilner, Peter" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 04:33:19 -0000 On Thu, Mar 27, 2014 at 1:00 PM, Alan Somers wrote: > I never thought about using eval to dynamically define testcases. Can > you please give an example? Not the prettiest code in the world, but the ipf tests in NetBSD are an example. See the h_common.sh[1] file for the helper functions and t_filter_exec.sh[2] for a specific case of their use and the definition of various tests. 1: http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/ipf/h_common.sh?rev=1.8&content-type=text/x-cvsweb-markup&only_with_tag=MAIN 2: http://cvsweb.netbsd.org/bsdweb.cgi/src/tests/ipf/t_filter_exec.sh?rev=1.8&content-type=text/x-cvsweb-markup&only_with_tag=MAIN From owner-freebsd-testing@FreeBSD.ORG Thu Mar 27 13:16:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F25F1AA for ; Thu, 27 Mar 2014 13:16:59 +0000 (UTC) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E42EEED3 for ; Thu, 27 Mar 2014 13:16:58 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id e16so4146161qcx.41 for ; Thu, 27 Mar 2014 06:16:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=HNKwXxqVfemK8RozLfYtHqi48gxIIxBKQeFW6CYKIBo=; b=JBqPm8Qt5rPr7fHYjBsRtkiOtle1P+C1KqxALqolk2JmtuINdlynLPO8OUXh+ULFtT QNXhFI92cAWVdiEce0ev8WxG5gMG5W+Er6e1t/g3GMLlM5DSLrBcSd6ghSxmS6UpRxS0 5lVU2HTyiWLi4h7sqf+oPGci0KzmRd+usAsREHIZFTRU0boJlc2i4D+uL/tY9uOCfPNR mvQmT/As5KRuCD7S7EMdiIickILNHCPQdsLXrPPk+cfHMIyeSVuuNUxHZ0jWginMpYAX v85RvL/f8vnxKljx63M5t8BpIC0SsazQkU461vfP11cWDspU8saKQc7NgI0hvP38E5sd Fivg== X-Gm-Message-State: ALoCoQk2xHcKRkpw+K+0hssU2JL3Uqdwnuse72Yer9t2bBRODskJnxO1db6XO5ni2xy2bt84FOMU X-Received: by 10.224.119.141 with SMTP id z13mr1973790qaq.48.1395926212575; Thu, 27 Mar 2014 06:16:52 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.83.102 with HTTP; Thu, 27 Mar 2014 06:16:32 -0700 (PDT) X-Originating-IP: [1.242.225.82] In-Reply-To: <16437CC5729B5345AF77F816513376E81298241F@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E81298241F@MX103CL02.corp.emc.com> From: Julio Merino Date: Thu, 27 Mar 2014 22:16:32 +0900 Message-ID: Subject: Re: fix for sbin/mdconfig/tests/legacy_test.sh To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 13:16:59 -0000 On Fri, Mar 21, 2014 at 1:09 AM, Peel, Casey wrote: > The sbin/mdconfig/tests/legacy_test.sh script fails because it is looking for the wrong runner (diff is against our internal code, but the change is trivial): Committed in r263814 and apologies for the delay; dunno how I missed this. Thank you! From owner-freebsd-testing@FreeBSD.ORG Thu Mar 27 13:19:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D58422D6 for ; Thu, 27 Mar 2014 13:19:18 +0000 (UTC) Received: from mail-qa0-f48.google.com (mail-qa0-f48.google.com [209.85.216.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9758EEF5 for ; Thu, 27 Mar 2014 13:19:18 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so3721292qaj.21 for ; Thu, 27 Mar 2014 06:19:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to:content-type; bh=1R9spzamw+KlRUusij1zKH88n/wemIsZnnNQH+6dPzM=; b=MQcoMJh26bfIhEN/U/q/pQatJ/g8bICo8Z4xNC1R4p4sc6AhHtASv/ndKdC+dpZW95 kWNhF4Zt9xuEKatotaqxSAM4l59G9zQT71nE+JGqp8SvQXQ+tBJkQXVu2Fz2CSOk7CK+ 34ibmVgvEMUJhjuQprGn0df/5gkiJXv2wa8+GEZpTe3/eR6xeHM0caHgpsZvv6Gu7RDL Y+nvXdH8S81LtasRvONuiIW42J7XmQGDAFwSleQJFH4z5klEFC9vjRLam3Oj/lDsnP/Y TNe5GKJdpF1oHNSgEBpgILuC56zLe72FJ9tt3qPZ3DXm2aHWdZGm0/o6XlAH/DMxZJ4k 3uEw== X-Gm-Message-State: ALoCoQkqtisnlNm6q0q6Ermz4GcZv125pgqASIB/4+1ApbtkUlyy01ZE/jRsuPXhRiVBvIDMDMFL X-Received: by 10.224.46.73 with SMTP id i9mr1892309qaf.65.1395926357082; Thu, 27 Mar 2014 06:19:17 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Thu, 27 Mar 2014 06:18:56 -0700 (PDT) X-Originating-IP: [1.242.225.82] From: Julio Merino Date: Thu, 27 Mar 2014 22:18:56 +0900 X-Google-Sender-Auth: Edzrd6hQq3tXHLNE5-3NC4SQZQk Message-ID: Subject: Changes to the test cluster To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 13:19:19 -0000 Hello, Background ========== Validating changes to the test cluster was pretty much impossible without causing downtime on users: every change to the configuration had to wait a full build cycle to see if it worked, and if it didn't the breakage was immediately visible to all of you. After having broken the test runs several times in a row because of this, I have been reorganizing the way things work so that breakages are not as visible. How does this affect you? ========================= If you access test runs from the http://kyua1.nyi.freebsd.org/ master page, you should see no difference. Some links have been updated, but that's about it. If you access specific test runs via a bookmark to http://kyua1.nyi.freebsd.org//, you should fix your bookmarks to point to kyua3 instead. You will notice that the front page now has a new "canary" entry. Use the results of that link at your own risk. It is likely for this link to be broken way more frequently than the others. Also, all of kyua[123] now serve the test suite front page so pointing your browser to any of the instances will work. Details ======= Basically, I have turned kyua1 into a canary host for changes to autotest itself and I have made that machine build FreeBSD with NO_CLEAN=yes for a faster feedback cycle. The previous role of kyua1 has moved to kyua3 which now builds amd64 just as kyua1 used to do. The front page now contains a link to this canary setup, which you may want to check. However, please be aware that you should not rely on the results reported here without comparing them to the other entries. The fact that this machine is using NO_CLEAN=yes will certainly result in false negatives at some point, which will require manual intervention on my part to resolve. With this in mind, whenever I have to work on changes to autotest, I will be pushing them to kyua1 first, waiting for a successful run, and only then pushing the changes to the other workers. More details can be found here: http://wiki.freebsd.org/TestSuite/Cluster Please let me know any thoughts and, especially, where the documentation is lacking the most. Cheers, Julio From owner-freebsd-testing@FreeBSD.ORG Thu Mar 27 22:25:55 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0624AFB1 for ; Thu, 27 Mar 2014 22:25:55 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd51.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4A9DE67 for ; Thu, 27 Mar 2014 22:25:54 +0000 (UTC) Received: from maildlpprd51.lss.emc.com (maildlpprd51.lss.emc.com [10.106.48.155]) by mailuogwprd51.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2RMPqmm027917 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 27 Mar 2014 18:25:52 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com s2RMPqmm027917 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1395959152; bh=KRCpnH6ecQlfHZUmtC9WlQwmJG0=; h=From:To:Subject:Date:Message-ID:Content-Type:MIME-Version; b=GQ6GlMOYOKQ6jJBaD+iAiQv+/pejODOeDHwsmKDePtuO/3b2w7Fbhx9lCKu9XGIKA xTNHxlBUAmanJmryc07k+/5xTMunfaBj73vF8kLiSd6qAf8LTD2TOGb7p5D7+tI5jo vtFAi4mbj1o/mDq9VBXw5+aIbYqSqxxmce/f0QLQ= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com s2RMPqmm027917 Received: from mailusrhubprd04.lss.emc.com (mailusrhubprd04.lss.emc.com [10.253.24.22]) by maildlpprd51.lss.emc.com (RSA Interceptor) for ; Thu, 27 Mar 2014 18:25:38 -0400 Received: from mxhub09.corp.emc.com (mxhub09.corp.emc.com [10.254.92.104]) by mailusrhubprd04.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2RMPc6Q030469 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Thu, 27 Mar 2014 18:25:38 -0400 Received: from MXHUB106.corp.emc.com (10.253.58.23) by mxhub09.corp.emc.com (10.254.92.104) with Microsoft SMTP Server (TLS) id 8.3.327.1; Thu, 27 Mar 2014 18:25:37 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB106.corp.emc.com ([10.253.58.23]) with mapi id 14.03.0158.001; Thu, 27 Mar 2014 18:25:37 -0400 From: "Peel, Casey" To: "freebsd-testing@freebsd.org" Subject: Makefiles for skip-level directories? Thread-Topic: Makefiles for skip-level directories? Thread-Index: Ac9KC3q0P6/AqbVZQcSVRmNVpzR91g== Date: Thu, 27 Mar 2014 22:25:36 +0000 Message-ID: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd04.lss.emc.com X-RSA-Classifications: public Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 22:25:55 -0000 We've created some ATF network tests in sys/net/, which involved creating t= he sys/net/tests directory and sys/net/tests/Makefile. There are currently = no tests under sys/ at all. In order for that Makefile to get picked up by = the build, I added sys/net/Makefile that looks like this: # $FreeBSD$ .include .if ${MK_TESTS} !=3D "no" SUBDIR+=3D tests .endif .include For that to get picked up, I added this to sys/Makefile before .include : .if ${MK_TESTS} !=3D "no" SUBDIR+=3D net .endif My tests build and get installed correctly in /usr/tests: [root@system /usr/tests]# find sys sys sys/net sys/net/t_lagg_smoke sys/net/Kyuafile You'll note that there is no Kyuafile in sys/ which prevents kyua from trav= ersing into sys/net/ to pick up the Kyuafile there and actually execute the= test from /usr/tests All that to say: what magic am I missing in the sys/Makefile to have the sy= s/Kyuafile generated? I've looked and can't find another example in the cur= rent tests that have this type of skip-level test layout. Thanks, -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 From owner-freebsd-testing@FreeBSD.ORG Thu Mar 27 22:52:30 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04E6FC5 for ; Thu, 27 Mar 2014 22:52:30 +0000 (UTC) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B7C2D1F8 for ; Thu, 27 Mar 2014 22:52:29 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id im17so5056100vcb.37 for ; Thu, 27 Mar 2014 15:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=isZ0dD0vd38XqrLkHVJh1/s0LY61/itptrxbV9AVzWg=; b=WjF+dEoo/q7GqJHj1SgL6plUDz/FE0owUY48/CjYhp/RzBWNCI+h0FQl4zG9bpGQJH qs8liljIXddKJMYGWEibpq3MAAAQMQuNT8r9WwfONgmrpg91/+pM57D8fuIw/gwUwcWx eZYFzKOZjI7YWrcgmgQMUAFqJmmuQInfaSr9/EqMEJDE6V0skC5K+r7UBIyrhmTb5HhW cwFnHB2ylw6aF6/02cVIc6x5V9eh1B3RqZXlx0pteivBYbSyBdaRjJxcEyL7ZPzOHcDy XxR8ecX5EGLuhA1NIebcsDmWUfJ5UyfJ9DXNWcfqyQoyoaZc0RKaW7goQd4n11iS9c2x +Ejg== MIME-Version: 1.0 X-Received: by 10.52.119.197 with SMTP id kw5mr3132916vdb.5.1395960748845; Thu, 27 Mar 2014 15:52:28 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.58.207.169 with HTTP; Thu, 27 Mar 2014 15:52:28 -0700 (PDT) In-Reply-To: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> Date: Thu, 27 Mar 2014 16:52:28 -0600 X-Google-Sender-Auth: aqL3WRBKW8usWWDDGq9CTd6ECEA Message-ID: Subject: Re: Makefiles for skip-level directories? From: Alan Somers To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 22:52:30 -0000 On Thu, Mar 27, 2014 at 4:25 PM, Peel, Casey wrote: > We've created some ATF network tests in sys/net/, which involved creating= the sys/net/tests directory and sys/net/tests/Makefile. There are currentl= y no tests under sys/ at all. In order for that Makefile to get picked up b= y the build, I added sys/net/Makefile that looks like this: > # $FreeBSD$ > .include > .if ${MK_TESTS} !=3D "no" > SUBDIR+=3D tests > .endif > .include > > For that to get picked up, I added this to sys/Makefile before .include <= bsd.subdir.mk>: > .if ${MK_TESTS} !=3D "no" > SUBDIR+=3D net > .endif > > My tests build and get installed correctly in /usr/tests: > [root@system /usr/tests]# find sys > sys > sys/net > sys/net/t_lagg_smoke > sys/net/Kyuafile What command are you using to build and install the tests? It is inappropriate to do this as part of buildkernel, because you could have multiple kernel configs installed on one machine, each build with different options. But there's no way to install ATF tests for different kernel configs. I tried putting tests in sys/kern/tests and building them with buildworld, but I had to hack up Makefile.inc1. In a thread in January, we agreed not to put any tests in sys/. Instead, we'll put them in tests/sys. Take a look at tests/sys/netinet; you can probably copy its structure for your tests. http://lists.freebsd.org/pipermail/freebsd-testing/2014-January/000134.html -Alan From owner-freebsd-testing@FreeBSD.ORG Fri Mar 28 01:49:24 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40A86764 for ; Fri, 28 Mar 2014 01:49:24 +0000 (UTC) Received: from mail-qc0-f171.google.com (mail-qc0-f171.google.com [209.85.216.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0131E68E for ; Fri, 28 Mar 2014 01:49:23 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id c9so5260814qcz.2 for ; Thu, 27 Mar 2014 18:49:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=sH2Zl7XVpmOrWpSGGPfAFT9ruxL4a87f09VWK5bh/EY=; b=cueea/7JO+sZXpYTSbk4SWH/RpXN7fXrmys8tckQgnPeLap2e00a82twYTk26q5whh iEwP2C0dQZme1/+O74BHaHd210iZ1/WnAxRqkpbrlCV8HwNiZhvh65Qs9CpbazMUsPOH xHtqWPf+yFZkzlv9nuFXSdjYKnr9oBgV+IGXVJJKDJzNxlyusDNPRm14skhAIgCUjS/f h2aei0F8gX1vF15UXNdoEngb2Y1RpUGq643po9ObbQqhxgmEXa9b+Kbb7N7RKkpRg164 VJ+0FVVhXX757JijkkrCrvDOj0jOgPdDjYIPxuttTCo41UGsAygXOoLinTZNIWO6/FKy qtBQ== X-Gm-Message-State: ALoCoQmkL+voEXLEZFRUA4DqEsPRbxjUtjw8vsYZTwK2XjQGSvQ5GfbbJIOtGwn78hi63XSUfYAF X-Received: by 10.140.98.233 with SMTP id o96mr5778650qge.86.1395971357590; Thu, 27 Mar 2014 18:49:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.83.102 with HTTP; Thu, 27 Mar 2014 18:48:57 -0700 (PDT) X-Originating-IP: [2401:fa00:d:c:c54e:32b8:1d2a:b775] In-Reply-To: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> From: Julio Merino Date: Fri, 28 Mar 2014 10:48:57 +0900 Message-ID: Subject: Re: Makefiles for skip-level directories? To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 01:49:24 -0000 On Fri, Mar 28, 2014 at 7:25 AM, Peel, Casey wrote: [...] > All that to say: what magic am I missing in the sys/Makefile to have the sys/Kyuafile generated? I've looked and can't find another example in the current tests that have this type of skip-level test layout. Alan already answered this in his email by clarifying that kernel-level tests ought to live in src/tests/sys. But, regardless, replying to this specific question for the record: The magic to install intermediate Kyuafiles lives in the 'tests' subdirectory corresponding to where they are installed! See src/bin/tests/Makefile, for example. Cheers From owner-freebsd-testing@FreeBSD.ORG Sun Mar 30 01:17:34 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 796B2CE6 for ; Sun, 30 Mar 2014 01:17:34 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B026FFF for ; Sun, 30 Mar 2014 01:17:34 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id ld13so7246474vcb.33 for ; Sat, 29 Mar 2014 18:17:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Dn8QLer5pf4ttw3FgUlHS/64IMibkQ7Hu+zIgfol7+A=; b=vJq7ATTd+njOXf23SGG4iZ03Jxrq9mPjbCJi7QfGa4vhPsq48agHuSecElGpI1+TI6 XSp2n96X13lsMvgvtV40gWyHbgW+bC1EI0uUYX252ckS8jqIKDeFdfchzbfN8pDwS79D ADTk3dqu72H000uWDiBLg0iMv8X3BSSeFPXh1TG9IDKE0L1k/2ocA8yTYfUqSiG60Jaj Jp+FwT11k5MkbwnDlKMom/jjy468yjJFGBhcFuJdhl0hbuF8tiqVs5aJfPwCZAn3Ylan EmApbKPJAzvLH/aQdAJX0AhkYmBvXzj5k3dT2eOLEO9BW2lwsGbgXCHl9EfeUYXGRck2 6aUQ== MIME-Version: 1.0 X-Received: by 10.52.241.106 with SMTP id wh10mr12683518vdc.16.1396142253250; Sat, 29 Mar 2014 18:17:33 -0700 (PDT) Received: by 10.220.239.84 with HTTP; Sat, 29 Mar 2014 18:17:33 -0700 (PDT) Date: Sat, 29 Mar 2014 21:17:33 -0400 Message-ID: Subject: Jenkins Dashboard From: Thomas Hoffmann To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Mar 2014 01:17:34 -0000 Just wanted to say what a great resource the Jenkins Dashboard is for me. I do near-nightly builds of -CURRENT and having a resource that I can use to virtually eliminate build false starts is priceless. Today's experience provides a good example. Just as I was planning for my nightly build, I noticed that FreeBSD_HEAD #370 failed. Looked in the log to see the error and offending item. Waited for the item to be fixed (committed to svn). Waited for build #371 to complete successfully. Proceeded with my nightly build. So, I was able to avoid a failed build (false start), avoid needless posts to the developer on the mailing lists, and avoid opening a needless PR, all because I knew exactly what was going on. Had I decided to build anyway. I could have easily checked out up to the revision causing the problem because I knew which commit caused the build fail. One question: are all the FreeBSD_HEAD builds for amd64? Thanks for this great resource. -Tom From owner-freebsd-testing@FreeBSD.ORG Sun Mar 30 02:02:40 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8B38C263 for ; Sun, 30 Mar 2014 02:02:40 +0000 (UTC) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 168BB380 for ; Sun, 30 Mar 2014 02:02:39 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id s7so4796856lbd.15 for ; Sat, 29 Mar 2014 19:02:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=M00UvPBUulp5REcBSNduZG1vgz0zGmwYPJMRFBszbLM=; b=vY5QmdHmnAn0K9NE6Wo4qGvFNDZk9aocNVct9uz5Q3Xej7zOzjd0MH2GoMTet9UVmz 39Fx7Rc3JCZuWoDXoE5qJ4XUIpvhG+BSxYmrWdwkvZD2WmVUMGzI6JWLPxU60dm/gtgK gdd3j7Q5n/LXR1+kk5iZhgKz1NuYaxH9da9W3ako5koNFd8jEIvTi5mX+Zf2awMiTtsu HpZYHkDpsTGuCUwHSC4Ptd1sYXvdrH/nICinpmC0mllY/AXloXprNIf5XkcdCc12o8aX W2AJ3mL3MJbkflfnBDUoU6XxgXsQJikFhMstl100ojZLhyod9QlHWQULWVQQUZi33cGr 4rGw== MIME-Version: 1.0 X-Received: by 10.112.28.82 with SMTP id z18mr11678462lbg.18.1396144956759; Sat, 29 Mar 2014 19:02:36 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Sat, 29 Mar 2014 19:02:36 -0700 (PDT) In-Reply-To: References: Date: Sat, 29 Mar 2014 19:02:36 -0700 X-Google-Sender-Auth: NuVTRrxKn36fEa2aDUo1f5PJCoY Message-ID: Subject: Re: Jenkins Dashboard From: Craig Rodrigues To: Thomas Hoffmann Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Mar 2014 02:02:40 -0000 On Sat, Mar 29, 2014 at 6:17 PM, Thomas Hoffmann wrote: > Just wanted to say what a great resource the Jenkins Dashboard is for me. I > do near-nightly builds of -CURRENT and having a resource that I can use to > virtually eliminate build false starts is priceless. I'm glad that you were able to save time based on looking at the Jenkins dashboard. That's cool!! > One question: are all the FreeBSD_HEAD builds for amd64? Currently yes. We will be setting up more builds on different architectures as we get more hardware from the FreeBSD cluster administration team. See our web page for the status of what we are working on: http://wiki.freebsd.org/Jenkins -- Craig From owner-freebsd-testing@FreeBSD.ORG Mon Mar 31 23:27:41 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 405D5305 for ; Mon, 31 Mar 2014 23:27:41 +0000 (UTC) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd01.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DED352A1 for ; Mon, 31 Mar 2014 23:27:40 +0000 (UTC) Received: from maildlpprd01.lss.emc.com (maildlpprd01.lss.emc.com [10.253.24.33]) by mailuogwprd01.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2VN7jXl006650 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 31 Mar 2014 19:07:45 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com s2VN7jXl006650 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1396307265; bh=LiYNOz74YFFLoD+f9PWlQBMKKpk=; h=From:To:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=Wb1cSIeBG2qpMQf0niBVHv8t8LFhKf36GiO6rTI0k7c1M8OC8CSqq3ELrXtuYk8KD PXWWYzf++9j/dkVk+9DA9fc8NK17CHJ5CGcxtAigekycb5kGFCDHGE5s+0k73C3tVu QqRbsUq6F8/ONgr+wFXp4jSiFddSPTNdndTElMr0= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd01.lss.emc.com s2VN7jXl006650 Received: from mailusrhubprd02.lss.emc.com (mailusrhubprd02.lss.emc.com [10.253.24.20]) by maildlpprd01.lss.emc.com (RSA Interceptor) for ; Mon, 31 Mar 2014 19:07:35 -0400 Received: from mxhub05.corp.emc.com (mxhub05.corp.emc.com [128.222.70.202]) by mailusrhubprd02.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s2VN7ZC9010803 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Mon, 31 Mar 2014 19:07:35 -0400 Received: from MXHUB107.corp.emc.com (10.253.50.23) by mxhub05.corp.emc.com (128.222.70.202) with Microsoft SMTP Server (TLS) id 8.3.327.1; Mon, 31 Mar 2014 19:07:35 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB107.corp.emc.com ([10.253.50.23]) with mapi id 14.03.0158.001; Mon, 31 Mar 2014 19:07:35 -0400 From: "Peel, Casey" To: "freebsd-testing@freebsd.org" Subject: RE: Makefiles for skip-level directories? Thread-Topic: Makefiles for skip-level directories? Thread-Index: Ac9KC3q0P6/AqbVZQcSVRmNVpzR91gAPe4yAALq6voA= Date: Mon, 31 Mar 2014 23:07:34 +0000 Message-ID: <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd02.lss.emc.com X-RSA-Classifications: DLM_1, public X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 23:27:41 -0000 > In a thread in January, we agreed not to put any tests in sys/. > Instead, we'll put them in tests/sys. Take a look at > tests/sys/netinet; you can probably copy its structure for your tests. Ah, that helped, thanks. I was able to get them working in test/sys/. > The magic to install intermediate Kyuafiles lives in the 'tests' > subdirectory corresponding to where they are installed! See > src/bin/tests/Makefile, for example. Thanks, but that's about as clear as mud. Is the "magic" the KYUAFILE=3Dyes= line? When is this required and when isn't it? Where is the documentation = on its usage? Is there a document that gives examples on how to correctly set up Makefile= s for various directory layouts? I just spent several hours to get tests pl= aced into src/sbin/geom/class/part/tests pulled into the build and have fai= led miserably. There is a decent amount of documentation on how to write AT= F tests themselves, but I've found no resource on how to correctly add/upda= te Makefiles to make them work. I don't think this is rocket science but it= sure feels like it at this point. -- Casey From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 09:00:33 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5084FB7C for ; Wed, 2 Apr 2014 09:00:33 +0000 (UTC) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.freebsd.org (Postfix) with SMTP id E54FED19 for ; Wed, 2 Apr 2014 09:00:32 +0000 (UTC) Received: (qmail 97457 invoked from network); 2 Apr 2014 08:53:50 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay01.pair.com with SMTP; 2 Apr 2014 08:53:50 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s328rnxG062881 for ; Wed, 2 Apr 2014 10:53:49 +0200 (CEST) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s328rnAc062880 for freebsd-testing@freebsd.org; Wed, 2 Apr 2014 10:53:49 +0200 (CEST) (envelope-from pho) Date: Wed, 2 Apr 2014 10:53:49 +0200 From: Peter Holm To: freebsd-testing@freebsd.org Subject: ATF work Message-ID: <20140402085349.GA61802@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 09:00:33 -0000 I have uploaded a snapshot of the ATF work done for EMC Isilon, based primarily on Garrett Cooper's work. http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz + chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr - Peter From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 16:31:46 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64E58164 for ; Wed, 2 Apr 2014 16:31:46 +0000 (UTC) Received: from mail-we0-x233.google.com (mail-we0-x233.google.com [IPv6:2a00:1450:400c:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F3E9C8E0 for ; Wed, 2 Apr 2014 16:31:45 +0000 (UTC) Received: by mail-we0-f179.google.com with SMTP id x48so495534wes.24 for ; Wed, 02 Apr 2014 09:31:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=pE64YYplw7/D1nGx1Gb/IraxqKtLr5yA3xd2k7FM5aU=; b=Q+pq2BIkSKfRyj+BZobl7LCZUccgbcxn471Rxty8f6w+ESMPYTzmuEEvFT7ENXBJji 1F8ymYniJri5MK7Pc5ecD0pzmM++/w0h3cNYtvr1radVu0pbMqcy9mBdT0cyUp4oDnZv thFBU78LLrrxoJh+vUpwVTkckAbBwZwOCXR7cyXnvjg1AELVZ35WJ2D8hbHu2h+oEpDd GWeDuUA+PCmFvo9kT0C/ulvTyS+pYj9/1i+fJmrrr6gyzSfZ0dBIrPDTQgR2u8RxbdxS uwjxXOeLFpzyMAXdw9Nu2aGCzbmFVMaXJ/vtT5TeDiMzqO5SVgch0d7zBej/ppPmgcm7 nzOw== MIME-Version: 1.0 X-Received: by 10.180.187.16 with SMTP id fo16mr30016165wic.26.1396456304174; Wed, 02 Apr 2014 09:31:44 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Wed, 2 Apr 2014 09:31:44 -0700 (PDT) In-Reply-To: <20140402085349.GA61802@x2.osted.lan> References: <20140402085349.GA61802@x2.osted.lan> Date: Wed, 2 Apr 2014 10:31:44 -0600 X-Google-Sender-Auth: 3_YJIa1QJuvh0krVHQDPLvfBozw Message-ID: Subject: Re: ATF work From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 16:31:46 -0000 On Wed, Apr 2, 2014 at 2:53 AM, Peter Holm wrote: > I have uploaded a snapshot of the ATF work done for EMC Isilon, > based primarily on Garrett Cooper's work. > > http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz > + > chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr Holy patch queue, Batman! There's fifty thousand lines of code in here! I'm not going to read every line, but here's what I noticed: All of the files from NetBSD should have a $FreeBSD:$ tag alongside the NetBSD tag. lib/libc/tests/net/getaddrinfo/Makefile has some commented out code that should be removed. So does lib/libc/tests/net/getaddrinfo/t_getaddrinfo.sh. Could lib/libc/tests/net/gen_ether_subr be converted to an awk script by replacing the first line with "#!/usr/bin/awk -f" and removing the invocation to awk? Then it would have the correct syntax highlighting in editors. lib/libc/tests/net/Makefile has one tested comment out with the comment "test uses rump". Would it be possible instead to leave the test in the build, but put "require.progs rump_server" in the relevant test cases' heads? It would be more idiomatic for lib/libc/tests/tls_dso/Makefile to define a TESTSDIR variable and use that in the definitions of LIBDIR and SHLIBDIR. lib/libc/tests/gen/Makefile has several tests commented out. Several of the comments are confusingly short (what does "XXX: F_MAXFD DNE" mean)? And the "#ATF_TESTS+= posix_spawn" line should be deleted. lib/libc/tests/gen/posix_spawn/Makefile contains commented out code. lib/libc/tests/arch contains subdirectories that FreeBSD doesn't support and never will, like Vax. They should be deleted. It's unfortunate that all of t_sha2 is disabled. I don't understand the comment, either. Is it disabled just because FreeBSD doesn't have sha384 functions? We have both sha1 and sha256 in libc. lib/libc/tests/termios/Makefile has WARNS=4 commented out. Why is that? Some Makefiles define "MAN= ". It is more idiomatic to define "NOMAN=". Check your whitespace in tests/sys/kern/kern_fork_test.c and tests/sys/kern/user_test.c. I see lines with leading spaces. I tried to test your patch, but I was unable to apply it. It looks like you did "diff -dur src /usr/src" or something like that. The problem is that the two paths need different levels of directories to be stripped. Neither patch nor svn patch can deal with that. The result is that either the newly created files appear in the wrong place, or the tool patches /usr/src/Makefile when it's supposed to patch /usr/src/tests/Makefile and /usr/src/libc/Makefile. Could you please try to regenerate the patch? -Alan > > - Peter > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 17:21:28 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9BF9CE46 for ; Wed, 2 Apr 2014 17:21:28 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 25596D9A for ; Wed, 2 Apr 2014 17:21:27 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so424318lbj.31 for ; Wed, 02 Apr 2014 10:21:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=l6qca9mZM/YXp7tOUYoX0WSWd6IPUZcHwlXrksi0BTc=; b=YoAMi+kfa/SG/fFtM0WFECw+VWZSWilxoRIN3p5dvlpJNGX7V4gf+IkD80zaSoLnTk /dGhBWfzdW3cl8Q00qQhFdnbQksK9CIImNnjzBudRbv9E1SSPvGG/xVg/NPfwaAdRcgq dJaN/gKvVwiDz0Dank494gaeJ9mGfQFc6SzV8BdPws6nQC+8o8NAsk9WJ/JI00HU9nSJ PJOVZPIyaeoi1LZFVCFz+85lOr7cPXZuwHaIhFQUWVVUsUchvNMUcqjHb5KPsVmQbPph 838kGpTbUwUCJWoGuiJXZE5EpbR4Cu7cUL8xiZqrPTWM2+7K9rRjsdyuniMlt882e/pW COtQ== MIME-Version: 1.0 X-Received: by 10.112.241.73 with SMTP id wg9mr770122lbc.48.1396459286153; Wed, 02 Apr 2014 10:21:26 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Wed, 2 Apr 2014 10:21:26 -0700 (PDT) In-Reply-To: References: <20140402085349.GA61802@x2.osted.lan> Date: Wed, 2 Apr 2014 10:21:26 -0700 X-Google-Sender-Auth: Yry-rxPp2ZULHr-NXenm1ENH3L0 Message-ID: Subject: Re: ATF work From: Craig Rodrigues To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 17:21:28 -0000 On Wed, Apr 2, 2014 at 9:31 AM, Alan Somers wrote: > On Wed, Apr 2, 2014 at 2:53 AM, Peter Holm wrote: >> I have uploaded a snapshot of the ATF work done for EMC Isilon, >> based primarily on Garrett Cooper's work. >> >> http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz >> + >> chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr > > Holy patch queue, Batman! There's fifty thousand lines of code in here! When a patch is that big, it becomes very difficult to review. You might want to consider forking a copy on github of the https://github.com/freebsd/freebsd repository and maintaining the patch there until it is MFC'd: https://help.github.com/articles/fork-a-repo https://help.github.com/articles/syncing-a-fork It's not perfect but some of the web interface tools around the github ecosystem make reviewing patches (and contributing updates) a bit easier. -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 17:34:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 982E3319 for ; Wed, 2 Apr 2014 17:34:14 +0000 (UTC) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.freebsd.org (Postfix) with SMTP id 32631ED1 for ; Wed, 2 Apr 2014 17:34:13 +0000 (UTC) Received: (qmail 55639 invoked from network); 2 Apr 2014 17:34:12 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay01.pair.com with SMTP; 2 Apr 2014 17:34:12 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s32HYB4e074736; Wed, 2 Apr 2014 19:34:11 +0200 (CEST) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s32HYB5d074735; Wed, 2 Apr 2014 19:34:11 +0200 (CEST) (envelope-from pho) Date: Wed, 2 Apr 2014 19:34:11 +0200 From: Peter Holm To: Alan Somers Subject: Re: ATF work Message-ID: <20140402173411.GA74066@x2.osted.lan> References: <20140402085349.GA61802@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 17:34:14 -0000 On Wed, Apr 02, 2014 at 10:31:44AM -0600, Alan Somers wrote: > On Wed, Apr 2, 2014 at 2:53 AM, Peter Holm wrote: > > I have uploaded a snapshot of the ATF work done for EMC Isilon, > > based primarily on Garrett Cooper's work. > > > > http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz > > + > > chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr > > Holy patch queue, Batman! There's fifty thousand lines of code in here! > > I'm not going to read every line, but here's what I noticed: > The patch is just a snapshot at what's being worked on, where the goal was to get the maximum number of working test scenarios. As you may have noticed the broken ones has just been marked as "skip" / WIP for now. I'll go thru your comments and fix the issues pointed out. Thank you for doing this! I just started looking at rump. It seems some tests needs the user mode tools like for example rump.ifconfig from rumpkernel/netbsd-userspace-src Do you know if anybody has looked at those? As for the format of the patch file; I use it all to time for tests? (cd /usr; patch -p2 < ~pho/FreeBSD-HEAD-ATF-Isilon-20140402.diff) > All of the files from NetBSD should have a $FreeBSD:$ tag alongside > the NetBSD tag. > > lib/libc/tests/net/getaddrinfo/Makefile has some commented out code > that should be removed. > > So does lib/libc/tests/net/getaddrinfo/t_getaddrinfo.sh. > > Could lib/libc/tests/net/gen_ether_subr be converted to an awk script > by replacing the first line with "#!/usr/bin/awk -f" and removing the > invocation to awk? Then it would have the correct syntax highlighting > in editors. > > lib/libc/tests/net/Makefile has one tested comment out with the > comment "test uses rump". Would it be possible instead to leave the > test in the build, but put "require.progs rump_server" in the relevant > test cases' heads? > > It would be more idiomatic for lib/libc/tests/tls_dso/Makefile to > define a TESTSDIR variable and use that in the definitions of LIBDIR > and SHLIBDIR. > > lib/libc/tests/gen/Makefile has several tests commented out. Several > of the comments are confusingly short (what does "XXX: F_MAXFD DNE" > mean)? And the "#ATF_TESTS+= posix_spawn" line should be deleted. > > lib/libc/tests/gen/posix_spawn/Makefile contains commented out code. > > lib/libc/tests/arch contains subdirectories that FreeBSD doesn't > support and never will, like Vax. They should be deleted. > > It's unfortunate that all of t_sha2 is disabled. I don't understand > the comment, either. Is it disabled just because FreeBSD doesn't have > sha384 functions? We have both sha1 and sha256 in libc. > > lib/libc/tests/termios/Makefile has WARNS=4 commented out. Why is that? > > Some Makefiles define "MAN= ". It is more idiomatic to define "NOMAN=". > > Check your whitespace in tests/sys/kern/kern_fork_test.c and > tests/sys/kern/user_test.c. I see lines with leading spaces. > > I tried to test your patch, but I was unable to apply it. It looks > like you did "diff -dur src /usr/src" or something like that. The > problem is that the two paths need different levels of directories to > be stripped. Neither patch nor svn patch can deal with that. The > result is that either the newly created files appear in the wrong > place, or the tool patches /usr/src/Makefile when it's supposed to > patch /usr/src/tests/Makefile and /usr/src/libc/Makefile. Could you > please try to regenerate the patch? > > -Alan > > > > > - Peter > > _______________________________________________ > > freebsd-testing@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" -- Peter From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 17:36:44 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A11E0523; Wed, 2 Apr 2014 17:36:44 +0000 (UTC) Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17A28F07; Wed, 2 Apr 2014 17:36:43 +0000 (UTC) Received: by mail-we0-f181.google.com with SMTP id q58so577914wes.26 for ; Wed, 02 Apr 2014 10:36:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=yxhDP2qlWdUpRRza6XHsM6bswmU+miwj4fttkXBtCko=; b=rISuXea2T92L2HEMqISvdk+1eReRQPT8X4ddeCeltJUrazJaF2SuR/HWYCUe+RmzeJ fL5hMmGumf2QgToAZl/0xZBA77iJuCyigVm6wz6WogoATuE6MQF4pV2ejxfM0gQ7gZs1 VQcMG08mfKrcDAZmkNxChL7MgzIaJxoeSU74hv1Bfq0h6rY1cgVLwlnwk5HGfoksNgtI RPuE3w3f9jjesGi9wHuWOL6NYuwUegRw+HaNvPkbB//LAhlqB7YW2w39/Bg8BUaIVsfe tLfAAPDo04DIgu+nfY5Pt689jgjUjYwincu4OYl1zQtJ4krQBgJhtDTFYRKoXQjtSW3T YbOg== MIME-Version: 1.0 X-Received: by 10.194.157.41 with SMTP id wj9mr2851006wjb.34.1396460202229; Wed, 02 Apr 2014 10:36:42 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Wed, 2 Apr 2014 10:36:42 -0700 (PDT) In-Reply-To: <20140402173411.GA74066@x2.osted.lan> References: <20140402085349.GA61802@x2.osted.lan> <20140402173411.GA74066@x2.osted.lan> Date: Wed, 2 Apr 2014 11:36:42 -0600 X-Google-Sender-Auth: ftx5VscQg7Ppm46L1WVADpV0rns Message-ID: Subject: Re: ATF work From: Alan Somers To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 17:36:44 -0000 On Wed, Apr 2, 2014 at 11:34 AM, Peter Holm wrote: > On Wed, Apr 02, 2014 at 10:31:44AM -0600, Alan Somers wrote: >> On Wed, Apr 2, 2014 at 2:53 AM, Peter Holm wrote: >> > I have uploaded a snapshot of the ATF work done for EMC Isilon, >> > based primarily on Garrett Cooper's work. >> > >> > http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz >> > + >> > chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr >> >> Holy patch queue, Batman! There's fifty thousand lines of code in here! >> >> I'm not going to read every line, but here's what I noticed: >> > > The patch is just a snapshot at what's being worked on, where the > goal was to get the maximum number of working test scenarios. As > you may have noticed the broken ones has just been marked as "skip" > / WIP for now. > > I'll go thru your comments and fix the issues pointed out. Thank you > for doing this! > > I just started looking at rump. It seems some tests needs the user > mode tools like for example rump.ifconfig from > rumpkernel/netbsd-userspace-src Do you know if anybody has looked > at those? rump requires extensive kernel support. It would be great if FreeBSD had it, but it's not going to happen any time soon. > > As for the format of the patch file; I use it all to time for tests? > (cd /usr; patch -p2 < ~pho/FreeBSD-HEAD-ATF-Isilon-20140402.diff) It only works if your source tree is at /usr/src. Mine isn't. What command are you using to produce the diff? > >> All of the files from NetBSD should have a $FreeBSD:$ tag alongside >> the NetBSD tag. >> >> lib/libc/tests/net/getaddrinfo/Makefile has some commented out code >> that should be removed. >> >> So does lib/libc/tests/net/getaddrinfo/t_getaddrinfo.sh. >> >> Could lib/libc/tests/net/gen_ether_subr be converted to an awk script >> by replacing the first line with "#!/usr/bin/awk -f" and removing the >> invocation to awk? Then it would have the correct syntax highlighting >> in editors. >> >> lib/libc/tests/net/Makefile has one tested comment out with the >> comment "test uses rump". Would it be possible instead to leave the >> test in the build, but put "require.progs rump_server" in the relevant >> test cases' heads? >> >> It would be more idiomatic for lib/libc/tests/tls_dso/Makefile to >> define a TESTSDIR variable and use that in the definitions of LIBDIR >> and SHLIBDIR. >> >> lib/libc/tests/gen/Makefile has several tests commented out. Several >> of the comments are confusingly short (what does "XXX: F_MAXFD DNE" >> mean)? And the "#ATF_TESTS+= posix_spawn" line should be deleted. >> >> lib/libc/tests/gen/posix_spawn/Makefile contains commented out code. >> >> lib/libc/tests/arch contains subdirectories that FreeBSD doesn't >> support and never will, like Vax. They should be deleted. >> >> It's unfortunate that all of t_sha2 is disabled. I don't understand >> the comment, either. Is it disabled just because FreeBSD doesn't have >> sha384 functions? We have both sha1 and sha256 in libc. >> >> lib/libc/tests/termios/Makefile has WARNS=4 commented out. Why is that? >> >> Some Makefiles define "MAN= ". It is more idiomatic to define "NOMAN=". >> >> Check your whitespace in tests/sys/kern/kern_fork_test.c and >> tests/sys/kern/user_test.c. I see lines with leading spaces. >> >> I tried to test your patch, but I was unable to apply it. It looks >> like you did "diff -dur src /usr/src" or something like that. The >> problem is that the two paths need different levels of directories to >> be stripped. Neither patch nor svn patch can deal with that. The >> result is that either the newly created files appear in the wrong >> place, or the tool patches /usr/src/Makefile when it's supposed to >> patch /usr/src/tests/Makefile and /usr/src/libc/Makefile. Could you >> please try to regenerate the patch? >> >> -Alan >> >> > >> > - Peter >> > _______________________________________________ >> > freebsd-testing@freebsd.org mailing list >> > https://lists.freebsd.org/mailman/listinfo/freebsd-testing >> > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" > > -- > Peter From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 17:54:36 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2BCC8E0 for ; Wed, 2 Apr 2014 17:54:36 +0000 (UTC) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.freebsd.org (Postfix) with SMTP id 90193170 for ; Wed, 2 Apr 2014 17:54:36 +0000 (UTC) Received: (qmail 62918 invoked from network); 2 Apr 2014 17:54:35 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay01.pair.com with SMTP; 2 Apr 2014 17:54:35 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s32HsYaq075416; Wed, 2 Apr 2014 19:54:34 +0200 (CEST) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s32HsY0v075415; Wed, 2 Apr 2014 19:54:34 +0200 (CEST) (envelope-from pho) Date: Wed, 2 Apr 2014 19:54:34 +0200 From: Peter Holm To: Craig Rodrigues Subject: Re: ATF work Message-ID: <20140402175434.GA75279@x2.osted.lan> References: <20140402085349.GA61802@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 17:54:37 -0000 On Wed, Apr 02, 2014 at 10:21:26AM -0700, Craig Rodrigues wrote: > On Wed, Apr 2, 2014 at 9:31 AM, Alan Somers wrote: > > On Wed, Apr 2, 2014 at 2:53 AM, Peter Holm wrote: > >> I have uploaded a snapshot of the ATF work done for EMC Isilon, > >> based primarily on Garrett Cooper's work. > >> > >> http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz > >> + > >> chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr > > > > Holy patch queue, Batman! There's fifty thousand lines of code in here! > > > When a patch is that big, it becomes very difficult to review. > > You might want to consider forking a copy on github of the > https://github.com/freebsd/freebsd repository and maintaining the patch there > until it is MFC'd: > > https://help.github.com/articles/fork-a-repo > https://help.github.com/articles/syncing-a-fork > > It's not perfect but some of the web interface tools around the github > ecosystem make reviewing patches (and contributing updates) a bit easier. > Thank you for the references. I will definitely look at this. -- Peter From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 18:07:05 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8ED4D90; Wed, 2 Apr 2014 18:07:05 +0000 (UTC) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56B442CE; Wed, 2 Apr 2014 18:07:05 +0000 (UTC) Received: by mail-vc0-f172.google.com with SMTP id la4so835994vcb.17 for ; Wed, 02 Apr 2014 11:07:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=uWAiy2MHYclfASwTAZAitn8a+/u/0CT6/ftcI9ZlYkg=; b=ygLlTGjM9JlNE+dvWWUoo5F97IEce7j5IK+BzzePxIu6oFbNPNUP7kt9cX2uJ/M5Z0 f8scedaA8o3JWNXXicLBOZAGqWA7/6iqlo1ZuzG+cYSo7+5nY+2Q8IBaKp0skouuAwfn 6RVjUvPGNh937HGyGEHpK2bqOmXURdNEgTa5P340FAEte6HC+29dCofGVFpyscC+ubKL JlduMr+iGGlAi78abNZ+sAlaiH1DqboMcxKXC/2im+ipbRZVNthCawbd6dzEig7AQswx +p0WOH2IIFwE2RRQ5Ujo5KDapt7OdgoGna/n5FJs7DI2n3ein2MzqfFTsOcELy7ACKzW +bcw== MIME-Version: 1.0 X-Received: by 10.52.164.175 with SMTP id yr15mr2024177vdb.59.1396462024505; Wed, 02 Apr 2014 11:07:04 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 2 Apr 2014 11:07:04 -0700 (PDT) In-Reply-To: References: <20140402085349.GA61802@x2.osted.lan> Date: Wed, 2 Apr 2014 11:07:04 -0700 Message-ID: Subject: Re: ATF work From: Garrett Cooper To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 18:07:05 -0000 (Just to fill in some context on some of the items here) On Wed, Apr 2, 2014 at 9:31 AM, Alan Somers wrote: ... > All of the files from NetBSD should have a $FreeBSD:$ tag alongside > the NetBSD tag. Yes, I should have added those. > lib/libc/tests/net/getaddrinfo/Makefile has some commented out code > that should be removed. Most likely code that didn't compile in my workspace. > So does lib/libc/tests/net/getaddrinfo/t_getaddrinfo.sh. Not sure, but it might have been code that was broken due to discrepancies between FreeBSD and NetBSD > Could lib/libc/tests/net/gen_ether_subr be converted to an awk script > by replacing the first line with "#!/usr/bin/awk -f" and removing the > invocation to awk? Then it would have the correct syntax highlighting > in editors. I try to avoid that without pushing the change up to NetBSD first as I tried to make as minimal changes as possible to make things work with FreeBSD without things blowing up. > lib/libc/tests/net/Makefile has one tested comment out with the > comment "test uses rump". Would it be possible instead to leave the > test in the build, but put "require.progs rump_server" in the relevant > test cases' heads? Probably carryover from NetBSD that should be pushed back to NetBSD. > It would be more idiomatic for lib/libc/tests/tls_dso/Makefile to > define a TESTSDIR variable and use that in the definitions of LIBDIR > and SHLIBDIR. Good catch; this should potentially be pushed back to NetBSD. > lib/libc/tests/gen/Makefile has several tests commented out. Several > of the comments are confusingly short (what does "XXX: F_MAXFD DNE" > mean)? And the "#ATF_TESTS+= posix_spawn" line should be deleted. DNE -> does not exist, e.g. it's a parameter/functionality that exists on NetBSD and doesn't exist on FreeBSD. (philosophical item) I'm not sure about the commented out lines for posix_spawn as this (again) is a mostly carryover from NetBSD. Whether or not it should persist with a better "descriptor" or should be deleted is a good idea. > lib/libc/tests/gen/posix_spawn/Makefile contains commented out code. Probably incompatibilities between FreeBSD and NetBSD. > lib/libc/tests/arch contains subdirectories that FreeBSD doesn't > support and never will, like Vax. They should be deleted. Good point. Agreed. > It's unfortunate that all of t_sha2 is disabled. I don't understand > the comment, either. Is it disabled just because FreeBSD doesn't have > sha384 functions? We have both sha1 and sha256 in libc. sha384 doesn't exist in FreeBSD; either the code should be adjusted and the adjustment should be contributed back to NetBSD, or leave it as-is. > lib/libc/tests/termios/Makefile has WARNS=4 commented out. Why is that? > > Some Makefiles define "MAN= ". It is more idiomatic to define "NOMAN=". Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 18:09:24 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EBD0FE5; Wed, 2 Apr 2014 18:09:24 +0000 (UTC) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B40330D; Wed, 2 Apr 2014 18:09:24 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id im17so860647vcb.9 for ; Wed, 02 Apr 2014 11:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ohAZlXNTLmtZoLx4J3vC3YqOBYtf2BTGzL+KLjE3XLg=; b=j4eQ+hjADYy5utFz7+kFZmCOBVOEuUDg8z0PJCWgu0zHMMivj+cc/EY1qvzT+Szztw mYTiTcFGE5ldh0gS+yhNgW15Yfc1ZyNFnZH/T32iIbT4b1mIH6RtEA6+c2ZAW498iRP1 SpH4UqioAQgF+nQdMLSvmn2ER0FZtmXINwSEfMuvWYaJo/kay82RiQVVGOz3Oa9JcqCc +tEXhZqp2FpXWEI7fK288pBNgwfUgdiGhkJoY/5sY8vVuPFYQyhc6GY7a5zBvS7KJh0r ehP0D8VNXUvwmhHO5Fm6pDWgEe8KyRJu5huos5KxJXI4EeG+Hb/MobfkT6accK0sy5kU RPcQ== MIME-Version: 1.0 X-Received: by 10.52.164.237 with SMTP id yt13mr2214210vdb.18.1396462163170; Wed, 02 Apr 2014 11:09:23 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 2 Apr 2014 11:09:23 -0700 (PDT) In-Reply-To: References: <20140402085349.GA61802@x2.osted.lan> Date: Wed, 2 Apr 2014 11:09:23 -0700 Message-ID: Subject: Re: ATF work From: Garrett Cooper To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 18:09:24 -0000 (Grammar police -- apologize for the noise) On Wed, Apr 2, 2014 at 11:07 AM, Garrett Cooper wrote: > (Just to fill in some context on some of the items here) ... > DNE -> does not exist, e.g. it's a parameter/functionality that exists > on NetBSD and doesn't exist on FreeBSD. e.g. -> i.e. > (philosophical item) I'm not sure about the commented out lines for > posix_spawn as this (again) is a mostly carryover from NetBSD. Whether > or not it should persist with a better "descriptor" or should be > deleted is a good idea. "is a good idea" -> "... is a good question" From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 18:31:16 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1BADC9A6 for ; Wed, 2 Apr 2014 18:31:16 +0000 (UTC) Received: from relay02.pair.com (relay02.pair.com [209.68.5.16]) by mx1.freebsd.org (Postfix) with SMTP id D5F87855 for ; Wed, 2 Apr 2014 18:31:15 +0000 (UTC) Received: (qmail 7094 invoked from network); 2 Apr 2014 18:31:07 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay02.pair.com with SMTP; 2 Apr 2014 18:31:07 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s32IV7DB076649; Wed, 2 Apr 2014 20:31:07 +0200 (CEST) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s32IV7l5076648; Wed, 2 Apr 2014 20:31:07 +0200 (CEST) (envelope-from pho) Date: Wed, 2 Apr 2014 20:31:07 +0200 From: Peter Holm To: Alan Somers Subject: Re: ATF work Message-ID: <20140402183107.GA76455@x2.osted.lan> References: <20140402085349.GA61802@x2.osted.lan> <20140402173411.GA74066@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 18:31:16 -0000 On Wed, Apr 02, 2014 at 11:36:42AM -0600, Alan Somers wrote: > On Wed, Apr 2, 2014 at 11:34 AM, Peter Holm wrote: > > On Wed, Apr 02, 2014 at 10:31:44AM -0600, Alan Somers wrote: > >> On Wed, Apr 2, 2014 at 2:53 AM, Peter Holm wrote: > >> > I have uploaded a snapshot of the ATF work done for EMC Isilon, > >> > based primarily on Garrett Cooper's work. > >> > > >> > http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz > >> > + > >> > chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr > >> > >> Holy patch queue, Batman! There's fifty thousand lines of code in here! > >> > >> I'm not going to read every line, but here's what I noticed: > >> > > > > The patch is just a snapshot at what's being worked on, where the > > goal was to get the maximum number of working test scenarios. As > > you may have noticed the broken ones has just been marked as "skip" > > / WIP for now. > > > > I'll go thru your comments and fix the issues pointed out. Thank you > > for doing this! > > > > I just started looking at rump. It seems some tests needs the user > > mode tools like for example rump.ifconfig from > > rumpkernel/netbsd-userspace-src Do you know if anybody has looked > > at those? > > rump requires extensive kernel support. It would be great if FreeBSD > had it, but it's not going to happen any time soon. > Oh, that's a shame. > > > > As for the format of the patch file; I use it all to time for tests? > > (cd /usr; patch -p2 < ~pho/FreeBSD-HEAD-ATF-Isilon-20140402.diff) > > It only works if your source tree is at /usr/src. Mine isn't. What > command are you using to produce the diff? > It's just a small script written for the occasion, but as Craig pointed out I *really* need to start using github. - Peter > > > >> All of the files from NetBSD should have a $FreeBSD:$ tag alongside > >> the NetBSD tag. > >> > >> lib/libc/tests/net/getaddrinfo/Makefile has some commented out code > >> that should be removed. > >> > >> So does lib/libc/tests/net/getaddrinfo/t_getaddrinfo.sh. > >> > >> Could lib/libc/tests/net/gen_ether_subr be converted to an awk script > >> by replacing the first line with "#!/usr/bin/awk -f" and removing the > >> invocation to awk? Then it would have the correct syntax highlighting > >> in editors. > >> > >> lib/libc/tests/net/Makefile has one tested comment out with the > >> comment "test uses rump". Would it be possible instead to leave the > >> test in the build, but put "require.progs rump_server" in the relevant > >> test cases' heads? > >> > >> It would be more idiomatic for lib/libc/tests/tls_dso/Makefile to > >> define a TESTSDIR variable and use that in the definitions of LIBDIR > >> and SHLIBDIR. > >> > >> lib/libc/tests/gen/Makefile has several tests commented out. Several > >> of the comments are confusingly short (what does "XXX: F_MAXFD DNE" > >> mean)? And the "#ATF_TESTS+= posix_spawn" line should be deleted. > >> > >> lib/libc/tests/gen/posix_spawn/Makefile contains commented out code. > >> > >> lib/libc/tests/arch contains subdirectories that FreeBSD doesn't > >> support and never will, like Vax. They should be deleted. > >> > >> It's unfortunate that all of t_sha2 is disabled. I don't understand > >> the comment, either. Is it disabled just because FreeBSD doesn't have > >> sha384 functions? We have both sha1 and sha256 in libc. > >> > >> lib/libc/tests/termios/Makefile has WARNS=4 commented out. Why is that? > >> > >> Some Makefiles define "MAN= ". It is more idiomatic to define "NOMAN=". > >> > >> Check your whitespace in tests/sys/kern/kern_fork_test.c and > >> tests/sys/kern/user_test.c. I see lines with leading spaces. > >> > >> I tried to test your patch, but I was unable to apply it. It looks > >> like you did "diff -dur src /usr/src" or something like that. The > >> problem is that the two paths need different levels of directories to > >> be stripped. Neither patch nor svn patch can deal with that. The > >> result is that either the newly created files appear in the wrong > >> place, or the tool patches /usr/src/Makefile when it's supposed to > >> patch /usr/src/tests/Makefile and /usr/src/libc/Makefile. Could you > >> please try to regenerate the patch? > >> > >> -Alan > >> > >> > > >> > - Peter > >> > _______________________________________________ > >> > freebsd-testing@freebsd.org mailing list > >> > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > >> > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" > > > > -- > > Peter From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 18:47:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B243CECE; Wed, 2 Apr 2014 18:47:09 +0000 (UTC) Received: from mail-vc0-x22a.google.com (mail-vc0-x22a.google.com [IPv6:2607:f8b0:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5EDA0A7A; Wed, 2 Apr 2014 18:47:09 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id hu19so941137vcb.1 for ; Wed, 02 Apr 2014 11:47:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=iLyF8+46Sq7YVLVB5LG9B6UkivZHRV4lfAqyNP2vQZk=; b=tPRuTjBvvajAJjO5SNrNsc4wfzXLxMkqi11lVmuDce2ehj2fwUaljniE8rqrnd1sI9 V5Bjz0+eywjZMjRr/oEKBIoWImCi2Jfx/9bDd+RaqPyKDU6ObMKy+YScZMyawJBjynJ2 uTy/SnczIHEdfknzjWvjG7v00573egfJQr8nDvVBS+JncHMyq3kViAzDiM+B1YG8GcVC pNoIa6WuEOPnk2cjeepaY/WDs+YaohmREPGghVik+v+oJXnglugVh7iBNMqX/8z1o8FH 5Ju6SchfT1QCRIEUvfxE+GraKlBdMtQEJ3RpPAETBPTNVrmN4PbvYLFy0Qn7+bCjT7vp F8fA== MIME-Version: 1.0 X-Received: by 10.52.119.178 with SMTP id kv18mr2317379vdb.39.1396464428434; Wed, 02 Apr 2014 11:47:08 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 2 Apr 2014 11:47:08 -0700 (PDT) In-Reply-To: References: <20140402085349.GA61802@x2.osted.lan> Date: Wed, 2 Apr 2014 11:47:08 -0700 Message-ID: Subject: Re: ATF work From: Garrett Cooper To: Craig Rodrigues Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 18:47:09 -0000 On Wed, Apr 2, 2014 at 10:21 AM, Craig Rodrigues wrote: > On Wed, Apr 2, 2014 at 9:31 AM, Alan Somers wrote: >> On Wed, Apr 2, 2014 at 2:53 AM, Peter Holm wrote: >>> I have uploaded a snapshot of the ATF work done for EMC Isilon, >>> based primarily on Garrett Cooper's work. >>> >>> http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz >>> + >>> chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr >> >> Holy patch queue, Batman! There's fifty thousand lines of code in here! > > > When a patch is that big, it becomes very difficult to review. > > You might want to consider forking a copy on github of the > https://github.com/freebsd/freebsd repository and maintaining the patch there > until it is MFC'd: > > https://help.github.com/articles/fork-a-repo > https://help.github.com/articles/syncing-a-fork > > It's not perfect but some of the web interface tools around the github > ecosystem make reviewing patches (and contributing updates) a bit easier. I was going to move master aside and make all of the ATF work into its own separate branch again as my fork's master still contains little tidbits of other fixes and hacks that I made to make things work on my system(s). When I originally did this on my fork I was dealing with a lot more branches (nowadays I mostly maintain my fork instead of develop/port new code). It might be easiest to move forward if pho forks my repo once it's been cleaned up, and the Isilon patches to my code get integrated into my fork as a GitHub pull request. I'll start working on that today and will talk with pho@ offline about getting things all herded together properly. Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 21:38:34 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ACC20A8D for ; Wed, 2 Apr 2014 21:38:34 +0000 (UTC) Received: from mail-qc0-f180.google.com (mail-qc0-f180.google.com [209.85.216.180]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A4B7F1C for ; Wed, 2 Apr 2014 21:38:34 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id w7so918178qcr.39 for ; Wed, 02 Apr 2014 14:38:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=AFgMmqXA3qVAiaeeCK/CqdrMfphZDJ2pHB3IXjbzpnA=; b=GaGut5OqidC+8eJ4ABDR1O5hZbp6psxX0H4FsJyIJfiyAbMBAU0ZAluyXq4fTg39HG T9/xgWubem7Dh8g2EGHTBVO5K+rxvsEICXJdtJEELzzZTHVyokyip8Hqy3L1VgFqxYGi eFlmrkmZ1xch+3aW1ffcFnKd/FrcOIfFogYUpRwfJGyaeF75Q4yPmlsbJUGFby+O7CQ9 wSFsNaCBSuUzXQAz/rvvfMThZzBRXt2NcDq5DAugpVByL/eIEAsewvuH0qVLEoZAUEip JlusqyU+TXnCenVGwNr7MaSwpreGCptfyXtYZ8Jx52LJvxl55/02S06BGkEQyxo01q3u CUyw== X-Gm-Message-State: ALoCoQldoQPJDjq4NFPHyDBa7PjNMdxir8iFqXfX1JiGMv/Cij/F6Ufshh39XbECdMbsm9l9q17o X-Received: by 10.140.29.130 with SMTP id b2mr3335069qgb.48.1396474706951; Wed, 02 Apr 2014 14:38:26 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.102 with HTTP; Wed, 2 Apr 2014 14:38:06 -0700 (PDT) X-Originating-IP: [2620:0:1003:1007:29ba:37ff:bd36:97ee] In-Reply-To: <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> From: Julio Merino Date: Wed, 2 Apr 2014 17:38:06 -0400 X-Google-Sender-Auth: R1ifANsC4cIqaDaS2RdTo_CxG3A Message-ID: Subject: Re: Makefiles for skip-level directories? To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 21:38:34 -0000 On Mon, Mar 31, 2014 at 7:07 PM, Peel, Casey wrote: >> In a thread in January, we agreed not to put any tests in sys/. >> Instead, we'll put them in tests/sys. Take a look at >> tests/sys/netinet; you can probably copy its structure for your tests. > > Ah, that helped, thanks. I was able to get them working in test/sys/. > >> The magic to install intermediate Kyuafiles lives in the 'tests' >> subdirectory corresponding to where they are installed! See >> src/bin/tests/Makefile, for example. > > Thanks, but that's about as clear as mud. Is the "magic" the KYUAFILE=yes line? When is this required and when isn't it? Where is the documentation on its usage? The *.test.mk files in share/mk/ are reasonably documented. The share/mk/README file also includes a few words on bsd.test.mk. There are also heavily commented examples in /usr/src/share/examples/tests/tests/ . And there is also https://wiki.freebsd.org/TestSuite/DeveloperHowTo . Please let me know where the documentation above is lacking and/or how to make it more discoverable. Cheers From owner-freebsd-testing@FreeBSD.ORG Wed Apr 2 23:31:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6CCAFDF; Wed, 2 Apr 2014 23:31:09 +0000 (UTC) Received: from mail-vc0-x229.google.com (mail-vc0-x229.google.com [IPv6:2607:f8b0:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 52DFEACE; Wed, 2 Apr 2014 23:31:09 +0000 (UTC) Received: by mail-vc0-f169.google.com with SMTP id ik5so1415760vcb.14 for ; Wed, 02 Apr 2014 16:31:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=YIa4u4Gk57/Dpmq7NJ+QaVwSgNCQqN1gI0Asqc9nbtE=; b=TmWuqFW7J1Ju086CO9vt5mZL0ehG6AsQRSXJdMtnEUhXn1oEkGW9kKKD1aPB2uVv16 RCVKTQzUYqIR2Z6QpisJ8PF0tthKq138xE4Om7CN0emQRXuDVQZkHVYYvqbTf7ccJ91g uzSYpxqtufg9DEjDRUzf8Bu72kfJ2kEwJlo59dM9BhGyqjTaqA/sc6gHW4pIFcJnlVJ8 g2rGFb59TgE8LOj2/NLayZ081a2fg0o+5OoVarR9+0TUN/rEiz8rXkSOQUE2r47ZXxVO 2lV9tQMLPRDCq7UEduRwevE55L2VoWmLtmBJ1EteQOFwwC9hzoO+3onhfozo/Pfuimo7 4Pbw== MIME-Version: 1.0 X-Received: by 10.52.12.36 with SMTP id v4mr4151906vdb.20.1396481468439; Wed, 02 Apr 2014 16:31:08 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 2 Apr 2014 16:31:08 -0700 (PDT) In-Reply-To: References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> Date: Wed, 2 Apr 2014 16:31:08 -0700 Message-ID: Subject: Re: Makefiles for skip-level directories? From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 23:31:09 -0000 On Wed, Apr 2, 2014 at 2:38 PM, Julio Merino wrote: > On Mon, Mar 31, 2014 at 7:07 PM, Peel, Casey wrote: >>> In a thread in January, we agreed not to put any tests in sys/. >>> Instead, we'll put them in tests/sys. Take a look at >>> tests/sys/netinet; you can probably copy its structure for your tests. >> >> Ah, that helped, thanks. I was able to get them working in test/sys/. >> >>> The magic to install intermediate Kyuafiles lives in the 'tests' >>> subdirectory corresponding to where they are installed! See >>> src/bin/tests/Makefile, for example. >> >> Thanks, but that's about as clear as mud. Is the "magic" the KYUAFILE=yes line? When is this required and when isn't it? Where is the documentation on its usage? > > The *.test.mk files in share/mk/ are reasonably documented. The > share/mk/README file also includes a few words on bsd.test.mk. There > are also heavily commented examples in > /usr/src/share/examples/tests/tests/ . And there is also > https://wiki.freebsd.org/TestSuite/DeveloperHowTo . > > Please let me know where the documentation above is lacking and/or how > to make it more discoverable. As a very brief sidenote, the Kyuafile that handles autodiscovery in FreeBSD head lives under src/test/Kyuafile ; it automatically gets installed when make install is run from src/tests . Cheers! -Garrett From owner-freebsd-testing@FreeBSD.ORG Fri Apr 4 17:14:58 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 30A044A1; Fri, 4 Apr 2014 17:14:58 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd53.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC69D92E; Fri, 4 Apr 2014 17:14:57 +0000 (UTC) Received: from maildlpprd54.lss.emc.com (maildlpprd54.lss.emc.com [10.106.48.158]) by mailuogwprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s34HEl7W017567 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 4 Apr 2014 13:14:49 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com s34HEl7W017567 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1396631689; bh=YCmAT2XQOqyrxq7txBbfhDyV+GA=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=s8WNOTPIAJbSWtkLX1oaVhUWwbrZwVzz40EyfzTXkv6A0bCO+c72enZUDNiKKxLT+ C2fl8S47s3ERVR4WOx6MmOpRDsIYMOu2GmtzvWPV/pw7gHNJd7Ova5WKdY/zpL9lPE ykzn5aWXG8+oxoTbquGYDtOyA4qndaYNAJC4Au7M= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com s34HEl7W017567 Received: from mailusrhubprd02.lss.emc.com (mailusrhubprd02.lss.emc.com [10.253.24.20]) by maildlpprd54.lss.emc.com (RSA Interceptor); Fri, 4 Apr 2014 13:14:39 -0400 Received: from mxhub09.corp.emc.com (mxhub09.corp.emc.com [10.254.92.104]) by mailusrhubprd02.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s34HEcK7019913 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 4 Apr 2014 13:14:39 -0400 Received: from MXHUB103.corp.emc.com (10.253.50.16) by mxhub09.corp.emc.com (10.254.92.104) with Microsoft SMTP Server (TLS) id 8.3.327.1; Fri, 4 Apr 2014 13:14:38 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB103.corp.emc.com ([::1]) with mapi id 14.03.0158.001; Fri, 4 Apr 2014 13:14:38 -0400 From: "Peel, Casey" To: Julio Merino Subject: RE: Makefiles for skip-level directories? Thread-Topic: Makefiles for skip-level directories? Thread-Index: Ac9KC3q0P6/AqbVZQcSVRmNVpzR91gAPe4yAALq6voAAakIbAABStweA Date: Fri, 4 Apr 2014 17:14:37 +0000 Message-ID: <16437CC5729B5345AF77F816513376E820BAC7D2@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd02.lss.emc.com X-RSA-Classifications: public Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 17:14:58 -0000 > Please let me know where the documentation above is lacking and/or how > to make it more discoverable. Well, lets start with the fact that the share/mk/bsd.README file isn't refe= renced anywhere on https://wiki.freebsd.org/TestSuite/DeveloperHowTo yet it= contains information on the Makefile variables that don't seem to be docum= ented anywhere else. It would be good to keep in mind that most testers aren't build engineers a= nd know just enough about Makefiles to get themselves in trouble (me includ= ed) so this is an area that needs to be over-documented. If we want to enab= le people to contribute test code, we need to lower th bar on how to wire t= hem up to the build. In an ideal world, the entire source tree would contai= n tests/ directories ready and primed for testcases to be added and the Mak= efile within them simply updated. In the meantime I'm going to acquire a Sonic Cherry Coke, put up my Do Not = Disturb sign, and see if I can't figure out how the hell this stuff works a= nd write up the definitive guide for n00bs like myself. -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 From owner-freebsd-testing@FreeBSD.ORG Fri Apr 4 19:05:56 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 688EEF28 for ; Fri, 4 Apr 2014 19:05:56 +0000 (UTC) Received: from mail-qa0-f41.google.com (mail-qa0-f41.google.com [209.85.216.41]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22F235EC for ; Fri, 4 Apr 2014 19:05:55 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id j5so3532209qaq.14 for ; Fri, 04 Apr 2014 12:05:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=OwK1xI/l3P8III7uqimBiMYYuoSKwupmT7Kww47IZfg=; b=Wo/LNVzsuUSavrn5c/jSzUKulrXRewbMFYbuwwl+cPOjLd89iGxd+qApDHGHjFuCFr Rvx2zcKpyOz1QuLOrjq5elKD8oBO7x9cZgNgHDywYcAQK1SjcKR/w/FrRxd0qxpFjO0a v984kw1gr1KG9Uc35XIyoXLS4h4f5CBg+GOQEghLhaOkP8S8/qqoyi6RKqdKmLyb4/k3 OPJg8O9P6r08zWBofXMWNjLoL2tmRGWU18NEPO8WWC8z6Hyskufu0yvvu7f+mmE9Yc9p UJ+06z/XvlW1E/D3KIew1gjz9sOTvUyD5QjsPHwDs/3yKUbJ1xUjZoRfEttXQiaqZq8T z85A== X-Gm-Message-State: ALoCoQkWexrGjscYSxQdFx9K4YFUtSt6Bav5rubkebAEvJCPWhK6dnKztn7e6Zs5CE8KfQpAyF0m X-Received: by 10.224.114.209 with SMTP id f17mr16587221qaq.40.1396638349432; Fri, 04 Apr 2014 12:05:49 -0700 (PDT) Received: from mastodon.meroh.net (cpe-184-153-117-174.rochester.res.rr.com. [184.153.117.174]) by mx.google.com with ESMTPSA id f3sm17887011qag.7.2014.04.04.12.05.48 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Apr 2014 12:05:48 -0700 (PDT) Sender: Julio Merino Date: Fri, 4 Apr 2014 15:05:43 -0400 From: Julio Merino To: Garrett Cooper Subject: Re: ATF work Message-ID: <20140404190543.GA652@mastodon.meroh.net> References: <20140402085349.GA61802@x2.osted.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 19:05:56 -0000 Just a couple of minor comments: On Wed, Apr 02, 2014 at 11:07:04AM -0700, Garrett Cooper wrote: > (Just to fill in some context on some of the items here) > > On Wed, Apr 2, 2014 at 9:31 AM, Alan Somers wrote: > > lib/libc/tests/net/Makefile has one tested comment out with the > > comment "test uses rump". Would it be possible instead to leave the > > test in the build, but put "require.progs rump_server" in the relevant > > test cases' heads? > > Probably carryover from NetBSD that should be pushed back to NetBSD. Why? rump is "standard" in NetBSD so that's probably not going to fly. It'd be like saying "require.progs = ls". I'd not keep commented code in the tree. There is no point: by the time FreeBSD gets rump (if ever, but it'd take a really long time regardless), the borrowed code may have significantly changed from upstream. And if rump is never imported, this code will remain commented out forever. > > lib/libc/tests/gen/posix_spawn/Makefile contains commented out code. > > Probably incompatibilities between FreeBSD and NetBSD. Then, like above, remove the code... unless the tests should work in FreeBSD but don't yet, in which case marking them as "expected failures" and accompanying them with a PR would be better. Cheers From owner-freebsd-testing@FreeBSD.ORG Fri Apr 4 19:19:55 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88BCD53A; Fri, 4 Apr 2014 19:19:55 +0000 (UTC) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE9C2759; Fri, 4 Apr 2014 19:19:54 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id z2so1903376wiv.12 for ; Fri, 04 Apr 2014 12:19:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=OMfSZbMfEQtaFHxcdjhgwpbc+MC6engxIe2NQIZdekc=; b=cqNsPWc1pw4En4yz0GlLQ6/I0p+wuGq0h8PONOB5H1jr902/PyYnT5GJawGlKvSONH mFNRWnpNnOWiKkXFd6CEpFqLcbp13CK2SAjximk+Wt+51CODY+TXflxzBohp6yEV8+fD SF8zC6JwEH8l3hn9IK2qkLIdQO/I0Q0WAvPoxrTjHeqxCzfeG+M/8mJ8QlCwdSINM103 rUGk/88YBvqtp9sRYAgKDV41ftVWmBKEABvqLCNpljUjbulEVo7T7fyNvecblIYRUgje cdRW1ncGtdrqSvxcg0uhPHT3fZJUOh+k+n316cZlFr8O9wwbhufiy3NB0nil8RtXFb0l 5Fbw== MIME-Version: 1.0 X-Received: by 10.180.73.19 with SMTP id h19mr6983007wiv.40.1396639193068; Fri, 04 Apr 2014 12:19:53 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Fri, 4 Apr 2014 12:19:52 -0700 (PDT) In-Reply-To: <20140404190543.GA652@mastodon.meroh.net> References: <20140402085349.GA61802@x2.osted.lan> <20140404190543.GA652@mastodon.meroh.net> Date: Fri, 4 Apr 2014 13:19:52 -0600 X-Google-Sender-Auth: sY9NvgNC-IkA7B1hfyuRzAcJ9ys Message-ID: Subject: Re: ATF work From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 19:19:55 -0000 On Fri, Apr 4, 2014 at 1:05 PM, Julio Merino wrote: > Just a couple of minor comments: > > On Wed, Apr 02, 2014 at 11:07:04AM -0700, Garrett Cooper wrote: >> (Just to fill in some context on some of the items here) >> >> On Wed, Apr 2, 2014 at 9:31 AM, Alan Somers wrote: >> > lib/libc/tests/net/Makefile has one tested comment out with the >> > comment "test uses rump". Would it be possible instead to leave the >> > test in the build, but put "require.progs rump_server" in the relevant >> > test cases' heads? >> >> Probably carryover from NetBSD that should be pushed back to NetBSD. > > Why? rump is "standard" in NetBSD so that's probably not going to fly. > It'd be like saying "require.progs = ls". I don't know about NetBSD, but FreeBSD has a lot of optional stuff in base that's compiled in by default. For example, the entire Bluetooth stack can be disabled by WITHOUT_BLUETOOTH=yes make buildworld. At $WORK, I disabled a whole bunch of stuff that way to slim down our product's image. A deeply embedded system, I'm sure, would disable even more. If rump can be disabled in a NetBSD build, then it would be worthwhile for rump-based ATF tests to identify themselves via require.progs. -Alan From owner-freebsd-testing@FreeBSD.ORG Fri Apr 4 19:26:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4AE0A641; Fri, 4 Apr 2014 19:26:09 +0000 (UTC) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [IPv6:2607:f8b0:400c:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8188806; Fri, 4 Apr 2014 19:26:08 +0000 (UTC) Received: by mail-ve0-f175.google.com with SMTP id oz11so1794531veb.20 for ; Fri, 04 Apr 2014 12:26:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=TAE+YUtvaToNR34c96HeTxvHzi+Zv7RZtjlK5DXOBcU=; b=SKiti+jjNmxXPY/NceMQsOqDUK/v3sEDDu7ue4NsZETWx/YNRsSQAEcPYsipZ7QUnr Y+ehSo4wb34di0LwmU9VaumxRZ0e+ZDPy5rTKOYwj/1w98FdWwdHVxJs1B+Z5FA/8gb7 CbOxwvcax3O7cqiTRpxV23k5PJLb2WX9EqOUbVbog84CYdKvNZ6fUbRRAF7DvaFSfcay c8J5sOI8GBKsFr2SIbHvNHFDLubId1RKEXeg/NCem0Q65L6xMF5n+1M+JPp4u2AS+WPY 9Mhr0bZqQYJ9ZjSxKjkvM5gqCK1GY6tSJ4rgMCCIjoOUTjnnyw2nGXkwuskG9HQSIwAI /v/w== MIME-Version: 1.0 X-Received: by 10.52.6.162 with SMTP id c2mr12234142vda.6.1396639567931; Fri, 04 Apr 2014 12:26:07 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Fri, 4 Apr 2014 12:26:07 -0700 (PDT) In-Reply-To: References: <20140402085349.GA61802@x2.osted.lan> <20140404190543.GA652@mastodon.meroh.net> Date: Fri, 4 Apr 2014 12:26:07 -0700 Message-ID: Subject: Re: ATF work From: Garrett Cooper To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 19:26:09 -0000 On Fri, Apr 4, 2014 at 12:19 PM, Alan Somers wrote: > On Fri, Apr 4, 2014 at 1:05 PM, Julio Merino wrote: >> Just a couple of minor comments: >> >> On Wed, Apr 02, 2014 at 11:07:04AM -0700, Garrett Cooper wrote: >>> (Just to fill in some context on some of the items here) >>> >>> On Wed, Apr 2, 2014 at 9:31 AM, Alan Somers wrote: >>> > lib/libc/tests/net/Makefile has one tested comment out with the >>> > comment "test uses rump". Would it be possible instead to leave the >>> > test in the build, but put "require.progs rump_server" in the relevant >>> > test cases' heads? >>> >>> Probably carryover from NetBSD that should be pushed back to NetBSD. >> >> Why? rump is "standard" in NetBSD so that's probably not going to fly. >> It'd be like saying "require.progs = ls". > > I don't know about NetBSD, but FreeBSD has a lot of optional stuff in > base that's compiled in by default. For example, the entire Bluetooth > stack can be disabled by WITHOUT_BLUETOOTH=yes make buildworld. At > $WORK, I disabled a whole bunch of stuff that way to slim down our > product's image. A deeply embedded system, I'm sure, would disable > even more. If rump can be disabled in a NetBSD build, then it would > be worthwhile for rump-based ATF tests to identify themselves via > require.progs. Rump is optional, so I think that Alan's point is valid (from http://netbsd.gw.com/cgi-bin/man-cgi?mk.conf++NetBSD-current): MKRUMP Can be set to ``yes'' or ``no''. Indicates whether the rump(3) headers, libraries and programs are to be installed. Default: ``yes'' I'll add that to my fork. Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Fri Apr 4 22:28:58 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E702FA2; Fri, 4 Apr 2014 22:28:58 +0000 (UTC) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd02.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B8155A9D; Fri, 4 Apr 2014 22:28:57 +0000 (UTC) Received: from maildlpprd01.lss.emc.com (maildlpprd01.lss.emc.com [10.253.24.33]) by mailuogwprd02.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s34MSnO2016881 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 4 Apr 2014 18:28:49 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd02.lss.emc.com s34MSnO2016881 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1396650529; bh=Y+dSB6hz2/eC5vDb34UwmGQ4Bwc=; h=From:To:CC:Subject:Date:Message-ID:References:Content-Type: MIME-Version; b=Cxp/jAl053Wq8F69HTgwm4WqBse+hqxHvPbCsbszRKSf9aYnugbueaaWm8eabRUKL Vtegpgq5iGrqR+JrPYDs2A49WRKuCUVVKpfRKsSZx3JeqtkQmSWEX8oUeZ8vTc1n9C bd5f7TlSx0pueOsc1JSjrSljJ5LRQpa5cE6RhKo0= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd02.lss.emc.com s34MSnO2016881 Received: from mailusrhubprd04.lss.emc.com (mailusrhubprd04.lss.emc.com [10.253.24.22]) by maildlpprd01.lss.emc.com (RSA Interceptor); Fri, 4 Apr 2014 18:28:39 -0400 Received: from mxhub27.corp.emc.com (mxhub27.corp.emc.com [10.254.110.183]) by mailusrhubprd04.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s34MSc4Q027691 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 4 Apr 2014 18:28:39 -0400 Received: from MXHUB107.corp.emc.com (10.253.50.23) by mxhub27.corp.emc.com (10.254.110.183) with Microsoft SMTP Server (TLS) id 8.3.327.1; Fri, 4 Apr 2014 18:28:38 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB107.corp.emc.com ([10.253.50.23]) with mapi id 14.03.0158.001; Fri, 4 Apr 2014 18:28:38 -0400 From: "Peel, Casey" To: Julio Merino Subject: RE: Makefiles for skip-level directories? Thread-Topic: Makefiles for skip-level directories? Thread-Index: Ac9KC3q0P6/AqbVZQcSVRmNVpzR91gAPe4yAALq6voAAakIbAABStweAAAsXjUA= Date: Fri, 4 Apr 2014 22:28:37 +0000 Message-ID: <16437CC5729B5345AF77F816513376E820BACB49@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] Content-Type: multipart/mixed; boundary="_002_16437CC5729B5345AF77F816513376E820BACB49MX103CL02corpem_" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd04.lss.emc.com X-RSA-Classifications: DLM_1, public Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 22:28:58 -0000 --_002_16437CC5729B5345AF77F816513376E820BACB49MX103CL02corpem_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Ok, I figured it out and wrote up the following document (that is published= on our internal wiki). You may take whatever you want of this for the Free= BSD wiki if you find it useful. Something rcattelan pointed out while doing the code review for my gpart ch= anges that I wanted to run past you: is the include bsd.own.mk to pick up MK_TESTS? yes do you need to do that in the files that are not look at MK_TESTS? geom/tests/Makefile ? I probably don't have to, but that's how the other Makefiles like t= hat are done in FreeBSD, eg: src/bin/tests/Makefile bsd.own.mk is included by bsd.init.mk which is included by bsd.test= .mk=20 so it seems redundant maybe we should ask the upstream if is correct ... might be better = to leave it off vs perpetuating something that might not be right -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 -----Original Message----- From: Peel, Casey=20 Sent: Friday, April 04, 2014 10:15 AM To: 'Julio Merino' Cc: freebsd-testing@freebsd.org Subject: RE: Makefiles for skip-level directories? > Please let me know where the documentation above is lacking and/or how > to make it more discoverable. Well, lets start with the fact that the share/mk/bsd.README file isn't refe= renced anywhere on https://wiki.freebsd.org/TestSuite/DeveloperHowTo yet it= contains information on the Makefile variables that don't seem to be docum= ented anywhere else. It would be good to keep in mind that most testers aren't build engineers a= nd know just enough about Makefiles to get themselves in trouble (me includ= ed) so this is an area that needs to be over-documented. If we want to enab= le people to contribute test code, we need to lower th bar on how to wire t= hem up to the build. In an ideal world, the entire source tree would contai= n tests/ directories ready and primed for testcases to be added and the Mak= efile within them simply updated. In the meantime I'm going to acquire a Sonic Cherry Coke, put up my Do Not = Disturb sign, and see if I can't figure out how the hell this stuff works a= nd write up the definitive guide for n00bs like myself. -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 --_002_16437CC5729B5345AF77F816513376E820BACB49MX103CL02corpem_ Content-Type: text/plain; name="freebsd_atf_makefiles_for_dummies.txt" Content-Description: freebsd_atf_makefiles_for_dummies.txt Content-Disposition: attachment; filename="freebsd_atf_makefiles_for_dummies.txt"; size=4404; creation-date="Fri, 04 Apr 2014 22:27:11 GMT"; modification-date="Fri, 04 Apr 2014 22:23:24 GMT" Content-Transfer-Encoding: base64 RnJlZUJTRCBBVEYgTWFrZWZpbGUgZm9yIER1bW1pZXMKClRoZSBwcmltYXJ5IEFURiBkb2N1bWVu dGF0aW9uIGlzIHNhZGx5IGxhY2tpbmcgaW4gZGV0YWlscyBmb3IgaG93IHlvdSB3aXJlIHVwIE1h a2VmaWxlcyBmb3Igc29tZSBkaXJlY3Rvcmllcy4gVGhpcyBhdHRlbXB0cyB0byBzaGVkIHNvbWUg bGlnaHQgb24gdGhpcy4KCj0gS2VybmVsIHRlc3RzIGRvbid0IGZvbGxvdyB0aGUgcGxhY2VtZW50 IHJ1bGVzID0KSWYgeW91IGFyZSB0cnlpbmcgdG8gYWRkIGFuIEFURiB0ZXN0IHRvIHNvbWUgY29k ZSB1bmRlciBzcmMvc3lzLyB5b3UgZG9uJ3QgcHV0IHRoZW0gaW4gc3JjL3N5cy4gSW5zdGVhZCwg dGhlc2UgYmVsb25nIGluIHNyYy90ZXN0cy9zeXMvLiBJbiB0aGlzIGxvY2F0aW9uIHRoZXkgZm9s bG93IHNpbWlsYXIgcnVsZXMgYXMgZWxzZXdoZXJlIC0tIGlmIHlvdSdyZSB3cml0aW5nIGEgdGVz dCBmb3Igc3JjL3N5cy9uZXQgdGhlIHRlc3RzIGJlbG9uZyBpbiBzcmMvdGVzdHMvc3lzL25ldC8K Cj0gV2lyaW5nIHVwIHRoZSB0ZXN0cy8gc3ViZGlyZWN0b3J5ID0KV2hlbiBhZGRpbmcgYSBuZXcg dGVzdHMvIGRpcmVjdG9yeSBzb21ld2hlcmUgaW4gdGhlIHRyZWUsIGl0cyBpbXBvcnRhbnQgdG8g d2lyZSBpdCB1cCB0byBpdHMgcGFyZW50IE1ha2VmaWxlIHNvIGl0IGNhbiBiZSBzZWVuIGJ5IHRo ZSBidWlsZC4KClRoZSBbaHR0cHM6Ly93aWtpLmZyZWVic2Qub3JnL1Rlc3RTdWl0ZS9EZXZlbG9w ZXJIb3dUbyBGcmVlQlNEIGluc3RydWN0aW9uc10gc2F5IHNpbXBseSB0byAiRWRpdCB0aGUgcGFy ZW50IE1ha2VmaWxlIHRvIHJlY3Vyc2UgaW50byB0aGUgbmV3IHN1YmRpcmVjdG9yeSIgYnV0IGl0 IGRvZXNuJ3QgdGVsbCB5b3UgaG93IHRvIGRvIHRoYXQuIEEgbmlldmUgcmVhZGluZyBzYXlzIGl0 J3MgYXMgc2ltcGxlIGFzIGVkaXRpbmcgdGhlIHBhcmVudCBkaXJlY3RvcnkncyBNYWtlZmlsZSBh bmQgYWRkaW5nIHRoaXM6CiAuaWYgJHtNS19URVNUU30gIT0gIm5vIgogU1VCRElSKz0gICAgdGVz dHMKIC5lbmRpZgogCiAuaW5jbHVkZSA8YnNkLnN1YmRpci5taz4KCkFuZCB0aGF0IHdvcmtzIC4u LiBzb21ldGltZXMuIFVuZm9ydHVuYXRlbHkgdGhhdCBtYWtlcyBzb21lIGh1Z2UgYXNzdW1wdGlv bnMgYWJvdXQgaG93IHRoZSBwYXJlbnQgZGlyZWN0b3J5J3MgTWFrZWZpbGUgaXMgY29kZWQuIEZv ciBzb21lIE1ha2VmaWxlcywgbGlrZSB0aG9zZSBpbiBzcmMvc2Jpbi9nZW9tL2NsYXNzL3BhcnQs IHRoaXMgd29uJ3Qgd29yay4gVGhhdCBNYWtlZmlsZSBsb29rcyBsaWtlIHRoaXM6CiAjICRGcmVl QlNEJAogCiAuUEFUSDogJHsuQ1VSRElSfS8uLi8uLi9taXNjCiAKIEdFT01fQ0xBU1M9CXBhcnQK IAogRFBBREQ9CSR7TElCVVRJTH0KIExEQUREPQktbHV0aWwKIAogLmluY2x1ZGUgPGJzZC5saWIu bWs+CgpBZGRpbmcgdGhlIFNVQkRJUiBzdHVmZiB0byB0aGUgZW5kIG9mIHRoaXMgZmlsZSB3b24n dCB3b3JrLiBic2QubGliLm1rIGRvZXMgc29tZSB0aGluZ3MgdGhhdCB3aWxsIHNraXAgc3ViZGly ZWN0b3J5IHByb2Nlc3Npbmcgc28geW91IGhhdmUgdG8gbW92ZSBpdCBhYm92ZSB0aGF0OgogIyAk RnJlZUJTRCQKIAogLlBBVEg6ICR7LkNVUkRJUn0vLi4vLi4vbWlzYwogCiBHRU9NX0NMQVNTPQlw YXJ0CiAKIERQQUREPQkke0xJQlVUSUx9CiBMREFERD0JLWx1dGlsCiAKIC5pZiAke01LX1RFU1RT fSAhPSAibm8iCiBTVUJESVIrPSAgICB0ZXN0cwogLmVuZGlmCiAgCiAuaW5jbHVkZSA8YnNkLnN1 YmRpci5taz4KIAogLmluY2x1ZGUgPGJzZC5saWIubWs+CgpUaGlzIHdvbid0IHdvcmsgZWl0aGVy IHRob3VnaCBiZWNhdXNlIE1LX1RFU1RTIGlzbid0IGRlZmluZWQgeWV0LiBUaGF0IGRvZXNuJ3Qg Z2V0IGRlZmluZWQgdW50aWwgYnNkLm93bi5tayBnZXRzIHB1bGxlZCBpbiB0aHJvdWdoIGJzZC5s aWIubWsuIFRvIHJlc29sdmUgdGhhdCwgeW91IG5lZWQgdG8gaW5jbHVkZSB0aGF0IGF0IHRoZSB0 b3Agb2YgdGhlIGZpbGU6CiAjICRGcmVlQlNEJAogCiAuaW5jbHVkZSA8YnNkLm93bi5taz4KIAog LlBBVEg6ICR7LkNVUkRJUn0vLi4vLi4vbWlzYwogCiBHRU9NX0NMQVNTPQlwYXJ0CiAKIERQQURE PQkke0xJQlVUSUx9CiBMREFERD0JLWx1dGlsCiAKIC5pZiAke01LX1RFU1RTfSAhPSAibm8iCiBT VUJESVIrPSAgICB0ZXN0cwogLmVuZGlmCiAgCiAuaW5jbHVkZSA8YnNkLnN1YmRpci5taz4KIAog LmluY2x1ZGUgPGJzZC5saWIubWs+CgpTdWNjZXNzISBUaGlzIGlzIG5vdyBwdWxsaW5nIGluIHRo ZSBwYXJ0L3Rlc3RzLyBkaXJlY3RvcnkgYW5kIHRoZSBmaWxlcyBhcmUgYmVpbmcgbGFpZCBvdXQg aW4gdGhlIC91c3IvdGVzdHMgZGlyZWN0b3J5OgogW3Jvb3QgL3Vzci90ZXN0c10jIGZpbmQgc2Jp bi9nZW9tCiBzYmluL2dlb20KIHNiaW4vZ2VvbS9jbGFzcwogc2Jpbi9nZW9tL2NsYXNzL3BhcnQK IHNiaW4vZ2VvbS9jbGFzcy9wYXJ0L3RfZ3BhcnQKIHNiaW4vZ2VvbS9jbGFzcy9wYXJ0L0t5dWFm aWxlCgpFeGNlcHQga3l1YSBjYW4ndCBmaW5kIHRoZW06CiBbcm9vdCAvdXNyL3Rlc3RzXSMga3l1 YSBsaXN0IHwgZ3JlcCBncGFydAogW3Jvb3QgL3Vzci90ZXN0c10jIAoKPSBXaXJpbmcgdXAgYWxs IHRlc3RzLyBkaXJlY3RvcnkgYW5jZXN0b3JzID0Ka3l1YSByZWxpZXMgb24gZWFjaCBkaXJlY3Rv cnkgdW5kZXIgL3Vzci90ZXN0cyB3aXRoIHN1YmRpcmVjdG9yaWVzIHRvIGNvbnRhaW4gYSBLeXVh ZmlsZS4gSWYgeW91J3ZlIGFkZGVkIGEgZGVlcCBzdWJkaXJlY3RvcnkgKGxpa2UgaW4gdGhlIHNi aW4vZ2VvbS9jbGFzcy9wYXJ0IGV4YW1wbGUgYWJvdmUpIHRoZXNlIGludGVybWVkaWFyeSBLeXVh ZmlsZXMgd29uJ3QgZXhpc3QuIFRvIGZpeCB0aGF0IGV2ZXJ5IGFuY2VzdG9yIGRpcmVjdG9yeSBv ZiB5b3VyIHRlc3RzIGRpcmVjdG9yeSBhbHNvIG5lZWQgYSB0ZXN0cy8gZGlyZWN0b3J5IGFuZCBh IE1ha2VmaWxlIHRoYXQgbG9va3Mgc2ltaWxhciB0byB0aGlzOgogIyAkRnJlZUJTRCQKIAogLmlu Y2x1ZGUgPGJzZC5vd24ubWs+CiAKIFRFU1RTRElSPSAgICAgICAke1RFU1RTQkFTRX0vc2Jpbi9n ZW9tL2NsYXNzCiAKIC5QQVRIOiAgICAgICAgICAkey5DVVJESVI6SDpIOkh9L3Rlc3RzCiBLWVVB RklMRT0gICAgICAgeWVzCiAKIC5pbmNsdWRlIDxic2QudGVzdC5taz4KClRFU1RTRElSIHNob3Vs ZCBwb2ludCB0byB0aGUgZGlyZWN0b3J5IHRoYXQgdGhlIEt5dWFmaWxlIHdpbGwgZW5kIHVwIGlu IC91c3IvdGVzdHMgKGp1c3QgbGlrZSBhbnkgb3RoZXIgVEVTVFNESVIpLgoKLlBBVEg6IG5lZWRz IHRvIHBvaW50IHRvIHRoZSBzcmMvdGVzdHMgZGlyZWN0b3J5IHdoZXJlIHRoZSBLeXVhZmlsZSB3 aWxsIGJlIHB1bGxlZCBpbiBmcm9tLiBUaGUgOkggc3ludGF4IGluICR7LkNVUkRJUn0gc3BlY2lm aWVzIGEgcmVsYXRpdmUgcGF0aCB0byAkey5DVVJESVJ9LiAkey5DVVJESVJ9IGlzIHRoZSBjdXJy ZW50IHdvcmtpbmcgZGlyZWN0b3J5LCAkey5DVVJESVI6SH0gaXMgdGhlIHBhcmVudCBkaXJlY3Rv cnkgb2YgdGhlIGN1cnJlbnQgd29ya2luZyBkaXJlY3RvcnksICR7LkNVUkRJUjpIOkh9IGlzIHRo ZSBwYXJlbnQncyBwYXJlbnQgZGlyZWN0b3J5LCBldGMuIFNvICR7LkNVUkRJUjpIOkg6SH0vdGVz dHMgcG9pbnRzIHRvIC4uLy4uLy4uL3Rlc3RzLiBJbiB0aGUgY2FzZSBvZiBzeXMvc2Jpbi9nZW9t L2NsYXNzLywgdGhhdCByZXNvbHZlcyB0byBzeXMvdGVzdHMvLgoKS1lVQUZJTEU9eWVzIHRlbGxz IHRoZSBNYWtlZmlsZSB0byBwdWxsIGFuIGV4aXN0aW5nIEt5dWFmaWxlLCBub3QgdG8gY3JlYXRl IG9uZSBpdHNlbGYuIEl0IGlzIGNyaXRpY2FsIHRvIGdldCAuUEFUSCBzZXQgdXAgY29ycmVjdGx5 IHNvIHRoZSBLeXVhZmlsZSBjYW4gYmUgZm91bmQgYnkgdGhlIGJ1aWxkLgoKTm93IGZvciBlYWNo IG9mIHRoZXNlIG5ldyB0ZXN0cy8gZGlyZWN0b3JpZXMsIHlvdSBuZWVkIHRvIHdpcmUgdGhlbSB1 cCB0byB0aGUgcGFyZW50IGRpcmVjdG9yeSdzIE1ha2VmaWxlIHBlciB0aGUgaW5zdHJ1Y3Rpb25z IGFib3ZlLgoKVGhpcyBtZWFucyB0aGF0IHRvIGFkZCB0ZXN0cyB3aXRoaW4gc2Jpbi9nZW9tL2Ns YXNzL3BhcnQvdGVzdHMgeW91J2xsIG5lZWQgdG8gY3JlYXRlIHRoZSBmb2xsb3dpbmcgYWRkaXRp b25hbCBkaXJlY3RvcmllczoKIHNiaW4vZ2VvbS90ZXN0cwogc2Jpbi9nZW9tL2NsYXNzL3Rlc3Rz CkFuZCBwb3B1bGF0ZSB0aGVtIHdpdGggY3VzdG9taXplZCBNYWtlZmlsZXM6CiBzYmluL2dlb20v dGVzdHMvTWFrZWZpbGUKIHNiaW4vZ2VvbS9jbGFzcy90ZXN0cy9NYWtlZmlsZQpUaGUgc2Jpbi90 ZXN0cyBkaXJlY3RvcnkgYWxyZWFkeSBleGlzdHMsIG9yIHdlIHdvdWxkIGhhdmUgbmVlZGVkIHRv IGFkZCB0aGF0IHRvby4K --_002_16437CC5729B5345AF77F816513376E820BACB49MX103CL02corpem_-- From owner-freebsd-testing@FreeBSD.ORG Fri Apr 4 23:28:21 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2BD27AA; Fri, 4 Apr 2014 23:28:21 +0000 (UTC) Received: from mail-pd0-x22b.google.com (mail-pd0-x22b.google.com [IPv6:2607:f8b0:400e:c02::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DF0CF5F; Fri, 4 Apr 2014 23:28:21 +0000 (UTC) Received: by mail-pd0-f171.google.com with SMTP id r10so3974202pdi.30 for ; Fri, 04 Apr 2014 16:28:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=Qn2AhW5t3hpCbV9JUsBYxq7WyRwa3QEtwxeHGY8ApHI=; b=GZbtsrTcPTo5BsQ+f0g5RkUMjIZBPRT2kSna8A3LDn4YZGDlYcKzf/W7LtKH9gv5tA 2BSPT/9Ewt94mlMLn0M1ktlSmXKxf/+vKyPUffkHqdkEtdEPs59de8Z0Of2WTcxMheqJ 0cF7aESI/U2sJOFWadKZ5OGVK/Wh9GHyPQvWlB952M/nGeG5rf0oIqwkQ/LWXqWXXLB+ qDhCKsTwKo05/jiy5lUO5L3R7pEhYYGWW0olT6BHQl0dBNIalQ93ZrxeCfAq/jBMT0zo AwxtSs3KG+JUluGrIyOLVZ9L0fysL61nT7QHDmwhyygFfs0iG7Tm5s9R0zb04bR4+c2+ 4kNQ== X-Received: by 10.68.211.164 with SMTP id nd4mr17759805pbc.44.1396654101293; Fri, 04 Apr 2014 16:28:21 -0700 (PDT) Received: from [10.80.66.181] ([166.170.41.109]) by mx.google.com with ESMTPSA id ac5sm20270576pbc.37.2014.04.04.16.28.20 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 04 Apr 2014 16:28:20 -0700 (PDT) References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BACB49@MX103CL02.corp.emc.com> Mime-Version: 1.0 (1.0) In-Reply-To: <16437CC5729B5345AF77F816513376E820BACB49@MX103CL02.corp.emc.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <2194A8F0-810B-471A-AD9C-C95C469EAD99@gmail.com> X-Mailer: iPhone Mail (11D167) From: Garrett Cooper Subject: Re: Makefiles for skip-level directories? Date: Fri, 4 Apr 2014 16:28:16 -0700 To: "Peel, Casey" Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Apr 2014 23:28:21 -0000 > On Apr 4, 2014, at 15:28, "Peel, Casey" wrote: >=20 > Ok, I figured it out and wrote up the following document (that is publishe= d on our internal wiki). You may take whatever you want of this for the Free= BSD wiki if you find it useful. >=20 > Something rcattelan pointed out while doing the code review for my gpart c= hanges that I wanted to run past you: > is the include bsd.own.mk to pick up MK_TESTS? > yes > do you need to do that in the files that are not look at MK_TESTS?= > geom/tests/Makefile ? > I probably don't have to, but that's how the other Makefiles like t= hat are done in FreeBSD, eg: src/bin/tests/Makefile > bsd.own.mk is included by bsd.init.mk which is included by bsd.tes= t.mk=20 > so it seems redundant > maybe we should ask the upstream if is correct ... might be better= to leave it off vs perpetuating something that might not be right Yeah, bsd.own.mk is redundant, but it doesn't do any harm minus a hit on the= cache, and increase the file sizes a small amount. In the event that the Makefile snippet would be removed from the included ma= kefiles then stuff would break horribly. Plus it makes it clear that MK_TEST= S is obtained from bsd.own.mk. Just a thought :/... Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Sat Apr 5 17:06:34 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D64537AD; Sat, 5 Apr 2014 17:06:34 +0000 (UTC) Received: from mail-pb0-x233.google.com (mail-pb0-x233.google.com [IPv6:2607:f8b0:400e:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A263C7E6; Sat, 5 Apr 2014 17:06:34 +0000 (UTC) Received: by mail-pb0-f51.google.com with SMTP id uo5so4863645pbc.10 for ; Sat, 05 Apr 2014 10:06:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=w3a8+Ss7FzGRwct9Jz8HIA6HjhEoVDdADp/WhgjzOuw=; b=LYZqI+lzMqj9Dw9RfH3tIYuE9Gh5mwXdoiVNlgdM6IpjZwWNL74d6OqETC6VH9Wp9m pjMVJ8oPBi+8GtyYf0Bsetn834xY/aUJSyrSucrMV+vGVaPfYD60bgEE0Q7IpzqF4ofE VfX/aRruowVa3FSQKhADVH4Rc4BtUTQ7Cp5wTvUED4gC5PpKIonB4JULGFiGLVYx2ynm MsfBDgqdKXMa/QYl1GkRmsKqJ4/JifUAE/7qFIpOn21RvzeTFsuxfahxHuozZTs+B1Rg fXAPDf9iomkKqt6pU8grcB/KAw7MZMU2awmzLk5usXDl0QQ9TqWPEK/O8f4W6ofc8R7i ar4g== X-Received: by 10.66.232.40 with SMTP id tl8mr3399pac.137.1396717594221; Sat, 05 Apr 2014 10:06:34 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:594:7736:96fd:de7e? ([2601:8:ab80:7d6:594:7736:96fd:de7e]) by mx.google.com with ESMTPSA id pv4sm25465400pbb.55.2014.04.05.10.06.32 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 05 Apr 2014 10:06:33 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: ATF work From: Garrett Cooper In-Reply-To: Date: Sat, 5 Apr 2014 10:06:30 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <74A4FAAB-88B8-4226-949A-11AA8D99D662@gmail.com> References: <20140402085349.GA61802@x2.osted.lan> To: Craig Rodrigues X-Mailer: Apple Mail (2.1874) Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 17:06:35 -0000 On Apr 2, 2014, at 11:47 AM, Garrett Cooper = wrote: > On Wed, Apr 2, 2014 at 10:21 AM, Craig Rodrigues = wrote: >> On Wed, Apr 2, 2014 at 9:31 AM, Alan Somers = wrote: >>> On Wed, Apr 2, 2014 at 2:53 AM, Peter Holm wrote: >>>> I have uploaded a snapshot of the ATF work done for EMC Isilon, >>>> based primarily on Garrett Cooper's work. >>>>=20 >>>> = http://people.freebsd.org/~pho/FreeBSD-HEAD-ATF-Isilon-20140401.diff.xz >>>> + >>>> chmod +x /usr/src/lib/libc/tests/net/gen_ether_subr >>>=20 >>> Holy patch queue, Batman! There's fifty thousand lines of code in = here! >>=20 >>=20 >> When a patch is that big, it becomes very difficult to review. >>=20 >> You might want to consider forking a copy on github of the >> https://github.com/freebsd/freebsd repository and maintaining the = patch there >> until it is MFC'd: >>=20 >> https://help.github.com/articles/fork-a-repo >> https://help.github.com/articles/syncing-a-fork >>=20 >> It's not perfect but some of the web interface tools around the = github >> ecosystem make reviewing patches (and contributing updates) a bit = easier. >=20 > I was going to move master aside and make all of the ATF work into its > own separate branch again as my fork's master still contains little > tidbits of other fixes and hacks that I made to make things work on my > system(s). When I originally did this on my fork I was dealing with a > lot more branches (nowadays I mostly maintain my fork instead of > develop/port new code). I=92ve master on my fork reflects FreeBSD master. =93atf=94 now = reflects all of my ATF work, so if you diff master and atf you=92ll see = the relevant changes I=92ve done for ATF in the past. I=92ll update the = docs on the wiki to reflect this. Thanks! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Wed Apr 9 22:51:54 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 645F6E09 for ; Wed, 9 Apr 2014 22:51:54 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd52.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A2741E60 for ; Wed, 9 Apr 2014 22:51:53 +0000 (UTC) Received: from maildlpprd56.lss.emc.com (maildlpprd56.lss.emc.com [10.106.48.160]) by mailuogwprd52.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s39Mppx0013104 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 9 Apr 2014 18:51:51 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com s39Mppx0013104 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1397083912; bh=jf0XxqupO7EtaluVuxcFa62gcP8=; h=From:To:Subject:Date:Message-ID:Content-Type:MIME-Version; b=vyOk83kTqVxlbOq4Zs3z047QiKiLW+qJ1UeswxKco0OWFPNpSujOJIFMFEzKWQP1V Wc/vJf+7RoIQO+LB3sH5efUmeLXJ8DNvPljD7F6q1OOnJfJwc0i1CBo7mssUUXbwpl 0K+XtWuly0PqQgkoC8LAYjFiKKv4MYIeqEwu8ln8= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com s39Mppx0013104 Received: from mailusrhubprd53.lss.emc.com (mailusrhubprd53.lss.emc.com [10.106.48.18]) by maildlpprd56.lss.emc.com (RSA Interceptor) for ; Wed, 9 Apr 2014 18:51:39 -0400 Received: from mxhub06.corp.emc.com (mxhub06.corp.emc.com [128.222.70.203]) by mailusrhubprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s39MpdaY018738 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Wed, 9 Apr 2014 18:51:39 -0400 Received: from MXHUB106.corp.emc.com (10.253.58.23) by mxhub06.corp.emc.com (128.222.70.203) with Microsoft SMTP Server (TLS) id 8.3.327.1; Wed, 9 Apr 2014 18:51:38 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB106.corp.emc.com ([10.253.58.23]) with mapi id 14.03.0158.001; Wed, 9 Apr 2014 18:51:38 -0400 From: "Peel, Casey" To: "freebsd-testing@freebsd.org" Subject: Diff to fix pjdfstest tests Thread-Topic: Diff to fix pjdfstest tests Thread-Index: Ac9UQ7bQLMqvk3xBQ5WGVtQa6VT5hQ== Date: Wed, 9 Apr 2014 22:51:38 +0000 Message-ID: <16437CC5729B5345AF77F816513376E820BAE6A1@MX103CL02.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] Content-Type: multipart/mixed; boundary="_004_16437CC5729B5345AF77F816513376E820BAE6A1MX103CL02corpem_" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd53.lss.emc.com X-RSA-Classifications: public X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 22:51:54 -0000 --_004_16437CC5729B5345AF77F816513376E820BAE6A1MX103CL02corpem_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable In attempting to get pjdfstest working I found that some of the tests have = easily-corrected failures: One of which is a plan error: ftruncate/00.t runs 26 tests but only specifi= es 21 in the plan. Tests that create a small memory disk and fill it to ENOSPC don't validate = that the memory disk mounts and will happily attempt to fill up your real p= artition. The core of the failing is that the memory disk isn't large enoug= h for the subsequent newfs to succeed and thus the mount fails. vbox1009# mdconfig -a -n -t malloc -s 256k 0 vbox1009# newfs /dev/md0 /dev/md0: 0.2MB (512 sectors) block size 32768, fragment size 4096 using 1 cylinder groups of 0.25MB, 8 blks, 128 inodes. super-block backups (for fsck -b #) at: 192 first cylinder group ran out of space vbox1009# mount /dev/md0 /mnt mount: /dev/md0: Invalid argument vbox1009# I've increased the size of the memdisk and added a hard exit to the test if= the mount fails. The open/16.t tests are all failing. The tests expect a ELOOP exit code but= I'm getting EMLINK. I haven't validated if this is a test error or an erro= r on the not-exactly-FreeBSD-10 code I'm running, so I haven't included tho= se tests in this diff and just made them TODO in my code until I figure it = out. root # prove -r --nocount /svn/branch/src/tools/regression/pjdfstest/tests All tests successful. Files=3D206, Tests=3D12100, 839 wallclock secs ( 3.04 usr 2.71 sys + 40.46= cusr 374.61 csys =3D 420.82 CPU) Result: PASS root # -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 --_004_16437CC5729B5345AF77F816513376E820BAE6A1MX103CL02corpem_ Content-Type: application/octet-stream; name="pjdfstest.diff" Content-Description: pjdfstest.diff Content-Disposition: attachment; filename="pjdfstest.diff"; size=3023; creation-date="Wed, 09 Apr 2014 22:41:03 GMT"; modification-date="Wed, 09 Apr 2014 22:31:19 GMT" Content-Transfer-Encoding: base64 SW5kZXg6IHRvb2xzL3JlZ3Jlc3Npb24vcGpkZnN0ZXN0L3Rlc3RzL2Z0cnVuY2F0ZS8wMC50Cj09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT0KLS0tIHRvb2xzL3JlZ3Jlc3Npb24vcGpkZnN0ZXN0L3Rlc3RzL2Z0cnVuY2F0ZS8w MC50CShyZXZpc2lvbiAyNjQzMDkpCisrKyB0b29scy9yZWdyZXNzaW9uL3BqZGZzdGVzdC90ZXN0 cy9mdHJ1bmNhdGUvMDAudAkod29ya2luZyBjb3B5KQpAQCAtNiw3ICs2LDcgQEAKIGRpcj1gZGly bmFtZSAkMGAKIC4gJHtkaXJ9Ly4uL21pc2Muc2gKIAotZWNobyAiMS4uMjEiCitlY2hvICIxLi4y NiIKIAogbjA9YG5hbWVnZW5gCiBuMT1gbmFtZWdlbmAKSW5kZXg6IHRvb2xzL3JlZ3Jlc3Npb24v cGpkZnN0ZXN0L3Rlc3RzL2xpbmsvMTUudAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSB0b29scy9yZWdyZXNzaW9u L3BqZGZzdGVzdC90ZXN0cy9saW5rLzE1LnQJKHJldmlzaW9uIDI2NDMwOSkKKysrIHRvb2xzL3Jl Z3Jlc3Npb24vcGpkZnN0ZXN0L3Rlc3RzL2xpbmsvMTUudAkod29ya2luZyBjb3B5KQpAQCAtMTUs OSArMTUsOSBAQAogbjI9YG5hbWVnZW5gCiAKIGV4cGVjdCAwIG1rZGlyICR7bjB9IDA3NTUKLW49 YG1kY29uZmlnIC1hIC1uIC10IG1hbGxvYyAtcyAyNTZrYAorbj1gbWRjb25maWcgLWEgLW4gLXQg bWFsbG9jIC1zIDUxMmtgCiBuZXdmcyAvZGV2L21kJHtufSA+L2Rldi9udWxsCi1tb3VudCAvZGV2 L21kJHtufSAke24wfQorbW91bnQgL2Rldi9tZCR7bn0gJHtuMH0gfHwgZXhpdCAxCiBleHBlY3Qg MCBjcmVhdGUgJHtuMH0vJHtuMX0gMDY0NAogaT0wCiB3aGlsZSA6OyBkbwpJbmRleDogdG9vbHMv cmVncmVzc2lvbi9wamRmc3Rlc3QvdGVzdHMvbWtkaXIvMTEudAo9PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSB0b29s cy9yZWdyZXNzaW9uL3BqZGZzdGVzdC90ZXN0cy9ta2Rpci8xMS50CShyZXZpc2lvbiAyNjQzMDkp CisrKyB0b29scy9yZWdyZXNzaW9uL3BqZGZzdGVzdC90ZXN0cy9ta2Rpci8xMS50CSh3b3JraW5n IGNvcHkpCkBAIC0xNCw5ICsxNCw5IEBACiBuMT1gbmFtZWdlbmAKIAogZXhwZWN0IDAgbWtkaXIg JHtuMH0gMDc1NQotbj1gbWRjb25maWcgLWEgLW4gLXQgbWFsbG9jIC1zIDI1NmtgCituPWBtZGNv bmZpZyAtYSAtbiAtdCBtYWxsb2MgLXMgNTEya2AKIG5ld2ZzIC9kZXYvbWQke259ID4vZGV2L251 bGwKLW1vdW50IC9kZXYvbWQke259ICR7bjB9Cittb3VudCAvZGV2L21kJHtufSAke24wfSB8fCBl eGl0IDEKIGk9MAogd2hpbGUgOjsgZG8KIAlta2RpciAke24wfS8ke2l9ID4vZGV2L251bGwgMj4m MQpJbmRleDogdG9vbHMvcmVncmVzc2lvbi9wamRmc3Rlc3QvdGVzdHMvbWtmaWZvLzExLnQKPT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PQotLS0gdG9vbHMvcmVncmVzc2lvbi9wamRmc3Rlc3QvdGVzdHMvbWtmaWZvLzExLnQJ KHJldmlzaW9uIDI2NDMwOSkKKysrIHRvb2xzL3JlZ3Jlc3Npb24vcGpkZnN0ZXN0L3Rlc3RzL21r Zmlmby8xMS50CSh3b3JraW5nIGNvcHkpCkBAIC0xNCw5ICsxNCw5IEBACiBuMT1gbmFtZWdlbmAK IAogZXhwZWN0IDAgbWtkaXIgJHtuMH0gMDc1NQotbj1gbWRjb25maWcgLWEgLW4gLXQgbWFsbG9j IC1zIDI1NmtgCituPWBtZGNvbmZpZyAtYSAtbiAtdCBtYWxsb2MgLXMgNTEya2AKIG5ld2ZzIC9k ZXYvbWQke259ID4vZGV2L251bGwKLW1vdW50IC9kZXYvbWQke259ICR7bjB9Cittb3VudCAvZGV2 L21kJHtufSAke24wfSB8fCBleGl0IDEKIGk9MAogd2hpbGUgOjsgZG8KIAlta2ZpZm8gJHtuMH0v JHtpfSA+L2Rldi9udWxsIDI+JjEKSW5kZXg6IHRvb2xzL3JlZ3Jlc3Npb24vcGpkZnN0ZXN0L3Rl c3RzL29wZW4vMTkudAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSB0b29scy9yZWdyZXNzaW9uL3BqZGZzdGVzdC90 ZXN0cy9vcGVuLzE5LnQJKHJldmlzaW9uIDI2NDMwOSkKKysrIHRvb2xzL3JlZ3Jlc3Npb24vcGpk ZnN0ZXN0L3Rlc3RzL29wZW4vMTkudAkod29ya2luZyBjb3B5KQpAQCAtMTQsOSArMTQsOSBAQAog bjE9YG5hbWVnZW5gCiAKIGV4cGVjdCAwIG1rZGlyICR7bjB9IDA3NTUKLW49YG1kY29uZmlnIC1h IC1uIC10IG1hbGxvYyAtcyAyNTZrYAorbj1gbWRjb25maWcgLWEgLW4gLXQgbWFsbG9jIC1zIDUx MmtgCiBuZXdmcyAvZGV2L21kJHtufSA+L2Rldi9udWxsCi1tb3VudCAvZGV2L21kJHtufSAke24w fQorbW91bnQgL2Rldi9tZCR7bn0gJHtuMH0gfHwgZXhpdCAxCiBpPTAKIHdoaWxlIDo7IGRvCiAJ dG91Y2ggJHtuMH0vJHtpfSA+L2Rldi9udWxsIDI+JjEKSW5kZXg6IHRvb2xzL3JlZ3Jlc3Npb24v cGpkZnN0ZXN0L3Rlc3RzL3N5bWxpbmsvMTEudAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSB0b29scy9yZWdyZXNz aW9uL3BqZGZzdGVzdC90ZXN0cy9zeW1saW5rLzExLnQJKHJldmlzaW9uIDI2NDMwOSkKKysrIHRv b2xzL3JlZ3Jlc3Npb24vcGpkZnN0ZXN0L3Rlc3RzL3N5bWxpbmsvMTEudAkod29ya2luZyBjb3B5 KQpAQCAtMTQsOSArMTQsOSBAQAogbjE9YG5hbWVnZW5gCiAKIGV4cGVjdCAwIG1rZGlyICR7bjB9 IDA3NTUKLW49YG1kY29uZmlnIC1hIC1uIC10IG1hbGxvYyAtcyAyNTZrYAorbj1gbWRjb25maWcg LWEgLW4gLXQgbWFsbG9jIC1zIDUxMmtgCiBuZXdmcyAvZGV2L21kJHtufSA+L2Rldi9udWxsCi1t b3VudCAvZGV2L21kJHtufSAke24wfQorbW91bnQgL2Rldi9tZCR7bn0gJHtuMH0gfHwgZXhpdCAx CiBpPTAKIHdoaWxlIDo7IGRvCiAJbG4gLXMgdGVzdCAke24wfS8ke2l9ID4vZGV2L251bGwgMj4m MQo= --_004_16437CC5729B5345AF77F816513376E820BAE6A1MX103CL02corpem_-- From owner-freebsd-testing@FreeBSD.ORG Wed Apr 9 22:57:10 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1936FE9A; Wed, 9 Apr 2014 22:57:10 +0000 (UTC) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [IPv6:2607:f8b0:400c:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BAFE41F9E; Wed, 9 Apr 2014 22:57:09 +0000 (UTC) Received: by mail-ve0-f175.google.com with SMTP id oz11so2730440veb.34 for ; Wed, 09 Apr 2014 15:57:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=+zinAmUcoFJzX6Nr9D0+IpbF86tVNgxdXvISDjfwT10=; b=Mq6Sej00f6F91IF/PcMBI2YCRrM2khMM56Adwr8JmTuVbDCyJY+8VOatQxEmRForjC y8qxf9+DZJ0tiGGSphcs2fjahF8u/NFlbvXCsw4PiaWpWck2vAqPrlGkh3C4hpUmDg4y Y9TfNFbq3zCLsqf2+90sgmpQ/6KelPaqNKGzOD07iGgiBgHbUlhyJxTHjrYjNzIBLONz FC+B9M1UbYq8bGlW7PXSAiAf5Ddawb6Ym2LqHMtwEqmXVHnKrCfrYMu8AYqQd47sIyoo fth8AkV29NQEv0wdGNkJ/5c0NuwzCD3BoViEIdRvuZcp6IBMDR5Ga7h+5Mp9/2kOFpj9 lsgA== MIME-Version: 1.0 X-Received: by 10.58.202.133 with SMTP id ki5mr10803181vec.19.1397084228900; Wed, 09 Apr 2014 15:57:08 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 9 Apr 2014 15:57:08 -0700 (PDT) In-Reply-To: <16437CC5729B5345AF77F816513376E820BAE6A1@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E820BAE6A1@MX103CL02.corp.emc.com> Date: Wed, 9 Apr 2014 15:57:08 -0700 Message-ID: Subject: Re: Diff to fix pjdfstest tests From: Garrett Cooper To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 22:57:10 -0000 (CCing the maintainer, pjd@) On Wed, Apr 9, 2014 at 3:51 PM, Peel, Casey wrote: > In attempting to get pjdfstest working I found that some of the tests hav= e easily-corrected failures: > > One of which is a plan error: ftruncate/00.t runs 26 tests but only speci= fies 21 in the plan. > > Tests that create a small memory disk and fill it to ENOSPC don't validat= e that the memory disk mounts and will happily attempt to fill up your real= partition. The core of the failing is that the memory disk isn't large eno= ugh for the subsequent newfs to succeed and thus the mount fails. > > vbox1009# mdconfig -a -n -t malloc -s 256k > 0 > vbox1009# newfs /dev/md0 > /dev/md0: 0.2MB (512 sectors) block size 32768, fragment size 4096 > using 1 cylinder groups of 0.25MB, 8 blks, 128 inodes. > super-block backups (for fsck -b #) at: > 192 > first cylinder group ran out of space > vbox1009# mount /dev/md0 /mnt > mount: /dev/md0: Invalid argument > vbox1009# > > I've increased the size of the memdisk and added a hard exit to the test = if the mount fails. > > The open/16.t tests are all failing. The tests expect a ELOOP exit code b= ut I'm getting EMLINK. I haven't validated if this is a test error or an er= ror on the not-exactly-FreeBSD-10 code I'm running, so I haven't included t= hose tests in this diff and just made them TODO in my code until I figure i= t out. > > root # prove -r --nocount /svn/branch/src/tools/regression/pjdfstest/test= s > > All tests successful. > Files=3D206, Tests=3D12100, 839 wallclock secs ( 3.04 usr 2.71 sys + 40.= 46 cusr 374.61 csys =3D 420.82 CPU) > Result: PASS > root # Have you looked at what I did on https://github.com/yaneurabeya/pjdfstest (in particular I was working on the onefs branch back when I was with EMC)? The EMLINK issue was a known problem with the tests: https://github.com/yaneurabeya/pjdfstest/commit/8035e2c5b9ea505ec42cd2f2aad= 69f4d40cd2a6d . There are some other commits you probably want to grab and apply to the local tree after validating them: https://github.com/yaneurabeya/pjdfstest/commits/master . It would be nice if these were in the upstream github repo/in FreeBSD, but I didn't get around to setting up a pull request. I'll see about doing that now. Cheers, -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Apr 9 23:15:28 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 226F340D; Wed, 9 Apr 2014 23:15:28 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd54.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0B2C1161; Wed, 9 Apr 2014 23:15:27 +0000 (UTC) Received: from maildlpprd51.lss.emc.com (maildlpprd51.lss.emc.com [10.106.48.155]) by mailuogwprd54.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s39NFPKH031242 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Apr 2014 19:15:25 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd54.lss.emc.com s39NFPKH031242 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1397085326; bh=F8d1h7bmq74KtVvkR1gaQVEri00=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=VZZFQlNu0ojyW/avm52CwWbGW3YvnPrQwrNKO6ok5NJXO5uk/sLNXgRV+J50ZXjdj bt+MprfEmf4dcRSdjfHTrIyKgWEgvFv9k57aw4rPTVlOYDooH0fA3nUcnSzuN0NyUT qh5DvZmTKSh8oFcaIfPfDTPs7iZ3mCl3c2C+WQJE= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd54.lss.emc.com s39NFPKH031242 Received: from mailusrhubprd02.lss.emc.com (mailusrhubprd02.lss.emc.com [10.253.24.20]) by maildlpprd51.lss.emc.com (RSA Interceptor); Wed, 9 Apr 2014 19:15:05 -0400 Received: from mxhub35.corp.emc.com (mxhub35.corp.emc.com [10.254.93.83]) by mailusrhubprd02.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s39NF5w5010572 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 9 Apr 2014 19:15:05 -0400 Received: from MXHUB105.corp.emc.com (10.253.50.22) by mxhub35.corp.emc.com (10.254.93.83) with Microsoft SMTP Server (TLS) id 8.3.327.1; Wed, 9 Apr 2014 19:15:05 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB105.corp.emc.com ([10.253.50.22]) with mapi id 14.03.0158.001; Wed, 9 Apr 2014 19:15:05 -0400 From: "Peel, Casey" To: Garrett Cooper Subject: RE: Diff to fix pjdfstest tests Thread-Topic: Diff to fix pjdfstest tests Thread-Index: Ac9UQ7bQLMqvk3xBQ5WGVtQa6VT5hQAJNjQAAAfcBLA= Date: Wed, 9 Apr 2014 23:15:04 +0000 Message-ID: <16437CC5729B5345AF77F816513376E820BAE703@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E820BAE6A1@MX103CL02.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd02.lss.emc.com X-RSA-Classifications: public Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 23:15:28 -0000 -----Original Message----- From: Garrett Cooper [mailto:yaneurabeya@gmail.com]=20 > Have you looked at what I did on > https://github.com/yaneurabeya/pjdfstest (in particular I was working > on the onefs branch back when I was with EMC)? Ummm.. no. How much of your work am I just recreating? :) At the moment I'm targeting UFS, but I can see I'll need your noexec change= s when I start targeting OneFS. So if it's clear that these things are wrong and you have them corrected, w= hy can't we get them pushed up to -CURRENT? -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 From owner-freebsd-testing@FreeBSD.ORG Wed Apr 9 23:18:22 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4975A552 for ; Wed, 9 Apr 2014 23:18:22 +0000 (UTC) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [IPv6:2607:f8b0:400c:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 091DE117D for ; Wed, 9 Apr 2014 23:18:21 +0000 (UTC) Received: by mail-ve0-f175.google.com with SMTP id oz11so2696410veb.20 for ; Wed, 09 Apr 2014 16:18:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=L++sP2oZu3azhhoo9U8hZHLEVd2gaiFy8piWjS72kGs=; b=t3wgmhKvYEMRtRISpZVKKv8rvKU9n5rUpoRj27jh1x5UAfHJwsyFRpz2e8VagXqOxJ BFi9Lec48mSM8dZtkS9KnoRn+y6ibJ+zBJmUYsiUSujjmTK44SCMeemMcfw+q/PLkQSJ RcgtFvP1z7ihiVBW9+ITEyZM775auVaN283MRCEGtFXFYptjW2kfnjixeiirldb+A0qH PCW2o4lWL4tMobo4McTzBl8CVTZuiRvN4U8SNyvozT5R/y+UOrNVcsT1Suog7qyabFVF O2aa3X/gPMj9qplGZvL/pyYcl3c2FIR2yVnHAv3zI0EfKvgTEKOyqLMj5ZMg7zk/64IV To6A== MIME-Version: 1.0 X-Received: by 10.220.103.141 with SMTP id k13mr22153vco.25.1397085501160; Wed, 09 Apr 2014 16:18:21 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 9 Apr 2014 16:18:21 -0700 (PDT) In-Reply-To: References: Date: Wed, 9 Apr 2014 16:18:21 -0700 Message-ID: Subject: Re: ATF Test Cases From: Garrett Cooper To: "Kilner, Peter" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 23:18:22 -0000 Hi again Peter! On Wed, Mar 26, 2014 at 3:26 PM, Kilner, Peter wrote= : > Hello, > > I hope I am posting this question to the correct list. > > Is there a way to have a single ATF test case report multiple pass/fail r= esults? Currently I have seen that one can include a number of test condit= ions in a single test case (for example many aft_checks). However the test= case will only report one pass/fail result for that case. > > I would like to build a test case that will iterate though many configura= tions and would like to see a pass/fail for each config. However this is d= ifficult to implement with multiply test cases because of the for loops tha= t I am using. > > Any suggestions would be appreciated. As others have suggested, your approach could be tweaked a bit to iterate over all of the different configurations you are trying to test. In general I would create either a generic library or test driver with common code and have a common validation method, then build off that with additional validation methods if needed. I do that all the time. I think that Julio's suggestion with dynamically eval'ing tests is spot on as well. There's an example here of how to do that (that we might want to extend to a more generic form and push into Kyua or FreeBSD's test infrastructure if it's ok): https://github.com/yaneurabeya/freebsd/blob/master/bin/sh/tests/functional_= test.sh . Important: this code is proof of concept -- please do _NOT_ copy-paste it if at all possible. Thanks, -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Apr 9 23:21:28 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 770C15B4; Wed, 9 Apr 2014 23:21:28 +0000 (UTC) Received: from mail-vc0-x236.google.com (mail-vc0-x236.google.com [IPv6:2607:f8b0:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 24C5A120B; Wed, 9 Apr 2014 23:21:28 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id ib6so2675941vcb.41 for ; Wed, 09 Apr 2014 16:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=g1wRCaJVn+q0tBvLvpHIXjQXraLwfm2EsFFfKh9V9us=; b=iYK5DebtM3H9x+uK4Z4KJ6DgD3MVzILBJURWEwfuAn6QASZvFHbhY/dX3qupHNi8p9 GE6+wFX3xLpOTUvs2poCp6dJwfkq9SkX3PelrHcMQJqoVAfb3Aypjs8JjoXdJd3PUbJF BQIiwxoWuNwbmkB0bYCA1r14iaS+24xAmLphbNy8bITdD0D/L+TQ+JUREBG6tOGMAVYY sindcXL/+FdU3spIuiqUoCv0tFU+zgHThGNErhYVE4oviqk7Oouo+2/KeVHRS79h24M+ G4UE3AedmdruPi301MdJYfPdpaH6fIa/XCRuIp2uhEHUmgZWEOZLYVLTbn0JZZTiC6wy X9Ug== MIME-Version: 1.0 X-Received: by 10.58.202.133 with SMTP id ki5mr10892064vec.19.1397085687288; Wed, 09 Apr 2014 16:21:27 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 9 Apr 2014 16:21:27 -0700 (PDT) In-Reply-To: <16437CC5729B5345AF77F816513376E820BAE703@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E820BAE6A1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAE703@MX103CL02.corp.emc.com> Date: Wed, 9 Apr 2014 16:21:27 -0700 Message-ID: Subject: Re: Diff to fix pjdfstest tests From: Garrett Cooper To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 23:21:28 -0000 On Wed, Apr 9, 2014 at 4:15 PM, Peel, Casey wrote: > -----Original Message----- > From: Garrett Cooper [mailto:yaneurabeya@gmail.com] > >> Have you looked at what I did on >> https://github.com/yaneurabeya/pjdfstest (in particular I was working >> on the onefs branch back when I was with EMC)? > > Ummm.. no. How much of your work am I just recreating? :) Not that much fortunately. Just a week or so of focused work :). > At the moment I'm targeting UFS, but I can see I'll need your noexec changes when I start targeting OneFS. It would affect UFS/ZFS as well if other non-standard mount options were turned on, like noexec. OneFS is its own special critter because of what mount flags are enabled by default. > So if it's clear that these things are wrong and you have them corrected, why can't we get them pushed up to -CURRENT? I hadn't filed bugs yet/done pull requests and back when I was working on this it was on 7.2. I'm re-validating things on 11-CURRENT and will send out a github PR and CC you on it. Cheers :)! -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Apr 9 23:30:37 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4FB126AB for ; Wed, 9 Apr 2014 23:30:37 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd54.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF32412C5 for ; Wed, 9 Apr 2014 23:30:36 +0000 (UTC) Received: from maildlpprd55.lss.emc.com (maildlpprd55.lss.emc.com [10.106.48.159]) by mailuogwprd54.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s39NUYQY001942 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Apr 2014 19:30:35 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd54.lss.emc.com s39NUYQY001942 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1397086235; bh=8/2CkPFVOrg0y8HI7SLYfEiO8Os=; h=From:To:CC:Date:Subject:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=DhgMBukAodw14d43C8x0pzxTq1U2VOyma9/vBy4K16hTAYzj0HH5QlkQSeQqnKVJr QssQc4uEOTbOM9RFR61PuAErn4MZoc8dxn8yQYYfaoR/fJ/FpmwDS9Ec77hOQ8lm4l q08RpipS+TaEgh4GTLKPQFeeLGpkdmG3MtmfRrlM= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd54.lss.emc.com s39NUYQY001942 Received: from mailusrhubprd03.lss.emc.com (mailusrhubprd03.lss.emc.com [10.253.24.21]) by maildlpprd55.lss.emc.com (RSA Interceptor); Wed, 9 Apr 2014 19:30:29 -0400 Received: from mxhub02.corp.emc.com (mxhub02.corp.emc.com [10.254.141.104]) by mailusrhubprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s39NUTV3018853 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 9 Apr 2014 19:30:29 -0400 Received: from MX01A.corp.emc.com ([10.254.141.71]) by mxhub02.corp.emc.com ([10.254.141.104]) with mapi; Wed, 9 Apr 2014 19:30:28 -0400 From: "Kilner, Peter" To: Garrett Cooper Date: Wed, 9 Apr 2014 19:30:27 -0400 Subject: Re: ATF Test Cases Thread-Topic: ATF Test Cases Thread-Index: Ac9US7DY5B6QDP1DQs+JcnljnQYtJw== Message-ID: <1C62F69C-F225-42D8-B56D-A22E5019892E@isilon.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd03.lss.emc.com X-RSA-Classifications: public Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" , "Kilner, Peter" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 23:30:37 -0000 Hi Garrett, Thanks for the example, this is great solution! I was starting to think ab= out creating a test case generator but this a much better way to go. Thank you, Peter K On Apr 9, 2014, at 4:18 PM, Garrett Cooper > wrote: Hi again Peter! On Wed, Mar 26, 2014 at 3:26 PM, Kilner, Peter > wrote: Hello, I hope I am posting this question to the correct list. Is there a way to have a single ATF test case report multiple pass/fail res= ults? Currently I have seen that one can include a number of test conditio= ns in a single test case (for example many aft_checks). However the test c= ase will only report one pass/fail result for that case. I would like to build a test case that will iterate though many configurati= ons and would like to see a pass/fail for each config. However this is dif= ficult to implement with multiply test cases because of the for loops that = I am using. Any suggestions would be appreciated. As others have suggested, your approach could be tweaked a bit to iterate over all of the different configurations you are trying to test. In general I would create either a generic library or test driver with common code and have a common validation method, then build off that with additional validation methods if needed. I do that all the time. I think that Julio's suggestion with dynamically eval'ing tests is spot on as well. There's an example here of how to do that (that we might want to extend to a more generic form and push into Kyua or FreeBSD's test infrastructure if it's ok): https://github.com/yaneurabeya/freebsd/blob/master/bin/sh/tests/functional_= test.sh . Important: this code is proof of concept -- please do _NOT_ copy-paste it if at all possible. Thanks, -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Apr 9 23:45:47 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39FA6850 for ; Wed, 9 Apr 2014 23:45:47 +0000 (UTC) Received: from mail-vc0-x22f.google.com (mail-vc0-x22f.google.com [IPv6:2607:f8b0:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED63513BC for ; Wed, 9 Apr 2014 23:45:46 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id lh14so2751754vcb.34 for ; Wed, 09 Apr 2014 16:45:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9tykYyRrQYMtjthMo1f7VmoQPjWGiKhFf/mYH/2b4WA=; b=upWIaXZm4GWmNVdecslPhIP28r9tJ8m0dZlulsGhJPCLRBEXYrV46F+2WBtjclLzki ZoNIl4ZDPvgO3RWaVwOjAZeJDxU3g+k+JYX7q92rlDosexu+EhDqnaTHxvOEI3Xnn6uK nhBj0QUKKeVsNFAMrsdFUOgd1gxYF0IMqmw4f/utWVKcfSoCZto6tPBOet4sZLU4tHQW Tse9m3vSFIdtA0cuTPMTOGTHJOpN9BJIagD+VIDE046P4BTGclsoY+g86Q1KiYSOvzOu oGWgpMQwECS334JX8VbgRFXlgctCerXikfTBU6bngtax0EdlC29/uEoT73HoI5LlC+K7 KLSg== MIME-Version: 1.0 X-Received: by 10.58.202.133 with SMTP id ki5mr10978825vec.19.1397087146135; Wed, 09 Apr 2014 16:45:46 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 9 Apr 2014 16:45:46 -0700 (PDT) In-Reply-To: <1C62F69C-F225-42D8-B56D-A22E5019892E@isilon.com> References: <1C62F69C-F225-42D8-B56D-A22E5019892E@isilon.com> Date: Wed, 9 Apr 2014 16:45:46 -0700 Message-ID: Subject: Re: ATF Test Cases From: Garrett Cooper To: "Kilner, Peter" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 23:45:47 -0000 On Wed, Apr 9, 2014 at 4:30 PM, Kilner, Peter wrote: > Hi Garrett, > > Thanks for the example, this is great solution! Credit goes to Julio fro the suggestion :). My original solution was a lot more hardcoded. > I was starting to think about creating a test case generator but this a much better way to go. Yeah.. it's one reason why I like nose better than a lot of other test infrastructures; it allows me to be lazy :). > Thank you, No problem :). Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Apr 10 19:18:53 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7A60C2B2; Thu, 10 Apr 2014 19:18:53 +0000 (UTC) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 267D318E5; Thu, 10 Apr 2014 19:18:53 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id im17so3770184vcb.23 for ; Thu, 10 Apr 2014 12:18:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GTYXiPt6JMIYdZWzw541G4TybAx3j6u1mYOX0YmRCTw=; b=IQbn0qPJSGOABdZmuNLo7QJ7pdigrQBhvKprfB1aB0WjkEMExakGXAy8BcnhVP1eIi Cd04n/aKf7/dEtSm5CWw9hqa5czB/EIf375EeIV/Gwjqyu8SoCcCI+G6JUx21tK0aJ9z Xp1k9IvIgCq4RJ3UXR3poioDG+BmNVSr6/wyw90ifL71leo4a1lf5qvmwTfFVu0sTkIs 6BhjawiHOPEyTrPLrjo5c6DHFxB64JLlm3Sox4uCSloiP3a7v2SrsZ9L7j1N5yDhMIto De4QBDx7IONb9on7VdofXMfj29IRs9c5FyQ9BehoShYI9HTn4EqW7etR7aEbhTs2iW2B 9y8w== MIME-Version: 1.0 X-Received: by 10.220.190.197 with SMTP id dj5mr4690314vcb.19.1397157532223; Thu, 10 Apr 2014 12:18:52 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Thu, 10 Apr 2014 12:18:52 -0700 (PDT) In-Reply-To: References: <16437CC5729B5345AF77F816513376E820BAE6A1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAE703@MX103CL02.corp.emc.com> Date: Thu, 10 Apr 2014 12:18:52 -0700 Message-ID: Subject: Re: Diff to fix pjdfstest tests From: Garrett Cooper To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 19:18:53 -0000 Hi Casey! The issues you found on 10.x are valid (I ran into this when rerunning the test suite on UFS -- it was passing cleanly before). I'll apply the changes, retest, then send out the PR as promised earlier. Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Apr 10 21:07:29 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C97C70B; Thu, 10 Apr 2014 21:07:29 +0000 (UTC) Received: from mail-vc0-x235.google.com (mail-vc0-x235.google.com [IPv6:2607:f8b0:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 272E414EC; Thu, 10 Apr 2014 21:07:29 +0000 (UTC) Received: by mail-vc0-f181.google.com with SMTP id id10so3884053vcb.40 for ; Thu, 10 Apr 2014 14:07:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K8c8on/wu6L4l3gE+ZFqXsb385fX+MXdW3qNPtHMSRc=; b=K+CBxLBQfD2ut9G9BG5LmGiuNWGoUhZokaIdr5PF0pocRrt3j1Kp9EPrM+1VjCBRuM 95Unie9LppLWUOsJrZymg+RhbV1ov4VsZWT8jhuEqj7lE/nKDCEeS8tx5Bxj4QmEfrkH u13qDzZ+oNITO6lFcmetRyjzYN9kAO0tsw7Pz+aXEdT6lNmAPXfxL7WIbUvIsmhK7hfq KZU53qYiMpYXVU/idh5Ni5et0fBVeR4N9QODKMTUrm8exZ1ir+xVF8cuPyJqrmCH7Zm/ 8z+AnLoGDPGTTyC9mFsmBnWVUtDnc8zB6vWZ/tdiGM+En82An5OxEwFIntGCF+VqngPV 7vUg== MIME-Version: 1.0 X-Received: by 10.52.173.165 with SMTP id bl5mr13351397vdc.13.1397164048250; Thu, 10 Apr 2014 14:07:28 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Thu, 10 Apr 2014 14:07:28 -0700 (PDT) In-Reply-To: References: <16437CC5729B5345AF77F816513376E820BAE6A1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAE703@MX103CL02.corp.emc.com> Date: Thu, 10 Apr 2014 14:07:28 -0700 Message-ID: Subject: Re: Diff to fix pjdfstest tests From: Garrett Cooper To: "Peel, Casey" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Pawel Jakub Dawidek X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 21:07:29 -0000 On Thu, Apr 10, 2014 at 12:18 PM, Garrett Cooper wrote: > Hi Casey! > The issues you found on 10.x are valid (I ran into this when > rerunning the test suite on UFS -- it was passing cleanly before). > I'll apply the changes, retest, then send out the PR as promised > earlier. Ok. All the issues identified have been fixed and a caught some more: https://github.com/yaneurabeya/pjdfstest/commit/b8e1e30dc3f44fb7619cf86884a981fba5476def . It turns out that a pull request was already open for my changes -- they just haven't been committed to the upstream master yet: https://github.com/pjd/pjdfstest/pull/1 . Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Apr 10 22:19:36 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5219378 for ; Thu, 10 Apr 2014 22:19:36 +0000 (UTC) Received: from mail-yh0-f50.google.com (mail-yh0-f50.google.com [209.85.213.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F4C51B05 for ; Thu, 10 Apr 2014 22:19:36 +0000 (UTC) Received: by mail-yh0-f50.google.com with SMTP id c41so4504696yho.23 for ; Thu, 10 Apr 2014 15:19:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=Rk4Az2ehqc/UbceVOYuemMPceoVnOIwESD2QaC/Ktv8=; b=IY/Py5MXUXGk8jvQPm1QndINa1XSCg0qVLE4ogUehC8VU6wIEje7YtGOjej0y0HF2q 3IkScGGiNmxMwXorKBLmgQ3cOpx5WppLTxGJ9PZSGxmzCMOvBkoVZdGuh2NYvQV9DyL3 eA8raZdA6PBHTN4laNZqQrHsp6jUlJthfX0CObr8aUfRQ8fTPj5hvCC7LuXTNnUpJ4nx rJkRvNdrX/NjytqKfxutNmrQxMbjxqNLWKDHJd+hEEySEpicgQm8PkkL8my0Q5a+GGUz xB1kJA87tgHdD3TU0G+RVUC1vOdry6EcB81c/sZdBehasa17c1kCYH2lElY8hgJAm+7G tbgA== X-Gm-Message-State: ALoCoQkTFBeHOlpEOXo3rF7pZOjTxuS/zifAibIUiVxeMRfp9CAIpHsl8ZeLG7fXgrRtW/MXSXoE X-Received: by 10.236.20.194 with SMTP id p42mr26611570yhp.56.1397168368987; Thu, 10 Apr 2014 15:19:28 -0700 (PDT) Received: from dhcp-172-26-107-241.nyc.corp.google.com (dhcp-172-26-107-241.nyc.corp.google.com [172.26.107.241]) by mx.google.com with ESMTPSA id z69sm10046153yha.26.2014.04.10.15.19.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 15:19:28 -0700 (PDT) Sender: Julio Merino Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Makefiles for skip-level directories? From: Julio Merino In-Reply-To: <2194A8F0-810B-471A-AD9C-C95C469EAD99@gmail.com> Date: Thu, 10 Apr 2014 18:19:02 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <2BC2387E-387B-46EC-832F-78ECEA51A206@freebsd.org> References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BACB49@MX103CL02.corp.emc.com> <2194A8F0-810B-471A-AD9C-C95C469EAD99@gmail.com> To: Garrett Cooper X-Mailer: Apple Mail (2.1874) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 22:19:36 -0000 On Apr 4, 2014, at 19:28, Garrett Cooper wrote: >=20 >> On Apr 4, 2014, at 15:28, "Peel, Casey" = wrote: >>=20 >> Ok, I figured it out and wrote up the following document (that is = published on our internal wiki). You may take whatever you want of this = for the FreeBSD wiki if you find it useful. >>=20 >> Something rcattelan pointed out while doing the code review for my = gpart changes that I wanted to run past you: >> is the include bsd.own.mk to pick up MK_TESTS? >> yes >> do you need to do that in the files that are not look at = MK_TESTS? >> geom/tests/Makefile ? >> I probably don't have to, but that's how the other Makefiles = like that are done in FreeBSD, eg: src/bin/tests/Makefile >> bsd.own.mk is included by bsd.init.mk which is included by = bsd.test.mk=20 >> so it seems redundant >> maybe we should ask the upstream if is correct ... might be = better to leave it off vs perpetuating something that might not be = right >=20 > Yeah, bsd.own.mk is redundant, but it doesn't do any harm minus a hit = on the cache, and increase the file sizes a small amount. >=20 > In the event that the Makefile snippet would be removed from the = included makefiles then stuff would break horribly. Plus it makes it = clear that MK_TESTS is obtained from bsd.own.mk. bsd.own.mk is not redundant if you are going to test for MK_TESTS; you = need that include. What I=92m not clear about is if bsd.own.mk ought to be bsd.init.mk or = not.= From owner-freebsd-testing@FreeBSD.ORG Thu Apr 10 22:24:39 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 173A04AF; Thu, 10 Apr 2014 22:24:39 +0000 (UTC) Received: from mail-vc0-x22c.google.com (mail-vc0-x22c.google.com [IPv6:2607:f8b0:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B52741BD0; Thu, 10 Apr 2014 22:24:38 +0000 (UTC) Received: by mail-vc0-f172.google.com with SMTP id la4so4013674vcb.17 for ; Thu, 10 Apr 2014 15:24:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Ka/7j/lAZUG5azJF9cd9liFsI9dZPkrcnDDXurFvPm8=; b=TuKKaxASLfb0v/CF9yDR+q6RiwNMoOg/0LIb+vRi41KvLheAZlnxaGgbaqQCmomZTI 0XYX8mlGhzuWHrfkm8RVNFdrwEIG0FPcal7pQap0yfZsDeyxF6dt5gLFJv8IRnOHLExd Mvob6hxzycvU2sZqs8R1HXu1/YJVNnEBoMhTcaMupZXzIhdOGsIRLOPOCAegv3cblB67 eapTsBqGBWfrbHaoxB/aJOOctYzSpNsu17PBq9gC4iArr4guxqAAyhnlNaKzmYVx9ZGt 4rCuJMA0XRFRx+ORdKBEtOA12CPJNpEcsyZk1RHTOqBoOsLl8+mDAlifLZz2kKpcVAg8 /m9g== MIME-Version: 1.0 X-Received: by 10.58.123.71 with SMTP id ly7mr16458549veb.11.1397168677911; Thu, 10 Apr 2014 15:24:37 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Thu, 10 Apr 2014 15:24:37 -0700 (PDT) In-Reply-To: <2BC2387E-387B-46EC-832F-78ECEA51A206@freebsd.org> References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BACB49@MX103CL02.corp.emc.com> <2194A8F0-810B-471A-AD9C-C95C469EAD99@gmail.com> <2BC2387E-387B-46EC-832F-78ECEA51A206@freebsd.org> Date: Thu, 10 Apr 2014 15:24:37 -0700 Message-ID: Subject: Re: Makefiles for skip-level directories? From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 22:24:39 -0000 On Thu, Apr 10, 2014 at 3:19 PM, Julio Merino wrote: >> >> Yeah, bsd.own.mk is redundant, but it doesn't do any harm minus a hit on the cache, and increase the file sizes a small amount. >> >> In the event that the Makefile snippet would be removed from the included makefiles then stuff would break horribly. Plus it makes it clear that MK_TESTS is obtained from bsd.own.mk. > > bsd.own.mk is not redundant if you are going to test for MK_TESTS; you need that include. > > What I'm not clear about is if bsd.own.mk ought to be bsd.init.mk or not. Unfortunately changing this would break a lot of things that have worked in the past, so I think it's best to leave this alone :/.. -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Apr 10 22:56:05 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 747E1BD5 for ; Thu, 10 Apr 2014 22:56:05 +0000 (UTC) Received: from mail-yh0-f52.google.com (mail-yh0-f52.google.com [209.85.213.52]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FF081E5D for ; Thu, 10 Apr 2014 22:56:04 +0000 (UTC) Received: by mail-yh0-f52.google.com with SMTP id c41so4539498yho.39 for ; Thu, 10 Apr 2014 15:56:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=9LO4TT1AKJdd03J1egoxS6Xra60u2vsxY0Wkj2/kEC8=; b=M9l49MMzv0wrOD2WrvFD5U/H9u5ExQ1/jWEE37n/kBgveGC4FFKnFvkUR121O8tyFf sONKJQPYMkpjottneeTufqVFCFz5U9WJvdcUw2zpgIvwe6oU92+omi3ohr3CJH98dqDI jQ+ZasOAzuCii6AaUL612P8oAmh93/O2sMW3AOXWT3O/IANMV0zQbA9mB1Q4ibkijVd2 EtUyBnQ6AlTzT32o31TW577xbyAv57xOQHq3g48D/wXLiE6YS3bJ9gCScQEXivIl8wO4 0vCcbTUBHeR3QhisKH1YY2vCnLhhDXa5XKeARm1FJEcGMqGlvlx53Ux0cIC1+t0JEoFU H++Q== X-Gm-Message-State: ALoCoQmw4kZFooo/9gQCH5727H29v9sVVLN7VL0AGTH1c6RGUTmHOyBCuf+uXCz1Y4ZhwUyjwCSK X-Received: by 10.236.168.42 with SMTP id j30mr7331767yhl.101.1397170145679; Thu, 10 Apr 2014 15:49:05 -0700 (PDT) Received: from ?IPv6:2620::1003:1007:e58e:92bd:23b1:5fcb? ([2620:0:1003:1007:e58e:92bd:23b1:5fcb]) by mx.google.com with ESMTPSA id 44sm10182380yhp.17.2014.04.10.15.49.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 15:49:05 -0700 (PDT) Sender: Julio Merino Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Makefiles for skip-level directories? From: Julio Merino In-Reply-To: <16437CC5729B5345AF77F816513376E820BACB49@MX103CL02.corp.emc.com> Date: Thu, 10 Apr 2014 18:49:03 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <16437CC5729B5345AF77F816513376E8129868F1@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BAA20A@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BACB49@MX103CL02.corp.emc.com> To: "Peel, Casey" X-Mailer: Apple Mail (2.1874) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 22:56:05 -0000 On Apr 4, 2014, at 18:28, Peel, Casey wrote: > Ok, I figured it out and wrote up the following document (that is = published on our internal wiki). You may take whatever you want of this = for the FreeBSD wiki if you find it useful. Added stuff along these lines to the = https://wiki.freebsd.org/TestSuite/DeveloperHowTo page. Please take a = look.= From owner-freebsd-testing@FreeBSD.ORG Thu Apr 10 22:58:48 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADDA2CF2 for ; Thu, 10 Apr 2014 22:58:48 +0000 (UTC) Received: from mail-yk0-f179.google.com (mail-yk0-f179.google.com [209.85.160.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 692141E7D for ; Thu, 10 Apr 2014 22:58:48 +0000 (UTC) Received: by mail-yk0-f179.google.com with SMTP id 9so4186968ykp.38 for ; Thu, 10 Apr 2014 15:58:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=XW91sEdsV1ybFipnNpEArVR1w5c5P3LVdhqmH1PIuuc=; b=Ms8E8LfO6xx31ZkxHRDvhASe6wdh510It4MIB2+amWImKfK/SBUvJfKyz6uGS9V5XB ihYSr6V0FolSNrtRuuL09l3yh72eaZc3il1+IjN9yMLivkwQd/bZAD4Cn8W090kA3Qis NcCEiduKXUe2UEac3KbwJUcsOa9w39G3emKYreiz9CIzhnvgNuGJaOmyq/TKnqYB7+YG iLWA58416ojbXSS02hnIgWYRf+JjJMrZMj8XDgd7bX94UdlpnPId4+v84Q0Sh8qdcnEF gEL/wYwu4chdPNJDstjGJToGiPnIZKzSQjvKBlLsQcEr+he52QSWUsDMwijICxP++40z A4nA== X-Gm-Message-State: ALoCoQm54J3+7QJX+KXu/1roaFwAJJ3H9mCP+pfcA04OcpEL8NNuqw7zweFU/IqcTk03TcMWbTKt X-Received: by 10.236.100.177 with SMTP id z37mr6900866yhf.123.1397170394224; Thu, 10 Apr 2014 15:53:14 -0700 (PDT) Received: from dhcp-172-26-107-241.nyc.corp.google.com (dhcp-172-26-107-241.nyc.corp.google.com [172.26.107.241]) by mx.google.com with ESMTPSA id d44sm10189957yhd.30.2014.04.10.15.53.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 15:53:13 -0700 (PDT) Sender: Julio Merino Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: ATF work From: Julio Merino In-Reply-To: <74A4FAAB-88B8-4226-949A-11AA8D99D662@gmail.com> Date: Thu, 10 Apr 2014 18:53:12 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <7075B84B-A2DD-473D-8ABC-E79BE8FFAB79@freebsd.org> References: <20140402085349.GA61802@x2.osted.lan> <74A4FAAB-88B8-4226-949A-11AA8D99D662@gmail.com> To: Garrett Cooper X-Mailer: Apple Mail (2.1874) Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 22:58:48 -0000 On Apr 5, 2014, at 13:06, Garrett Cooper wrote: > I=92ve master on my fork reflects FreeBSD master. =93atf=94 now = reflects all of my ATF work, so if you diff master and atf you=92ll see = the relevant changes I=92ve done for ATF in the past. I=92ll update the = docs on the wiki to reflect this. Thanks for the split; it makes reviewing the changes somewhat simpler. But if you are interested in getting this into the tree, it'd be much = better if you could split the changes in semantical and *self-contained* = diffs and submit them one by one. Reviewing a gigantic, whole-tree diff = is too hard to digest. Obviously it'd be awesome to incorporate all the = work, but with so limited time (at least on my side) it's hard to keep = up even with email...= From owner-freebsd-testing@FreeBSD.ORG Fri Apr 11 17:57:03 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64701B6D; Fri, 11 Apr 2014 17:57:03 +0000 (UTC) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd04.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 10C981971; Fri, 11 Apr 2014 17:57:02 +0000 (UTC) Received: from maildlpprd05.lss.emc.com (maildlpprd05.lss.emc.com [10.253.24.37]) by mailuogwprd04.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s3BHuxaq022165 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 11 Apr 2014 13:57:00 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd04.lss.emc.com s3BHuxaq022165 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1397239020; bh=lTJu7fn8S7lQB0YoqhSCfVuZL3s=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=sk2PZrnXM8VKmTyP8eLGcYVlHOPK8TciXO5RgAqxQhODJlnOauBPo4xwTk5cpwTK4 bVGyzjDEGfoGEZR2as2gxvnwAxmkNxfQnvBj1n8CRXkVxT5LMENbIErFnTCtkhGV6c +AjS+rDxAGoNTGzRnzWD1+CEJxD30EBAysG2zS1A= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd04.lss.emc.com s3BHuxaq022165 Received: from mailusrhubprd53.lss.emc.com (mailusrhubprd53.lss.emc.com [10.106.48.18]) by maildlpprd05.lss.emc.com (RSA Interceptor); Fri, 11 Apr 2014 13:56:50 -0400 Received: from mxhub27.corp.emc.com (mxhub27.corp.emc.com [10.254.110.183]) by mailusrhubprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s3BHuoCL008102 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 11 Apr 2014 13:56:50 -0400 Received: from MXHUB106.corp.emc.com (10.253.58.23) by mxhub27.corp.emc.com (10.254.110.183) with Microsoft SMTP Server (TLS) id 8.3.327.1; Fri, 11 Apr 2014 13:56:49 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB106.corp.emc.com ([10.253.58.23]) with mapi id 14.03.0158.001; Fri, 11 Apr 2014 13:56:49 -0400 From: "Peel, Casey" To: "freebsd-testing@freebsd.org" Subject: Please provide process for small, targeted fixes in tools/regression Thread-Topic: Please provide process for small, targeted fixes in tools/regression Thread-Index: Ac9VrITJRfNb0i8ZTXWISpxKRMnAiQ== Date: Fri, 11 Apr 2014 17:56:48 +0000 Message-ID: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd53.lss.emc.com X-RSA-Classifications: public Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 17:57:03 -0000 I have several small, targted fixes to files in src/tools/regression/ to: 1) enable existing *.t tests to run that were failing 2) add new *.t files for directories that enable running TAP-enabled b= inaries through prove 3) *.c updates to remove clang compiler warnings Can someone please provide a process for getting these approved and committ= ed in a timely manner? I would prefer to get these upstream and then pull t= hem down if at all possible, but I simply don't have time for these to lang= uish for weeks in a committee. Isilon has BSD committers (eg: bdrewery) I can leverage if that would be mo= re expedient but given these are all testing-centric it seems like getting = approval from one or more people on freebsd-testing would be best. Thanks, -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 From owner-freebsd-testing@FreeBSD.ORG Fri Apr 11 19:06:44 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DFF7FE22 for ; Fri, 11 Apr 2014 19:06:43 +0000 (UTC) Received: from mail-yk0-f172.google.com (mail-yk0-f172.google.com [209.85.160.172]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D22811EA for ; Fri, 11 Apr 2014 19:06:43 +0000 (UTC) Received: by mail-yk0-f172.google.com with SMTP id 200so5266399ykr.17 for ; Fri, 11 Apr 2014 12:06:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=4zFRZy0kZpMYze9CnJO0eJTJ8Q2tR33Nac08s2jYZsM=; b=gLRRuWGdnq9/WST5/wyKAOZjdL6JZ5bZUjVqrtYFso+TuH8U4q1kRlyefgq+l7Q8E6 fN3wF39sT2kuXpdp3NKrHqKSZdNWVMog0VWGK+CKgwz+9KCCKNVA47FIwBzxHl8bh2wF LhGUHCXx3gDb8zfCs9WTrBEt93NHlg1DtOwkJyUs3FwDA4zH4139klnhcpbn673AJ620 TXEYvNgDt5QYBgMesrqrJlDwivXLY9r4Jgm5YVEL29kwI7rlukW1SBgWkcp0g2dOEXvf 1kFP2YCnjePxfaUyf8mlAvqglwxl6DJuPP4z28oRc8AIYptwq2bLoil2PpuTG9KO6apb ZuKA== X-Gm-Message-State: ALoCoQkJYE/HduhchMiAgveCGDa0xW12KCJ+dyS//l087eYJ6VTRX2egW4faLsOJN28YzOm1Z0SP X-Received: by 10.236.220.72 with SMTP id n68mr15177103yhp.102.1397242895484; Fri, 11 Apr 2014 12:01:35 -0700 (PDT) Received: from dhcp-172-26-107-241.nyc.corp.google.com (dhcp-172-26-107-241.nyc.corp.google.com [172.26.107.241]) by mx.google.com with ESMTPSA id s26sm10731834yhg.4.2014.04.11.12.01.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Apr 2014 12:01:34 -0700 (PDT) Sender: Julio Merino Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Please provide process for small, targeted fixes in tools/regression From: Julio Merino In-Reply-To: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> Date: Fri, 11 Apr 2014 15:01:33 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> To: "Peel, Casey" X-Mailer: Apple Mail (2.1874) Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 19:06:44 -0000 Hello Casey, On Apr 11, 2014, at 13:56, Peel, Casey wrote: > I have several small, targted fixes to files in src/tools/regression/ = to: >=20 > 1) enable existing *.t tests to run that were failing This is not "too hard" but is certainly tricky. I did a bunch of them about a month ago but didn't write a lot of = documentation on the topic. You can find some useful notes in = http://goo.gl/rWpdDR though, which includes slides and accompanying = notes for the tutorial I gave at AsiaBSDCon. Most of that ought to be = converted to actual documentation of course. The very first step, which is arguably the hardest, is to get the tests = to work while running them with the prove(1) tool. This has several = implications (an important one being that any testing logic must be = removed from the Makefile). > 2) add new *.t files for directories that enable running = TAP-enabled binaries through prove That's good as a second step. I suspect these new *.t files will just = invoke another script (the actual test), which means the .t files will = be removed when the tests are hooked into the test suite. But that's = fine: it's better to get them running with prove(1) first because then = the move is simple enough. > 3) *.c updates to remove clang compiler warnings Yes please. Once the tests run with prove(1) and pass, we can bundle them into the = test suite. This roughly involves moving the code to corresponding = 'tests/' subdirectories, writing Makefiles and updating the mtree. = Mostly mechanical (but very annoying when the tests provide tons of data = files; see usr.bin/make/tests/). > Can someone please provide a process for getting these approved and = committed in a timely manner? I would prefer to get these upstream and = then pull them down if at all possible, but I simply don't have time for = these to languish for weeks in a committee. >=20 > Isilon has BSD committers (eg: bdrewery) I can leverage if that would = be more expedient but given these are all testing-centric it seems like = getting approval from one or more people on freebsd-testing would be = best. If you can get self-contained diffs, I can try to get them in for you = (but can't make any promises as my time for FreeBSD varies significantly = week to week...) For simplicity of review, I'd appreciate at least one patch for every = subdirectory within tools/regression/foo/bar/ with at least one patch to = get the first tests up and running first with prove(1). More patches = are better! The move of the fixed tests to the new infrastructure (aka the = boilerplate work) is something I may be able to help with.= From owner-freebsd-testing@FreeBSD.ORG Fri Apr 11 20:10:45 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C0B5286 for ; Fri, 11 Apr 2014 20:10:45 +0000 (UTC) Received: from mail-yk0-f173.google.com (mail-yk0-f173.google.com [209.85.160.173]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 59948177C for ; Fri, 11 Apr 2014 20:10:44 +0000 (UTC) Received: by mail-yk0-f173.google.com with SMTP id 10so5314232ykt.4 for ; Fri, 11 Apr 2014 13:10:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=4zFRZy0kZpMYze9CnJO0eJTJ8Q2tR33Nac08s2jYZsM=; b=BK5Xq/Szv+RYc1E1EXsvFY0a67wIi8HKB4PXQOcBJT+FyGMs4UB2XEh6y80kzSmukb Gx1bojmuRqJ27FMBbVtf6A4L3FZj8lB/TxWtLI6a49sJdG/rnyPNw7O+Fxy42sQKjh+j XCBSihBTOprNw2rBN5XgRN3FCFL0iLz4p69/R1vDPIa/b0uzOTRr1GGq5Ta0kLiGvxSN ov8BgLViEMeFhdwPFIdvLOK8YkGrxcU4YI/m44ZpDkwtF0Tzp3qygVzolr+rdV7jlCGJ LIxvEEgpyxfB8ROV1lMgsPk9/n0WddjRep1EX3MmymWuryFdQ4R9atCjXnwTxOUMwoR1 6rlw== X-Gm-Message-State: ALoCoQmv12h+98vhLGlfLhziBHmZzZnY28TbA1+hBTm4Fe1Dcr1V3HPLpqSqOPOpYkx99uYAHpMV X-Received: by 10.236.61.45 with SMTP id v33mr35257703yhc.20.1397246651117; Fri, 11 Apr 2014 13:04:11 -0700 (PDT) Received: from ?IPv6:2620::1003:1007:a0c2:bdf:88e5:e142? ([2620:0:1003:1007:a0c2:bdf:88e5:e142]) by mx.google.com with ESMTPSA id q66sm15354291yhj.44.2014.04.11.13.04.10 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Apr 2014 13:04:10 -0700 (PDT) Sender: Julio Merino Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Please provide process for small, targeted fixes in tools/regression From: Julio Merino In-Reply-To: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> Date: Fri, 11 Apr 2014 16:04:08 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <7EEDDB7A-A8E2-4E8D-AD98-0BD2AB0A9C80@freebsd.org> References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> To: "Peel, Casey" X-Mailer: Apple Mail (2.1874) Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 20:10:45 -0000 Hello Casey, On Apr 11, 2014, at 13:56, Peel, Casey wrote: > I have several small, targted fixes to files in src/tools/regression/ = to: >=20 > 1) enable existing *.t tests to run that were failing This is not "too hard" but is certainly tricky. I did a bunch of them about a month ago but didn't write a lot of = documentation on the topic. You can find some useful notes in = http://goo.gl/rWpdDR though, which includes slides and accompanying = notes for the tutorial I gave at AsiaBSDCon. Most of that ought to be = converted to actual documentation of course. The very first step, which is arguably the hardest, is to get the tests = to work while running them with the prove(1) tool. This has several = implications (an important one being that any testing logic must be = removed from the Makefile). > 2) add new *.t files for directories that enable running = TAP-enabled binaries through prove That's good as a second step. I suspect these new *.t files will just = invoke another script (the actual test), which means the .t files will = be removed when the tests are hooked into the test suite. But that's = fine: it's better to get them running with prove(1) first because then = the move is simple enough. > 3) *.c updates to remove clang compiler warnings Yes please. Once the tests run with prove(1) and pass, we can bundle them into the = test suite. This roughly involves moving the code to corresponding = 'tests/' subdirectories, writing Makefiles and updating the mtree. = Mostly mechanical (but very annoying when the tests provide tons of data = files; see usr.bin/make/tests/). > Can someone please provide a process for getting these approved and = committed in a timely manner? I would prefer to get these upstream and = then pull them down if at all possible, but I simply don't have time for = these to languish for weeks in a committee. >=20 > Isilon has BSD committers (eg: bdrewery) I can leverage if that would = be more expedient but given these are all testing-centric it seems like = getting approval from one or more people on freebsd-testing would be = best. If you can get self-contained diffs, I can try to get them in for you = (but can't make any promises as my time for FreeBSD varies significantly = week to week...) For simplicity of review, I'd appreciate at least one patch for every = subdirectory within tools/regression/foo/bar/ with at least one patch to = get the first tests up and running first with prove(1). More patches = are better! The move of the fixed tests to the new infrastructure (aka the = boilerplate work) is something I may be able to help with.= From owner-freebsd-testing@FreeBSD.ORG Sat Apr 12 02:00:16 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31E01989; Sat, 12 Apr 2014 02:00:16 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd54.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2B7817C9; Sat, 12 Apr 2014 02:00:15 +0000 (UTC) Received: from maildlpprd54.lss.emc.com (maildlpprd54.lss.emc.com [10.106.48.158]) by mailuogwprd54.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s3C20552030650 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 11 Apr 2014 22:00:08 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd54.lss.emc.com s3C20552030650 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1397268008; bh=ZCCF+s58Bx0HEt/hOsmqnoefWhY=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=N7CZRr4bh2r6+5l/o3ZPCaoJ0HGHx3FheqMbR4aq/OSxdVA/nhifZ0e6V+BC2Sdfn JBJqz4Vs3C6vtom1e99rl0JEdRxfHGLrKdU8TQ4/xlQqHYY3wlPc0+nog+HWKqlPxI SLVG0g3VD/i5DkjEzeMAPv50QW454X75Zsj1oKcM= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd54.lss.emc.com s3C20552030650 Received: from mailusrhubprd03.lss.emc.com (mailusrhubprd03.lss.emc.com [10.253.24.21]) by maildlpprd54.lss.emc.com (RSA Interceptor); Fri, 11 Apr 2014 21:59:44 -0400 Received: from mxhub22.corp.emc.com (mxhub22.corp.emc.com [128.222.70.134]) by mailusrhubprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s3C1xhJg010419 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 11 Apr 2014 21:59:44 -0400 Received: from MXHUB106.corp.emc.com (10.253.58.23) by mxhub22.corp.emc.com (128.222.70.134) with Microsoft SMTP Server (TLS) id 8.3.327.1; Fri, 11 Apr 2014 21:59:43 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.182]) by MXHUB106.corp.emc.com ([10.253.58.23]) with mapi id 14.03.0158.001; Fri, 11 Apr 2014 21:59:43 -0400 From: "Peel, Casey" To: Julio Merino Subject: RE: Please provide process for small, targeted fixes in tools/regression Thread-Topic: Please provide process for small, targeted fixes in tools/regression Thread-Index: Ac9VrITJRfNb0i8ZTXWISpxKRMnAiQALXaOAAAXxSeA= Date: Sat, 12 Apr 2014 01:59:42 +0000 Message-ID: <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> In-Reply-To: <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] Content-Type: multipart/mixed; boundary="_004_16437CC5729B5345AF77F816513376E820BAFE8EMX103CL02corpem_" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd03.lss.emc.com X-RSA-Classifications: Source Code, public Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 02:00:16 -0000 --_004_16437CC5729B5345AF77F816513376E820BAFE8EMX103CL02corpem_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Exellent, thanks Julio. Attached are 3 self-contained diffs, one per directory. Descriptions of the= changes: * flock - .c changes to fix compiler errors; .t to wrap flock binary for in= tegrating with prove * geom_mirror - gmirror won't let you remove the last partition in a mirror= which all of these testcases do and thus fail (note that I've got these te= stcases plus other gmirror testcases already moved over to ATF and going th= rough internal code reviews now, so maybe we just want to discard this) * kqueue - kqtest outputs files in TAP-consumable format, this is a .t file= to run it through prove These were an intial salvo at getting existing tests working for our intern= al regression. I'm sure we'll be looking at others as well. If someone can = just review the changes, I can get someone internally to commit them if nec= ssary (although these will take more time to review than commit given how s= mall they are). -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 -----Original Message----- From: Julio Merino [mailto:jmmv@meroh.net] On Behalf Of Julio Merino Sent: Friday, April 11, 2014 12:02 PM To: Peel, Casey Cc: freebsd-testing@freebsd.org; bdrewery@FreeBSD.org Subject: Re: Please provide process for small, targeted fixes in tools/regr= ession Hello Casey, On Apr 11, 2014, at 13:56, Peel, Casey wrote: > I have several small, targted fixes to files in src/tools/regression/ to: >=20 > 1) enable existing *.t tests to run that were failing This is not "too hard" but is certainly tricky. I did a bunch of them about a month ago but didn't write a lot of documenta= tion on the topic. You can find some useful notes in http://goo.gl/rWpdDR = though, which includes slides and accompanying notes for the tutorial I gav= e at AsiaBSDCon. Most of that ought to be converted to actual documentatio= n of course. The very first step, which is arguably the hardest, is to get the tests to = work while running them with the prove(1) tool. This has several implicati= ons (an important one being that any testing logic must be removed from the= Makefile). > 2) add new *.t files for directories that enable running TAP-enabled= binaries through prove That's good as a second step. I suspect these new *.t files will just invo= ke another script (the actual test), which means the .t files will be remov= ed when the tests are hooked into the test suite. But that's fine: it's be= tter to get them running with prove(1) first because then the move is simpl= e enough. > 3) *.c updates to remove clang compiler warnings Yes please. Once the tests run with prove(1) and pass, we can bundle them into the test= suite. This roughly involves moving the code to corresponding 'tests/' su= bdirectories, writing Makefiles and updating the mtree. Mostly mechanical = (but very annoying when the tests provide tons of data files; see usr.bin/m= ake/tests/). > Can someone please provide a process for getting these approved and commi= tted in a timely manner? I would prefer to get these upstream and then pull= them down if at all possible, but I simply don't have time for these to la= nguish for weeks in a committee. >=20 > Isilon has BSD committers (eg: bdrewery) I can leverage if that would be = more expedient but given these are all testing-centric it seems like gettin= g approval from one or more people on freebsd-testing would be best. If you can get self-contained diffs, I can try to get them in for you (but = can't make any promises as my time for FreeBSD varies significantly week to= week...) For simplicity of review, I'd appreciate at least one patch for every subdi= rectory within tools/regression/foo/bar/ with at least one patch to get the= first tests up and running first with prove(1). More patches are better! The move of the fixed tests to the new infrastructure (aka the boilerplate = work) is something I may be able to help with. --_004_16437CC5729B5345AF77F816513376E820BAFE8EMX103CL02corpem_ Content-Type: application/octet-stream; name="flock.diff" Content-Description: flock.diff Content-Disposition: attachment; filename="flock.diff"; size=1239; creation-date="Sat, 12 Apr 2014 01:56:33 GMT"; modification-date="Fri, 11 Apr 2014 19:10:05 GMT" Content-Transfer-Encoding: base64 SW5kZXg6IGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2ZpbGUvZmxvY2svZmxvY2suYwo9 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09Ci0tLSBmcmVlYnNkLWhlYWQub3JpZy90b29scy9yZWdyZXNzaW9uL2ZpbGUvZmxv Y2svZmxvY2suYworKysgZnJlZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Npb24vZmlsZS9mbG9jay9m bG9jay5jCkBAIC01NSw3ICs1NSw3IEBACiAjZW5kaWYKICNlbmRpZgogCi1pbnQgdmVyYm9zZSA9 IDA7CitzdGF0aWMgaW50IHZlcmJvc2UgPSAwOwogCiBzdGF0aWMgaW50CiBtYWtlX2ZpbGUoY29u c3QgY2hhciAqcGF0aG5hbWUsIG9mZl90IHN6KQpAQCAtMTUxNCw3ICsxNTE0LDcgQEAgc3RydWN0 IHRlc3QgewogCWludCBpbnRyOwkJLyogbm9uLXplcm8gaWYgdGhlIHRlc3QgaW50ZXJydXB0cyBh IGxvY2sgKi8KIH07CiAKLXN0cnVjdCB0ZXN0IHRlc3RzW10gPSB7CitzdGF0aWMgc3RydWN0IHRl c3QgdGVzdHNbXSA9IHsKIAl7CXRlc3QxLAkJMSwJMAl9LAogCXsJdGVzdDIsCQkyLAkwCX0sCiAJ ewl0ZXN0MywJCTMsCTEJfSwKQEAgLTE1MzIsNyArMTUzMiw3IEBAIHN0cnVjdCB0ZXN0IHRlc3Rz W10gPSB7CiAJewl0ZXN0MTUsCQkxNSwJMQl9LAogCXsJdGVzdDE2LAkJMTYsCTEJfSwKIH07Ci1p bnQgdGVzdF9jb3VudCA9IHNpemVvZih0ZXN0cykgLyBzaXplb2YodGVzdHNbMF0pOworc3RhdGlj IGludCB0ZXN0X2NvdW50ID0gc2l6ZW9mKHRlc3RzKSAvIHNpemVvZih0ZXN0c1swXSk7CiAKIGlu dAogbWFpbihpbnQgYXJnYywgY29uc3QgY2hhciAqYXJndltdKQpJbmRleDogZnJlZWJzZC1oZWFk L3Rvb2xzL3JlZ3Jlc3Npb24vZmlsZS9mbG9jay9mbG9jay50Cj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIC9kZXYv bnVsbAorKysgZnJlZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Npb24vZmlsZS9mbG9jay9mbG9jay50 CkBAIC0wLDAgKzEsMTQgQEAKKyMhL2Jpbi9zaAorCitlY2hvICIxLi4xNiIKKworbWFrZSA+IC9k ZXYvbnVsbCB8fCBleGl0IDEKKworZm9yIG4gaW4gYHNlcSAxIDE2YDsgZG8KKwkuL2Zsb2NrIC4g JG4gfCBncmVwIC1xIFNVQ0NFRUQKKwlpZiBbICQ/IC1lcSAwIF07IHRoZW4KKwkJZWNobyAib2sg JG4iCisJZWxzZQorCQllY2hvICJub3Qgb2sgJG4iCisJZmkKK2RvbmUK --_004_16437CC5729B5345AF77F816513376E820BAFE8EMX103CL02corpem_ Content-Type: application/octet-stream; name="geom_mirror.diff" Content-Description: geom_mirror.diff Content-Disposition: attachment; filename="geom_mirror.diff"; size=4399; creation-date="Sat, 12 Apr 2014 01:56:33 GMT"; modification-date="Fri, 11 Apr 2014 19:10:05 GMT" Content-Transfer-Encoding: base64 SW5kZXg6IGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2dlb21fbWlycm9yL3Rlc3QtMS50 Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT0KLS0tIGZyZWVic2QtaGVhZC5vcmlnL3Rvb2xzL3JlZ3Jlc3Npb24vZ2VvbV9t aXJyb3IvdGVzdC0xLnQKKysrIGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2dlb21fbWly cm9yL3Rlc3QtMS50CkBAIC0yNiw5ICsyNiw3IEBAIGVsc2UKIAllY2hvICJub3Qgb2sgMSIKIGZp CiAKLWdtaXJyb3IgcmVtb3ZlICRuYW1lIG1kJHt1czB9Ci1nbWlycm9yIHJlbW92ZSAkbmFtZSBt ZCR7dXMxfQotZ21pcnJvciByZW1vdmUgJG5hbWUgbWQke3VzMn0KK2dtaXJyb3IgZGVzdHJveSAk bmFtZQogbWRjb25maWcgLWQgLXUgJHVzMAogbWRjb25maWcgLWQgLXUgJHVzMQogbWRjb25maWcg LWQgLXUgJHVzMgpJbmRleDogZnJlZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Npb24vZ2VvbV9taXJy b3IvdGVzdC0yLnQKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQotLS0gZnJlZWJzZC1oZWFkLm9yaWcvdG9vbHMvcmVncmVz c2lvbi9nZW9tX21pcnJvci90ZXN0LTIudAorKysgZnJlZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Np b24vZ2VvbV9taXJyb3IvdGVzdC0yLnQKQEAgLTUyLDcgKzUyLDcgQEAgZWxzZQogCWVjaG8gIm9r IDQiCiBmaQogCi1nbWlycm9yIHJlbW92ZSAkbmFtZSBtZCR7dXMwfSBtZCR7dXMxfSBtZCR7dXMy fQorZ21pcnJvciBkZXN0cm95ICRuYW1lCiBtZGNvbmZpZyAtZCAtdSAkdXMwCiBtZGNvbmZpZyAt ZCAtdSAkdXMxCiBtZGNvbmZpZyAtZCAtdSAkdXMyCkluZGV4OiBmcmVlYnNkLWhlYWQvdG9vbHMv cmVncmVzc2lvbi9nZW9tX21pcnJvci90ZXN0LTMudAo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBmcmVlYnNkLWhl YWQub3JpZy90b29scy9yZWdyZXNzaW9uL2dlb21fbWlycm9yL3Rlc3QtMy50CisrKyBmcmVlYnNk LWhlYWQvdG9vbHMvcmVncmVzc2lvbi9nZW9tX21pcnJvci90ZXN0LTMudApAQCAtMyw3ICszLDcg QEAKIAogLiBgZGlybmFtZSAkMGAvY29uZi5zaAogCi1lY2hvICIxLi41IgorZWNobyAiMS4uNCIK IAogYmFsYW5jZT0icm91bmQtcm9iaW4iCiB1czA9NDUKQEAgLTQ5LDE5ICs0OSwxMyBAQCBlbHNl CiAJZWNobyAib2sgMyIKIGZpCiAKLWdtaXJyb3IgcmVtb3ZlICRuYW1lIG1kJHt1czJ9Ci1kZCBp Zj0vZGV2L21pcnJvci8ke25hbWV9IG9mPSR7ZHN0fSBicz0kZGRicyBjb3VudD0kbmJsb2NrczIg Pi9kZXYvbnVsbCAyPiYxCi1pZiBbIGBtZDUgLXEgJHtzcmN9YCAhPSBgbWQ1IC1xICR7ZHN0fWAg XTsgdGhlbgotCWVjaG8gIm5vdCBvayA0IgotZWxzZQotCWVjaG8gIm9rIDQiCi1maQorZ21pcnJv ciBkZXN0cm95ICRuYW1lCiAKICMgbWlycm9yLyR7bmFtZX0gc2hvdWxkIGJlIHJlbW92ZWQuCiBp ZiBbIC1jIC9kZXYvJHtuYW1lfSBdOyB0aGVuCi0JZWNobyAibm90IG9rIDUiCisJZWNobyAibm90 IG9rIDQiCiBlbHNlCi0JZWNobyAib2sgNSIKKwllY2hvICJvayA0IgogZmkKIAogbWRjb25maWcg LWQgLXUgJHVzMApJbmRleDogZnJlZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Npb24vZ2VvbV9taXJy b3IvdGVzdC00LnQKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQotLS0gZnJlZWJzZC1oZWFkLm9yaWcvdG9vbHMvcmVncmVz c2lvbi9nZW9tX21pcnJvci90ZXN0LTQudAorKysgZnJlZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Np b24vZ2VvbV9taXJyb3IvdGVzdC00LnQKQEAgLTMsNyArMyw3IEBACiAKIC4gYGRpcm5hbWUgJDBg L2NvbmYuc2gKIAotZWNobyAiMS4uNSIKK2VjaG8gIjEuLjQiCiAKIGJhbGFuY2U9ImxvYWQiCiB1 czA9NDUKQEAgLTQ5LDE5ICs0OSwxMyBAQCBlbHNlCiAJZWNobyAib2sgMyIKIGZpCiAKLWdtaXJy b3IgcmVtb3ZlICRuYW1lIG1kJHt1czJ9Ci1kZCBpZj0vZGV2L21pcnJvci8ke25hbWV9IG9mPSR7 ZHN0fSBicz0kZGRicyBjb3VudD0kbmJsb2NrczIgPi9kZXYvbnVsbCAyPiYxCi1pZiBbIGBtZDUg LXEgJHtzcmN9YCAhPSBgbWQ1IC1xICR7ZHN0fWAgXTsgdGhlbgotCWVjaG8gIm5vdCBvayA0Igot ZWxzZQotCWVjaG8gIm9rIDQiCi1maQorZ21pcnJvciBkZXN0cm95ICRuYW1lCiAKICMgbWlycm9y LyR7bmFtZX0gc2hvdWxkIGJlIHJlbW92ZWQuCiBpZiBbIC1jIC9kZXYvJHtuYW1lfSBdOyB0aGVu Ci0JZWNobyAibm90IG9rIDUiCisJZWNobyAibm90IG9rIDQiCiBlbHNlCi0JZWNobyAib2sgNSIK KwllY2hvICJvayA0IgogZmkKIAogbWRjb25maWcgLWQgLXUgJHVzMApJbmRleDogZnJlZWJzZC1o ZWFkL3Rvb2xzL3JlZ3Jlc3Npb24vZ2VvbV9taXJyb3IvdGVzdC01LnQKPT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0g ZnJlZWJzZC1oZWFkLm9yaWcvdG9vbHMvcmVncmVzc2lvbi9nZW9tX21pcnJvci90ZXN0LTUudAor KysgZnJlZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Npb24vZ2VvbV9taXJyb3IvdGVzdC01LnQKQEAg LTMsNyArMyw3IEBACiAKIC4gYGRpcm5hbWUgJDBgL2NvbmYuc2gKIAotZWNobyAiMS4uNSIKK2Vj aG8gIjEuLjQiCiAKIGJhbGFuY2U9InNwbGl0IgogdXMwPTQ1CkBAIC00OSwxOSArNDksMTMgQEAg ZWxzZQogCWVjaG8gIm9rIDMiCiBmaQogCi1nbWlycm9yIHJlbW92ZSAkbmFtZSBtZCR7dXMyfQot ZGQgaWY9L2Rldi9taXJyb3IvJHtuYW1lfSBvZj0ke2RzdH0gYnM9JGRkYnMgY291bnQ9JG5ibG9j a3MyID4vZGV2L251bGwgMj4mMQotaWYgWyBgbWQ1IC1xICR7c3JjfWAgIT0gYG1kNSAtcSAke2Rz dH1gIF07IHRoZW4KLQllY2hvICJub3Qgb2sgNCIKLWVsc2UKLQllY2hvICJvayA0IgotZmkKK2dt aXJyb3IgZGVzdHJveSAkbmFtZQogCiAjIG1pcnJvci8ke25hbWV9IHNob3VsZCBiZSByZW1vdmVk LgogaWYgWyAtYyAvZGV2LyR7bmFtZX0gXTsgdGhlbgotCWVjaG8gIm5vdCBvayA1IgorCWVjaG8g Im5vdCBvayA0IgogZWxzZQotCWVjaG8gIm9rIDUiCisJZWNobyAib2sgNCIKIGZpCiAKIG1kY29u ZmlnIC1kIC11ICR1czAKSW5kZXg6IGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2dlb21f bWlycm9yL3Rlc3QtNi50Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGZyZWVic2QtaGVhZC5vcmlnL3Rvb2xzL3Jl Z3Jlc3Npb24vZ2VvbV9taXJyb3IvdGVzdC02LnQKKysrIGZyZWVic2QtaGVhZC90b29scy9yZWdy ZXNzaW9uL2dlb21fbWlycm9yL3Rlc3QtNi50CkBAIC00NSw3ICs0NSw3IEBAIGVsc2UKIAllY2hv ICJvayAyIgogZmkKIAotZ21pcnJvciByZW1vdmUgJG5hbWUgbWQke3VzMH0gbWQke3VzMX0gbWQk e3VzMn0KK2dtaXJyb3IgZGVzdHJveSAkbmFtZQogbWRjb25maWcgLWQgLXUgJHVzMAogbWRjb25m aWcgLWQgLXUgJHVzMQogbWRjb25maWcgLWQgLXUgJHVzMgpJbmRleDogZnJlZWJzZC1oZWFkL3Rv b2xzL3JlZ3Jlc3Npb24vZ2VvbV9taXJyb3IvdGVzdC03LnQKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gZnJlZWJz ZC1oZWFkLm9yaWcvdG9vbHMvcmVncmVzc2lvbi9nZW9tX21pcnJvci90ZXN0LTcudAorKysgZnJl ZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Npb24vZ2VvbV9taXJyb3IvdGVzdC03LnQKQEAgLTMsNyAr Myw3IEBACiAKIC4gYGRpcm5hbWUgJDBgL2NvbmYuc2gKIAotZWNobyAiMS4uNSIKK2VjaG8gIjEu LjQiCiAKIGJhbGFuY2U9InByZWZlciIKIHVzMD00NQpAQCAtNDksMTkgKzQ5LDEzIEBAIGVsc2UK IAllY2hvICJvayAzIgogZmkKIAotZ21pcnJvciByZW1vdmUgJG5hbWUgbWQke3VzMn0KLWRkIGlm PS9kZXYvbWlycm9yLyR7bmFtZX0gb2Y9JHtkc3R9IGJzPSRkZGJzIGNvdW50PSRuYmxvY2tzMiA+ L2Rldi9udWxsIDI+JjEKLWlmIFsgYG1kNSAtcSAke3NyY31gICE9IGBtZDUgLXEgJHtkc3R9YCBd OyB0aGVuCi0JZWNobyAibm90IG9rIDQiCi1lbHNlCi0JZWNobyAib2sgNCIKLWZpCitnbWlycm9y IGRlc3Ryb3kgJG5hbWUKIAogIyBtaXJyb3IvJHtuYW1lfSBzaG91bGQgYmUgcmVtb3ZlZC4KIGlm IFsgLWMgL2Rldi8ke25hbWV9IF07IHRoZW4KLQllY2hvICJub3Qgb2sgNSIKKwllY2hvICJub3Qg b2sgNCIKIGVsc2UKLQllY2hvICJvayA1IgorCWVjaG8gIm9rIDQiCiBmaQogCiBtZGNvbmZpZyAt ZCAtdSAkdXMwCg== --_004_16437CC5729B5345AF77F816513376E820BAFE8EMX103CL02corpem_ Content-Type: application/octet-stream; name="kqueue.diff" Content-Description: kqueue.diff Content-Disposition: attachment; filename="kqueue.diff"; size=474; creation-date="Sat, 12 Apr 2014 01:56:33 GMT"; modification-date="Fri, 11 Apr 2014 19:10:05 GMT" Content-Transfer-Encoding: base64 SW5kZXg6IGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2txdWV1ZS9rcXRlc3QudAo9PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09Ci0tLSAvZGV2L251bGwKKysrIGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2tx dWV1ZS9rcXRlc3QudApAQCAtMCwwICsxLDE2IEBACisjIS9iaW4vc2gKKworbWFrZSA+IC9kZXYv bnVsbCB8fCBleGl0IDEKKworLi9rcXRlc3QgfCB3aGlsZSByZWFkIGxpbmU7IGRvCisJZWNobyAk bGluZSB8IGdyZXAgLXEgcGFzc2VkCisJaWYgWyAkPyAtZXEgMCBdOyB0aGVuCisJCWVjaG8gIm9r IgorCWZpCisKKwllY2hvICRsaW5lIHwgZ3JlcCAtcSAndGVzdHMgY29tcGxldGVkJworCWlmIFsg JD8gLWVxIDAgXTsgdGhlbgorCQllY2hvIC1uICIxLi4iCisJCWVjaG8gJGxpbmUgfCBjdXQgLWQn ICcgLWYzCisJZmkKK2RvbmUK --_004_16437CC5729B5345AF77F816513376E820BAFE8EMX103CL02corpem_-- From owner-freebsd-testing@FreeBSD.ORG Tue Apr 15 17:43:21 2014 Return-Path: Delivered-To: freebsd-testing@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 77AA5C15 for ; Tue, 15 Apr 2014 17:43:21 +0000 (UTC) Received: from mail-qg0-x22f.google.com (mail-qg0-x22f.google.com [IPv6:2607:f8b0:400d:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D91E1773 for ; Tue, 15 Apr 2014 17:43:18 +0000 (UTC) Received: by mail-qg0-f47.google.com with SMTP id i50so2694467qgf.34 for ; Tue, 15 Apr 2014 10:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=De38ra3Fa3nR8flzmVYB8Towkr7KSRq2TUhmViOsf4E=; b=MtGHZ5rT7TcORMCxaUTDxPnGU0RxoL9mDcPRNFOgmr0ZZJMUkTKrfdRBBxOwd6Jcmm gVv2lUlylQSc8uzjtmIyJpG0c+8JhK6huog7mnMm/XKyv9E3b4JHygM0VMoIsy187c0/ /jZsCLRd9H71u/Cd0HVcjtqE0AvSW0FR8AhFFQtp7bJX+MkpxqGjmPM1Vm2ZPlKXfgvO 3mx1G6Eu5vURFoBaHqZmuNkYMAGLb1BAnAZfAQEirBzQAS3QXGuxjVIwkwXAXVFROA3o 9nu9IecBIod6qxkHv72x91Rm/vowKv6gBqaLKEF/jD3VbHdKIT2HYtyC09lWvsHgslKE HTBw== MIME-Version: 1.0 X-Received: by 10.140.101.99 with SMTP id t90mr4361985qge.115.1397583797263; Tue, 15 Apr 2014 10:43:17 -0700 (PDT) Received: by 10.140.91.162 with HTTP; Tue, 15 Apr 2014 10:43:17 -0700 (PDT) Date: Tue, 15 Apr 2014 19:43:17 +0200 Message-ID: Subject: Coverity scans on Jenkins From: =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= To: freebsd-testing@FreeBSD.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 17:43:21 -0000 Hey all so how do I get on this CI bandwagon? I have some scripts that set up the coverity environment (a one-time thing) then those should run at most once per week (preferably when -CURRENT is actually building) and then tar up the results and upload them to scan.coverity.com. As there are some passwords/tokens that are required for the upload, I'd like it if not all of the scripts are visible to everyone that has a Jenkins account. Is that possible? Cheers, Uli From owner-freebsd-testing@FreeBSD.ORG Tue Apr 15 18:10:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D7435AA for ; Tue, 15 Apr 2014 18:10:59 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8ADDC1A08 for ; Tue, 15 Apr 2014 18:10:58 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id gf5so7212271lab.35 for ; Tue, 15 Apr 2014 11:10:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=LGk6xJHSPFNiBqCbA26yp0oNkbF5/E4fF3PgmpH3Tz8=; b=sA1QDvZWvO/M4X6SR9cxIdqcd4LAxjHqAy52gv/WyELfcN1CTlUdk+40TC1tINaxdW Zv5lPyph2WXUWSLCSYxHZvZ48sp3+jivDxtDTOu8782shqgftrS4U+VRhrRfLNeRliti Knvj930MiQngSXONESfCCyk0asC7+pnSS6N2ighY521qlgMoli8fW1i+yo6kDt4d29d6 hQhqExq5vcU6Mb6BoefzBKFuhb+0W9u2RMclD9WB+OMNzPTdrg2v4w2gKOS0c3iukF7s Ycipe5NUDSl0NgJmBbOu1OO3NmXllobr8XAgfCrEg0EUB16F3R6jR4aRSxg0I5W/dmDC YJ1g== MIME-Version: 1.0 X-Received: by 10.152.246.43 with SMTP id xt11mr2238539lac.34.1397585456420; Tue, 15 Apr 2014 11:10:56 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Tue, 15 Apr 2014 11:10:56 -0700 (PDT) In-Reply-To: References: Date: Tue, 15 Apr 2014 11:10:56 -0700 X-Google-Sender-Auth: eydC8rsqTxwoITLLpqcOhv23paw Message-ID: Subject: Re: Coverity scans on Jenkins From: Craig Rodrigues To: =?ISO-8859-1?Q?Ulrich_Sp=F6rlein?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 18:10:59 -0000 Hi, Yes, what you want is totally doable. If you look at https://jenkins.freebsd.org , you will see that we have several bhyve VM's that build several src branches, and we also have several builds that use poudriere to build ports. We can set up another build configuration, either in an existing VM, or a n= ew VM, to run your scripts once a week. Will you be at BSDCan? There is a working group on May 15 where it would be a good place to coordinate on this: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups -- Craig On Tue, Apr 15, 2014 at 10:43 AM, Ulrich Sp=F6rlein = wrote: > Hey all > > so how do I get on this CI bandwagon? I have some scripts that set up > the coverity environment (a one-time thing) then those should run at > most once per week (preferably when -CURRENT is actually building) and > then tar up the results and upload them to scan.coverity.com. > > As there are some passwords/tokens that are required for the upload, > I'd like it if not all of the scripts are visible to everyone that has > a Jenkins account. > > Is that possible? > > Cheers, > Uli > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org= " From owner-freebsd-testing@FreeBSD.ORG Thu Apr 17 09:35:14 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C314698; Thu, 17 Apr 2014 09:35:14 +0000 (UTC) Received: from acme.spoerlein.net (unknown [IPv6:2a02:2528:fa:1000::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.spoerlein.net", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9AE5512D8; Thu, 17 Apr 2014 09:35:13 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a02:2528:fa:1000::1]) by acme.spoerlein.net (8.14.8/8.14.7) with ESMTP id s3H9Z9Ac090804 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 17 Apr 2014 11:35:10 +0200 (CEST) (envelope-from uqs@FreeBSD.org) Date: Thu, 17 Apr 2014 11:35:09 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Craig Rodrigues Subject: Re: Coverity scans on Jenkins Message-ID: <20140417093509.GA3620@acme.spoerlein.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 09:35:14 -0000 No BSDCan for me this year, I'll be on the road: http://igg.me/at/aor2014/x/5664674 So do I need a special account to get the scripts and coverity binaries uploaded? I know we use LDAP now at the cluster, but I've never set up a password, how would I do that? A new VM might be easier, as it requires special setup with the coverity binaries and curl, etc. Cheers, Uli On Tue, 2014-04-15 at 11:10:56 -0700, Craig Rodrigues wrote: > Hi, > > Yes, what you want is totally doable. > If you look at https://jenkins.freebsd.org , you will see that > we have several bhyve VM's that build several src branches, > and we also have several builds that use poudriere to build ports. > > We can set up another build configuration, either in an existing VM, or a new > VM, to run your scripts once a week. > > Will you be at BSDCan? There is a working group on May 15 where it would > be a good place to coordinate on this: > https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups > > -- > Craig > > On Tue, Apr 15, 2014 at 10:43 AM, Ulrich Spörlein wrote: > > Hey all > > > > so how do I get on this CI bandwagon? I have some scripts that set up > > the coverity environment (a one-time thing) then those should run at > > most once per week (preferably when -CURRENT is actually building) and > > then tar up the results and upload them to scan.coverity.com. > > > > As there are some passwords/tokens that are required for the upload, > > I'd like it if not all of the scripts are visible to everyone that has > > a Jenkins account. > > > > Is that possible? > > > > Cheers, > > Uli > > _______________________________________________ > > freebsd-testing@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" From owner-freebsd-testing@FreeBSD.ORG Thu Apr 17 13:09:49 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A102469; Thu, 17 Apr 2014 13:09:49 +0000 (UTC) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D437A19CF; Thu, 17 Apr 2014 13:09:48 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id rd3so359250pab.25 for ; Thu, 17 Apr 2014 06:09:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=kb/op3nqRuBGNzpPh/wJWP1FdiM0kXtBLNjYrpqOn/I=; b=mwhHC8bzOo5COWLltgYA63Z9f+l0hEsPk5xRDBvh79eJvoxFsiRmQQtQrevFMZiNSE iSgtaTDwOGSW/YKPcr85Ch9Zpu6M2fWnv34WCxojJ0KyBhnhSQRgq9OB3Wf1Zqp00LQX T1uF0s20vHpOdUooSHhQInaBm78Kqk2GARH5uQI+MiAEP0gFQhDyeboBlJ25SCysWzgl UiJPY2yz9VHEDP7q7vMnFpBLufR9+J6Q6mqwyqDlxJIv3eooDfnPbRQNAoQGnp4Ocsq8 OXDnFTDvSoSBPrUKM3+bPvSyD2b02czxzNELGiZNbiK+EcymCYMka/PQfP/8/sZAIvq+ 6tUA== MIME-Version: 1.0 X-Received: by 10.66.144.102 with SMTP id sl6mr15322638pab.96.1397740171413; Thu, 17 Apr 2014 06:09:31 -0700 (PDT) Sender: lwhsu.freebsd@gmail.com Received: by 10.70.65.73 with HTTP; Thu, 17 Apr 2014 06:09:31 -0700 (PDT) In-Reply-To: <20140417093509.GA3620@acme.spoerlein.net> References: <20140417093509.GA3620@acme.spoerlein.net> Date: Thu, 17 Apr 2014 21:09:31 +0800 X-Google-Sender-Auth: x9M4zcfT_24cNUSGab3ea-H7iZY Message-ID: Subject: Re: Coverity scans on Jenkins From: Li-Wen Hsu To: =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 13:09:49 -0000 I've asked for setup scan.FreeBSD.org for publishing result of clang scan-build. It turns up that I still need to open a ticket in cluster admin's issue tracker. Do you think these two stuff can be hosted in the same host? Since they are both in scan.freebsd.your.org before. If so, I'll apply that later and we can work together on making it a job on jenkins.FreeBSD.org and published the results on scan.FreeBSD.org . Li-Wen On Thursday, April 17, 2014, Ulrich Sp=C3=B6rlein wrote: > No BSDCan for me this year, I'll be on the road: > http://igg.me/at/aor2014/x/5664674 > > So do I need a special account to get the scripts and coverity binaries > uploaded? > > I know we use LDAP now at the cluster, but I've never set up a password, > how would I do that? > > A new VM might be easier, as it requires special setup with the coverity > binaries and curl, etc. > > Cheers, > Uli > > On Tue, 2014-04-15 at 11:10:56 -0700, Craig Rodrigues wrote: > > Hi, > > > > Yes, what you want is totally doable. > > If you look at https://jenkins.freebsd.org , you will see that > > we have several bhyve VM's that build several src branches, > > and we also have several builds that use poudriere to build ports. > > > > We can set up another build configuration, either in an existing VM, or > a new > > VM, to run your scripts once a week. > > > > Will you be at BSDCan? There is a working group on May 15 where it wou= ld > > be a good place to coordinate on this: > > https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups > > > > -- > > Craig > > > > On Tue, Apr 15, 2014 at 10:43 AM, Ulrich Sp=C3=B6rlein > > wrote: > > > Hey all > > > > > > so how do I get on this CI bandwagon? I have some scripts that set up > > > the coverity environment (a one-time thing) then those should run at > > > most once per week (preferably when -CURRENT is actually building) an= d > > > then tar up the results and upload them to scan.coverity.com. > > > > > > As there are some passwords/tokens that are required for the upload, > > > I'd like it if not all of the scripts are visible to everyone that ha= s > > > a Jenkins account. > > > > > > Is that possible? > > > > > > Cheers, > > > Uli > > > _______________________________________________ > > > freebsd-testing@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > > > To unsubscribe, send any mail to " > freebsd-testing-unsubscribe@freebsd.org " > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org= > " --=20 Li-Wen Hsu http://lwhsu.org From owner-freebsd-testing@FreeBSD.ORG Thu Apr 17 14:52:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 683D3CE5; Thu, 17 Apr 2014 14:52:18 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C21617E9; Thu, 17 Apr 2014 14:52:17 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id ec20so456321lab.25 for ; Thu, 17 Apr 2014 07:52:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=GAP7QMdt9yYjxf7IQQR7WWk05LTgb3qpCDM5VBonXLQ=; b=1JZb/LOCQUMmDJ4QiINXTszgObQ94wuH7b1qo6/hsq+Y5yAkwEuCxcw9m/ne0W8Tzd LqJ23ippAla52vIrShxr2OaYS2Jj6rQl1TyDjUN85AYBALNDzwQcljPFIqhHvZEtgXQo EJgfd0YxvyGLshiqlCCujqYJOGuvyaNF4YcGgSlxmOhlMi8aaD7rikWPNR8Nxpe/n9Ca zvUp1Wvm/AfbfqHGN59n5JHQaCTmrn5mFtIaV0ymqowDskyQMnQJHv2I0TxxJ1cbR1E/ YHYMXZIavp2No4NBJcd4NiGcCZXWAMlWfWeVS1FL1IRxCYN5Ak3rnKZ2x4deVGzJhBj7 3aWg== MIME-Version: 1.0 X-Received: by 10.152.184.66 with SMTP id es2mr10287188lac.22.1397746335319; Thu, 17 Apr 2014 07:52:15 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Thu, 17 Apr 2014 07:52:15 -0700 (PDT) In-Reply-To: <20140417093509.GA3620@acme.spoerlein.net> References: <20140417093509.GA3620@acme.spoerlein.net> Date: Thu, 17 Apr 2014 07:52:15 -0700 X-Google-Sender-Auth: hjdUqiLCUYHF6S0csiz_iMXZO5k Message-ID: Subject: Re: Coverity scans on Jenkins From: Craig Rodrigues To: =?ISO-8859-1?Q?Ulrich_Sp=F6rlein?= , Li-Wen Hsu Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 14:52:18 -0000 On Thu, Apr 17, 2014 at 2:35 AM, Ulrich Sp=F6rlein wrote: > So do I need a special account to get the scripts and coverity binaries > uploaded? > > I know we use LDAP now at the cluster, but I've never set up a password, > how would I do that? We ( jenkins-admin@freebsd.org ) will need to set up a new VM for you inside the FreeBSD cluster, and work with the clusteradm@freebsd.or= g team to do it. To set your LDAP password, you need to follow these instructions: http://www.freebsd.org/doc/en/articles/committers-guide/kerberos-ldap.html > > A new VM might be easier, as it requires special setup with the coverity > binaries and curl, etc. We have been setting up multiple VM's running in bhyve inside the FreeBSD cluster and it has worked out very well in terms of performance: https://wiki.freebsd.org/Jenkins/#Setup_notes Li-Wen, can you take the lead on this item? Can you add a task item on http://wiki.freebsd.org/Jenkins ? We should set up a new VM, scan.freebsd.org, and have the Coverity binaries plus your clang-scan build running in that VM. I can help you with setting up the VM. -- Craig From owner-freebsd-testing@FreeBSD.ORG Thu Apr 17 15:11:23 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 55B50230; Thu, 17 Apr 2014 15:11:23 +0000 (UTC) Received: from acme.spoerlein.net (unknown [IPv6:2a02:2528:fa:1000::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.spoerlein.net", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D8E7F1A29; Thu, 17 Apr 2014 15:11:22 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a02:2528:fa:1000::1]) by acme.spoerlein.net (8.14.8/8.14.7) with ESMTP id s3HFBK3S098660 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 17 Apr 2014 17:11:20 +0200 (CEST) (envelope-from uqs@FreeBSD.org) Date: Thu, 17 Apr 2014 17:11:20 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Craig Rodrigues Subject: Re: Coverity scans on Jenkins Message-ID: <20140417151120.GB3620@acme.spoerlein.net> References: <20140417093509.GA3620@acme.spoerlein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Li-Wen Hsu X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 15:11:23 -0000 On Thu, 2014-04-17 at 07:52:15 -0700, Craig Rodrigues wrote: > On Thu, Apr 17, 2014 at 2:35 AM, Ulrich Spörlein wrote: > > So do I need a special account to get the scripts and coverity binaries > > uploaded? > > > > I know we use LDAP now at the cluster, but I've never set up a password, > > how would I do that? > > We ( jenkins-admin@freebsd.org ) will need to set up a new VM > for you inside the FreeBSD cluster, and work with the clusteradm@freebsd.org > team to do it. > > To set your LDAP password, you need to follow these instructions: > http://www.freebsd.org/doc/en/articles/committers-guide/kerberos-ldap.html > > > > > A new VM might be easier, as it requires special setup with the coverity > > binaries and curl, etc. > > We have been setting up multiple VM's running in bhyve inside the FreeBSD > cluster and it has worked out very well in terms of performance: > https://wiki.freebsd.org/Jenkins/#Setup_notes > > > Li-Wen, can you take the lead on this item? Can you add a task item > on http://wiki.freebsd.org/Jenkins ? We should set up > a new VM, scan.freebsd.org, and have the Coverity binaries plus > your clang-scan build running in that VM. I can help you with setting up > the VM. A shared VM sounds good to me! I would need zsh, vim, tmux, curl on that box. The clang stuff needs a webserver and fdupes would be helpful. Oh, it should be a -CURRENT machine, so that buildworld of head doesn't break every other week :/ Cheers, Uli From owner-freebsd-testing@FreeBSD.ORG Sun Apr 20 21:14:34 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CD0E3AC; Sun, 20 Apr 2014 21:14:34 +0000 (UTC) Received: from FreeBSD.cs.nctu.edu.tw (FreeBSD.cs.nctu.edu.tw [140.113.17.209]) by mx1.freebsd.org (Postfix) with ESMTP id 125E115C9; Sun, 20 Apr 2014 21:14:33 +0000 (UTC) Received: by FreeBSD.cs.nctu.edu.tw (Postfix, from userid 1058) id 786D523E8; Mon, 21 Apr 2014 05:14:27 +0800 (CST) Date: Mon, 21 Apr 2014 05:14:27 +0800 From: Li-Wen Hsu To: Ulrich =?utf-8?B?U3DDtnJsZWlu?= Subject: Re: Coverity scans on Jenkins Message-ID: <20140420211427.GA12085@FreeBSD.cs.nctu.edu.tw> References: <20140417093509.GA3620@acme.spoerlein.net> <20140417151120.GB3620@acme.spoerlein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20140417151120.GB3620@acme.spoerlein.net> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 21:14:34 -0000 On Thu, Apr 17, 2014 at 17:11:20 +0200, Ulrich Spörlein wrote: > On Thu, 2014-04-17 at 07:52:15 -0700, Craig Rodrigues wrote: > > On Thu, Apr 17, 2014 at 2:35 AM, Ulrich Spörlein wrote: > > > So do I need a special account to get the scripts and coverity binaries > > > uploaded? > > > > > > I know we use LDAP now at the cluster, but I've never set up a password, > > > how would I do that? > > > > We ( jenkins-admin@freebsd.org ) will need to set up a new VM > > for you inside the FreeBSD cluster, and work with the clusteradm@freebsd.org > > team to do it. > > > > To set your LDAP password, you need to follow these instructions: > > http://www.freebsd.org/doc/en/articles/committers-guide/kerberos-ldap.html > > > > > > > > A new VM might be easier, as it requires special setup with the coverity > > > binaries and curl, etc. > > > > We have been setting up multiple VM's running in bhyve inside the FreeBSD > > cluster and it has worked out very well in terms of performance: > > https://wiki.freebsd.org/Jenkins/#Setup_notes > > > > > > Li-Wen, can you take the lead on this item? Can you add a task item > > on http://wiki.freebsd.org/Jenkins ? We should set up > > a new VM, scan.freebsd.org, and have the Coverity binaries plus > > your clang-scan build running in that VM. I can help you with setting up > > the VM. > > A shared VM sounds good to me! I would need zsh, vim, tmux, curl on that > box. The clang stuff needs a webserver and fdupes would be helpful. > > Oh, it should be a -CURRENT machine, so that buildworld of head doesn't > break every other week :/ OK, I've added two items on http://wiki.freebsd.org/Jenkins . And I'll follow https://wiki.freebsd.org/Jenkins/#Setup_notes to setup a -CURRENT VM for scan.FreeBSD.org on the same host of jenkins.FreeBSD.org , or do you suggest better place to host new VM? Cheers, Li-Wen -- Li-Wen Hsu http://lwhsu.org From owner-freebsd-testing@FreeBSD.ORG Sun Apr 20 22:21:45 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B0AACD7; Sun, 20 Apr 2014 22:21:45 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D9BF1A78; Sun, 20 Apr 2014 22:21:44 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id p9so2667830lbv.24 for ; Sun, 20 Apr 2014 15:21:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6zJeXukE8QAOdphYvWJ+hu4I/VCGe8Ti+nuCSD4D+0k=; b=DyAwz9k8C9i+0VAhTC5h7KFwy0eSN2sdXeIsOguVB4UWJTDZYt8OMjNRC70eSh3iZJ DTzuBXOKZ3rsdQoXk+++US5d4pP+yIH3+pWoASLw8l6PNmHpD1PF3LzpfgGe2iZ78pic KiSYIcrP11ikcwhRr23pGoFcuglFXAw4BQ9OsEpghFBm50IMleqrOADWcwLxvsS9yfAb 0UwhD5Ths/iO+OYKpe2vq60bpDnIgevkG2Ng0wxsmbY+0y3SSwChO6iLuyXq4hB191/j vgS0218oQKc9E/GBheYk6T/a5TLeW8VhSumwi5p6OlipRkIEnjpk8gLIFiTkv1MMUlFq SFeA== MIME-Version: 1.0 X-Received: by 10.152.36.73 with SMTP id o9mr3976816laj.30.1398032501299; Sun, 20 Apr 2014 15:21:41 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Sun, 20 Apr 2014 15:21:41 -0700 (PDT) In-Reply-To: <20140420211427.GA12085@FreeBSD.cs.nctu.edu.tw> References: <20140417093509.GA3620@acme.spoerlein.net> <20140417151120.GB3620@acme.spoerlein.net> <20140420211427.GA12085@FreeBSD.cs.nctu.edu.tw> Date: Sun, 20 Apr 2014 15:21:41 -0700 X-Google-Sender-Auth: O_jtPCuMgopYmVSFjV22BQiHE-Q Message-ID: Subject: Re: Coverity scans on Jenkins From: Craig Rodrigues To: Li-Wen Hsu Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , =?ISO-8859-1?Q?Ulrich_Sp=F6rlein?= X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2014 22:21:45 -0000 On Sun, Apr 20, 2014 at 2:14 PM, Li-Wen Hsu wrote: > OK, I've added two items on http://wiki.freebsd.org/Jenkins . And I'll > follow https://wiki.freebsd.org/Jenkins/#Setup_notes to setup a -CURRENT > VM for scan.FreeBSD.org on the same host of jenkins.FreeBSD.org , or do > you suggest better place to host new VM? > Putting new VM on the same host is good enough to get started. We can move the VM around later if needed. -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Apr 22 00:21:23 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 630FF733; Tue, 22 Apr 2014 00:21:23 +0000 (UTC) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BEE119AD; Tue, 22 Apr 2014 00:21:22 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id im17so1810045vcb.9 for ; Mon, 21 Apr 2014 17:21:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=aAI5U+SG+71aSMvX7lukHVqJ9Y4pq20ks3Xn+zJ5NRY=; b=fjn+EuqtZelYvBsReqH/mS6Bhh09NrnU86PBgjAN9pmvOM4xhLFkkQ5Cy5tZLGmVWL VguSCYJIJc4+f15KbXkz8buKBU09WluumEKT7/x9qc8JBUb7+TB4skNiij9NoERnc/26 AKUh2kpfrT+I1n9fZS4iwZ4sz0Ef0j78yi/5hi/riMaAEN/fmt/IFRXPCtfaPK1OLMDN UuZthLMCzQ0GrnKG9ly5ctU472OSQpFQ+cVYoBcdafyLFT47S1ImPQvEsdbFkIgRjZqA Q4ojXQrvka4yNqnchRRUbGSbGyuPZDnvluhN3AYcXVkxDpKyyhhiBoSDy5MMYu6Ld2dU SFXw== MIME-Version: 1.0 X-Received: by 10.52.37.196 with SMTP id a4mr2723232vdk.33.1398126082167; Mon, 21 Apr 2014 17:21:22 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Mon, 21 Apr 2014 17:21:22 -0700 (PDT) In-Reply-To: References: <20140124162759.GB90996@stack.nl> <0A68C9FE-6A3A-4DCF-92A9-A506D7581D69@gmail.com> <96373EF8-77EA-4AA6-8422-63881C32E455@gmail.com> Date: Mon, 21 Apr 2014 17:21:22 -0700 Message-ID: Subject: Re: [PATCH v2] convert /bin/sh tests over to ATF From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , Jilles Tjoelker X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 00:21:23 -0000 On Sun, Mar 9, 2014 at 1:42 PM, Julio Merino wrote: ... > The tests need to populate /usr/tests/bin/sh/ with the > functional_test.sh driver and all its supporting data files. As I > understand it, your driver code assumes that the data files are > external (i.e. not bundled into the script itself), so those files > must be available from the same location as where the driver lives. > Hence you still need to use FILES to ensure the data files get > installed into /usr/tests/. > > ... but maybe I'm missing something from your patch; haven't reviewed > it again to double-check this. I've reduced the diff quite a bit. The end result of it can be found here: https://github.com/yaneurabeya/freebsd/pull/2 (and it will evolve over time if needed because github is additive in pull request reviews). FWIW, it might be better to actually review it on github instead of using this list as patches attached to any mailing list can become stale over time. You can get the diff like so: https://github.com/yaneurabeya/freebsd/pull/2.patch Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Tue Apr 22 02:08:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AC4EADF; Tue, 22 Apr 2014 02:08:09 +0000 (UTC) Received: from mail-vc0-x22a.google.com (mail-vc0-x22a.google.com [IPv6:2607:f8b0:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C7C6F12D8; Tue, 22 Apr 2014 02:08:08 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id hr9so545906vcb.1 for ; Mon, 21 Apr 2014 19:08:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9fElN21hm9i5mkkCilGD2v7EBVZjV5xNTKBZd1eLG6M=; b=djjOCSDc+03LJ4cgK3mrxWlZfGIJGAiSs3E7RrKI4NrHXncbMK+XuSPRBGoU5snAYS IGSSC3jSmFz7WY/VXTP9pr2NsrsaLfcmsHVPzV6N/vib8K6/kGkU9Pv6fnwQNYsI26xU /IeL8UT0q/ZShUJa9reDgN+QrcCDG20UVUxyrDaFgZsAS+fGDNKwMMfSGUyE6KP/gSRc YXVrxwOa1CV9yHe/elDY67omxhlHzwNuYJo3seqkH0KlcaDeVyyATQX2MOUlMkByLSeC Nrqou1LhbHDeXVEhZfInG8m1Kj5OHICUY7mZim28dpucI/9ph4B67vWkgdoXtwbAv1+2 wcxg== MIME-Version: 1.0 X-Received: by 10.220.159.4 with SMTP id h4mr34614259vcx.1.1398132487921; Mon, 21 Apr 2014 19:08:07 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Mon, 21 Apr 2014 19:08:07 -0700 (PDT) In-Reply-To: References: <6079AD8F-5EBB-431C-A06B-9B51E2729F5A@gmail.com> <4A25ADF9-3714-4006-ACB6-F8C745CB38AE@gmail.com> Date: Mon, 21 Apr 2014 19:08:07 -0700 Message-ID: Subject: Re: [PATCH v2] convert bin/date over to ATF From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , Giorgos Keramidas X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2014 02:08:09 -0000 Hi Julio! I've made the requested changes and posted a new version of the diff up to github here: https://github.com/yaneurabeya/freebsd/pull/3 (the patch can be obtained via https://github.com/yaneurabeya/freebsd/pull/3.patch ). Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Fri Apr 25 21:03:11 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0BC6212 for ; Fri, 25 Apr 2014 21:03:11 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9439E1709 for ; Fri, 25 Apr 2014 21:03:11 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id ik5so1119880vcb.4 for ; Fri, 25 Apr 2014 14:03:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=HXw9opm3kdLfV5PyBuQiFxivfFRzE4W4enIsrml1cxc=; b=IdEpR2IWDKdQz+Gquj+XUBcmJuAWupFY012d38xEsg4YJyhrBrQPoNnO5fu1U5/7KF J7ONY9eK8s54giqsJdDyHcTqcTu3UmvmYRIc/UxMsMWnaLtxcW2Fkcx6KOfIxx7mFXfL XqXg8wI/b6BBFMDPwuASF0drGLa0SaHQ9ygKa20K283Azmj0XlEJBW1f5FyFHi7REtfW hfTKUjGURbqnHotsVj0HmRHbwMLlgUHI2HmB3bn3jCMn0iCxj7JjZ0jOC4u/GuMqRnbK xjWVyHe1QlTqtmIGZR5MDU5GOevvb4jCSYiS2DXw4bCrtQWYtEgvEcI4TW6ADd7sUnVK YrEg== MIME-Version: 1.0 X-Received: by 10.52.183.228 with SMTP id ep4mr7066116vdc.30.1398459790754; Fri, 25 Apr 2014 14:03:10 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Fri, 25 Apr 2014 14:03:10 -0700 (PDT) Date: Fri, 25 Apr 2014 14:03:10 -0700 Message-ID: Subject: Failures on head From: Garrett Cooper To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 21:03:11 -0000 Hi, While looking at http://kyua3.nyi.freebsd.org/head/data/0-LATEST/results/index.html , I noticed that a handful of testcases are failing, in particular: Broken test cases usr.bin/sed/inplace_race_test:main Failed test cases bin/pkill/pgrep-j_test:main bin/pkill/pkill-j_test:main lib/atf/libatf-c/macros_test:use usr.bin/sed/multi_test:main usr.bin/yacc/legacy_test:main Are these known issues (I know about the pgrep/pkill issue, but I'm not sure about the rest). I'll write up patches to fix these if they aren't known. Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Apr 30 17:03:43 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD8C6FB5; Wed, 30 Apr 2014 17:03:42 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd52.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7993119DA; Wed, 30 Apr 2014 17:03:42 +0000 (UTC) Received: from maildlpprd55.lss.emc.com (maildlpprd55.lss.emc.com [10.106.48.159]) by mailuogwprd52.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s3UH3cjj019251 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 30 Apr 2014 13:03:39 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com s3UH3cjj019251 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1398877420; bh=Mh/1xrfh5kC9AZhIQono9VJpByQ=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=iQ/8ggAmPs2iRT8DMETsd42/NEjqyagXszF1oWTDPuMQ7xQUcvqKEwLwDkdSKSOab j3xMG/rxOUkEQ0BDlKG2oOSPs1kn2BJYtZg3dJYEaKP0c6xAMO0IFDyFxrQHljjVoS wWzFd+n7nVHS8ZG8ToRYBUg2FEPGHoNgsA6/yoV0= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com s3UH3cjj019251 Received: from mailusrhubprd03.lss.emc.com (mailusrhubprd03.lss.emc.com [10.253.24.21]) by maildlpprd55.lss.emc.com (RSA Interceptor); Wed, 30 Apr 2014 13:03:17 -0400 Received: from mxhub05.corp.emc.com (mxhub05.corp.emc.com [128.222.70.202]) by mailusrhubprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s3UH3Grh023253 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 30 Apr 2014 13:03:16 -0400 Received: from MXHUB104.corp.emc.com (10.253.58.16) by mxhub05.corp.emc.com (128.222.70.202) with Microsoft SMTP Server (TLS) id 8.3.327.1; Wed, 30 Apr 2014 13:03:16 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.21]) by MXHUB104.corp.emc.com ([::1]) with mapi id 14.03.0158.001; Wed, 30 Apr 2014 13:03:16 -0400 From: "Peel, Casey" To: Julio Merino Subject: RE: Please provide process for small, targeted fixes in tools/regression Thread-Topic: Please provide process for small, targeted fixes in tools/regression Thread-Index: Ac9VrITJRfNb0i8ZTXWISpxKRMnAiQALXaOAAAXxSeADqKQsMA== Date: Wed, 30 Apr 2014 17:03:15 +0000 Message-ID: <16437CC5729B5345AF77F816513376E820BCBE25@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> In-Reply-To: <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.7.188.119] Content-Type: multipart/mixed; boundary="_004_16437CC5729B5345AF77F816513376E820BCBE25MX103CL02corpem_" MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd03.lss.emc.com Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 17:03:43 -0000 --_004_16437CC5729B5345AF77F816513376E820BCBE25MX103CL02corpem_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Julio, here are 3 additional changes in tools/regression to get tests in th= ere to run and pass via prove. * libutil.diff - fixes test-humanize_number.c to print 1-based test numbers= instead of 0-based test numbers as prove expects. * mmap.diff - adds .t file to allow running mmap test through prove * fifo.diff - adds a top-level Makefile to compile the code in the fifo/ su= bdirectories. Adds .t to run the tests via prove. Any progress on the prior set I tossed you? -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 -----Original Message----- From: owner-freebsd-testing@freebsd.org [mailto:owner-freebsd-testing@freeb= sd.org] On Behalf Of Peel, Casey Sent: Friday, April 11, 2014 7:00 PM To: Julio Merino Cc: freebsd-testing@freebsd.org; bdrewery@FreeBSD.org Subject: RE: Please provide process for small, targeted fixes in tools/regr= ession Exellent, thanks Julio. Attached are 3 self-contained diffs, one per directory. Descriptions of the= changes: * flock - .c changes to fix compiler errors; .t to wrap flock binary for in= tegrating with prove * geom_mirror - gmirror won't let you remove the last partition in a mirror= which all of these testcases do and thus fail (note that I've got these te= stcases plus other gmirror testcases already moved over to ATF and going th= rough internal code reviews now, so maybe we just want to discard this) * kqueue - kqtest outputs files in TAP-consumable format, this is a .t file= to run it through prove These were an intial salvo at getting existing tests working for our intern= al regression. I'm sure we'll be looking at others as well. If someone can = just review the changes, I can get someone internally to commit them if nec= ssary (although these will take more time to review than commit given how s= mall they are). -- Casey -- Performance Test Ninja casey.peel@isilon.com / 206.777.7945 -----Original Message----- From: Julio Merino [mailto:jmmv@meroh.net] On Behalf Of Julio Merino Sent: Friday, April 11, 2014 12:02 PM To: Peel, Casey Cc: freebsd-testing@freebsd.org; bdrewery@FreeBSD.org Subject: Re: Please provide process for small, targeted fixes in tools/regr= ession Hello Casey, On Apr 11, 2014, at 13:56, Peel, Casey wrote: > I have several small, targted fixes to files in src/tools/regression/ to: >=20 > 1) enable existing *.t tests to run that were failing This is not "too hard" but is certainly tricky. I did a bunch of them about a month ago but didn't write a lot of documenta= tion on the topic. You can find some useful notes in http://goo.gl/rWpdDR = though, which includes slides and accompanying notes for the tutorial I gav= e at AsiaBSDCon. Most of that ought to be converted to actual documentatio= n of course. The very first step, which is arguably the hardest, is to get the tests to = work while running them with the prove(1) tool. This has several implicati= ons (an important one being that any testing logic must be removed from the= Makefile). > 2) add new *.t files for directories that enable running TAP-enabled= binaries through prove That's good as a second step. I suspect these new *.t files will just invo= ke another script (the actual test), which means the .t files will be remov= ed when the tests are hooked into the test suite. But that's fine: it's be= tter to get them running with prove(1) first because then the move is simpl= e enough. > 3) *.c updates to remove clang compiler warnings Yes please. Once the tests run with prove(1) and pass, we can bundle them into the test= suite. This roughly involves moving the code to corresponding 'tests/' su= bdirectories, writing Makefiles and updating the mtree. Mostly mechanical = (but very annoying when the tests provide tons of data files; see usr.bin/m= ake/tests/). > Can someone please provide a process for getting these approved and commi= tted in a timely manner? I would prefer to get these upstream and then pull= them down if at all possible, but I simply don't have time for these to la= nguish for weeks in a committee. >=20 > Isilon has BSD committers (eg: bdrewery) I can leverage if that would be = more expedient but given these are all testing-centric it seems like gettin= g approval from one or more people on freebsd-testing would be best. If you can get self-contained diffs, I can try to get them in for you (but = can't make any promises as my time for FreeBSD varies significantly week to= week...) For simplicity of review, I'd appreciate at least one patch for every subdi= rectory within tools/regression/foo/bar/ with at least one patch to get the= first tests up and running first with prove(1). More patches are better! The move of the fixed tests to the new infrastructure (aka the boilerplate = work) is something I may be able to help with. --_004_16437CC5729B5345AF77F816513376E820BCBE25MX103CL02corpem_ Content-Type: application/octet-stream; name="fifo.diff" Content-Description: fifo.diff Content-Disposition: attachment; filename="fifo.diff"; size=795; creation-date="Wed, 30 Apr 2014 17:02:52 GMT"; modification-date="Wed, 30 Apr 2014 16:50:19 GMT" Content-Transfer-Encoding: base64 SW5kZXg6IGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2ZpZm8vTWFrZWZpbGUKPT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PQotLS0gL2Rldi9udWxsCisrKyBmcmVlYnNkLWhlYWQvdG9vbHMvcmVncmVzc2lvbi9maWZv L01ha2VmaWxlCkBAIC0wLDAgKzEsOCBAQAorIyAkRnJlZUJTRCQKKworU1VCRElSKz1maWZvX2Ny ZWF0ZQorU1VCRElSKz1maWZvX2lvCitTVUJESVIrPWZpZm9fbWlzYworU1VCRElSKz1maWZvX29w ZW4KKworLmluY2x1ZGUgPGJzZC5zdWJkaXIubWs+CkluZGV4OiBmcmVlYnNkLWhlYWQvdG9vbHMv cmVncmVzc2lvbi9maWZvL2ZpZm8udAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSAvZGV2L251bGwKKysrIGZyZWVi c2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2ZpZm8vZmlmby50CkBAIC0wLDAgKzEsMjAgQEAKKyMh L2Jpbi9zaAorIyAkRnJlZUJTRCQKKworbWFrZSA+L2Rldi9udWxsIDI+JjEgfHwgZXhpdCAxCisK K3Rlc3RzPSJmaWZvX2NyZWF0ZSBmaWZvX2lvIGZpZm9fbWlzYyBmaWZvX29wZW4iCisKK249MAor CitlY2hvICIxLi40IgorCitmb3IgYXRlc3QgaW4gJHt0ZXN0c30KK2RvCisJbj1gZXhwciAke259 ICsgMWAKKwlpZiAkYXRlc3QvJGF0ZXN0OyB0aGVuCisJCWVjaG8gIm9rICR7bn0gLSAke2F0ZXN0 fSIKKwllbHNlCisJCWVjaG8gIm5vdCBvayAke259IC0gJHthdGVzdH0iCisJZmkKK2RvbmUK --_004_16437CC5729B5345AF77F816513376E820BCBE25MX103CL02corpem_ Content-Type: application/octet-stream; name="libutil.diff" Content-Description: libutil.diff Content-Disposition: attachment; filename="libutil.diff"; size=1504; creation-date="Wed, 30 Apr 2014 17:02:52 GMT"; modification-date="Wed, 30 Apr 2014 16:50:19 GMT" Content-Transfer-Encoding: base64 SW5kZXg6IGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL2xpYi9saWJ1dGlsL3Rlc3QtaHVt YW5pemVfbnVtYmVyLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gZnJlZWJzZC1oZWFkLm9yaWcvdG9vbHMvcmVn cmVzc2lvbi9saWIvbGlidXRpbC90ZXN0LWh1bWFuaXplX251bWJlci5jCisrKyBmcmVlYnNkLWhl YWQvdG9vbHMvcmVncmVzc2lvbi9saWIvbGlidXRpbC90ZXN0LWh1bWFuaXplX251bWJlci5jCkBA IC01MzcsMTIgKzUzNywxMiBAQCBtYWluKGludCBhcmdjLCBjaGFyICogY29uc3QgYXJndltdKQog CiAJCWlmICh0ZXN0X2FyZ3NbaV0uc2NhbGUgPCAwICYmICEgaW5jbHVkZU5lZ1NjYWxlKSB7CiAJ CQlza2lwcGVkKys7Ci0JCQl0ZXN0c2tpcHBlZChpKTsKKwkJCXRlc3Rza2lwcGVkKGkgKyAxKTsK IAkJCWNvbnRpbnVlOwogCQl9CiAJCWlmICh0ZXN0X2FyZ3NbaV0ubnVtID49IGhhbGZFeGFieXRl ICYmICEgaW5jbHVkZUV4YWJ5dGVUZXN0cykgewogCQkJc2tpcHBlZCsrOwotCQkJdGVzdHNraXBw ZWQoaSk7CisJCQl0ZXN0c2tpcHBlZChpICsgMSk7CiAJCQljb250aW51ZTsKIAkJfQogCkBAIC01 NTgsNyArNTU4LDcgQEAgbWFpbihpbnQgYXJnYywgY2hhciAqIGNvbnN0IGFyZ3ZbXSkKIAkJCQkg ICAgdGVzdF9hcmdzW2ldLnJlcywgdGVzdF9hcmdzW2ldLm51bSwKIAkJCQkgICAgc2NhbGVfc3Ry LCBmbGFnX3N0cik7CiAJCQllbHNlCi0JCQkJcHJpbnRmKCJub3Qgb2sgJWx1ICMgcmV0dXJuICVk ICE9ICVkXG4iLCBpLCByLAorCQkJCXByaW50Zigibm90IG9rICVsdSAjIHJldHVybiAlZCAhPSAl ZFxuIiwgaSArIDEsIHIsCiAJCQkJICAgIHRlc3RfYXJnc1tpXS5yZXR2YWwpOwogCQkJZXJyY250 Kys7CiAJCX0gZWxzZSBpZiAoc3RyY21wKGJ1ZiwgdGVzdF9hcmdzW2ldLnJlcykgIT0gMCkgewpA QCAtNTY3LDcgKzU2Nyw3IEBAIG1haW4oaW50IGFyZ2MsIGNoYXIgKiBjb25zdCBhcmd2W10pCiAJ CQkJICAgIGksIGJ1ZiwgdGVzdF9hcmdzW2ldLnJlcywgdGVzdF9hcmdzW2ldLm51bSwKIAkJCQkg ICAgc2NhbGVfc3RyLCBmbGFnX3N0cik7CiAJCQllbHNlCi0JCQkJcHJpbnRmKCJub3Qgb2sgJWx1 ICMgYnVmIFwiJXNcIiAhPSBcIiVzXCJcbiIsIGksCisJCQkJcHJpbnRmKCJub3Qgb2sgJWx1ICMg YnVmIFwiJXNcIiAhPSBcIiVzXCJcbiIsIGkgKyAxLAogCQkJCSAgICBidWYsIHRlc3RfYXJnc1tp XS5yZXMpOwogCQkJZXJyY250Kys7CiAJCX0gZWxzZSB7CkBAIC01NzYsNyArNTc2LDcgQEAgbWFp bihpbnQgYXJnYywgY2hhciAqIGNvbnN0IGFyZ3ZbXSkKIAkJCQkgICAgaSwgciwgYnVmLCB0ZXN0 X2FyZ3NbaV0ubnVtLCBzY2FsZV9zdHIsCiAJCQkJICAgIGZsYWdfc3RyKTsKIAkJCWVsc2UKLQkJ CQlwcmludGYoIm9rICVsdVxuIiwgaSk7CisJCQkJcHJpbnRmKCJvayAlbHVcbiIsIGkgKyAxKTsK IAkJfQogCQl0ZXN0ZWQrKzsKIAl9Cg== --_004_16437CC5729B5345AF77F816513376E820BCBE25MX103CL02corpem_ Content-Type: application/octet-stream; name="mmap.diff" Content-Description: mmap.diff Content-Disposition: attachment; filename="mmap.diff"; size=448; creation-date="Wed, 30 Apr 2014 17:02:52 GMT"; modification-date="Wed, 30 Apr 2014 16:50:19 GMT" Content-Transfer-Encoding: base64 SW5kZXg6IGZyZWVic2QtaGVhZC90b29scy9yZWdyZXNzaW9uL21tYXAvbW1hcC50Cj09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT0KLS0tIC9kZXYvbnVsbAorKysgZnJlZWJzZC1oZWFkL3Rvb2xzL3JlZ3Jlc3Npb24vbW1hcC9t bWFwLnQKQEAgLTAsMCArMSwxNyBAQAorIyEvYmluL3NoCisKK21ha2UgPi9kZXYvbnVsbCAyPiYx IHx8IGV4aXQgMQorCit0ZXN0bnVtPTAKKworZWNobyAiMS4uOSIKKworLi9tbWFwIDI+JjEgfCB3 aGlsZSByZWFkIGxpbmU7IGRvCisJZWNobyAkbGluZSB8IGdyZXAgLXEgT0sKKwl0ZXN0bnVtPSQo KCR0ZXN0bnVtICsgMSkpCisJaWYgWyAkPyAtZXEgMCBdOyB0aGVuCisJCWVjaG8gIm9rICR0ZXN0 bnVtIgorCWVsc2UKKwkJZWNobyAibm90IG9rICR0ZXN0bnVtIgorCWZpCitkb25lCg== --_004_16437CC5729B5345AF77F816513376E820BCBE25MX103CL02corpem_-- From owner-freebsd-testing@FreeBSD.ORG Wed Apr 30 17:16:11 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 207684D4; Wed, 30 Apr 2014 17:16:11 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE9F81AEE; Wed, 30 Apr 2014 17:16:10 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id ib6so2580262vcb.19 for ; Wed, 30 Apr 2014 10:16:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=aN0iSWboJq87e9CYZ/R5T4GPlvREqyRaQJ0IB6z7RaY=; b=EQBhpXckoW6UBbuUUsULh8QSx3IPGzrg+1ohnEHCPqEMoQHLejZgGceCrgO/PAe4QN 8HbTIN3tZujuWF1XF65P/VM/Q6WJRCVLVhV4aBkdS/s9CPPtUvqyRnjz4biZzn42MW99 IPnwLswo9uEfEaNgyLuQOMRvwdmG+5KjDsPIeUewk5sUTR232oT/3msSZehph/NBiUh+ TusoYhWAbTZ4itDXkkFDUEOgJgYtDMsXiLhYSgEQygOJU3VNUQl/n825ksTeeKSsm7Yk DhuA11dZHlrLJeY1r8jfHfigttE7LRm9Ub4+4xp5DW3jXBdk7SJAwE7VCU8N5bX7coiv 5hrg== MIME-Version: 1.0 X-Received: by 10.58.219.233 with SMTP id pr9mr4811658vec.10.1398878169164; Wed, 30 Apr 2014 10:16:09 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 30 Apr 2014 10:16:09 -0700 (PDT) In-Reply-To: <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> Date: Wed, 30 Apr 2014 10:16:09 -0700 Message-ID: Subject: Re: Please provide process for small, targeted fixes in tools/regression From: Garrett Cooper To: "Peel, Casey" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 17:16:11 -0000 On Fri, Apr 11, 2014 at 6:59 PM, Peel, Casey wrote: > Exellent, thanks Julio. > > Attached are 3 self-contained diffs, one per directory. Descriptions of t= he changes: > * flock - .c changes to fix compiler errors; .t to wrap flock binary for = integrating with prove > * geom_mirror - gmirror won't let you remove the last partition in a mirr= or which all of these testcases do and thus fail (note that I've got these = testcases plus other gmirror testcases already moved over to ATF and going = through internal code reviews now, so maybe we just want to discard this) > * kqueue - kqtest outputs files in TAP-consumable format, this is a .t fi= le to run it through prove > > These were an intial salvo at getting existing tests working for our inte= rnal regression. I'm sure we'll be looking at others as well. If someone ca= n just review the changes, I can get someone internally to commit them if n= ecssary (although these will take more time to review than commit given how= small they are). The test drivers seem ok for TAP, but out of curiosity why are you removing the geom remove testing in the geom_mirror testcases? Does the test not work out of the box on vanilla FreeBSD? Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Apr 30 17:28:04 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA059754; Wed, 30 Apr 2014 17:28:04 +0000 (UTC) Received: from mail-ve0-x230.google.com (mail-ve0-x230.google.com [IPv6:2607:f8b0:400c:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72FC71C4D; Wed, 30 Apr 2014 17:28:04 +0000 (UTC) Received: by mail-ve0-f176.google.com with SMTP id db11so2556160veb.35 for ; Wed, 30 Apr 2014 10:28:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MMuT+uazAjJMSRm1JfD8KQXHo7ButS/cGY8uzOVtOQw=; b=XMOfomqyDTZE4XENZEbiQdQnYo42TTC6SqohMl//p8Nmpu5ZbrMKWZuMa+dRJU1qK6 AAnYSj9Gg1/b6TR/AfEawxesvo/dY32NzHZf8HPvmoqXemzb7MSEfHSw5ZZ2tGJpK7B0 ZD/rr/H52daOceBNwFZyivPdBq/hgxtE9HEhZfehv7Kwa04Y92kME3M4YvI6MsMtEyn/ C/ttPO3ffOaoT3YKXQLRU+C0BpJRuiO1PJjgXA++TvUrQN3KLfuRL/wMpPDqilGdSLTd 6rQ17Z0v5V+bRGA70sU/Gd+7XyYj4caUHZgHxIBvekooO045TNfFboZCnlTeH57t+SmG R8eA== MIME-Version: 1.0 X-Received: by 10.221.59.194 with SMTP id wp2mr749149vcb.59.1398878883585; Wed, 30 Apr 2014 10:28:03 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Wed, 30 Apr 2014 10:28:03 -0700 (PDT) In-Reply-To: <16437CC5729B5345AF77F816513376E820BCBE25@MX103CL02.corp.emc.com> References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> <16437CC5729B5345AF77F816513376E820BCBE25@MX103CL02.corp.emc.com> Date: Wed, 30 Apr 2014 10:28:03 -0700 Message-ID: Subject: Re: Please provide process for small, targeted fixes in tools/regression From: Garrett Cooper To: "Peel, Casey" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 17:28:04 -0000 On Wed, Apr 30, 2014 at 10:03 AM, Peel, Casey wrote: > Julio, here are 3 additional changes in tools/regression to get tests in there to run and pass via prove. > > * libutil.diff - fixes test-humanize_number.c to print 1-based test numbers instead of 0-based test numbers as prove expects. > * mmap.diff - adds .t file to allow running mmap test through prove > * fifo.diff - adds a top-level Makefile to compile the code in the fifo/ subdirectories. Adds .t to run the tests via prove. These changes look ok. The only thing (just a consistency nit) is that I would choose a consistent way of incrementing in loops, i.e. i=$(expr $i + 1) : $(( i += 1 )) i=$(( $i + 1 )) e.g. $ cat increment.sh ; sh increment.sh #!/bin/sh set -x sh -c 'i=$(expr $i + 1); echo $i' sh -c ': $(( i += 1 )); echo $i' sh -c 'i=$(( $i + 1 )); echo $i' + sh -c 'i=$(expr $i + 1); echo $i' 1 + sh -c ': $(( i += 1 )); echo $i' 1 + sh -c 'i=$(( $i + 1 )); echo $i' 1 On the other hand, if it's all going to be converted over to ATF in the near future, it doesn't really matter :). If you (or Peter) commit the work to isilon-atf, then I'll do the work required to convert it (minus the libutil testcases -- these need to be merged into what's in isilon-atf from NetBSD). Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu May 1 22:44:35 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CC459FC4; Thu, 1 May 2014 22:44:35 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd51.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72FCF1651; Thu, 1 May 2014 22:44:34 +0000 (UTC) Received: from maildlpprd53.lss.emc.com (maildlpprd53.lss.emc.com [10.106.48.157]) by mailuogwprd51.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s41MiW9C001395 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 1 May 2014 18:44:32 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com s41MiW9C001395 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1398984272; bh=G3CTpKOoVibsYp7KRCLr2pEs3vA=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-ID:Content-Transfer-Encoding:MIME-Version; b=chbr8WFvtq6KpBgPT8nQHbU0C+fYCLO63a/BuFVM0+/fc1sEqmvLIExBp96YAtH1b 4S+RwRRwaKEpb5kFbhIKPAW6X0TE+XfzhFP3/WI+YMnSaZNZXVP9CVlBynWndJS6RN Ik+MP+g6Lm7mMcjwpO59HI2e5T9i+5HsG0XbpQyM= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com s41MiW9C001395 Received: from mailusrhubprd53.lss.emc.com (mailusrhubprd53.lss.emc.com [10.106.48.18]) by maildlpprd53.lss.emc.com (RSA Interceptor); Thu, 1 May 2014 18:44:20 -0400 Received: from mxhub37.corp.emc.com (mxhub37.corp.emc.com [128.222.70.104]) by mailusrhubprd53.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s41MiKTj019587 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 1 May 2014 18:44:20 -0400 Received: from MXHUB104.corp.emc.com (10.253.58.16) by mxhub37.corp.emc.com (128.222.70.104) with Microsoft SMTP Server (TLS) id 8.3.327.1; Thu, 1 May 2014 18:44:20 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.21]) by MXHUB104.corp.emc.com ([::1]) with mapi id 14.03.0158.001; Thu, 1 May 2014 18:44:19 -0400 From: "Peel, Casey" To: Garrett Cooper Subject: Re: Please provide process for small, targeted fixes in tools/regression Thread-Topic: Please provide process for small, targeted fixes in tools/regression Thread-Index: Ac9VrITJRfNb0i8ZTXWISpxKRMnAiQALXaOAAAXxSeADsenUgAAvFKeA Date: Thu, 1 May 2014 22:44:19 +0000 Message-ID: References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [10.13.52.90] Content-Type: text/plain; charset="us-ascii" Content-ID: <4055668E9E526143B031A14A3B06EEFF@mail.corp.emc.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd53.lss.emc.com Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 22:44:35 -0000 > but out of curiosity why are you > removing the geom remove testing in the geom_mirror testcases? Does > the test not work out of the box on vanilla FreeBSD? To be honest, I haven't tested on a stock FreeBSD system, but on our internal almost-but-not-quite FreeBSD-CURRENT system removing the last partition of a gmirror fails. To remove the last partition you have to destroy the gmirror. It should be easy to confirm this is still true on FreeBSD-CURRENT. That said, I have the gmirror tests entirely moved over to ATF so I'm not sure its worth messing with. > These changes look ok. The only thing (just a consistency nit) is that > I would choose a consistent way of incrementing in loops, i.e. Good point, will do! -- Casey --=20 Performance Test Ninja casey.peel@isilon.com / 206.777.7945 On 4/30/14 10:16 AM, "Garrett Cooper" wrote: >On Fri, Apr 11, 2014 at 6:59 PM, Peel, Casey >wrote: >> Exellent, thanks Julio. >> >> Attached are 3 self-contained diffs, one per directory. Descriptions of >>the changes: >> * flock - .c changes to fix compiler errors; .t to wrap flock binary >>for integrating with prove >> * geom_mirror - gmirror won't let you remove the last partition in a >>mirror which all of these testcases do and thus fail (note that I've got >>these testcases plus other gmirror testcases already moved over to ATF >>and going through internal code reviews now, so maybe we just want to >>discard this) >> * kqueue - kqtest outputs files in TAP-consumable format, this is a .t >>file to run it through prove >> >> These were an intial salvo at getting existing tests working for our >>internal regression. I'm sure we'll be looking at others as well. If >>someone can just review the changes, I can get someone internally to >>commit them if necssary (although these will take more time to review >>than commit given how small they are). > > The test drivers seem ok for TAP, but out of curiosity why are you >removing the geom remove testing in the geom_mirror testcases? Does >the test not work out of the box on vanilla FreeBSD? >Thanks! >-Garrett > From owner-freebsd-testing@FreeBSD.ORG Thu May 1 22:53:05 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F15521DC; Thu, 1 May 2014 22:53:05 +0000 (UTC) Received: from mail-vc0-x22f.google.com (mail-vc0-x22f.google.com [IPv6:2607:f8b0:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B52C1710; Thu, 1 May 2014 22:53:05 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id lh4so4568701vcb.6 for ; Thu, 01 May 2014 15:53:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=i06cpXGdEU27obqPAefE8x7fbpCSzITcIH5ZQRANdpM=; b=ryJntd0fcN+wR3m31RErZaK2OgzzQPLaPjqM/kOQJdNUuWZ3IjuC7ymqjuag9lVxeq PGPA7izvf0jkOKokhXb0NpFmpYojUAtIg7Vx1HjLC8camj6wYCOJXaa8636gJO4EavPl 6x4dAoc3xFw0QF2nrCVgswzVlUAmEFyjNTRE3mzqrnW7YxI9C3s6CbcVILHuK1NGzPJO TPKD7oh4WNy6AWFxS3Y1ubfp27Z6JXjyURzGNJFhSs21P9DrxZTSvQrKSbXzBkt5oNJe e1bt/D+vIJnbJb3JiO2aqnsavU0iNc6z7hqWcTfNDKz7366FT/xKKnQFVmlyDNHZx7ch D/sw== MIME-Version: 1.0 X-Received: by 10.58.116.175 with SMTP id jx15mr4235306veb.9.1398984784610; Thu, 01 May 2014 15:53:04 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Thu, 1 May 2014 15:53:04 -0700 (PDT) In-Reply-To: References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> Date: Thu, 1 May 2014 15:53:04 -0700 Message-ID: Subject: Re: Please provide process for small, targeted fixes in tools/regression From: Garrett Cooper To: "Peel, Casey" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 22:53:06 -0000 On Thu, May 1, 2014 at 3:44 PM, Peel, Casey wrote: >> but out of curiosity why are you >> removing the geom remove testing in the geom_mirror testcases? Does >> the test not work out of the box on vanilla FreeBSD? > > To be honest, I haven't tested on a stock FreeBSD system, but on our > internal almost-but-not-quite FreeBSD-CURRENT system removing the last > partition of a gmirror fails. To remove the last partition you have to > destroy the gmirror. It should be easy to confirm this is still true on > FreeBSD-CURRENT. That said, I have the gmirror tests entirely moved over > to ATF so I'm not sure its worth messing with. Knowing now what I knew then, it's probably one of the modifications that was made to geom_mirror to be more robust by automatically rebuilding the mirrors with Isilon secret sauce. I wonder if turning off the secret sauce (kernel bits, middleware) would allow the test to pass without issue. Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu May 1 23:16:22 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DDC585B6; Thu, 1 May 2014 23:16:21 +0000 (UTC) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd03.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8367018C6; Thu, 1 May 2014 23:16:20 +0000 (UTC) Received: from maildlpprd03.lss.emc.com (maildlpprd03.lss.emc.com [10.253.24.35]) by mailuogwprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s41NGITc026817 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 1 May 2014 19:16:19 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd03.lss.emc.com s41NGITc026817 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1398986179; bh=kSft6nhINmjG0RWtcdGETjoqWoE=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-ID:Content-Transfer-Encoding:MIME-Version; b=orgzJLRlABYszMwDjcRDZUxrWKUEfqk0kEPtdUmCWp7yDBug1X0M8Y2M20bUwj4ML 8lFnDFkZMA8YT4XRy4MbysZkZQFJGC67HbIlvO752wAiqvld4JdMl84vGstqahUXxJ U2toAr36FLaqWjgpS0D0sDkpVuxnXbdqFzGTCXpc= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd03.lss.emc.com s41NGITc026817 Received: from mailusrhubprd02.lss.emc.com (mailusrhubprd02.lss.emc.com [10.253.24.20]) by maildlpprd03.lss.emc.com (RSA Interceptor); Thu, 1 May 2014 19:16:09 -0400 Received: from mxhub10.corp.emc.com (mxhub10.corp.emc.com [10.254.92.105]) by mailusrhubprd02.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s41NG8wV029519 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 1 May 2014 19:16:08 -0400 Received: from MXHUB106.corp.emc.com (10.253.58.23) by mxhub10.corp.emc.com (10.254.92.105) with Microsoft SMTP Server (TLS) id 8.3.327.1; Thu, 1 May 2014 19:16:08 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.21]) by MXHUB106.corp.emc.com ([10.253.58.23]) with mapi id 14.03.0158.001; Thu, 1 May 2014 19:16:08 -0400 From: "Peel, Casey" To: Garrett Cooper Subject: Re: Please provide process for small, targeted fixes in tools/regression Thread-Topic: Please provide process for small, targeted fixes in tools/regression Thread-Index: Ac9VrITJRfNb0i8ZTXWISpxKRMnAiQALXaOAAAXxSeADsenUgAAvFKeAAA76OAD//5EQgA== Date: Thu, 1 May 2014 23:16:07 +0000 Message-ID: References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [10.13.52.90] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd02.lss.emc.com Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 23:16:22 -0000 > Knowing now what I knew then, it's probably one of the modifications > that was made to geom_mirror to be more robust by automatically Unlikely since this was failing well before we brought in our gmirror changes. # mdconfig -a -t malloc -s 10m -u 0 # mdconfig -a -t malloc -s 10m -u 1 # gmirror label mirror /dev/md0GEOM_MIRROR: Device mirror/mirror launched (1/1). # gmirror insert mirror /dev/md1 GEOM_MIRROR: Device mirror: rebuilding provider md1. GEOM_MIRROR: Device mirror: rebuilding provider md1 finished. # gmirror remove mirror /dev/md0GEOM_MIRROR: Device mirror: provider md0 destroyed. # gmirror remove mirror /dev/md1 gmirror: mirror: Can't remove the last ACTIVE component /dev/md1. -- Casey --=20 Performance Test Ninja casey.peel@isilon.com / 206.777.7945 On 5/1/14 3:53 PM, "Garrett Cooper" wrote: >On Thu, May 1, 2014 at 3:44 PM, Peel, Casey wrote: >>> but out of curiosity why are you >>> removing the geom remove testing in the geom_mirror testcases? Does >>> the test not work out of the box on vanilla FreeBSD? >> >> To be honest, I haven't tested on a stock FreeBSD system, but on our >> internal almost-but-not-quite FreeBSD-CURRENT system removing the last >> partition of a gmirror fails. To remove the last partition you have to >> destroy the gmirror. It should be easy to confirm this is still true on >> FreeBSD-CURRENT. That said, I have the gmirror tests entirely moved over >> to ATF so I'm not sure its worth messing with. > >Knowing now what I knew then, it's probably one of the modifications >that was made to geom_mirror to be more robust by automatically >rebuilding the mirrors with Isilon secret sauce. I wonder if turning >off the secret sauce (kernel bits, middleware) would allow the test to >pass without issue. > >Thanks! >-Garrett > From owner-freebsd-testing@FreeBSD.ORG Thu May 1 23:39:51 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33560A97; Thu, 1 May 2014 23:39:51 +0000 (UTC) Received: from mail-vc0-x229.google.com (mail-vc0-x229.google.com [IPv6:2607:f8b0:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C00711A6B; Thu, 1 May 2014 23:39:50 +0000 (UTC) Received: by mail-vc0-f169.google.com with SMTP id im17so4601794vcb.14 for ; Thu, 01 May 2014 16:39:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=/hnR7B1tVVg4Jv3Yys4Z9U2JTK7emIEzayu4w6sR87w=; b=JYcIPHZWZ1hB5NF8OYPy4KI3RLvF9/L56SOj0ZrJ5sS34O3b2qKGzOnDQ3Yvs1NDA0 bMJD/OQpc6pTIcswYFFRWcmLkp4GFaT2UN6GHOgwrfP6tO9toWvRw2RlkjvGtPK2c2/u b/ihcX6jW1z2lmOP9jJUGrGrHV12RlSPYcHD118WWJKitztmZMV2Fcm59gUt/An5bT97 AnxhWDeZ2XIcpZYYdWzn5CgA7hHIZZh9VmVfjDcEc82ezNS1mZUVglUledv4i6Ral4Ct h5MywnXX2iSBq3QbXZr8572lB9K2Ns2iOZqOz5TW3HcgST+qE9O493o79T9qu7lCbwew OyKg== MIME-Version: 1.0 X-Received: by 10.52.6.162 with SMTP id c2mr8722965vda.6.1398987589951; Thu, 01 May 2014 16:39:49 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Thu, 1 May 2014 16:39:49 -0700 (PDT) In-Reply-To: References: <16437CC5729B5345AF77F816513376E820BAF854@MX103CL02.corp.emc.com> <5F1D5D49-5F39-4EAC-89D5-E4D10FB3B01E@freebsd.org> <16437CC5729B5345AF77F816513376E820BAFE8E@MX103CL02.corp.emc.com> Date: Thu, 1 May 2014 16:39:49 -0700 Message-ID: Subject: Re: Please provide process for small, targeted fixes in tools/regression From: Garrett Cooper To: "Peel, Casey" Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "bdrewery@FreeBSD.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2014 23:39:51 -0000 On Thu, May 1, 2014 at 4:16 PM, Peel, Casey wrote: >> Knowing now what I knew then, it's probably one of the modifications >> that was made to geom_mirror to be more robust by automatically > > Unlikely since this was failing well before we brought in our gmirror > changes. > > # mdconfig -a -t malloc -s 10m -u 0 > # mdconfig -a -t malloc -s 10m -u 1 > > # gmirror label mirror /dev/md0GEOM_MIRROR: Device mirror/mirror launched > (1/1). > # gmirror insert mirror /dev/md1 > GEOM_MIRROR: Device mirror: rebuilding provider md1. > GEOM_MIRROR: Device mirror: rebuilding provider md1 finished. > > # gmirror remove mirror /dev/md0GEOM_MIRROR: Device mirror: provider md0 > destroyed. > # gmirror remove mirror /dev/md1 > gmirror: mirror: Can't remove the last ACTIVE component /dev/md1. Confirmed on relatively vanilla CURRENT. Cheers! -Garrett % gmirror remove mirror /dev/md1 gmirror: mirror: Can't remove the last ACTIVE component /dev/md1. % uname -a FreeBSD fbsd-vm.zonarsystems.net 11.0-CURRENT FreeBSD 11.0-CURRENT #5 aa42836(atf): Tue Apr 29 13:47:16 PDT 2014 root@fbsd-vm.zonarsystems.net:/usr/obj/usr/src/sys/GENERIC i386 From owner-freebsd-testing@FreeBSD.ORG Fri May 9 21:26:48 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E2B10656 for ; Fri, 9 May 2014 21:26:47 +0000 (UTC) Received: from mailuogwhop.emc.com (mailuogwhop.emc.com [168.159.213.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailuogwprd03.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8ED6A838 for ; Fri, 9 May 2014 21:26:46 +0000 (UTC) Received: from maildlpprd06.lss.emc.com (maildlpprd06.lss.emc.com [10.253.24.38]) by mailuogwprd03.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s49LQj8h006533 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 9 May 2014 17:26:45 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd03.lss.emc.com s49LQj8h006533 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1399670805; bh=2IOQTZZiaFK+NrvqiJETp/qHs/w=; h=From:To:Subject:Date:Message-ID:Content-Type:MIME-Version; b=KzO7fo1rGYNr43dE0YFJgcfT8qObhFKpwD72oNnKHP+1qZsv6tOdjeYy9IvbOJrGa +EdqZS8bPgxdOSAc5Iq7J7hY+BXY7GUNT5ZuJ/if+/4EpYu8pLt79pV00Zv9jbs9KQ 2owMamSl5ler3ijsp+Mbtq8aBPbAHAnMTweEujYw= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd03.lss.emc.com s49LQj8h006533 Received: from mailusrhubprd51.lss.emc.com (mailusrhubprd51.lss.emc.com [10.106.48.24]) by maildlpprd06.lss.emc.com (RSA Interceptor) for ; Fri, 9 May 2014 14:26:30 -0700 Received: from mxhub26.corp.emc.com (mxhub26.corp.emc.com [10.254.110.182]) by mailusrhubprd51.lss.emc.com (Sentrion-MTA-4.3.0/Sentrion-MTA-4.3.0) with ESMTP id s49LQUcn025173 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Fri, 9 May 2014 17:26:30 -0400 Received: from MXHUB108.corp.emc.com (10.253.58.24) by mxhub26.corp.emc.com (10.254.110.182) with Microsoft SMTP Server (TLS) id 8.3.327.1; Fri, 9 May 2014 17:26:29 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.81]) by MXHUB108.corp.emc.com ([10.253.58.24]) with mapi id 14.03.0158.001; Fri, 9 May 2014 17:22:12 -0400 From: "Walker, Myron" To: "freebsd-testing@freebsd.org" Subject: Adding Python Wrapper for ATF Thread-Topic: Adding Python Wrapper for ATF Thread-Index: Ac9rzNPnQAl3MGTjRV+kuWlhlaRiVw== Date: Fri, 9 May 2014 21:26:28 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.7.181.135] MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd51.lss.emc.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 21:26:48 -0000 I have created a python wrapper atfpy-sh that will act as an atf program an= d wrap a python module that contains unittest test cases. I have included = a zip of the wrapper script and the man page for it. I believe it will be = very efficient to create python unittest test cases and debug them in an fu= ll IDE environment like eclipse and PyDev on any linux or unix desktop syst= em against shared libraries that are common between operating systems. The= n just add them to the FreeBSD source tree as ATF wrapped tests once they a= re fully operational tests. See the attached zip for the atfpy-sh script, manpage, and an example unitt= est python module. I was putting it under contrib/atf/atfpy-sh and adding = a Makefile under usr.bin/atf/atfpy-sh to copy atfpy-sh to /usr/bin. Then j= ust stick the test file in a folder under /user/tests, modify the Kyuafile = for the folder and it should work. I think this would make a great addition to ATF. I would really like to be= able to use it. Thanks, Myron From owner-freebsd-testing@FreeBSD.ORG Fri May 9 21:44:48 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6738DC0 for ; Fri, 9 May 2014 21:44:48 +0000 (UTC) Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60B389B6 for ; Fri, 9 May 2014 21:44:48 +0000 (UTC) Received: by mail-we0-f181.google.com with SMTP id w61so4436424wes.40 for ; Fri, 09 May 2014 14:44:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=TZA3sfmwVW0z+GtrU/t4wylJTKTJGv2VMRraTfyyHPk=; b=gfVv9WHRYB9Kc/eh6+pf8VXxEeA3td16GKDAB8Bx0nZQbcSN/t3ZEoM6kgtyRhsAuW HaFXd9U5R+k+fmcxd6+Ul4p3Ftdh9vZG7JcRPHdARIcm9Gvzx4CQTt6QoWCsEiFl9htO TGuqQzXppneihB0+VRwasxK211nhTfK+P1VPWjBfZFP+nmHMcwVH/hCnFM3AxY7V8e32 zQOd0DYi3GYrCINTniYrOoobWUFbjqFSnJHIhgurSeCRnsck+RHpBbiuzsqlPNkkK3RS nO6/niODrkwfFd/2Mgfzqx5e34qQLzHQcJMqqfZWeTHUPAMIsX3Iztj//gBYUc9e/Dlg PuFQ== MIME-Version: 1.0 X-Received: by 10.194.205.35 with SMTP id ld3mr193161wjc.82.1399671886315; Fri, 09 May 2014 14:44:46 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.130 with HTTP; Fri, 9 May 2014 14:44:46 -0700 (PDT) In-Reply-To: References: Date: Fri, 9 May 2014 15:44:46 -0600 X-Google-Sender-Auth: pWjnSZCfIvebdS3S-aPgREewkC0 Message-ID: Subject: Re: Adding Python Wrapper for ATF From: Alan Somers To: "Walker, Myron" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 21:44:48 -0000 On Fri, May 9, 2014 at 3:26 PM, Walker, Myron wro= te: > I have created a python wrapper atfpy-sh that will act as an atf program = and wrap a python module that contains unittest test cases. I have include= d a zip of the wrapper script and the man page for it. I believe it will b= e very efficient to create python unittest test cases and debug them in an = full IDE environment like eclipse and PyDev on any linux or unix desktop sy= stem against shared libraries that are common between operating systems. T= hen just add them to the FreeBSD source tree as ATF wrapped tests once they= are fully operational tests. > > See the attached zip for the atfpy-sh script, manpage, and an example uni= ttest python module. I was putting it under contrib/atf/atfpy-sh and addin= g a Makefile under usr.bin/atf/atfpy-sh to copy atfpy-sh to /usr/bin. Then= just stick the test file in a folder under /user/tests, modify the Kyuafil= e for the folder and it should work. > > I think this would make a great addition to ATF. I would really like to = be able to use it. > > Thanks, > Myron Firstly, it looks like the list server stripped out your attachment. Could you try posting it online somewhere? Secondly, this sounds like it would affect more then just FreeBSD. I suggest moving the thread to kyua-discuss@googlegroups.com. See also a previous discussion about a python API at https://groups.google.com/forum/#!topic/kyua-discuss/ji8fWB9chKg . I never finished that API. Partly, it's because it was really just a toy for me; I have no serious applications for Python and Kyua. But it also reflected a philosophical disagreement between myself and Julio, which I never explicitly raised. Now seems like a good time: What is the proper amount of test case isolation for Kyua tests written in Python? Julio would say that the Python API should offer exactly the same isolation features as are found in the C and sh APIs. That would be most convenient if one were using Python as an alternative to sh, to write system-level functional tests. But a career Pythoner would probably disagree. If your intention is to use Kyua to execute the unit tests for some existing Python program, then you would probably want exactly the same isolation features as are offered by Python's standard unittest module. For example, if you use a Python program from ports, then you might want to install its unit tests into /usr/tests and run them along with FreeBSD's own tests. In such a case, Kyua's extra isolation features would be superfluous because the tests were written without them in mind. The isolation features might even cause some test cases to fail, if they use the class and module level fixtures introduced in Python 2.7. Even if your tests would work fine with all of the isolation features, then would run much slower without. Kyua creates a separate process for each test case and each test case teardown. That works fine in sh and C, but incurs considerable overhead in languages like Python, Ruby, and Java. If we can agree on an answer to this question, then I think we'll be able to add APIs for many more languages, and greatly increase Kyua's usefulness. My personal preference is for APIs that predate Kyua, like Python's unittest, Ruby's unit/test, or C++'s googletest to execute the tests as similarly as possible whether or not Kyua is involved. That would mean no extra isolation. People who do want the isolation, for example to test external binaries using Python as glue logic, can always use the ATF sh API and call a python program. But there's certainly room for disagreement. I could almost go the other way for googletest, since its startup time is so low. -Alan > > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org= " From owner-freebsd-testing@FreeBSD.ORG Fri May 9 21:45:07 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7975DDD8; Fri, 9 May 2014 21:45:07 +0000 (UTC) Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C103D9BA; Fri, 9 May 2014 21:45:06 +0000 (UTC) Received: by mail-we0-f181.google.com with SMTP id w61so4570613wes.12 for ; Fri, 09 May 2014 14:45:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=FglI1ZPJhMbjh9yvUYeS7rsGAvq7BFgbzmoTqWAUzwI=; b=JW4ib6pXWNPweUhaOBnwxhx6o1CHx9fALJuDd9E6vTAMDq+f7nk1Qva10/U8eiEhDy jFRMdvW9uogM1c5UTFJaGbEKfp9Pn3TUNGqjdt/KX7eiqV2osOroiTDDkdFRhTHfzCpS +HbGWerno2qgv14mh4DU/sYYLos4FKYJHV6cl69bpWeYWTzBx1mBNy1WjQq8hx0UDehr doQIGVptd8myVxGVyleHw8qIp5iwqOKB4xUyVWViJhppolS9Dhh9tNWbl39nRUjsTddE UTwgYRZ8HCCnGRh5VV0U79fSB6kvwTjixTFmvR7k8Ccut1tTny049EEupTwSpaE+Cm+t hs8A== MIME-Version: 1.0 X-Received: by 10.180.85.10 with SMTP id d10mr5219533wiz.0.1399671905139; Fri, 09 May 2014 14:45:05 -0700 (PDT) Received: by 10.194.89.226 with HTTP; Fri, 9 May 2014 14:45:05 -0700 (PDT) In-Reply-To: References: Date: Fri, 9 May 2014 14:45:05 -0700 Message-ID: Subject: Re: Adding Python Wrapper for ATF From: Garrett Cooper To: "Walker, Myron" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 21:45:07 -0000 On Fri, May 9, 2014 at 2:26 PM, Walker, Myron wro= te: > I have created a python wrapper atfpy-sh that will act as an atf program = and wrap a python module that contains unittest test cases. I have include= d a zip of the wrapper script and the man page for it. I believe it will b= e very efficient to create python unittest test cases and debug them in an = full IDE environment like eclipse and PyDev on any linux or unix desktop sy= stem against shared libraries that are common between operating systems. T= hen just add them to the FreeBSD source tree as ATF wrapped tests once they= are fully operational tests. > > See the attached zip for the atfpy-sh script, manpage, and an example uni= ttest python module. I was putting it under contrib/atf/atfpy-sh and addin= g a Makefile under usr.bin/atf/atfpy-sh to copy atfpy-sh to /usr/bin. Then= just stick the test file in a folder under /user/tests, modify the Kyuafil= e for the folder and it should work. > > I think this would make a great addition to ATF. I would really like to = be able to use it. Hi Myron, I think the attachment was stripped from the email. You might want to contact Alan Somers about his unittest backend for Kyua [1] and when it might be included in the devel/kyua-testers port; Julio manages the upstream source [2]. HTH! -Garrett 1. http://lists.freebsd.org/pipermail/freebsd-testing/2013-November/000107.= html 2. https://github.com/jmmv/kyua/ From owner-freebsd-testing@FreeBSD.ORG Sun May 11 15:32:07 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 213F3E17; Sun, 11 May 2014 15:32:07 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D27432C81; Sun, 11 May 2014 15:32:06 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id DFE4CB808F; Sun, 11 May 2014 17:32:03 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id C107728497; Sun, 11 May 2014 17:32:03 +0200 (CEST) Date: Sun, 11 May 2014 17:32:03 +0200 From: Jilles Tjoelker To: Garrett Cooper Subject: Re: [PATCH v2] convert /bin/sh tests over to ATF Message-ID: <20140511153203.GA81002@stack.nl> References: <20140124162759.GB90996@stack.nl> <0A68C9FE-6A3A-4DCF-92A9-A506D7581D69@gmail.com> <96373EF8-77EA-4AA6-8422-63881C32E455@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 May 2014 15:32:07 -0000 On Mon, Apr 21, 2014 at 05:21:22PM -0700, Garrett Cooper wrote: > On Sun, Mar 9, 2014 at 1:42 PM, Julio Merino wrote: > ... > > The tests need to populate /usr/tests/bin/sh/ with the > > functional_test.sh driver and all its supporting data files. As I > > understand it, your driver code assumes that the data files are > > external (i.e. not bundled into the script itself), so those files > > must be available from the same location as where the driver lives. > > Hence you still need to use FILES to ensure the data files get > > installed into /usr/tests/. > > ... but maybe I'm missing something from your patch; haven't reviewed > > it again to double-check this. > I've reduced the diff quite a bit. The end result of it can be > found here: https://github.com/yaneurabeya/freebsd/pull/2 (and it will > evolve over time if needed because github is additive in pull request > reviews). FWIW, it might be better to actually review it on github > instead of using this list as patches attached to any mailing list can > become stale over time. > You can get the diff like so: > https://github.com/yaneurabeya/freebsd/pull/2.patch This is OK with me as long as important use cases for me are not broken: 1. Allow running new tests on a potentially broken new sh without installing anything. This used to be possible using 'make regress' in src/bin/sh/ but this was removed. I now use SH=$PWD/obj/sh sh tests/legacy_test.sh (I have an obj/ subdirectory to make the built objects more easily accessible). This should also work on stable/9 although it is acceptable to need a few local hacks for that (as is already the case for MK_TESTS and src.opts.mk). On head, it should work without local hacks. 2. Allow installing using a read-only /usr/obj (mounted over NFS). This means that everything needs to be built during buildworld. Installworld should not build anything. (I currently have local hacks: Kyuafile.auto is not built so I added all: Kyuafile.auto to some tests/Makefiles; I guess this should be in .) -- Jilles Tjoelker From owner-freebsd-testing@FreeBSD.ORG Mon May 12 04:20:19 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82696A53; Mon, 12 May 2014 04:20:19 +0000 (UTC) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A0308251A; Mon, 12 May 2014 04:20:18 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id p9so6930567lbv.7 for ; Sun, 11 May 2014 21:20:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=T9oKazMWWpYDreCsPcaMIDo3/twEbRc0/2/XDJ+Q6Zc=; b=mAzufY1vsJpU27rpcrfsmRMZ9us4pvkjJJQo2j859MNZRwpOpbL+rZvCY99p7tuRWX BHdgatlS9wYYCxk+V5YTva77/PcFTm5evZYYst4t3Eu9H64/rXB5Q2WVHHsnAQX5uDyo zXSIgp/qZqqYtsTeYs7E3fEBQqJekpjeYBoS0ufSU58OjuK9woYqBY5Rk1FGjxFk8Kq9 6v8IRasbi0sDQmWwOHCb9orkVXvSZnpH7ZjnyqICFeYF31dylqIQE+BebFZ2FYdziTby m7ZBlo4Q0R1bwYFO2nnabp9va8AmnT+6zZs/3vJ8F9GVlwIJyhR9r3MtewLcbZHhrw8c yeBw== MIME-Version: 1.0 X-Received: by 10.112.218.99 with SMTP id pf3mr22744707lbc.5.1399868416495; Sun, 11 May 2014 21:20:16 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Sun, 11 May 2014 21:20:16 -0700 (PDT) Received: by 10.112.67.73 with HTTP; Sun, 11 May 2014 21:20:16 -0700 (PDT) In-Reply-To: References: <20140225033526.GN85115@kiwi> Date: Sun, 11 May 2014 21:20:16 -0700 X-Google-Sender-Auth: Z4EOTDgBqq_Ll3A3iNh1L3WIwm4 Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-testing@freebsd.org, kyua-discuss@googlegroups.com, jenkins-admin@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 04:20:19 -0000 Hi, Have you made progress in adding the ability to output Kyua results in JUnit XML format? At the end of BSDCan I want to be able to demo kicking off Kyua tests from Jenkins, and have the results display in the Jenkins UI for tests. Thanks. -- Craig From owner-freebsd-testing@FreeBSD.ORG Mon May 12 21:06:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0FB5F87 for ; Mon, 12 May 2014 21:06:38 +0000 (UTC) Received: from mail-vc0-x229.google.com (mail-vc0-x229.google.com [IPv6:2607:f8b0:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 91DF32097 for ; Mon, 12 May 2014 21:06:38 +0000 (UTC) Received: by mail-vc0-f169.google.com with SMTP id ij19so9851804vcb.28 for ; Mon, 12 May 2014 14:06:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6OXzWyYhi/rnQXbtZODu6v9qqB1KQtijVdypFGrMm6k=; b=GonhC/XPLOE5ylIDZ+yFxBwc4JjdL7pdgEQIBzyOXO/zUfK5oXBA+9/ekgZnvr7Eze w2EnykDNWeYFXC9scrmZXTlq++R7eaPdrO/Np2cvMUDIyrCFuhLfjEtHITsAQ2jnXHfC pByN9pa0ZWY45oDSBNsBPhFrlJzhfjMicem0ywpp9FONC5QgjAdaKjm5KZllS0XuVJWi Mxfm+XV6WJiNa4wEWEcT7x5rKmrbEAEH02R59x0xiYFgfiI712mVjFfViwQ31p/EUzEI 3jI5T2ij1clr4GcIsibzOW6SgQC4KV3j9hlzfMETOStZq4vFhV9JxiHuD3R05CEQDzVv vqyQ== MIME-Version: 1.0 X-Received: by 10.220.5.10 with SMTP id 10mr2693706vct.40.1399928797719; Mon, 12 May 2014 14:06:37 -0700 (PDT) Received: by 10.221.67.136 with HTTP; Mon, 12 May 2014 14:06:37 -0700 (PDT) In-Reply-To: References: Date: Mon, 12 May 2014 14:06:37 -0700 Message-ID: Subject: Re: Failures on head From: Garrett Cooper To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 21:06:38 -0000 On Fri, Apr 25, 2014 at 2:03 PM, Garrett Cooper wrote: > Hi, > While looking at > http://kyua3.nyi.freebsd.org/head/data/0-LATEST/results/index.html , I > noticed that a handful of testcases are failing, in particular: > > Broken test cases > > usr.bin/sed/inplace_race_test:main > > Failed test cases > > bin/pkill/pgrep-j_test:main > bin/pkill/pkill-j_test:main > lib/atf/libatf-c/macros_test:use > usr.bin/sed/multi_test:main > usr.bin/yacc/legacy_test:main The yacc failures are because the tests in /usr/src/tools/regression are super stale. The tests should be picked up from contrib/byacc/test . I'll work on that. Thanks, -Garrett From owner-freebsd-testing@FreeBSD.ORG Sat May 17 13:26:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A1B61DF; Sat, 17 May 2014 13:26:18 +0000 (UTC) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 21198215A; Sat, 17 May 2014 13:26:18 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id D6BC5358C62; Sat, 17 May 2014 15:26:14 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id C2C5D28497; Sat, 17 May 2014 15:26:14 +0200 (CEST) Date: Sat, 17 May 2014 15:26:14 +0200 From: Jilles Tjoelker To: freebsd-testing@freebsd.org Subject: build Kyuafile.auto during buildworld, not installworld Message-ID: <20140517132614.GA85230@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 13:26:18 -0000 For some time, I have needed local hacks to make installworld work with a read-only /usr/obj (over NFS). It is also philosophically wrong to build things during installworld. I found two bugs that cause FILES not to be built during buildworld: * bsd.files.mk does not create its buildfiles target when a buildfiles target already exists from bsd.subdir.mk. * bsd.progs.mk does not include bsd.files.mk if an install target already exists (otherwise it does). The below patch appears to fix both problems in a somewhat ugly way. Index: share/mk/bsd.files.mk =================================================================== --- share/mk/bsd.files.mk (revision 266086) +++ share/mk/bsd.files.mk (working copy) @@ -6,11 +6,10 @@ FILESGROUPS?= FILES -.if !target(buildfiles) +buildfiles: buildfiles0 .for group in ${FILESGROUPS} -buildfiles: ${${group}} +buildfiles0: ${${group}} .endfor -.endif all: buildfiles Index: share/mk/bsd.test.mk =================================================================== --- share/mk/bsd.test.mk (revision 266086) +++ share/mk/bsd.test.mk (working copy) @@ -82,7 +82,9 @@ .if !empty(PROGS) || !empty(PROGS_CXX) || !empty(SCRIPTS) .include -.elif !empty(FILES) +.endif +# may or may not include . +.if !empty(FILES) && !target(buildfiles0) .include .endif -- Jilles Tjoelker From owner-freebsd-testing@FreeBSD.ORG Sun May 18 03:21:40 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 929F0C3B; Sun, 18 May 2014 03:21:40 +0000 (UTC) Received: from mail-la0-x22c.google.com (mail-la0-x22c.google.com [IPv6:2a00:1450:4010:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E39C6225F; Sun, 18 May 2014 03:21:39 +0000 (UTC) Received: by mail-la0-f44.google.com with SMTP id hr17so3098794lab.17 for ; Sat, 17 May 2014 20:21:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=mxJMh9g4leS5vVo8cXFMa0SBAn2zwte7rsdviQhfe9A=; b=ZzcRaR1dW7w0fnbuFKZAt4V7fAYhJpoXrLbGuc2aTkeAaa3eXsdSrmjPSCEL1mJ712 gQWw+HD7SEwQKanNhJAZEWdU/UrftITGsj4/vjX2QD1AL3NhOH9gsrVebYQetZG/ZiZU auvV9OEYldx5MF6JUMc8df7n9xustEuslcTJRCxy2C9TNwz288IUog37tsSQfsms3YvC EKECR3jw0pPdbpds722vsEMbBip54WvKOajv0FqKV1HHZ0FKoc2j7+vQ/TVelIOVtdXi v8+aCjBvvIfh8FoIULj8glzGK+5ptHlQmqv3DHAQnovPsAn5dYPdSrAfQl7TJDzknY1N J+Jw== MIME-Version: 1.0 X-Received: by 10.112.34.43 with SMTP id w11mr18232165lbi.31.1400383297720; Sat, 17 May 2014 20:21:37 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Sat, 17 May 2014 20:21:37 -0700 (PDT) Received: by 10.112.67.73 with HTTP; Sat, 17 May 2014 20:21:37 -0700 (PDT) In-Reply-To: References: Date: Sat, 17 May 2014 20:21:37 -0700 X-Google-Sender-Auth: BMRZ-yhP4gT-ru1bkIfUwwV1ucU Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: freebsd-testing@freebsd.org, jenkins-admin@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 03:21:40 -0000 Hi, At BSDCan 2014, I sat with Julio Merino and helped him get set up with Jenkins. Julio then did some hacking on Kyua so that it could generate JUnit XML output. Julio then imported this JUnit XML test output in Jenkins. If you look at slides 26-27 of Julio's BSDCan presentation: http://goo.gl/1SCfgv you can see the Kyua test results appear in the native Jenkins test result UI. This is very promising work. For me, proving that Jenkins and Kyua could be integrated was worth the entire trip of going to BSDCan. Thanks Julio! It was great to meet you and see you hack! This is a huge step forward for FreeBSD testing. -- Craig From owner-freebsd-testing@FreeBSD.ORG Sat May 24 20:38:30 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8434137 for ; Sat, 24 May 2014 20:38:30 +0000 (UTC) Received: from mail-qc0-f179.google.com (mail-qc0-f179.google.com [209.85.216.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6904E2B79 for ; Sat, 24 May 2014 20:38:30 +0000 (UTC) Received: by mail-qc0-f179.google.com with SMTP id r5so1712713qcx.10 for ; Sat, 24 May 2014 13:38:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:content-type; bh=UCxrOOGnuELlwzo0FM+2Dc11kLIsqgL5O8gA2CKn7L0=; b=lhGSm9Pnvu5xigPz3ngWqMx27hsNDIyLmqLGdI1tW3rlPHaJU2Hho4pSv5IvJgmOQi g/o/RvjjTQrE3d2FspAdUt1v6TeIyPF0BT2tbrKDyumL9pjIWIRfLhlGJ7LMsznv8y6K EcrzoEYWC5eKWJe1lucRwCgEQy6xhF3jdwGuSaEMqkrAWA5l1AMTHdLr+Xq4ZvbcJxgM KctwrMOL2Mn58s6E4i6Nz6cZH10SSYQCVv5NXHtSnvqEPYYTavGoyRh6jAfGFvNy3y1e YhTn3TxXxV/HGKKnIbXoIfCguya4EQb3skYFxa+t8mPq7JQJC2lpSkcQTPZSSQBs2N3r 8TcA== X-Gm-Message-State: ALoCoQlRj3bgWwgBnyH5d4GUhvSl6EV+dHY/l1zT3vM9k9nPHRuYfEulpiFtCVZwnw/Gi2hgGFMG X-Received: by 10.140.81.146 with SMTP id f18mr18543433qgd.47.1400963909333; Sat, 24 May 2014 13:38:29 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Sat, 24 May 2014 13:38:09 -0700 (PDT) X-Originating-IP: [184.153.117.174] In-Reply-To: References: From: Julio Merino Date: Sat, 24 May 2014 16:38:09 -0400 X-Google-Sender-Auth: sYsL_gFaWa9yyE2ELS2fRgw2j4c Message-ID: Subject: Re: Make atf libraries private To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2014 20:38:30 -0000 => Moving to freebsd-testing. On Thu, May 15, 2014 at 11:16 AM, Julio Merino wrote: > Hello, > > I'd like to make the libatf-c and libatf-c++ private, both in CURRENT _and_ in stable/10. > > These libraries are not built by default yet. Only when WITH_TESTS=yes is defined the libraries are built and installed. > > Once installed, nothing outside of the base system should use them with one exception. The exception are a bunch of atf-related ports (lutok, kyua*) that can use these libraries if and only if the TESTS ports option is enabled. > > So... I don't think moving the library impacts binary compatibility in any significant way. > > Plan: bring atf port up to date, move libraries to the private directory, make tests link to the libraries in the new location and add the old libraries to the obsolete list. > > I will move the libraries sometime next week unless there are objections to be discussed. Or submit earlier if somebody confirms this is OK. Ports have been changed to use devel/atf instead of the library in base so we should now be safe to do this. I have put a patch under https://phabric.freebsd.org/D49 for review and intend to submit it either tomorrow or Monday. If anyone wants to review this, please be my guest; don't forget to add yourself as a reviewer in Phabricator if you do! Instructions in https://wiki.freebsd.org/CodeReview Thanks From owner-freebsd-testing@FreeBSD.ORG Wed May 28 21:20:45 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E29FB2FE; Wed, 28 May 2014 21:20:45 +0000 (UTC) Received: from mail-lb0-x231.google.com (mail-lb0-x231.google.com [IPv6:2a00:1450:4010:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 157A92691; Wed, 28 May 2014 21:20:44 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id s7so6066800lbd.22 for ; Wed, 28 May 2014 14:20:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=81MWKyHhx6QJ+BUNEjrMsCkW/2c3ZklnCsDLCc9JehA=; b=QfD2Qtp9i0gQmoFZmVILDZgxXgiVFblKESdasN9lQYdmEC8B5bBe2XW4dqWClE4ioy fsEQ4jVUtITpRysuCd/NFXC94pERUPnOTuUdgadPUAlG2RsK5IFoyO5JCbUPEMSPKgfQ kqIis9wPf11cXOCTMZq7BUMAqGLKITf/vhBd4FP1SApZUoBt3x0REZ/OwU94d3AX3NgS 7mG9kW/WsivABoEMfBIPGKfogX0q89AJI5avpZtdIXh+Ywj3XbFKd0A45TRwmsmIaVKY w7UUJe5ARcuZNMyhtuEbNCFb2UZWZgNgRKOrKwZT5z8VYMHxSYcPTXVJ08naOvMIoZdA 9K4g== MIME-Version: 1.0 X-Received: by 10.112.218.74 with SMTP id pe10mr2196927lbc.3.1401312042870; Wed, 28 May 2014 14:20:42 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 28 May 2014 14:20:42 -0700 (PDT) Date: Wed, 28 May 2014 14:20:42 -0700 X-Google-Sender-Auth: TCuvrTbK_3nddur1sB7726L9ogo Message-ID: Subject: libvirt + Jenkins proof of concept demo? From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 21:20:46 -0000 Roman, As part of ongoing continuous integration and testing efforts, we'd like to use your libvirt + bhyve work. I don't have time to work on it though, because I am busy with a few things. Do you think you have time in the next month or so to: (1) set up a simple Jenkins setup on your own machine somewhere (2) make sure that libvirt with all the necessary bhyve stuff is installed on that machine (3) under Jenkins, install the libvirt slaves plugin: https://wiki.jenkins-ci.org/display/JENKINS/Libvirt+Slaves+Plugin (4) set up a simple Jenkins job which starts a bhyve instance via libvirt, and runs some commands (5) If steps (1)-(4) work, blog about it. This would be seriously helpful if you could do it. Do you think you (or someone in the community) can take this on? You can e-mail here for help, or join the #freebsd-ci channel on EFNet IRC (I don't frequent IRC, because I'm busy). Thanks!!!!!!!! -- Craig From owner-freebsd-testing@FreeBSD.ORG Fri May 30 22:03:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8661EEFF for ; Fri, 30 May 2014 22:03:38 +0000 (UTC) Received: from mail-qg0-f43.google.com (mail-qg0-f43.google.com [209.85.192.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44369252E for ; Fri, 30 May 2014 22:03:37 +0000 (UTC) Received: by mail-qg0-f43.google.com with SMTP id 63so7118723qgz.16 for ; Fri, 30 May 2014 15:03:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=gXZ27SRstshbA5ivl6v+3Tc9f2QYXqkFxQTiSG1zuPE=; b=IejpiHKd9VC2XSRFSjDnQkDDy4UEqDy89plIa/AWjt848kW7drXOlP52dbgDG0RH8e 8jnfzwdxT+hp/e1idAqMfFxTIs8UtiZeltXFVlQ+0Px9xfcRahYjodbci7qgZDU/IKOJ gOWpBDh/VYeGVZPjx99OwolEi/wiwhsFws+c77QVTyqtwRNnvKcAeBSTePAT5EGtxZCu 635gqOZyli+g7Ms1MnTzeSRABQNttUoIex230DwEX9YpKC4VQKUuv7L6NOdDyTmpI16v eUlDtu96GMZtnfBIkmYDYZAJGg9uHzHBB49Ljl2NglH94TN92bmeKa4o33/J9uIteP7F 13rw== X-Gm-Message-State: ALoCoQmfUuzmRLXDPdmKxFK1MsI+XFgbx084cxltbhKdVLOrplT5T7bRyOkglEpeedXkPMoe77hV X-Received: by 10.224.167.70 with SMTP id p6mr25053360qay.48.1401487416873; Fri, 30 May 2014 15:03:36 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Fri, 30 May 2014 15:03:16 -0700 (PDT) X-Originating-IP: [2620:0:1003:1007:6c82:fc01:5943:b916] In-Reply-To: References: From: Julio Merino Date: Fri, 30 May 2014 18:03:16 -0400 X-Google-Sender-Auth: d7dW4jbphNWudwzres31tYVZa-4 Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 22:03:38 -0000 On Sat, May 17, 2014 at 11:21 PM, Craig Rodrigues wrote: > Hi, > > At BSDCan 2014, I sat with Julio Merino and helped him get set up with > Jenkins. Julio then did some hacking on Kyua so that it could generate > JUnit XML output. Julio then imported this JUnit XML test output in > Jenkins. If you look at slides 26-27 of Julio's BSDCan presentation: > > http://goo.gl/1SCfgv > > you can see the Kyua test results appear in the native Jenkins test result > UI. This is very promising work. For me, proving that Jenkins and Kyua > could be integrated was worth the entire trip of going to BSDCan. Thanks > Julio! It was great to meet you and see you hack! Same thing! As you probably read in my BSDCan report[1] and my post about Kyua[2], it was very good to learn about and play with Jenkins. FYI, I have just pushed a much-improved version of the JUnit report generation code to GitHub[3]. The new "report-junit" command generates XML reports that can be read by Jenkins and they should include all of the details also available in all other report types. Let's get this running in the FreeBSD Jenkins setup! 1: http://julipedia.meroh.net/2014/05/bsdcan-2014-summary.html 2: http://julipedia.meroh.net/2014/05/refocusing-kyua-maybe.html 3: https://github.com/jmmv/kyua/commits/master/kyua-cli From owner-freebsd-testing@FreeBSD.ORG Wed Jun 4 19:47:56 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A592CBD; Wed, 4 Jun 2014 19:47:56 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C010728E4; Wed, 4 Jun 2014 19:47:55 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id mc6so4706577lab.16 for ; Wed, 04 Jun 2014 12:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ycA2Re9eoitYvQ42K+QZPp2js+I9vTLfpqe3McK+w9w=; b=G3elBT+HY8Z7n7+4LA6Y48SnGpeKPaBJNN1/kfarz5CrVXXreTHrK56OoSu2YH7s5l SPxsFudQd0tpIlqxBgWoBVetSImOcb6c1Dto9Ewuba2tDqdbG95kOzOwXTC9KQ9X0Bzc zhYL3ANeXYCJnh8Mvki+oO2kyEeSbU7zmUcWj9y8xgGB5JlGllsCQpUCLc4QXUSCXGse 9J6qNWJGDUYY8iZSgYZIASIgFCWmEdT8H7feC8lBLgywAMHct/LPrvp+zmaoKlkcxLc1 Cu6+sM/B0Xgr251Sm11UIUyoXTl99+qCsMpcPD1klU45MlLSNwkjCVnJjW5k7sLblpPO 0i4Q== MIME-Version: 1.0 X-Received: by 10.112.63.65 with SMTP id e1mr4321802lbs.81.1401911273535; Wed, 04 Jun 2014 12:47:53 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 4 Jun 2014 12:47:53 -0700 (PDT) In-Reply-To: References: Date: Wed, 4 Jun 2014 12:47:53 -0700 X-Google-Sender-Auth: ht5evMzG3uULLosXdBi5rASfsdQ Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? From: Craig Rodrigues To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jun 2014 19:47:56 -0000 On Fri, May 30, 2014 at 3:03 PM, Julio Merino wrote: > FYI, I have just pushed a much-improved version of the JUnit report > generation code to GitHub[3]. The new "report-junit" command generates > XML reports that can be read by Jenkins and they should include all of > the details also available in all other report types. Has this made it into the FreeBSD port of Kyua, or do I need to download and build the sources to get this? Either way is fine, but I would like to try this out at my work on FreeNAS. -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Jun 4 20:35:05 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36EEF26D for ; Wed, 4 Jun 2014 20:35:05 +0000 (UTC) Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E7EA42E94 for ; Wed, 4 Jun 2014 20:35:04 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id j7so53985qaq.31 for ; Wed, 04 Jun 2014 13:34:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=sdoow0abHbSL0hLxVqPZI0maFD0WiQ9WOIjLPcfRs08=; b=iSY0R840X0qR7Hq5W9vfpV/8buhts/1qWHQTwaSTQQs5EzgJFxDkGLVz/gYi5QzJrD KoZfJsyL2d1IBawI2sofftL4/DThyjWAyqO06Y3pxftbGW9LJ4YVG7Aoi4qLuCkBe4ih F8ePrDch3sY+vrRO9BgzXmhFE3d1dRJD0jL3mqOqqdUUk7If5jG48qb9WQZ5qzGa9B3R Hne9mE2YdKG5Yb1jrDKJYg3OCpBHqX6S8OE9hkUv2F/8tfDyvTWNG2MNfbq+3OW53RaI Le327heZUuirxzvLuqthgXCV638lys+tO/QxnlExUrmbpuLsOVdnzT81vVv8RdsrlyGy uunw== X-Gm-Message-State: ALoCoQmRGD7L53CV5MG0SMsIknBQAYg2keXDPpGMUxHLGQXL5DociFRKZ9b85s5FfjnfnHuTkTla X-Received: by 10.224.167.70 with SMTP id p6mr9627609qay.48.1401914098308; Wed, 04 Jun 2014 13:34:58 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Wed, 4 Jun 2014 13:34:37 -0700 (PDT) X-Originating-IP: [2620:0:1003:1007:fd71:fc40:b23e:91dc] In-Reply-To: References: From: Julio Merino Date: Wed, 4 Jun 2014 16:34:37 -0400 X-Google-Sender-Auth: 45Ml5l-sNcIAJRitAYx00FlbR1E Message-ID: Subject: Re: Generating Kyua test result output in JUnit XML format for Jenkins integration? To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jun 2014 20:35:05 -0000 On Wed, Jun 4, 2014 at 3:47 PM, Craig Rodrigues wrote: > On Fri, May 30, 2014 at 3:03 PM, Julio Merino wrote: >> FYI, I have just pushed a much-improved version of the JUnit report >> generation code to GitHub[3]. The new "report-junit" command generates >> XML reports that can be read by Jenkins and they should include all of >> the details also available in all other report types. > > Has this made it into the FreeBSD port of Kyua, > or do I need to download and build the sources to get this? Still in Git only; haven't cut a release. From owner-freebsd-testing@FreeBSD.ORG Wed Jun 4 21:52:26 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A416336 for ; Wed, 4 Jun 2014 21:52:26 +0000 (UTC) Received: from mail-qa0-f42.google.com (mail-qa0-f42.google.com [209.85.216.42]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D67928F9 for ; Wed, 4 Jun 2014 21:52:25 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id j5so194475qaq.1 for ; Wed, 04 Jun 2014 14:52:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to:content-type; bh=R5lYeaY05xmk5hQ4yDK9Gb/nrfzKq4q65hx/Wk7joXc=; b=hql+5UkqpR/q0nUykKJHN1VKFyH4ivhp48SKTNSPvcpdlDmW8zxMwu4z7Puy0EK3sH KIyUDcSVS368Qoebk1n7grgVPGeF/0Bb1arl2vj2PnkrN1+IzHVJL/hXNB6md3Zc38XU GaVcjhWHUmxwXp22Yoge/Fmdij4WaxggZfRkbaoR3yn2EUvpq3UYotbv2lWa702XfZtu uUlWU/elAhlkTAr8wNJJSt7K0Xs45JbJ0qzEEONAQS8qFwWtVP7SKlNJyRuLr88E8vg/ cTxAowA2O8GEriE3oSgYj6VL2fuDf3mtmHyRyFL0v7wyceL4/a33T+VtZUzHogFVbRlY nEyg== X-Gm-Message-State: ALoCoQlIuRyr1V+l8gqR5XhDjA7OuTNq0ksJuXXO5l462HH8NYxtkpNkoQIqPRxT6MWypZdcLtcW X-Received: by 10.140.81.146 with SMTP id f18mr74561096qgd.47.1401918331474; Wed, 04 Jun 2014 14:45:31 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Wed, 4 Jun 2014 14:45:11 -0700 (PDT) X-Originating-IP: [2620:0:1003:1007:fd71:fc40:b23e:91dc] From: Julio Merino Date: Wed, 4 Jun 2014 17:45:11 -0400 X-Google-Sender-Auth: o7Am3v5pcZlVF0jftQ8qZGDVZ38 Message-ID: Subject: Enabling MK_TESTS by default To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jun 2014 21:52:26 -0000 Hello all, I think we are ready to enter the new world so I would like to take the final step and make MK_TESTS the default. But before that, I would like to commit https://phabric.freebsd.org/D174 . I think the change is safe for current (even though there is the unfortunate need of an entry in UPDATING). The impact on stable/10 will be evaluated later along with the move of the libraries to private (which I thought I had done but hadn't). Feel free to rubber-stamp that diff. So, unless I hear otherwise: - Tomorrow I will submit D174 referenced above. - Flip MK_TESTS locally and run make tinderbox ad nauseam to ensure things work. - On Sunday or thereabouts commit the MK_TESTS setting to yes by default. Cheers From owner-freebsd-testing@FreeBSD.ORG Fri Jun 6 17:21:59 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2702C774; Fri, 6 Jun 2014 17:21:59 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7AB2129EC; Fri, 6 Jun 2014 17:21:58 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id pv20so1722116lab.36 for ; Fri, 06 Jun 2014 10:21:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=xqmVp/FvKyZdcA7NDH74g6MkmTNIYV1SMRLRjQgB2r0=; b=pn16inHeseKUcFDZ/URDCZn2R6V0Uz5r42nMKPr9FAtIlmR1orFUEs7tNg7uwvGAJW w2kZIad/9qu6km21efBkcaF0fSyi+KxY5oYHC1MsFRwu6zzymJE7kohHjpVhRbKZXdim LZXwe/YyhMGwKwCIG/xxUJ8gLD81j3jlaOn2ji58CABrKfvWAGrfQmVw1LFDJFcf68Ja EHXj7GPzFzU94qllhZzRDw0wMT9r0XfqnQFnsYhSFDwC2GDInMEiUzBV+jeBUltwi4MZ a7P+ug8cnSZDIRHYSuRTNFb7Zzb/LqJMA3fJa/u2wHwg/hQnfhjJLDZ8eGQvlFiYUUQi tawQ== MIME-Version: 1.0 X-Received: by 10.152.28.194 with SMTP id d2mr4685461lah.25.1402075316368; Fri, 06 Jun 2014 10:21:56 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Fri, 6 Jun 2014 10:21:56 -0700 (PDT) In-Reply-To: References: Date: Fri, 6 Jun 2014 10:21:56 -0700 X-Google-Sender-Auth: YtviZ5qBxL0iKDvMXJyENDpKQ64 Message-ID: Subject: Re: Enabling MK_TESTS by default From: Craig Rodrigues To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 17:21:59 -0000 On Wed, Jun 4, 2014 at 2:45 PM, Julio Merino wrote: > Hello all, > > I think we are ready to enter the new world so I would like to take > the final step and make MK_TESTS the default. Can you explain what is the impact of this? What will folks doing "make buildworld", "make buildkernel", "make release", "make universe" see after this change? You'll probably want to send a heads-up e-mail to current@ to let folks know it is coming. -- Craig From owner-freebsd-testing@FreeBSD.ORG Fri Jun 6 17:29:06 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A17EA19 for ; Fri, 6 Jun 2014 17:29:06 +0000 (UTC) Received: from mail-qg0-f41.google.com (mail-qg0-f41.google.com [209.85.192.41]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EDCF42A63 for ; Fri, 6 Jun 2014 17:29:05 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id j5so5123442qga.14 for ; Fri, 06 Jun 2014 10:28:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=Zw1A/0Qo+zbHwBaKOHyCOy9f3u6A12a/wMft3+6CJ/4=; b=Lh9d08JdRZCXRhSu0TBW28GFWDrHEgZYSbWSmegS2uS5EQt+ZhvQ+5trw5ZysRqbcE iB70Kat6TLBI+vDY8pp1ao3yeqY8dM4godtn39VcVUHR01v2RQ2cKzdG3KlEmtA6BUlq ab52hinMueyaftOAUvveeL7LxmSac8QxVeXX6kc2xGzFixRWqO2D4zR+3Y5EmAJrlGTz DvCFsl/7rI1/Sqk6KTiqkvvilcjv3uzNxyu0z9rQqlwKD07dQ3BUq8ilpdP8ctej6gkp Tcvk/py1bxeb0H8xujkFReCIiS9b96r06i2pW5WLdh+egdISOuXM7dnul/kW0xUkkS1j 3KNw== X-Gm-Message-State: ALoCoQn0glaswCmMBc/XEZLyWCi00paDDauQAE7g5SGFQXmeQCFdeF+1v8l/t367jDXd+5/89arq X-Received: by 10.140.91.161 with SMTP id z30mr10061043qgd.65.1402075739485; Fri, 06 Jun 2014 10:28:59 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Fri, 6 Jun 2014 10:28:39 -0700 (PDT) X-Originating-IP: [2620:0:1003:1021:fd73:211d:1f18:17ea] In-Reply-To: References: From: Julio Merino Date: Fri, 6 Jun 2014 13:28:39 -0400 X-Google-Sender-Auth: ACjwqUR7daknHscbxWLy8FWIMZg Message-ID: Subject: Re: Enabling MK_TESTS by default To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 17:29:06 -0000 On Fri, Jun 6, 2014 at 1:21 PM, Craig Rodrigues wrote: > On Wed, Jun 4, 2014 at 2:45 PM, Julio Merino wrote: >> Hello all, >> >> I think we are ready to enter the new world so I would like to take >> the final step and make MK_TESTS the default. > > Can you explain what is the impact of this? > What will folks doing "make buildworld", "make buildkernel", > "make release", "make universe" see after this change? /usr/tests being populated and atf being installed. > You'll probably want to send a heads-up e-mail to current@ to let > folks know it is coming. That's probably worth it. From owner-freebsd-testing@FreeBSD.ORG Fri Jun 6 18:07:21 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 485C77A7 for ; Fri, 6 Jun 2014 18:07:21 +0000 (UTC) Received: from mail-qa0-f49.google.com (mail-qa0-f49.google.com [209.85.216.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B4A92E7E for ; Fri, 6 Jun 2014 18:07:20 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id cm18so4345966qab.8 for ; Fri, 06 Jun 2014 11:07:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to:content-type; bh=lPw3BCo/HrkWr/w6fCVcsgGiUsxrvsVzFnRw2w0qsUE=; b=LUnjsb6Kci849wjCsBL4o+Dy6pHzsQxmIX64iUFwN+0Js/GdLVu7DSdmuD/5U7YHDt jGTr6Gcd4sV41eiBBYtgu9SsMsQ1OQZEkANYt1q5Mtj0fcnrgXGtP9yXFF/f95JosIQX 5GV+xV/68OXslABlZV7CBmrVI10/fKm6/aRuimIf2LjC8MGU/B5Oph/jFPgz/TVjqGYn 9oI1dCFkD+5ZahklR0MDgFYPUa7cWWL5rJu4W7BK3z8+w9UUTwIEEIl2JDNC8fO96Iov gcOI/PV5QbrMpX/GrArToW2k89m9ttkWZXAfhObaNB/N3xNmP7NRI+tUJRIm3CAAxQG7 n1BA== X-Gm-Message-State: ALoCoQmTGBWc+B9H82Ypctfhw5EkU1M21Rp3nQ42RGywEhMkTSgDRvmNFU9alamKdiwPveAsDOuq X-Received: by 10.224.11.70 with SMTP id s6mr11135804qas.65.1402078034546; Fri, 06 Jun 2014 11:07:14 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Fri, 6 Jun 2014 11:06:54 -0700 (PDT) X-Originating-IP: [2620:0:1003:1021:fd73:211d:1f18:17ea] From: Julio Merino Date: Fri, 6 Jun 2014 14:06:54 -0400 X-Google-Sender-Auth: res89YEKmPG1xNO58irj6sPmO4k Message-ID: Subject: Turning atf-sh into a private binary To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 18:07:21 -0000 Hello, Just as we turned libatf-c and libatf-c++ into private libraries with r266650, I am going to do the same with the "shell library". I.e. I am going to move atf-sh from /usr/bin into /usr/libexec so that nobody can depend on atf-sh without intentionally doing so. Effectively, this turns all of atf into a "private" component of the base system, as it should be. The whole point of this change is to prevent people depending on atf-sh inadvertently: they will still be able to do so by using the binary from /usr/libexec (just as they can link against libraries in /usr/lib/private), but then they should understand that they are depending on internal implementation details that can change at will. I consider this to be a prerequisite for enabling WITH_TESTS by default given that moving the binary afterwards would be trickier. If you want to take a look: https://phabric.freebsd.org/D187 However, I plan to commit this *today* after I confirm the change working. The reason is to ride the UPDATING warning that I just committed with r267172 so that users don't have to clean their object trees twice in quick succession. Apologies for not giving enough time to review upfront, and yell if you think this is a horrible idea! Thanks From owner-freebsd-testing@FreeBSD.ORG Fri Jun 6 23:00:33 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D62553E8; Fri, 6 Jun 2014 23:00:33 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD14429A9; Fri, 6 Jun 2014 23:00:32 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id c11so1927864lbj.10 for ; Fri, 06 Jun 2014 16:00:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=DjkBtsV2kJmUsC4yvGCZVJoWBheqNMJAmXC3dyTPcDg=; b=VFjdNlyj+ElHm0dZ67C2OGjmf9loQP2YvUH3Oj0VM4DbsQlATI+uwXBQVIzYFRJ8A0 DVQvAau4NMXPkIdJ4Ge2WJ5xfcEOAb4R4QSS8N/QvPxEZB0FhodDnwmCaNL41bk3yb0f RREEuzIyS1n2/0b7GC1jJWLFS7kuoiOM4MGpcRGtmvhnPuimv3Y+4emsiN+UO6h2iGB3 XpP27grhSi21iCesh3dlpVfSO0Jixs5OXNx8G/kpneIX/y+DEZb1zzeEn+jKAgNhqzjA 7cblAJC85/Hwfgx4bkddcg7B/j5B1fsZv0+4/7XKwMeUEbUiL1bGeMeSHtR/+6flyHpO tiLA== MIME-Version: 1.0 X-Received: by 10.112.218.74 with SMTP id pe10mr5578782lbc.3.1402095630682; Fri, 06 Jun 2014 16:00:30 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Fri, 6 Jun 2014 16:00:30 -0700 (PDT) Date: Fri, 6 Jun 2014 16:00:30 -0700 X-Google-Sender-Auth: ufQom1G_kH0JpX28i1elHXjClZc Message-ID: Subject: JNA fix for libpam4j affecting Jenkins From: Craig Rodrigues To: "R. Tyler Croy" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Anthony Williams , Jung-uk Kim , "freebsd-java@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 23:00:33 -0000 Tyler, Anthony Williams took this patch from Jung-uk: http://people.freebsd.org/~jkim/jna.diff and tried to push it upstream to Jenkins via this pull request: https://github.com/jenkinsci/jna/pull/2 Can you use your connections in the Jenkins community to push this change into the Jenkins codebase? This addresses this Jenkins issue which we have seen in FreeBSD when trying to use PAM authentication: https://issues.jenkins-ci.org/browse/JENKINS-21507 Thanks. -- Craig From owner-freebsd-testing@FreeBSD.ORG Sun Jun 8 06:40:44 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D89F3D3C; Sun, 8 Jun 2014 06:40:44 +0000 (UTC) Received: from mail-ie0-x235.google.com (mail-ie0-x235.google.com [IPv6:2607:f8b0:4001:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A6612901; Sun, 8 Jun 2014 06:40:41 +0000 (UTC) Received: by mail-ie0-f181.google.com with SMTP id y20so641521ier.40 for ; Sat, 07 Jun 2014 23:40:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:date:message-id :cc:to:mime-version; bh=4fAXctjMW98g9GbxA6fVSdhVbqVYOfDqoO48vZnYURM=; b=Cd26c/7aEObDiRSSUua7IO6qWMLIHjQ1sf0OnwF+NbxMkBbmbv2VVdmVukqvVXOnAl 2Ibmp7+ZudtK/2tw649D57xN1Rn8inofiEdD0Xt8gVGWKFxFpyZFCuN7z6LuJToR7ygd be0sWOk3o4kg/V74YGDxz8Hlj7r4+KbkKV5FlwwgwIJiJdU+8RLtajzRgCbymLBYV/z2 vZ+WK5Ww1M/LHzN9dqsNLp5f9DGb3mcrl9B3s5mA28ALHx7LF3FaGjl0A3oe/QFTqVP0 Xf+GMmQLsKfsG1RXBiiJwXfssawy2M/XSX8w5G+HT4wo4LzzC8NyB40AXAo4agp5R2Wd FHkQ== X-Received: by 10.50.73.228 with SMTP id o4mr21393142igv.6.1402209640584; Sat, 07 Jun 2014 23:40:40 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:2005:d24f:fa1c:d46d? ([2601:8:ab80:7d6:2005:d24f:fa1c:d46d]) by mx.google.com with ESMTPSA id qo12sm76083509igb.21.2014.06.07.23.40.38 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 07 Jun 2014 23:40:38 -0700 (PDT) From: Garrett Cooper X-Google-Original-From: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: "Smoke tests" for FreeBSD VM images? Date: Sat, 7 Jun 2014 23:40:30 -0700 Message-Id: To: Craig Rodrigues , Julio Merino Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) X-Mailer: Apple Mail (2.1878.2) Cc: "freebsd-testing@freebsd.org" , freebsd-jenkins@Freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 06:40:44 -0000 Hello Craig and Julio, One of the things I=92ll be working on in the upcoming weeks is = automating merges to my fork(s). In order to make sure I don=92t merge = broken code, I would (at the very least) make sure that I am not booting = up a broken image. Is there a smoke test that has been added for = determining that basic bits are functional (boot, basic networking, = etc), and if so, where could I find said results [in Jenkins, etc]? Thank you! -Garrett= From owner-freebsd-testing@FreeBSD.ORG Sun Jun 8 21:00:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09F19949 for ; Sun, 8 Jun 2014 21:00:38 +0000 (UTC) Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BCC8B28D0 for ; Sun, 8 Jun 2014 21:00:37 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id j7so7105963qaq.31 for ; Sun, 08 Jun 2014 14:00:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=rzmkOuBQdmEZ9GhC7s03t93xnT/AyyZ9bB49ZHInDcM=; b=T/nKXq3ox2WlIrc52rmgq8Ao5ckx9O/aPd7BX703VRl4DgCAH/sCJnoOOp4TDAEvey 7t45kLMIUoDNSbFYYQQisf4O2xoYbpwVkzhjxSVUZliQoHXdirU+VTq4Y46tifEyXWoA tJkNr8E/klntgxlOquLIUNULnxwpaRj4zfv6OWZL99cK6DmNUg6JtR9Yd0IbOF0J2lGL F0mPycMWmLyl9osneeJpucs/nE8b+lBh2GD/8KgQ41bmESzcP0kFCZCZ/WjGd1YcyPud S//D5ApCu88vIVmY59vjEJzEcMoDC6SLigwHSID84qPW0chxVNLm+k0tGegVOaov15aW Vnvg== X-Gm-Message-State: ALoCoQnqGEGeM5JawgA7bF0vkgBxlMkzMZ9FS9q1+t364RH30ggQ74NfxcvmN2TbskovypNoHwNU X-Received: by 10.224.152.5 with SMTP id e5mr6714331qaw.65.1402261231077; Sun, 08 Jun 2014 14:00:31 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Sun, 8 Jun 2014 14:00:10 -0700 (PDT) X-Originating-IP: [89.101.236.178] In-Reply-To: References: From: Julio Merino Date: Sun, 8 Jun 2014 17:00:10 -0400 X-Google-Sender-Auth: YyE_AGtZ0vaW5tDkDSv1BvZRT0s Message-ID: Subject: Re: "Smoke tests" for FreeBSD VM images? To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , freebsd-jenkins@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 21:00:38 -0000 On Sun, Jun 8, 2014 at 2:40 AM, Garrett Cooper wrote: > Hello Craig and Julio, > One of the things I=E2=80=99ll be working on in the upcoming week= s is automating merges to my fork(s). In order to make sure I don=E2=80=99t= merge broken code, I would (at the very least) make sure that I am not boo= ting up a broken image. Is there a smoke test that has been added for deter= mining that basic bits are functional (boot, basic networking, etc), and if= so, where could I find said results [in Jenkins, etc]? The current test suite is not large enough to warrant having a separate set of smoke tests, in my opinion. We can use everything for now. But no, this has not been added to Jenkins yet. Details on how to deal with permissions need to be sorted out first... From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 05:40:02 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70D55E20; Tue, 10 Jun 2014 05:40:02 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 964602A3B; Tue, 10 Jun 2014 05:40:01 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id b8so3660945lan.9 for ; Mon, 09 Jun 2014 22:39:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=eKFRhzc9a7+eZZfxvSNsujNiT1HZvo1EXq+YZi6NblQ=; b=wDJOWcZHhYyxXNMMwqkwi+2Ffyzv2DT9KZh48Y8W/EqNhYNXfwzX7rH9560Qjx1rOq jekwijF9JFfHtQBIKTUP26CqwPPU47ShEAOKgpxoVXfKl1gtjLqLbpiqWs8KknLDAnOv 93/7dasIn/cIhm81/ptUjxJnUbSViKTHrg3i+A9BCuuhw2rmWl68GmNWUBzEdm2wU71N k19cg7qZH+dolOKEP28DdlFZF4KhfWJ6ddpZ8q7DVRhFwJng/DgzCoeJ5QD9Sqsd5t// ePfb8RO7T/nm1ceabb87qDjUKtpvigyzVqskfzEdp1NyDWqpJiSks6epZ7ezb7dsjhsw TaqA== MIME-Version: 1.0 X-Received: by 10.153.5.37 with SMTP id cj5mr542032lad.48.1402378799580; Mon, 09 Jun 2014 22:39:59 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Mon, 9 Jun 2014 22:39:59 -0700 (PDT) Date: Mon, 9 Jun 2014 22:39:59 -0700 X-Google-Sender-Auth: Z42-S537V34VLd4BOGoGcInKyn0 Message-ID: Subject: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy , Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 05:40:02 -0000 Roman, I have been reading your blog posts on using libvirt + bhyve here: http://empt1e.blogspot.com/ and the libvirt bhyve documentation here: http://libvirt.org/drvbhyve.html but am having problems reproducing the steps. I have a machine running: FreeBSD 11.0-CURRENT FreeBSD 11.0-CURRENT #6 r266934M: Sat May 31 20:24:37 PDT 2014 root@crodrigues.org:/usr/obj/opt2/branches/head/sys/GENERIC amd64 I applied the following patches to the devel/libvirt port to fix compilation problems for bhyve support: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002586.html I then did the following: BUILD AND CONFIGURE =================== cd /usr/ports/devel/libport make config (I made sure bhyve and QEMU support were enabled make make install LIBVIRT CONFIG ============== I created the following /tmp/bhyve.xml file: bhyve df3be7e7-a104-11e3-aeb0-50e5492bd3dc 219136 219136 1 hvm RUNNING ======== I did the following as root: service libvirtd restart virsh -c 'bhyve:///system' Then I ran these commands from the virsh prompt: virsh # define /tmp/bhyve.xml Domain bhyve defined from /tmp/bhyve.xml virsh # start bhyve error: Failed to start domain bhyve error: End of file while reading data: Input/output error error: Failed to reconnect to the hypervisor At this point libvirtd process crashed with a /libvirtd.core process , but for some reason I could not get a valid stacktrace. Can you provide assistance to get this working? Ultimately, I want to get the Jenkins libvirt plugin ( https://wiki.jenkins-ci.org/display/JENKINS/Libvirt+Slaves+Plugin ) to work. Thanks. -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 11:43:52 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50050CB8; Tue, 10 Jun 2014 11:43:52 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 71E032A9B; Tue, 10 Jun 2014 11:43:51 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id l4so3845599lbv.17 for ; Tue, 10 Jun 2014 04:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=q1K2dN0s2PNQSLofvWRY8FFgz38C/BLjjopc3TQBddI=; b=wovUoA9Wu9UuPxyHTRCMaIeokA7+Cn0FgGQlXY3x8acyMg5lFUtSevzxgiRMq/1qzx 0Lx52MxankEo9y33uPjAUtkXg/uYWlkVcdYybkfRiApSUMZD7J2B1hq8IYbXfCS5VC2Z Z8R1pj00wgp8KWWiSazwt09Mc/qmR8lMFcNxnyA+VnW7UFtAi4yeybhlooO8EPe5Yo0l fl2VdDgOz3g8Xbl9xQbDJgmtsBoYQO/tPckKc9xASOSJM+Rpp7b2f6WWZSrXnfdb3MKW IGBbRE189/BkARPDaEj/NCzEs3tYCAxIZr+InLBooanl65YbhF9ukammzVdMxfVToq9f hLbQ== X-Received: by 10.152.44.170 with SMTP id f10mr17291lam.87.1402400629311; Tue, 10 Jun 2014 04:43:49 -0700 (PDT) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id b6sm18446425laa.20.2014.06.10.04.43.48 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Jun 2014 04:43:48 -0700 (PDT) Sender: Roman Bogorodskiy Date: Tue, 10 Jun 2014 15:43:43 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140610114342.GA11194@dev.san.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 11:43:52 -0000 Craig Rodrigues wrote: > Roman, >=20 > I have been reading your blog posts on using libvirt + bhyve > here: >=20 > http://empt1e.blogspot.com/ >=20 > and the libvirt bhyve documentation here: > http://libvirt.org/drvbhyve.html >=20 > but am having problems reproducing the steps. >=20 > I have a machine running: > FreeBSD 11.0-CURRENT FreeBSD 11.0-CURRENT #6 r266934M: Sat May 31 > 20:24:37 PDT 2014 > root@crodrigues.org:/usr/obj/opt2/branches/head/sys/GENERIC amd64 >=20 >=20 > I applied the following patches to the devel/libvirt port to fix > compilation problems for bhyve support: >=20 > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/00258= 6.html >=20 > I then did the following: >=20 > BUILD AND CONFIGURE > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > cd /usr/ports/devel/libport > make config > (I made sure bhyve and QEMU support were enabled > make > make install >=20 >=20 > LIBVIRT CONFIG > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > I created the following /tmp/bhyve.xml file: >=20 >=20 > > bhyve > df3be7e7-a104-11e3-aeb0-50e5492bd3dc > 219136 > 219136 > 1 > > hvm > > > > > > > > > > > > > > > > > > >=20 >=20 > RUNNING > =3D=3D=3D=3D=3D=3D=3D=3D >=20 > I did the following as root: >=20 > service libvirtd restart > virsh -c 'bhyve:///system' >=20 > Then I ran these commands from the virsh prompt: >=20 > virsh # define /tmp/bhyve.xml > Domain bhyve defined from /tmp/bhyve.xml >=20 > virsh # start bhyve > error: Failed to start domain bhyve > error: End of file while reading data: Input/output error > error: Failed to reconnect to the hypervisor >=20 >=20 > At this point libvirtd process crashed with a /libvirtd.core process , > but for some reason I could not get a valid stacktrace. >=20 > Can you provide assistance to get this working? >=20 > Ultimately, I want to get the Jenkins libvirt plugin ( > https://wiki.jenkins-ci.org/display/JENKINS/Libvirt+Slaves+Plugin ) > to work. >=20 > Thanks. Hi Craig, It's a little unfortunate that there's no stacktrace, though we can try to see what's going on without it. Could you please edit libvirtd.conf (should be something like /usr/local/etc/libvirtd.conf if installed from port) to add: log_level =3D 2 then run libvirtd by hand in foreground: # libvirtd=20 and then try to start the domain again and post a couple of lines it prints before crashing? Thanks, Roman Bogorodskiy From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 15:58:04 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E0611A9; Tue, 10 Jun 2014 15:58:04 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41D5B24E6; Tue, 10 Jun 2014 15:58:03 +0000 (UTC) Received: by mail-la0-f46.google.com with SMTP id hz20so3889994lab.5 for ; Tue, 10 Jun 2014 08:58:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6mychWtN/91vFiEVBdrg28FvlRW00r3KQNg9JVWvQqk=; b=awLGWtyc3I8GuQ6jxP8sgp/m7CjFvmKRHGaI1fLB2td7HOKwAgEJfpMuLMO4l6uhOc N1TDg4siFdC/E37L/tyYFfxTs6nJg7tJBdBFpvhmpJaBAhPv0Mc201V641vz2uLURyRe SVnenltjF9x7JCkgF0fw/4efb0GGXPwnbb8McuLrfw58WYHw77w2hvxrqiG1gotWg2mA 821LVicVWP6VG+Eqf2ruzQ1C0uLs+fD3CUxMYs2fdcjVBz2fsVoAH2UGzZZMx5Ly7J7R j815JzqJLAUWTBs4g9oTDf5MyaS6bYlzLmo5I/W8ZoYKmlM3zsOqH9S7JFaGKy9olbZ6 COkw== MIME-Version: 1.0 X-Received: by 10.112.63.65 with SMTP id e1mr1763453lbs.81.1402415880970; Tue, 10 Jun 2014 08:58:00 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 08:58:00 -0700 (PDT) In-Reply-To: <20140610114342.GA11194@dev.san.ru> References: <20140610114342.GA11194@dev.san.ru> Date: Tue, 10 Jun 2014 08:58:00 -0700 X-Google-Sender-Auth: GZf-cv8HpjyZMTNLAFOetPKltFA Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 15:58:04 -0000 On Tue, Jun 10, 2014 at 4:43 AM, Roman Bogorodskiy wrote: > Hi Craig, > > It's a little unfortunate that there's no stacktrace, though we can try > to see what's going on without it. > > Could you please edit libvirtd.conf (should be something like > /usr/local/etc/libvirtd.conf if installed from port) to add: > > log_level = 2 > > then run libvirtd by hand in foreground: OK, I added log_level = 2 to /usr/local/etc/libvirt/libvirtd.conf and ran libvirtd in the foreground: 2014-06-10 15:53:26.846+0000: 34485593088: info : libvirt version: 1.2.5 2014-06-10 15:53:26.846+0000: 34485593088: warning : virDriverLoadModule:63 : Module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so not accessible 2014-06-10 15:53:27.023+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-10 15:53:27.023+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-10 15:53:27.034+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-10 15:53:27.034+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-10 15:53:27.130+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 15:53:53.668+0000: 34485598208: error : virNetDevGetMTU:404 : Cannot get interface MTU on 'virbr0': Device not configured Segmentation fault (core dumped) I got that coredump after doing: virsh -c 'bhyve:///system' virsh # define /tmp/bhyve.xml virsh # start bhyve From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 17:39:41 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92FB8D07; Tue, 10 Jun 2014 17:39:41 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B3F9F2F04; Tue, 10 Jun 2014 17:39:40 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id b8so4106474lan.23 for ; Tue, 10 Jun 2014 10:39:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=iUsX0MPEBUdNQbHcyU/bQuifIQIBlZFm7mUAQvwOSdE=; b=xNtV/iynAY4mFbpHVKDNWmLuun5jvZUDwvMFdvqwsGjnPSTnf2vXQX2OqGY7L9yJ7A BfYHxX8ryqtB0xHOGJy2qO4AgaJ4uJP4oEPy7I0HKwvjlo4LNYzliqJ+NQM5pNroSEQT 2wfOYRVky/q9M7rOx0opwrgqgszqxWENSc4jYBeMDjsrWZ1Nx70L600ygNPG+WhspWlt X1DEqzXv0nxJ8DaRPo1q7b+nZCaPWNfBXFST4z8VxXnWof0G9t5dPnqWCx8O76sVm8ym hcLIuNWvjznwFF+yyTxqz9ewfNwRC9iDbViG5MrqG3wS7Kob4rV1952y3hk0fBPCe1V7 OQXg== MIME-Version: 1.0 X-Received: by 10.112.146.202 with SMTP id te10mr2292463lbb.75.1402421978558; Tue, 10 Jun 2014 10:39:38 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 10:39:38 -0700 (PDT) In-Reply-To: References: <20140610114342.GA11194@dev.san.ru> Date: Tue, 10 Jun 2014 10:39:38 -0700 X-Google-Sender-Auth: R3ak9HTibl_6mmGOKIyEaRCddDE Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 17:39:41 -0000 On Tue, Jun 10, 2014 at 8:58 AM, Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 4:43 AM, Roman Bogorodskiy wrote: >> Hi Craig, >> >> It's a little unfortunate that there's no stacktrace, though we can try >> to see what's going on without it. >> >> Could you please edit libvirtd.conf (should be something like >> /usr/local/etc/libvirtd.conf if installed from port) to add: >> >> log_level = 2 >> >> then run libvirtd by hand in foreground: > > OK, I added log_level = 2 to /usr/local/etc/libvirt/libvirtd.conf > and ran libvirtd in the foreground: > > 2014-06-10 15:53:26.846+0000: 34485593088: info : libvirt version: 1.2.5 > 2014-06-10 15:53:26.846+0000: 34485593088: warning : > virDriverLoadModule:63 : Module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so not > accessible I saw the above warning, so I tried patching the port with this: --- pkg-plist.orig 2014-06-10 10:36:22.084757789 -0700 +++ pkg-plist 2014-06-10 09:27:25.280043239 -0700 @@ -274,6 +274,8 @@ lib/libvirt/connection-driver/libvirt_driver_secret.la lib/libvirt/connection-driver/libvirt_driver_storage.so lib/libvirt/connection-driver/libvirt_driver_storage.la +lib/libvirt/connection-driver/libvirt_driver_uml.so +lib/libvirt/connection-driver/libvirt_driver_uml.la lib/libvirt/connection-driver/libvirt_driver_vbox.so lib/libvirt/connection-driver/libvirt_driver_vbox.la However, when I restarted libvirtd, I got different errors: 2014-06-10 17:38:50.269+0000: 34485593088: info : libvirt version: 1.2.5 2014-06-10 17:38:50.269+0000: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so: Undefined symbol "inotify_add_watch" 2014-06-10 17:38:50.563+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-10 17:38:50.563+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-10 17:38:50.575+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-10 17:38:50.575+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-10 17:38:50.663+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 17:38:50.663+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 17:56:32 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FE6F578; Tue, 10 Jun 2014 17:56:32 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9CBE820F4; Tue, 10 Jun 2014 17:56:31 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id ty20so4213951lab.12 for ; Tue, 10 Jun 2014 10:56:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=LybjP7zymltthPdonftR2oJF/ppY/zhjyo0rBp/UuFI=; b=vYDc/pz0CJz4JPNUbwqK1LRITuifOHOp85VzWwrH2Z/NDBdphFYnBb3fGpgx1ouIRs BQbwJHPBtnHnowoaryaaw08BO9lvHCUgkyOWeOjYDAl2hG7+jNwsQJNCQ6gnmqEnOZLT VHUGybC2dR1qV3wojydbkqBoqKbmxal0bxMsCUamOSaYaB2rL3oQoeW6VzIRhkXrOwLL hOP3jO/fAfiENiBtoSICMsp/hpGkUF96xTyBYvXywbtzRT5UkpAe5CF5+B8To0vCkUxt ScvqcFA55ose1wyvBtJQZ2zCgB/rEzfvTB/yzbawHS4/0sqqT0vEyGxImnNLMoC4C0rV J/Tw== X-Received: by 10.152.23.136 with SMTP id m8mr23843973laf.2.1402422989518; Tue, 10 Jun 2014 10:56:29 -0700 (PDT) Received: from kloomba ([77.94.196.164]) by mx.google.com with ESMTPSA id k1sm22170264lbb.9.2014.06.10.10.56.28 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Jun 2014 10:56:28 -0700 (PDT) Sender: Roman Bogorodskiy Date: Tue, 10 Jun 2014 21:54:22 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140610175420.GC61743@kloomba> References: <20140610114342.GA11194@dev.san.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="c3bfwLpm8qysLVxt" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 17:56:32 -0000 --c3bfwLpm8qysLVxt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 8:58 AM, Craig Rodrigues wr= ote: > > On Tue, Jun 10, 2014 at 4:43 AM, Roman Bogorodskiy = wrote: > >> Hi Craig, > >> > >> It's a little unfortunate that there's no stacktrace, though we can try > >> to see what's going on without it. > >> > >> Could you please edit libvirtd.conf (should be something like > >> /usr/local/etc/libvirtd.conf if installed from port) to add: > >> > >> log_level =3D 2 > >> > >> then run libvirtd by hand in foreground: > > > > OK, I added log_level =3D 2 to /usr/local/etc/libvirt/libvirtd.conf > > and ran libvirtd in the foreground: > > > > 2014-06-10 15:53:26.846+0000: 34485593088: info : libvirt version: 1.2.5 > > 2014-06-10 15:53:26.846+0000: 34485593088: warning : > > virDriverLoadModule:63 : Module > > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so not > > accessible >=20 > I saw the above warning, so I tried patching the port with this: >=20 > --- pkg-plist.orig 2014-06-10 10:36:22.084757789 -0700 > +++ pkg-plist 2014-06-10 09:27:25.280043239 -0700 > @@ -274,6 +274,8 @@ > lib/libvirt/connection-driver/libvirt_driver_secret.la > lib/libvirt/connection-driver/libvirt_driver_storage.so > lib/libvirt/connection-driver/libvirt_driver_storage.la > +lib/libvirt/connection-driver/libvirt_driver_uml.so > +lib/libvirt/connection-driver/libvirt_driver_uml.la > lib/libvirt/connection-driver/libvirt_driver_vbox.so > lib/libvirt/connection-driver/libvirt_driver_vbox.la This warning is not critical. Actually, UML should not be built on FreeBSD and I would need to disable it by default. For now I think the proper fix is to configure with '--without-uml' flag. As for the segfault, I think I have a guess. Could you please drop '--without-network' from CONFIGURE_ARGS in the port's Makefile and try to rebuild? Roman Bogorodskiy --c3bfwLpm8qysLVxt Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTl0ZMAAoJEMltX/4IwiJqyuIIAJymYJPw21w6t4UyMmJHSfBk 42AgLt7jGgH1CiGuXjguLgtX/NUaaPlcHoen4KBVq+d7pPjKZvyEjbQBXfYURIjw 22p+ZjIz/g4mkFcel0OCMc0CSHitR2m8TUnk89kg5vljbK52DM+lnYUE7oaXanxc 9s+GYouBmzfisCbdjOcc++zQqOWz0QqR/WSzAYULsye59a0SycjEChRUQ7ZLILfK eMeBpCVq+cU7YXA8yEbA5kYDorWsAKS/IOQhpB0kEwYxaTiYC1CppltDT8a8smmi 6U/wPGBHa97Owf3cWTLSxTNbF3Jqk1UIpQZZyGFK/lQ10oB5KnI1ncntF08f2v4= =hNbZ -----END PGP SIGNATURE----- --c3bfwLpm8qysLVxt-- From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 19:01:54 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 494FCA3E; Tue, 10 Jun 2014 19:01:54 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D87627F6; Tue, 10 Jun 2014 19:01:53 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id pv20so4222182lab.36 for ; Tue, 10 Jun 2014 12:01:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5769kDRIX79tWcyli7KRrjvXrsTrWOuF8qmznu6hi8A=; b=JdSsv9gBCv8zIPeu4nqEUyXm9J96+0q63ek0vfiH4lzx7q33oxMPxmpvlOwIbt+gbg pwfRT6Q+oPEjZgxpa6FvA+cvAjrLpwARmThLll32lANgQvNAlQ2ZjsI5zDMDLd8cfn04 bS8pOsjpNJ7uxk202/4EjqR3XBDKpYyBH0oTUgbSwVgQXkgL31qeLqhPwnQ0AIIbf/aL X77+RWe7NFSnqzmXSiUv/WNsVj3HMCnJzK9kErSRMcVjqPTwXYmY0a7RmSEtpfrQlDYW 7VTNN2fXWJJ9bG/ZowRtGtye2uMGNIeagcc/ouhFgBTKreTz/71efZ6/XBR349OfpllO aGDQ== MIME-Version: 1.0 X-Received: by 10.152.87.20 with SMTP id t20mr2397001laz.79.1402426911351; Tue, 10 Jun 2014 12:01:51 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 12:01:51 -0700 (PDT) In-Reply-To: <20140610175420.GC61743@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> Date: Tue, 10 Jun 2014 12:01:51 -0700 X-Google-Sender-Auth: H-_EjKoONKpXFj6JhiWuajk2xZw Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 19:01:54 -0000 On Tue, Jun 10, 2014 at 10:54 AM, Roman Bogorodskiy wrote: > This warning is not critical. Actually, UML should not be built on > FreeBSD and I would need to disable it by default. For now I think the > proper fix is to configure with '--without-uml' flag. > > As for the segfault, I think I have a guess. > > Could you please drop '--without-network' from CONFIGURE_ARGS in the > port's Makefile and try to rebuild? > > Roman Bogorodskiy OK, I dropped --without-network from CONFIGURE_ARGS and rebuilt the port. I am still getting the coredump, but am getting different errors: 2014-06-10 18:59:22.713+0000: 34485593088: info : libvirt version: 1.2.5 2014-06-10 18:59:22.713+0000: 34485593088: warning : virDriverLoadModule:63 : Module /usr/local/lib/libvirt/connection-driver/libvirt_driver_network.so not accessible 2014-06-10 18:59:22.715+0000: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so: Undefined symbol "networkAllocateActualDevice" 2014-06-10 18:59:22.716+0000: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so: Undefined symbol "inotify_add_watch" 2014-06-10 18:59:22.845+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 18:59:22.845+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-10 18:59:22.847+0000: 34485605376: error : do_open:1166 : no connection driver available for qemu:///system 2014-06-10 18:59:42.273+0000: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet7: Invalid argument Segmentation fault (core dumped) From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 19:17:15 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0397B133; Tue, 10 Jun 2014 19:17:15 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19DE52941; Tue, 10 Jun 2014 19:17:13 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id u10so843366lbd.10 for ; Tue, 10 Jun 2014 12:17:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=7XvX+ESCIjQFv1erfiOllgxNETthdLWgwB0S1pL3ROE=; b=PZFP/+e0W8ZI58cm0MRJxNzBeRVIH1kdLhG71K/xv26ZditEUB1z8Z5ANr1qu0gMDe 1YAEBVB9WHsudq0AkRMKO3CY+XUr3wF0+FSgNK2DJlY8SztEt4Va8PXZMmEC3QE8FNMq HdRlqiU7ZmlX4dOonUUtsLEWYxNVKQkIDwqrlsX52HeBGgIlIjP1Dymypau/XL5I3D/n tdV8T5m2qzeR4PojwbopObQxRp1LOv+GAVA9f1h908jj5bn4M/fqFE9VGdvaSFRrRuiH 3A3bSMfWYXR+NpNT60/Nn6Ivf/tErDVWLWTocFECDRcOouMllAFbzYDhow1mheVexTOV oFqg== X-Received: by 10.112.14.42 with SMTP id m10mr2328900lbc.84.1402427832003; Tue, 10 Jun 2014 12:17:12 -0700 (PDT) Received: from kloomba ([77.94.196.164]) by mx.google.com with ESMTPSA id lq20sm22351683lbb.24.2014.06.10.12.17.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Jun 2014 12:17:11 -0700 (PDT) Sender: Roman Bogorodskiy Date: Tue, 10 Jun 2014 23:15:01 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140610191439.GA220@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="JgQwtEuHJzHdouWu" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 19:17:15 -0000 --JgQwtEuHJzHdouWu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 10:54 AM, Roman Bogorodskiy w= rote: > > This warning is not critical. Actually, UML should not be built on > > FreeBSD and I would need to disable it by default. For now I think the > > proper fix is to configure with '--without-uml' flag. > > > > As for the segfault, I think I have a guess. > > > > Could you please drop '--without-network' from CONFIGURE_ARGS in the > > port's Makefile and try to rebuild? > > > > Roman Bogorodskiy >=20 > OK, I dropped --without-network from CONFIGURE_ARGS and rebuilt the port. > I am still getting the coredump, but am getting different errors: >=20 > 2014-06-10 18:59:22.713+0000: 34485593088: info : libvirt version: 1.2.5 > 2014-06-10 18:59:22.713+0000: 34485593088: warning : > virDriverLoadModule:63 : Module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_network.so not > accessible ^^^^ Hmmm, this sounds weird. I think I need to try the port version myself to see if I could reproduce it this way (I'm usually using git version of libvirt instead of port). In the meantime, I'd really appreciate if you could build the port with WITH_DEBUG flag and try to get a stacktrace. > 2014-06-10 18:59:22.715+0000: 34485593088: error : > virDriverLoadModule:71 : failed to load module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so > /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so: > Undefined symbol "networkAllocateActualDevice" > 2014-06-10 18:59:22.716+0000: 34485593088: error : > virDriverLoadModule:71 : failed to load module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so: > Undefined symbol "inotify_add_watch" > 2014-06-10 18:59:22.845+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-10 18:59:22.845+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-10 18:59:22.847+0000: 34485605376: error : do_open:1166 : no > connection driver available for qemu:///system > 2014-06-10 18:59:42.273+0000: 34485598208: error : > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet7: > Invalid argument This last line also sounds strange. I'm wondering if you execute 'net-list --all' will you see the 'default' network? And if it's there, will 'net-show default' should that Autostart is enabled for it? > Segmentation fault (core dumped) Thanks, Roman Bogorodskiy --JgQwtEuHJzHdouWu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTl1kwAAoJEMltX/4IwiJqmFwIAMFDi5JMJQLIEMY5dIRX422i yfyDgeejj0mOssN2+v0XnqDlBReRSGhx6BmpP2fh/eVxUVnZEWcsMmq9DKcmLIkr Z93j4BR+7CoEc6hc045LwYcJpgU7aH5lp0fBnthYZDIDDerep5kIT5DbQgFdTSdW yatRI/MIrX1EpQmYXczr4y6WNh/0ZffQOM4JrYzzPRwcsGNx6J6Eo1kJzNPVnJ9j oqV5pWNdKuil9yNoTaNXnCzUJ22qL79uwG71RIG4n8/f1mhi8h68IldnkFWwXSxa 2xLbqtJgvp2YjbM/bzXnNCU6Mfpm3j3dBXCrGq0iOAM2YtX3XFYLKFtKWtoPnJ0= =IA6e -----END PGP SIGNATURE----- --JgQwtEuHJzHdouWu-- From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 19:25:24 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0555499; Tue, 10 Jun 2014 19:25:23 +0000 (UTC) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C3D72A04; Tue, 10 Jun 2014 19:25:22 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id 10so4234344lbg.18 for ; Tue, 10 Jun 2014 12:25:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=uAB/nAjMZ5ZUj4gVy+jkeVb7YvJ0e4/4pSpFKmDfpmc=; b=RGeuizuPF9VcPQRewei4WmWbwD5tAQFJA3+/S91aZLzKWMwnWWwlwYqrE+RysYsgme Opmasjlwi5CSrrqBcCGL2mMHWakKSMJmILgBF3UrZFE36rTMxfPDyrk1h2UL41VscjDM EhWGy4waqhz3ptkraDtSo+tUFa85Gi3KSt0BMugwebVoIDD96bDvOjElvlnYlr9zTyvh qus02H6nXFnGUfaek0dn2I6FKMVzs9XMS5qR0XrwxYduCKmzJtHDTQyxhndpJD26w9mM 531grJrMpvr18kfJ3Bo9RZiVYEYkanD+27Q9d57SRPOFCKtGHQ1oQhNe+B1u18oeheMw tPaw== MIME-Version: 1.0 X-Received: by 10.152.198.129 with SMTP id jc1mr2881226lac.63.1402428321065; Tue, 10 Jun 2014 12:25:21 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 12:25:20 -0700 (PDT) In-Reply-To: <20140610191439.GA220@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> Date: Tue, 10 Jun 2014 12:25:20 -0700 X-Google-Sender-Auth: rd4x9ybya1csiRclI_62D45fDyI Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 19:25:24 -0000 On Tue, Jun 10, 2014 at 12:15 PM, Roman Bogorodskiy wrote: > > This last line also sounds strange. > > I'm wondering if you execute 'net-list --all' will you see the 'default' > network? And if it's there, will 'net-show default' should that > Autostart is enabled for it? If I do: # virsh -c 'bhyve:///system' Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # net-list --all error: Failed to get the number of active networks error: this function is not supported by the connection driver: virConnectNumOfNetworks The past few times, I have been compiling the port using WITH_DEBUG=yes. However, I am not getting useful stacktraces: /usr/ports/devel/libvirt/work/libvirt-1.2.5]# file /usr/local/sbin/libvirtd /usr/local/sbin/libvirtd: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 11.0 (1100022), not stripped # gdb /usr/local/sbin/libvirtd libvirtd.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `libvirtd'. Program terminated with signal 11, Segmentation fault. #0 0x0000000801aa8513 in ?? () (gdb) where #0 0x0000000801aa8513 in ?? () #1 0x0000001800000008 in ?? () #2 0x00007fffff3f9330 in ?? () #3 0x0000000809e5510c in ?? () #4 0x0000000000000007 in ?? () #5 0x0000000809e5531b in ?? () #6 0x0000000000000000 in ?? () BTW, can you give me: (1) exact git command to checkout libvirt (2) configure command you typed (3) make command you typed I want to get a working setup like you have. It looks like the port needs some fixing before it can be used fully. -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 20:54:20 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BACE5253; Tue, 10 Jun 2014 20:54:20 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6F512268; Tue, 10 Jun 2014 20:54:19 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id el20so4349659lab.29 for ; Tue, 10 Jun 2014 13:54:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=n93JJS7xNnkWxz67beRPIDfMBc5EkcAJtsW6DXoiRu8=; b=JpCzkf1U1LkHSnGF3WR8Vco8dEbdF2MyNvmOQPng2/dRKnqmtZM2na940+BeaNn4Sb BztkDGPjLYiIMTxTyQ3qizzMo230os0NqFSpXDdN6wrKvJHbtb+o06DswcgHW8Zqaax2 2tQnqpyr7YvgQoI3JAXeUFpOTvxUbFDZnTh06Wb+4v96DPLg9j4MzfT9NYgj8jJ+6kpV JJdecymP2FoCAcq+QnujhoCkflGcswE/ib/b/mcxlABueI8+Zb3CinTCq9o8b6jRebo3 4pWUx8S2dcZcSnDfwXjNTetKkuqIf9h/sQbXA+3ZMFTgy6z9TJmKyy8oOOKuB1u+hbYL rJmQ== MIME-Version: 1.0 X-Received: by 10.152.36.38 with SMTP id n6mr24413266laj.0.1402433657667; Tue, 10 Jun 2014 13:54:17 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 13:54:17 -0700 (PDT) In-Reply-To: References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> Date: Tue, 10 Jun 2014 13:54:17 -0700 X-Google-Sender-Auth: 1Lp3IvBfMmszNDsK1mN7lNIhWTM Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 20:54:20 -0000 On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues wrote: > > > # gdb /usr/local/sbin/libvirtd libvirtd.core > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > Core was generated by `libvirtd'. > Program terminated with signal 11, Segmentation fault. > #0 0x0000000801aa8513 in ?? () > (gdb) where > #0 0x0000000801aa8513 in ?? () > #1 0x0000001800000008 in ?? () > #2 0x00007fffff3f9330 in ?? () > #3 0x0000000809e5510c in ?? () > #4 0x0000000000000007 in ?? () > #5 0x0000000809e5531b in ?? () > #6 0x0000000000000000 in ?? () OK, so it looks like in CURRENT, at least for the devel/libvirt port, the gdb in base, and also gdb in ports cannot debug libvirtd.core. I enabled WITH_LLDB=yes in /etc/src.conf and recompiled to install lldb. I am not so familiar with lldb, but at least I can get a stacktrace with it. lldb /usr/local/sbin/libvirtd libvirtd.core (lldb) bt * thread #1: tid = 0, 0x0000000801aa8513 libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x000000080a4211c8, ifname=0x000000080a88d1e0, macaddr=0x000000080a88d144, vmuuid=0x000000080a81b808, tapfd=0x0000000000000000, tapfdSize=1, virtPortProfile=0x0000000000000000, virtVlan=0x0000000000000000, flags=9) + 547 at virnetdevtap.c:583, name = 'libvirtd', stop reason = signal SIGSEGV * frame #0: 0x0000000801aa8513 libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x000000080a4211c8, ifname=0x000000080a88d1e0, macaddr=0x000000080a88d144, vmuuid=0x000000080a81b808, tapfd=0x0000000000000000, tapfdSize=1, virtPortProfile=0x0000000000000000, virtVlan=0x0000000000000000, flags=9) + 547 at virnetdevtap.c:583 frame #1: 0x0000000809e20569 libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=0x000000080a81b800, cmd=0x000000080a4db200, dryRun=false) + 841 at bhyve_command.c:82 frame #2: 0x0000000809e20196 libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=0x000000080a48a040, def=0x000000080a81b800, dryRun=false) + 310 at bhyve_command.c:248 frame #3: 0x0000000809e24d6a libvirt_driver_bhyve.so`virBhyveProcessStart(conn=0x000000080a4db100, driver=0x000000080a48a040, vm=0x000000080a48b0f0, reason=VIR_DOMAIN_RUNNING_BOOTED, flags=0) + 746 at bhyve_process.c:134 frame #4: 0x0000000809e2334a libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=0x000000080a48a280, flags=0) + 490 at bhyve_driver.c:847 frame #5: 0x0000000809e2315a libvirt_driver_bhyve.so`bhyveDomainCreate(dom=0x000000080a48a280) + 26 at bhyve_driver.c:867 frame #6: 0x0000000801b96aad libvirt.so.1002`virDomainCreate(domain=0x000000080a48a280) + 637 at libvirt.c:8968 frame #7: 0x0000000001075427 libvirtd`remoteDispatchDomainCreate(server=0x000000080781c400, client=0x00000008078df480, msg=0x00000008078791d0, rerr=0x00007fffff9fcc30, args=0x0000000807934340) + 231 at remote_dispatch.h:3048 frame #8: 0x000000000103b84e libvirtd`remoteDispatchDomainCreateHelper(server=0x000000080781c400, client=0x00000008078df480, msg=0x00000008078791d0, rerr=0x00007fffff9fcc30, args=0x0000000807934340, ret=0x000000080a421058) + 254 at remote_dispatch.h:3026 frame #9: 0x0000000801c26b20 libvirt.so.1002`virNetServerProgramDispatchCall(prog=0x000000080790f400, server=0x000000080781c400, client=0x00000008078df480, msg=0x00000008078791d0) + 784 at virnetserverprogram.c:437 frame #10: 0x0000000801c26641 libvirt.so.1002`virNetServerProgramDispatch(prog=0x000000080790f400, server=0x000000080781c400, client=0x00000008078df480, msg=0x00000008078791d0) + 705 at virnetserverprogram.c:307 frame #11: 0x000000000108f97c libvirtd`virNetServerProcessMsg(srv=0x000000080781c400, client=0x00000008078df480, prog=0x000000080790f400, msg=0x00000008078791d0) + 332 at virnetserver.c:172 frame #12: 0x000000000108cd9c libvirtd`virNetServerHandleJob(jobOpaque=0x0000000807934360, opaque=0x000000080781c400) + 204 at virnetserver.c:193 frame #13: 0x0000000801ac84b8 libvirt.so.1002`virThreadPoolWorker(opaque=0x0000000807901d60) + 680 at virthreadpool.c:145 frame #14: 0x0000000801ac7b9b libvirt.so.1002`virThreadHelper(data=0x000000080781b520) + 75 at virthread.c:197 frame #15: 0x0000000803c614f5 libthr.so.3`??? + 277 -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 21:36:49 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AA397D8A; Tue, 10 Jun 2014 21:36:49 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB92E25AF; Tue, 10 Jun 2014 21:36:48 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id b8so4271905lan.23 for ; Tue, 10 Jun 2014 14:36:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UirTTnAYIlu17tjb4zygEEj05bq+JUIYUEyrVxnvvb8=; b=AVdHK0CmdzqV5tBjE3mmkBqMeFnSPnyPyV1kgzGwcVF9cTdkTgd1sakOOIJ3bFgBKh N+xMZTiAug9Wyn4smqjho8sp8XQIbPUe3yskoWXcpIdTHZZOYWdISM4Iij6WT3hpvEnC 0FIuSTZxRx5ubEUGs0UuyH85NfMszcsFMEHMw48C4CoUNbR6zpaDNtWtRF5Gm/YDRyHx sEYCku692xVc4aRfHH+FNr0C1cBdhzUDQtElBzTxf/cwDR9dcPOwSQ1ip4AyXgZqyJpb bdFuDA/D0KKrFO+868x0tKHw/P8Nc6COWtoNbnkKPNZ5AK96VkyLW8l/P9lCwOBFQcdw +SLQ== MIME-Version: 1.0 X-Received: by 10.152.29.168 with SMTP id l8mr24364001lah.35.1402436206546; Tue, 10 Jun 2014 14:36:46 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 14:36:46 -0700 (PDT) In-Reply-To: References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> Date: Tue, 10 Jun 2014 14:36:46 -0700 X-Google-Sender-Auth: ODkdSs4u1jWrrW_ZwEaSZ3hbxz0 Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 21:36:49 -0000 On Tue, Jun 10, 2014 at 1:54 PM, Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues wrote: > lldb /usr/local/sbin/libvirtd libvirtd.core > > (lldb) bt > * thread #1: tid = 0, 0x0000000801aa8513 > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x000000080a4211c8, > ifname=0x000000080a88d1e0, macaddr=0x000000080a88d144, > vmuuid=0x000000080a81b808, tapfd=0x0000000000000000, tapfdSize=1, > virtPortProfile=0x0000000000000000, virtVlan=0x0000000000000000, > flags=9) + 547 at virnetdevtap.c:583, name = 'libvirtd', stop reason = > signal SIGSEGV > * frame #0: 0x0000000801aa8513 > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x000000080a4211c8, > ifname=0x000000080a88d1e0, macaddr=0x000000080a88d144, > vmuuid=0x000000080a81b808, tapfd=0x0000000000000000, tapfdSize=1, > virtPortProfile=0x0000000000000000, virtVlan=0x0000000000000000, > flags=9) + 547 at virnetdevtap.c:583 > frame #1: 0x0000000809e20569 > libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=0x000000080a81b800, > cmd=0x000000080a4db200, dryRun=false) + 841 at bhyve_command.c:82 > frame #2: 0x0000000809e20196 > libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=0x000000080a48a040, > def=0x000000080a81b800, dryRun=false) + 310 at bhyve_command.c:248 > frame #3: 0x0000000809e24d6a > libvirt_driver_bhyve.so`virBhyveProcessStart(conn=0x000000080a4db100, > driver=0x000000080a48a040, vm=0x000000080a48b0f0, > reason=VIR_DOMAIN_RUNNING_BOOTED, flags=0) + 746 at > bhyve_process.c:134 > frame #4: 0x0000000809e2334a > libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=0x000000080a48a280, > flags=0) + 490 at bhyve_driver.c:847 > frame #5: 0x0000000809e2315a > libvirt_driver_bhyve.so`bhyveDomainCreate(dom=0x000000080a48a280) + 26 > at bhyve_driver.c:867 > frame #6: 0x0000000801b96aad > libvirt.so.1002`virDomainCreate(domain=0x000000080a48a280) + 637 at > libvirt.c:8968 > frame #7: 0x0000000001075427 > libvirtd`remoteDispatchDomainCreate(server=0x000000080781c400, > client=0x00000008078df480, msg=0x00000008078791d0, > rerr=0x00007fffff9fcc30, args=0x0000000807934340) + 231 at > remote_dispatch.h:3048 > frame #8: 0x000000000103b84e > libvirtd`remoteDispatchDomainCreateHelper(server=0x000000080781c400, > client=0x00000008078df480, msg=0x00000008078791d0, > rerr=0x00007fffff9fcc30, args=0x0000000807934340, > ret=0x000000080a421058) + 254 at remote_dispatch.h:3026 > frame #9: 0x0000000801c26b20 > libvirt.so.1002`virNetServerProgramDispatchCall(prog=0x000000080790f400, > server=0x000000080781c400, client=0x00000008078df480, > msg=0x00000008078791d0) + 784 at virnetserverprogram.c:437 > frame #10: 0x0000000801c26641 > libvirt.so.1002`virNetServerProgramDispatch(prog=0x000000080790f400, > server=0x000000080781c400, client=0x00000008078df480, > msg=0x00000008078791d0) + 705 at virnetserverprogram.c:307 > frame #11: 0x000000000108f97c > libvirtd`virNetServerProcessMsg(srv=0x000000080781c400, > client=0x00000008078df480, prog=0x000000080790f400, > msg=0x00000008078791d0) + 332 at virnetserver.c:172 > frame #12: 0x000000000108cd9c > libvirtd`virNetServerHandleJob(jobOpaque=0x0000000807934360, > opaque=0x000000080781c400) + 204 at virnetserver.c:193 > frame #13: 0x0000000801ac84b8 > libvirt.so.1002`virThreadPoolWorker(opaque=0x0000000807901d60) + 680 > at virthreadpool.c:145 > frame #14: 0x0000000801ac7b9b > libvirt.so.1002`virThreadHelper(data=0x000000080781b520) + 75 at > virthread.c:197 > frame #15: 0x0000000803c614f5 libthr.so.3`??? + 277 > > -- > Craig OK, I think I found the problem. In this file: http://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/bhyve/bhyve_command.c;hb=HEAD We have: static int bhyveBuildNetArgStr(const virDomainDef *def, virCommandPtr cmd, bool dryRun) { virDomainNetDefPtr net = NULL; char *brname = NULL; char *realifname = NULL; int *tapfd = NULL; tapfd is never initialized after this. In the same function, we have this: if (!dryRun) if (virNetDevTapCreateInBridgePort(brname, &net->ifname, &net->mac, def->uuid, tapfd, 1, virDomainNetGetActualVirtPortProfile(net), virDomainNetGetActualVlan(net), VIR_NETDEV_TAP_CREATE_IFUP | VIR_NETDEV_TAP_CREATE_PERSIST) < 0) { VIR_FREE(net->ifname); VIR_FREE(brname); return -1; } virNetDevTapCreateinBridgePort() is being passed a NULL tapfd, but the next parameter is 1 instead of 0. Then further along in this file: http://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/util/virnetdevtap.c;hb=HEAD we have: error: for (i = 0; i < tapfdSize && tapfd[i] >= 0; i++) VIR_FORCE_CLOSE(tapfd[i]); tapfd is NULL, but tapfdSize is 1, so the coredump happens here. What is the best way to fix this? -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Jun 10 22:06:53 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C0DEACD; Tue, 10 Jun 2014 22:06:53 +0000 (UTC) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A0B722838; Tue, 10 Jun 2014 22:06:52 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id 10so4312432lbg.16 for ; Tue, 10 Jun 2014 15:06:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=ecHGgpND28V6CDN8VOuN19Pb+qKUEkUmP5kCA8nWToI=; b=I0dxS8ZjpI0BlTa9MVoSg2TTRfZC0GJ43aoeCLGpQ0J9Jk0MKobmrc2qos0SpOD1+l XXBO7z1rf9O7CBp0OnOEI35wk9xkWN6hRI0FIJ0qmxdazBNj3MkE4pUrdAeIWWZ+k0YR 46YKfXHQHmooTV1WvogxhVik5BnxHefhunHjCshBJlDmezvh/sJcuJs6Ousm4odwdzXR EGkv36g3qML1uRpBCSzW3NsV9Uf5cyIJbIe1mG9CewiblWYMrjzpCwDwDBF6H8Q5gMy0 GazEcnp3R4bLaLtI1/aAOPE8py8/4iLQZDBKeuWYy1hjf4ec6+PYcr4GxgeZ6g4sqHN+ vgPA== MIME-Version: 1.0 X-Received: by 10.112.159.7 with SMTP id wy7mr23506498lbb.4.1402438010489; Tue, 10 Jun 2014 15:06:50 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 15:06:50 -0700 (PDT) Date: Tue, 10 Jun 2014 15:06:50 -0700 X-Google-Sender-Auth: jb-eo8nJv_hNezTD0rIyUbXRABs Message-ID: Subject: Best place to learn how to write tests which work under kyua? From: Craig Rodrigues To: "freebsd-testing@freebsd.org" , Julio Merino Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 22:06:53 -0000 Hi, Where is the best place to learn about starting from a clean slate, writing new tests which execute under kyua, and how to structure the Kyuafile? Specifically, I want to use kyua in my own project, which is not part of the FreeBSD source tree. I looked at these links: http://wiki.netbsd.org/kyua/ https://wiki.freebsd.org/TestingFreeBSD https://wiki.freebsd.org/TestSuite http://www.bsdcan.org/2014/schedule/events/462.en.html While those links gave good general overviews of kyua, and also how to use it within the structure of /usr/tests on FreeBSD, I did not get a clear idea of how to use kyua in my own project. The only thing that started to clear things up for me was by looking in: src/share/examples/tests/tests/plain/ and reading some of the kyua man pages. Is there another source of information that I can look at for how to write tests which work under kyua -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 01:49:33 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9FF0D7C0; Wed, 11 Jun 2014 01:49:33 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 01FFB2970; Wed, 11 Jun 2014 01:49:32 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id z11so4469361lbi.13 for ; Tue, 10 Jun 2014 18:49:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=2tIZ+aoo/Bri5aAOd3kDwgXKynS/KPCQjc3SumDS+lk=; b=NUgxedfujtQqBWoakyyxHbWu/hPOZGsAmRSbZV7h4SGXpoD7Zhlj7zMA9Xm+4weL12 QNtrqE8NayFrQHG+k4Mbr6Oxsb/5iXZ35FCp59CgTjqXDQlIhs1QUYKVxcfGjfstpHQU e8fX5EjBYTQRHsoWWKJ5HpB3U1ljNX6t+5FmuZ+Wt3+Lt7GqtSf75y8yMlajVJIOjh1C Uv5vhuA8ZWdPt31Nuego4Wi2NqjXvyx1iDF0mCGruC3Sb5o9HLAaq27jZLtpemldV3Af mxFdYhmiYTsUR/xSBiLoTBv9d0GfjGfCeL7FqmxUjG5c9SdmPqPbF/sRC+vHIXsphFML D66A== MIME-Version: 1.0 X-Received: by 10.112.159.7 with SMTP id wy7mr24201563lbb.4.1402451370597; Tue, 10 Jun 2014 18:49:30 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 10 Jun 2014 18:49:30 -0700 (PDT) Date: Tue, 10 Jun 2014 18:49:30 -0700 X-Google-Sender-Auth: dbIhXntFGtS8JRwky9CSHlRoVAw Message-ID: Subject: Kyua + Jenkins integration From: Craig Rodrigues To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 Cc: "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 01:49:33 -0000 Hi, At BSDCan 2014, I met with Julio Merino, and we discussed how we could better integrate Kyua and Jenkins. To summarize, Julio modified Kyua, so that someone can do: kyua test (runs the tests) kyua report-junit (creates test output in JUnit XML format) The JUnit XML can be directly imported into Jenkins, which has a UI for reporting the status of tests. I ran the Kyua tests in the /usr/tests directory in FreeBSD and imported the output into Jenkins here: https://jenkins.freebsd.org/jenkins/job/Kyua_Integration/5/testReport/ This is a significant milestone! We will definitely build on this moving forward Thanks to Julio for working on the code, and blogging about it here: http://julipedia.meroh.net/2014/05/refocusing-kyua-maybe.html -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 02:00:47 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F2B4A81 for ; Wed, 11 Jun 2014 02:00:47 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 2833C2A1D for ; Wed, 11 Jun 2014 02:00:46 +0000 (UTC) Received: from [10.1.1.2] (S01060001abad1dea.hm.shawcable.net [50.70.146.73]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id B4D8E85DF2 for ; Wed, 11 Jun 2014 02:00:45 +0000 (UTC) Message-ID: <5397B84E.302@freebsd.org> Date: Tue, 10 Jun 2014 22:00:46 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-testing@freebsd.org Subject: Re: [HEADSUP] Kyua + Jenkins integration: milestone 1 achieved References: In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="p3232W86LWpoNnuS0N7VUmmV9sB9ocwU3" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 02:00:47 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --p3232W86LWpoNnuS0N7VUmmV9sB9ocwU3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-06-10 21:53, Craig Rodrigues wrote: > Hi, >=20 > FYI: >=20 > http://lists.freebsd.org/pipermail/freebsd-testing/2014-June/000409.htm= l >=20 > Please send followups to freebsd-testing@freebsd.org >=20 > -- > Craig >=20 Some of the test, like this one: https://jenkins.freebsd.org/jenkins/job/Kyua_Integration/5/testReport/usr= =2Ebin.truncate/truncate_test/bad_truncate/ Are failing because of ZFS. In this case, chflags uimmutable cannot be done on ZFS. Likely a problem with the kyua test, but, it explains that specific failu= re. --=20 Allan Jude --p3232W86LWpoNnuS0N7VUmmV9sB9ocwU3 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTl7hRAAoJEJrBFpNRJZKfSA0P/AmYOtbi6HQFe0A+R1kF3z/X 8nMDcVA3hw9MproOpnClhQdwsXqb5/SWi27AHg/ERjjeGPpYTcSF373034MxsXeC 4KX9cH4yjHy3X9uMkgwgC4svCy7M8EK4k/+4MzpHXLxoBGEVo1t3K3phuYWhg7uK RPNUqwVVBPQbYz+rPiPyl4K0gCfi8SqDW022+h+l8CyYqdsmaap9tt4+5OIIdYnl +/MYDRKLTnZVsd1jetfFqjZhL0N8QyVxwkMyqsMvftb78ohynJJCS2/fWDhwQHqV h+uzdHW640Yv0lwryjk88SK2uamuTmSW2tDsH3g9DvSTiwicaVUCOgy1mQvNGHXC Ksrnxr2kWDmZlL3PLKUZR9DQ/tpxfACcKfTvxOwOnPwXJq9hEMs1hd745BHe8dyg dlHeE+uL21r96R5ePh8y8JM77VhyOh8wluxiL+nuv45+GM9+QdwizqSQ/1krdQKu PvkXRAzj55gYaUEbQGtmfUmVNFPAqATKExmL/Zsk8Mc6x0ZW0eBIPGAkiEzrJyky Q0S274Tk2YFhqU4qcDRxrrHaaNUYfBA4VYEnl+U7xiWH2yWvRXp4wL5alMPrkuOn 3YRu23MBQhUUdoAg7rGRjZ1FR0aDpIMc7bU03OUVN1CoZDxcIPHXIXSsqdrCjEak Dsxs4TrRWKJ1CPWRYPc5 =MMPC -----END PGP SIGNATURE----- --p3232W86LWpoNnuS0N7VUmmV9sB9ocwU3-- From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 03:44:33 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32F7FE98; Wed, 11 Jun 2014 03:44:33 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 77F1E229F; Wed, 11 Jun 2014 03:44:32 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id k14so7289288wgh.6 for ; Tue, 10 Jun 2014 20:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=fPZEVqa6XF5mni4SmEPCPmRYtm3KelAWAV62uDkDswY=; b=KIYg0I2tkkQKq13Rllg6z3HJmTBiAPTpF5GfPWHGdsAwGTQWfSdOoZ5P6vjzIODMIt St0dr334F4k32FmaE+jM8zHQbeg+0Oa2TcNJsP0UvPBndzaXUpPEq8oey1TJqHMyMieF IE+W307F65vW1ULUraxopVnee4ytPFMIT5Lg2P9CBiKNqoGY6UTm93svPXIyS9R4KAB/ 2fMrsC3l9oRWzeityOUQpF1YbmiQD6bt8kgUVw15UEH9DSYOYX5ZyAZyyyFKgEMY5Dkj fiGFSm/TU6rccINX7E8uQOKPOgq9/oxFKka+w7EZmVJPzd+kDQ+of2W3O/1j+hIkUi3j mx4g== MIME-Version: 1.0 X-Received: by 10.180.101.98 with SMTP id ff2mr43803610wib.40.1402458270721; Tue, 10 Jun 2014 20:44:30 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.202 with HTTP; Tue, 10 Jun 2014 20:44:30 -0700 (PDT) In-Reply-To: References: Date: Tue, 10 Jun 2014 21:44:30 -0600 X-Google-Sender-Auth: ApBQkKp4sReFzqK4Qjtw95DbvH8 Message-ID: Subject: Re: Kyua + Jenkins integration From: Alan Somers To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 03:44:33 -0000 On Tue, Jun 10, 2014 at 7:49 PM, Craig Rodrigues wrote: > Hi, > > At BSDCan 2014, I met with Julio Merino, > and we discussed how we could better integrate Kyua and Jenkins. > > To summarize, Julio modified Kyua, so that someone can do: > > kyua test (runs the tests) > kyua report-junit (creates test output in JUnit XML format) > > The JUnit XML can be directly imported into Jenkins, > which has a UI for reporting the status of tests. > > I ran the Kyua tests in the /usr/tests directory in FreeBSD > and imported the output into Jenkins here: > > https://jenkins.freebsd.org/jenkins/job/Kyua_Integration/5/testReport/ > > > This is a significant milestone! > We will definitely build on this moving forward > > Thanks to Julio for working on the code, and blogging about it here: > > http://julipedia.meroh.net/2014/05/refocusing-kyua-maybe.html > > -- > Craig > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" Nice! Also, the "diff" columns will be very useful, if they do what I think they do. But I do see one problem: there's no message for why a test was skipped. Plain "kyua report" will give an explanation like "skipped: required configuration variable Foo was undefined". But that doesn't show up in the Jenkins output. -Alan From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 10:23:50 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E0F5C614 for ; Wed, 11 Jun 2014 10:23:50 +0000 (UTC) Received: from mail-qc0-f169.google.com (mail-qc0-f169.google.com [209.85.216.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A0276268D for ; Wed, 11 Jun 2014 10:23:50 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id c9so4221294qcz.0 for ; Wed, 11 Jun 2014 03:23:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=9RPCDr1xqxSUbCiTstEia9dr629q3dr67eowf0B4WAc=; b=GcGaHTpE2x7+aHJvh3pYAVqOnSMea9Nmp22LEpRDsaYsbV1xPNk4gBa2YI2qcQz4j/ zpZNCCyPuTbPkL1WWrpcGUz/OnYN6n4GByL/MvW6EjN9wP7+OdhysWatQWmw795HPZVS oHv7p4Esi2v7RNA/OIa/4Uta8fG3xycRXwyCgsZIAguk+gv9vhJygYvpZ6p9Ctwjr3dg NnhLbSnVBzShteYqT55yJ6NaocF82be70g38bSgwUneDD/GrUYzqLInCjT3l3qIYfodg veCwb9nWFQKzLPsx7uXbaq9pPbNFq7OSYO3f1cisdgBzInbkdvElJKkYsqDRHSUtzFLf zlhQ== X-Gm-Message-State: ALoCoQkUCMcmQpUXKjMHfF1IyxQOFP2XbK5lKJ2TxxgJ4rsPx5C40kMYVBxaEJpUBs1AGtGfNCpA X-Received: by 10.224.152.5 with SMTP id e5mr29087068qaw.65.1402482222686; Wed, 11 Jun 2014 03:23:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.83.99 with HTTP; Wed, 11 Jun 2014 03:23:22 -0700 (PDT) X-Originating-IP: [2620:0:1040:404:c910:9fd5:8657:ec41] In-Reply-To: References: From: Julio Merino Date: Wed, 11 Jun 2014 11:23:22 +0100 Message-ID: Subject: Re: Kyua + Jenkins integration To: Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 10:23:51 -0000 On Wed, Jun 11, 2014 at 4:44 AM, Alan Somers wrote: > Nice! Also, the "diff" columns will be very useful, if they do what I > think they do. But I do see one problem: there's no message for why a > test was skipped. Plain "kyua report" will give an explanation like > "skipped: required configuration variable Foo was undefined". But > that doesn't show up in the Jenkins output. It does now with: https://github.com/jmmv/kyua/commit/7acd1a1fa05e600ac41094857ee1344f4ae7adb5 Thanks for reporting. From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 18:23:16 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14E1797C; Wed, 11 Jun 2014 18:23:16 +0000 (UTC) Received: from mail-la0-x22c.google.com (mail-la0-x22c.google.com [IPv6:2a00:1450:4010:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 222192557; Wed, 11 Jun 2014 18:23:14 +0000 (UTC) Received: by mail-la0-f44.google.com with SMTP id hr17so72994lab.31 for ; Wed, 11 Jun 2014 11:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=2bXKTQjGBs4PbPFOr1Va9Rjgi4hItZb44w0vqmOpPMo=; b=hk529SRVnDtGWRJUp6Gy9iv/hZ+NU9l2zdt1Vb+BzC1ritE08Bd9r+5OZwx2nO82+v je48n3X7iL3h+ldbhmzo62Svh1e9doNVTQOEIv8hkFry5FvlKc+0rfvrws/3SdUb87oh C+IT5V6MvNQ434fmp1IFTnUvnNN8Sc7ABHX2VRGi7ThLUhH6cQu5snbFTNIJEwtwzbxB r2hfku985FQDrCRJE1ZRgVtcWHig9rjt4y3m+HOeoKUC7CM8OxpWIF7N3K5TzvCd+wgn wpcr0wcDU9y0YJJD1ARsNkHnAqmDCfVSXuiWVdXFjFUEbk2LTwBEf/4tHxhcXBM1aG9D 5jMw== X-Received: by 10.152.26.99 with SMTP id k3mr2828239lag.68.1402510992576; Wed, 11 Jun 2014 11:23:12 -0700 (PDT) Received: from kloomba ([95.104.132.28]) by mx.google.com with ESMTPSA id d9sm22196463lag.19.2014.06.11.11.23.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jun 2014 11:23:11 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 11 Jun 2014 22:21:02 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140611182101.GD2855@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W5WqUoFLvi1M7tJE" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 18:23:16 -0000 --W5WqUoFLvi1M7tJE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 1:54 PM, Craig Rodrigues wr= ote: > > On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues = wrote: > > lldb /usr/local/sbin/libvirtd libvirtd.core > > > > (lldb) bt > > * thread #1: tid =3D 0, 0x0000000801aa8513 > > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=3D0x000000080a421= 1c8, > > ifname=3D0x000000080a88d1e0, macaddr=3D0x000000080a88d144, > > vmuuid=3D0x000000080a81b808, tapfd=3D0x0000000000000000, tapfdSize=3D1, > > virtPortProfile=3D0x0000000000000000, virtVlan=3D0x0000000000000000, > > flags=3D9) + 547 at virnetdevtap.c:583, name =3D 'libvirtd', stop reaso= n =3D > > signal SIGSEGV > > * frame #0: 0x0000000801aa8513 > > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=3D0x000000080a421= 1c8, > > ifname=3D0x000000080a88d1e0, macaddr=3D0x000000080a88d144, > > vmuuid=3D0x000000080a81b808, tapfd=3D0x0000000000000000, tapfdSize=3D1, > > virtPortProfile=3D0x0000000000000000, virtVlan=3D0x0000000000000000, > > flags=3D9) + 547 at virnetdevtap.c:583 > > frame #1: 0x0000000809e20569 > > libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=3D0x000000080a81b800, > > cmd=3D0x000000080a4db200, dryRun=3Dfalse) + 841 at bhyve_command.c:82 > > frame #2: 0x0000000809e20196 > > libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=3D0x0000000= 80a48a040, > > def=3D0x000000080a81b800, dryRun=3Dfalse) + 310 at bhyve_command.c:248 > > frame #3: 0x0000000809e24d6a > > libvirt_driver_bhyve.so`virBhyveProcessStart(conn=3D0x000000080a4db100, > > driver=3D0x000000080a48a040, vm=3D0x000000080a48b0f0, > > reason=3DVIR_DOMAIN_RUNNING_BOOTED, flags=3D0) + 746 at > > bhyve_process.c:134 > > frame #4: 0x0000000809e2334a > > libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=3D0x000000080a48= a280, > > flags=3D0) + 490 at bhyve_driver.c:847 > > frame #5: 0x0000000809e2315a > > libvirt_driver_bhyve.so`bhyveDomainCreate(dom=3D0x000000080a48a280) + 26 > > at bhyve_driver.c:867 > > frame #6: 0x0000000801b96aad > > libvirt.so.1002`virDomainCreate(domain=3D0x000000080a48a280) + 637 at > > libvirt.c:8968 > > frame #7: 0x0000000001075427 > > libvirtd`remoteDispatchDomainCreate(server=3D0x000000080781c400, > > client=3D0x00000008078df480, msg=3D0x00000008078791d0, > > rerr=3D0x00007fffff9fcc30, args=3D0x0000000807934340) + 231 at > > remote_dispatch.h:3048 > > frame #8: 0x000000000103b84e > > libvirtd`remoteDispatchDomainCreateHelper(server=3D0x000000080781c400, > > client=3D0x00000008078df480, msg=3D0x00000008078791d0, > > rerr=3D0x00007fffff9fcc30, args=3D0x0000000807934340, > > ret=3D0x000000080a421058) + 254 at remote_dispatch.h:3026 > > frame #9: 0x0000000801c26b20 > > libvirt.so.1002`virNetServerProgramDispatchCall(prog=3D0x000000080790f4= 00, > > server=3D0x000000080781c400, client=3D0x00000008078df480, > > msg=3D0x00000008078791d0) + 784 at virnetserverprogram.c:437 > > frame #10: 0x0000000801c26641 > > libvirt.so.1002`virNetServerProgramDispatch(prog=3D0x000000080790f400, > > server=3D0x000000080781c400, client=3D0x00000008078df480, > > msg=3D0x00000008078791d0) + 705 at virnetserverprogram.c:307 > > frame #11: 0x000000000108f97c > > libvirtd`virNetServerProcessMsg(srv=3D0x000000080781c400, > > client=3D0x00000008078df480, prog=3D0x000000080790f400, > > msg=3D0x00000008078791d0) + 332 at virnetserver.c:172 > > frame #12: 0x000000000108cd9c > > libvirtd`virNetServerHandleJob(jobOpaque=3D0x0000000807934360, > > opaque=3D0x000000080781c400) + 204 at virnetserver.c:193 > > frame #13: 0x0000000801ac84b8 > > libvirt.so.1002`virThreadPoolWorker(opaque=3D0x0000000807901d60) + 680 > > at virthreadpool.c:145 > > frame #14: 0x0000000801ac7b9b > > libvirt.so.1002`virThreadHelper(data=3D0x000000080781b520) + 75 at > > virthread.c:197 > > frame #15: 0x0000000803c614f5 libthr.so.3`??? + 277 > > > > -- > > Craig >=20 >=20 >=20 > OK, I think I found the problem. >=20 > In this file: http://libvirt.org/git/?p=3Dlibvirt.git;a=3Dblob_plain;f=3D= src/bhyve/bhyve_command.c;hb=3DHEAD >=20 > We have: >=20 > static int > bhyveBuildNetArgStr(const virDomainDef *def, virCommandPtr cmd, bool dryR= un) > { > virDomainNetDefPtr net =3D NULL; > char *brname =3D NULL; > char *realifname =3D NULL; > int *tapfd =3D NULL; >=20 >=20 > tapfd is never initialized after this. >=20 >=20 > In the same function, we have this: >=20 > if (!dryRun) > if (virNetDevTapCreateInBridgePort(brname, &net->ifname, &net= ->mac, > def->uuid, tapfd, 1, >=20 > virDomainNetGetActualVirtPortProfile(net), > virDomainNetGetActualVlan(= net), >=20 > VIR_NETDEV_TAP_CREATE_IFUP | VIR_NETDEV_TAP_CREATE_PERSIST) < 0) { > VIR_FREE(net->ifname); > VIR_FREE(brname); > return -1; > } >=20 >=20 >=20 > virNetDevTapCreateinBridgePort() is being passed a NULL tapfd, but the > next parameter is 1 instead of 0. >=20 >=20 >=20 > Then further along in this file: > http://libvirt.org/git/?p=3Dlibvirt.git;a=3Dblob_plain;f=3Dsrc/util/virne= tdevtap.c;hb=3DHEAD > we have: >=20 > error: > for (i =3D 0; i < tapfdSize && tapfd[i] >=3D 0; i++) > VIR_FORCE_CLOSE(tapfd[i]); >=20 >=20 > tapfd is NULL, but tapfdSize is 1, so the coredump happens here. >=20 > What is the best way to fix this? Thanks for the analysis! I'll think what would be a proper fix for that problem. Also, while thinking how to reproduce it, I did 'kldunload if_bridge' and started to see the same problem. So, as a temporary fix, could you check if you have bridge support available (in kernel or via module)? Bridges are needed to get networking anyway. Thanks, Roman Bogorodskiy --W5WqUoFLvi1M7tJE Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmJ4NAAoJEMltX/4IwiJqXJUH/j+Oy+peyJse6WJYItF+CHnF DfQcz9YsuPxEOrFJIHMVz6MzTbfyXqXaxYLPtMfa0ly8qqgBuGWW3sCnQtYeZokG UxroKlFVKdgxBx5FlByNTHUn0Onj3sqK3oECp6lSKk/3e9HXTFUTsNF2q/5dueDN 66axiddJA4aKhb4RxxOsjBKU/jLSmDATfGp6d3GUentp7tqs4LGlgle4llNd+zNn LjFSN5ncXnCSKiQ3tCWqtGlBUQW5VfY8MhA5EeZe/8Hp65OxNurwDi7eWHdUQQD9 BfxysIiJiFWM1cJ2LBwTsdpbjxYv8JcuVvxLOB7epYQB+PJKQSxYJXNypn/WEWk= =lg/Q -----END PGP SIGNATURE----- --W5WqUoFLvi1M7tJE-- From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 18:56:36 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 031B4B63; Wed, 11 Jun 2014 18:56:36 +0000 (UTC) Received: from mail-lb0-x231.google.com (mail-lb0-x231.google.com [IPv6:2a00:1450:4010:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22170287B; Wed, 11 Jun 2014 18:56:34 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id u10so95405lbd.8 for ; Wed, 11 Jun 2014 11:56:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=8pKE9HiuPvhKU2zujnAsjeRGxu/3KEypiPBsZRMqPTQ=; b=wx7jtQziIOBlGyF88xaooPwM/JnhyHgysXpwj+80dxr4WpJid3EQrtpO0xcCr08aIO P7+As9Rg3BZrXYn4fBZRIGOPYO//qBqS7KIeXXGVIE3qviAecvFsFdyuz4rIBqO+cmyG ZI5TXTlfuqS5ou4VojIdytN+2oAvRLPIgESy4BsKG7xErJKABcVeYFleZJD0pCAJ7Dst 6pOf7lFuFp1Xp6hFbdAfVN38wbphM2pCzzYzTTbCdl9yH8evXHroWnNYpl5v8T8b2LgY L2xXmdFVJhDtXtOEBNsL5OqXwGqnM2NwMdeneKuJrXhqZxjgrTrV66Q3N4yK0IoFtdgU bmkQ== MIME-Version: 1.0 X-Received: by 10.112.129.135 with SMTP id nw7mr51908lbb.89.1402512992942; Wed, 11 Jun 2014 11:56:32 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 11:56:32 -0700 (PDT) In-Reply-To: <20140611182101.GD2855@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> Date: Wed, 11 Jun 2014 11:56:32 -0700 X-Google-Sender-Auth: NswZkeS0f7_sp_agdBO02X2c80Y Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 18:56:36 -0000 On Wed, Jun 11, 2014 at 11:21 AM, Roman Bogorodskiy wrote: > > Thanks for the analysis! I'll think what would be a proper fix for that > problem. > > Also, while thinking how to reproduce it, I did 'kldunload if_bridge' > and started to see the same problem. So, as a temporary fix, could you > check if you have bridge support available (in kernel or via module)? > > Bridges are needed to get networking anyway. Hi, My output of kldstat is: 1 29 0xffffffff80200000 19d6218 kernel 2 1 0xffffffff81bd7000 2e44b0 zfs.ko 3 2 0xffffffff81ebc000 8208 opensolaris.ko 4 1 0xffffffff81ec5000 4d68 nmdm.ko 5 1 0xffffffff81eca000 1a2110 vmm.ko 6 1 0xffffffff82211000 53e7 fdescfs.ko 7 1 0xffffffff82217000 9aff if_bridge.ko 8 1 0xffffffff82221000 5851 bridgestp.ko 9 1 0xffffffff82227000 5673 if_tap.ko 10 1 0xffffffff8222d000 2a94 uhid.ko 11 1 0xffffffff82230000 3592 ums.ko so I do have if_bridge.ko loaded. If you are doing development on CURRENT, make sure that you have WITH_LLDB=yes in /etc/src.conf when you rebuild the system. Looks like gdb in base cannot debug the cores. -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 19:27:30 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84AA457D; Wed, 11 Jun 2014 19:27:30 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C2E82B96; Wed, 11 Jun 2014 19:27:29 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id pv20so120323lab.13 for ; Wed, 11 Jun 2014 12:27:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=TEvaFTXrP5NiWlUZ7wBDhy7D9Mhb3V+3Ga8Xm6sjeGY=; b=YraM6fIw6W4qVjOJM1l6SYIevPwEJHliqA5O7EWavc2+B4MAaBaanP1lifR2toqFHl rSmcRQOTrP9yBAZMvSS2MThHxn/ue7jYVYh9m14ksnOug4DENuBrLIbobPnnkoDNMj2v P7zp6/yNPFqFRJwh+KSz09LJWfJI7B5wuZ4ordcytBUpnteisngHoLHb1xiLhS5Pq4oU 3jiyHrUGQfFtjc0ql/2XyG5TeJNMasuYGKw2yj8Ro27Vjf3qZL7StfsvZmmBtZEvkuR/ InW1FQsDDEDT89FpWJZCFnGLAV1vSQH9T4Kpmv6kEziNb3Zoew41cYW31DrG5LW3UP8h mMKA== X-Received: by 10.152.23.136 with SMTP id m8mr29195227laf.2.1402514847517; Wed, 11 Jun 2014 12:27:27 -0700 (PDT) Received: from kloomba ([95.104.132.28]) by mx.google.com with ESMTPSA id bg4sm25730885lbb.6.2014.06.11.12.27.25 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jun 2014 12:27:26 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 11 Jun 2014 23:25:17 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140611192514.GE2855@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lkTb+7nhmha7W+c3" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 19:27:30 -0000 --lkTb+7nhmha7W+c3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Wed, Jun 11, 2014 at 11:21 AM, Roman Bogorodskiy w= rote: > > > > Thanks for the analysis! I'll think what would be a proper fix for that > > problem. > > > > Also, while thinking how to reproduce it, I did 'kldunload if_bridge' > > and started to see the same problem. So, as a temporary fix, could you > > check if you have bridge support available (in kernel or via module)? > > > > Bridges are needed to get networking anyway. >=20 >=20 > Hi, >=20 > My output of kldstat is: >=20 > 1 29 0xffffffff80200000 19d6218 kernel > 2 1 0xffffffff81bd7000 2e44b0 zfs.ko > 3 2 0xffffffff81ebc000 8208 opensolaris.ko > 4 1 0xffffffff81ec5000 4d68 nmdm.ko > 5 1 0xffffffff81eca000 1a2110 vmm.ko > 6 1 0xffffffff82211000 53e7 fdescfs.ko > 7 1 0xffffffff82217000 9aff if_bridge.ko > 8 1 0xffffffff82221000 5851 bridgestp.ko > 9 1 0xffffffff82227000 5673 if_tap.ko > 10 1 0xffffffff8222d000 2a94 uhid.ko > 11 1 0xffffffff82230000 3592 ums.ko >=20 >=20 > so I do have if_bridge.ko loaded. I've attached a patch which should fix the segfault. Could you please let me know if it fixes the problem? However, we still need to figure out why networking is not being enabled. The root cause of that is that libvirt should start the default network when the daemon starts up. This network should create a bridge device (virbr0). Later on, when one starts a guest, it should create tap device and add it to the bridge. Obviously, it fails at this step as there's no bridge device. Could you please send me config.log file and also a port directory with your fixes so I could test it as well? As for your previous question on my setup, I'm using master branch of the libvirt git repo and my configure args are: $ ./configure --with-bhyve --without-polkit --with-hal \ CFLAGS=3D"-g -O0 -I/usr/local/include" LDFLAGS=3D-L/usr/local/lib \ --with-xml-catalog-file=3D/usr/local/share/xml/catalog I'm working on 10-STABLE. However, it's really bad if it works only this way. :-( I would love to fix the port. > If you are doing development on CURRENT, make sure that you have > WITH_LLDB=3Dyes in /etc/src.conf when you rebuild the system. > Looks like gdb in base cannot debug the cores. Thanks for the hint! Roman Bogorodskiy --lkTb+7nhmha7W+c3 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmK0aAAoJEMltX/4IwiJqlIcH/2yf8/YdVZY2SYbQy6dSkrKD ZK+b3hs0Q4ja6WFefdKbwpQvXq6d8O1+6bLtjNj9IV58o+7Nx/4ZLV+uZvhgfniX 6DekqwLYPtpbsSS0sfL2rnxwY0NQaIkxp6LzTbLp/kCiXiiG+FyEHqq0g3VImtuo xWJdgqck3JiFjmkOF/QfPJGoaavgNF3M729AqNkYgTTlbODUTcyoH3Mfg8kcZ8/m ksmLnFZghPjaONtiG1dCyanRewFaNI4RyRw2arGfZBl0uWtv0VY6cGyLocPMotAC j1nA1Z7iZVDlPqDNG5dHLBPVj7QccHTc7IyZS3kZFK5BozSwoz+8tOAjMyWrgGI= =tZZz -----END PGP SIGNATURE----- --lkTb+7nhmha7W+c3-- From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 19:28:26 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4FDA56CD; Wed, 11 Jun 2014 19:28:26 +0000 (UTC) Received: from mail-lb0-x22e.google.com (mail-lb0-x22e.google.com [IPv6:2a00:1450:4010:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F5752BAC; Wed, 11 Jun 2014 19:28:25 +0000 (UTC) Received: by mail-lb0-f174.google.com with SMTP id n15so116493lbi.33 for ; Wed, 11 Jun 2014 12:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Dch8AmgtcJagUb5CBsQanX5oPCOCTV7Kqp0SjwF9/to=; b=mslAJMZBON3ABtLW3P0k0lrUd52pBdESMnw7UG6kCgdbZlCpkFjphiQOWv+grSHyyk uhsya2EJoIYFJSCJlr+h8L2qMiyQPR7GvV0tTqO7EJUmeicIojzA2ESnQlvx9NpDw+42 1AuZiLwBF7fszYLCpP3xNARj84EE6Ytet9EGs1THHKB0cSoxXigHrdKZhG1N4Bk2KSTS BKbER9conaz9kOJp/Ce3QDXD/l2qerwJIbBySYSubelRMuDv7krz96iCzIpPWMW9gnYw RXwwB+72pi34XiFS5PVJaanzTefFW2fVUeGko7sU7OwrxEvSneeEXE4a1pFM8JcEe9y6 zl6Q== X-Received: by 10.152.6.74 with SMTP id y10mr29012815lay.11.1402514903380; Wed, 11 Jun 2014 12:28:23 -0700 (PDT) Received: from kloomba ([95.104.132.28]) by mx.google.com with ESMTPSA id g3sm22329057lam.18.2014.06.11.12.28.21 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jun 2014 12:28:21 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 11 Jun 2014 23:26:13 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140611192611.GF2855@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pE2VAHO2njSJCslu" Content-Disposition: inline In-Reply-To: <20140611192514.GE2855@kloomba> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 19:28:26 -0000 --pE2VAHO2njSJCslu Content-Type: multipart/mixed; boundary="JcvBIhDvR6w3jUPA" Content-Disposition: inline --JcvBIhDvR6w3jUPA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Roman Bogorodskiy wrote: > Craig Rodrigues wrote: >=20 > > On Wed, Jun 11, 2014 at 11:21 AM, Roman Bogorodskiy = wrote: > > > > > > Thanks for the analysis! I'll think what would be a proper fix for th= at > > > problem. > > > > > > Also, while thinking how to reproduce it, I did 'kldunload if_bridge' > > > and started to see the same problem. So, as a temporary fix, could you > > > check if you have bridge support available (in kernel or via module)? > > > > > > Bridges are needed to get networking anyway. > >=20 > >=20 > > Hi, > >=20 > > My output of kldstat is: > >=20 > > 1 29 0xffffffff80200000 19d6218 kernel > > 2 1 0xffffffff81bd7000 2e44b0 zfs.ko > > 3 2 0xffffffff81ebc000 8208 opensolaris.ko > > 4 1 0xffffffff81ec5000 4d68 nmdm.ko > > 5 1 0xffffffff81eca000 1a2110 vmm.ko > > 6 1 0xffffffff82211000 53e7 fdescfs.ko > > 7 1 0xffffffff82217000 9aff if_bridge.ko > > 8 1 0xffffffff82221000 5851 bridgestp.ko > > 9 1 0xffffffff82227000 5673 if_tap.ko > > 10 1 0xffffffff8222d000 2a94 uhid.ko > > 11 1 0xffffffff82230000 3592 ums.ko > >=20 > >=20 > > so I do have if_bridge.ko loaded. >=20 > I've attached a patch which should fix the segfault. Could you please > let me know if it fixes the problem? >=20 > However, we still need to figure out why networking is not being > enabled. The root cause of that is that libvirt should start the default > network when the daemon starts up. This network should create a bridge > device (virbr0). >=20 > Later on, when one starts a guest, it should create tap device and add > it to the bridge. Obviously, it fails at this step as there's no bridge > device. >=20 > Could you please send me config.log file and also a port directory with > your fixes so I could test it as well? >=20 > As for your previous question on my setup, I'm using master branch of the > libvirt git repo and my configure args are: >=20 > $ ./configure --with-bhyve --without-polkit --with-hal \ > CFLAGS=3D"-g -O0 -I/usr/local/include" LDFLAGS=3D-L/usr/local/lib \ > --with-xml-catalog-file=3D/usr/local/share/xml/catalog >=20 > I'm working on 10-STABLE. >=20 > However, it's really bad if it works only this way. :-( I would love to > fix the port. >=20 > > If you are doing development on CURRENT, make sure that you have > > WITH_LLDB=3Dyes in /etc/src.conf when you rebuild the system. > > Looks like gdb in base cannot debug the cores. >=20 > Thanks for the hint! Sorry, forgot the actual attachment. :-( Roman Bogorodskiy --JcvBIhDvR6w3jUPA Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="libvirt_no_bridge.diff" Content-Transfer-Encoding: quoted-printable diff --git a/src/bhyve/bhyve_command.c b/src/bhyve/bhyve_command.c index 91a8731..ba7a7ec 100644 --- a/src/bhyve/bhyve_command.c +++ b/src/bhyve/bhyve_command.c @@ -44,7 +44,7 @@ bhyveBuildNetArgStr(const virDomainDef *def, virCommandPt= r cmd, bool dryRun) virDomainNetDefPtr net =3D NULL; char *brname =3D NULL; char *realifname =3D NULL; - int *tapfd =3D NULL; + int tapfd =3D -1; char macaddr[VIR_MAC_STRING_BUFLEN]; =20 if (def->nnets !=3D 1) { @@ -80,7 +80,7 @@ bhyveBuildNetArgStr(const virDomainDef *def, virCommandPt= r cmd, bool dryRun) =20 if (!dryRun) if (virNetDevTapCreateInBridgePort(brname, &net->ifname, &net-= >mac, - def->uuid, tapfd, 1, + def->uuid, &tapfd, 1, virDomainNetGetActualVirtPo= rtProfile(net), virDomainNetGetActualVlan(n= et), VIR_NETDEV_TAP_CREATE_IFUP = | VIR_NETDEV_TAP_CREATE_PERSIST) < 0) { diff --git a/src/bhyve/bhyve_process.c b/src/bhyve/bhyve_process.c index 9f02a49..5e7d603 100644 --- a/src/bhyve/bhyve_process.c +++ b/src/bhyve/bhyve_process.c @@ -77,10 +77,12 @@ bhyveNetCleanup(virDomainObjPtr vm) int actualType =3D virDomainNetGetActualType(net); =20 if (actualType =3D=3D VIR_DOMAIN_NET_TYPE_BRIDGE) { - ignore_value(virNetDevBridgeRemovePort( - virDomainNetGetActualBridgeName(net), - net->ifname)); - ignore_value(virNetDevTapDelete(net->ifname)); + if (net->ifname) { + ignore_value(virNetDevBridgeRemovePort( + virDomainNetGetActualBridgeName(net), + net->ifname)); + ignore_value(virNetDevTapDelete(net->ifname)); + } } } } --JcvBIhDvR6w3jUPA-- --pE2VAHO2njSJCslu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmK1TAAoJEMltX/4IwiJqn8UIAIUivJv+T4xtI4WegKxiSITj oKCnk55jxrhO4maLg6mZcDNTp1U3zAFIpl1+YKMyk5D9Z4hJ1xTYUu6lyNxBZk/B H9eKiC51PO7Kn0RQE3/Bv79S3pLg5ltszbvWnemCLFX4v7+QLDXgL4+ldCN9+qLc AHZ3XlCqLIXg6NZ9C44pQbXYravoUdrTGCKHiYp4pEdRXRkUJwx/kbHXB0YBdLJH 9YZDcN7VhcC/o3ky33L2u3UjDnfl4+CnIvR6JSPGOurreyuhaHi906glZws2vQA+ 5fsdqSPeDXoaTnLRO+KlO8ECwl6x/VPZsFxXOMFNMq2K2pmIbkEBipQcOmH306c= =1S7U -----END PGP SIGNATURE----- --pE2VAHO2njSJCslu-- From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 21:37:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F6ECC2 for ; Wed, 11 Jun 2014 21:37:38 +0000 (UTC) Received: from mail-ie0-x232.google.com (mail-ie0-x232.google.com [IPv6:2607:f8b0:4001:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6303F2A00 for ; Wed, 11 Jun 2014 21:37:38 +0000 (UTC) Received: by mail-ie0-f178.google.com with SMTP id rd18so347510iec.23 for ; Wed, 11 Jun 2014 14:37:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=SaPTU18Vm0aVe6TLyirQieoZZGGyTgtq4trIoW+Wnh8=; b=Y6Qxgi441gGhmEz36U+JwEQ7NBlVkZeBLnhlXyK6i5XvQgErozqV9NGqflwhLg0XTL ClEFVpaf4L4/HZmZb42GyIOrdOiDtyrcQTjUKn9SMTUeDYGSaPCrnMVhArLeE5wFqzr9 rkvti4YunfbwXYagbuTJ5u40EbLJ7kZXVxyNBI+9oebiRzzRmGw/BWt9/3FmsLNMv6tQ PdVrHbjCYCKTZnbx9T/UonMdSHX8roVWCd4TNvMfvloJJNOJuF3dFMwW1SCaLqElfqX7 Ps0rsVEX3i3lOhrj6cYLkbt1ModHa4abq1NE4JI391oq6HQQk3NFqpVoTuWREvrn6moV 08qA== MIME-Version: 1.0 X-Received: by 10.51.17.99 with SMTP id gd3mr1030237igd.2.1402522657846; Wed, 11 Jun 2014 14:37:37 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Wed, 11 Jun 2014 14:37:37 -0700 (PDT) Date: Wed, 11 Jun 2014 14:37:37 -0700 Message-ID: Subject: Generic infrastructure for detecting kernel modules [for ATF/Kyua]? From: Garrett Cooper To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 21:37:38 -0000 Hello Julio, et al! The EMC Isilon tests have some assumptions as to what drivers are loaded in the tests, and in order to bypass a lot of failed tests, I figured it would be nice to have something like 'require.device "aio"', etc in the Kyuafile/tests. I was wondering where FreeBSD-specific test infrastructure should go -- in the port(s) or in the base system? Should it be in ATF or Kyua or both? Thank you :)! -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 21:56:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D6937FA for ; Wed, 11 Jun 2014 21:56:18 +0000 (UTC) Received: from mail-ie0-x235.google.com (mail-ie0-x235.google.com [IPv6:2607:f8b0:4001:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D466A2BC5 for ; Wed, 11 Jun 2014 21:56:17 +0000 (UTC) Received: by mail-ie0-f181.google.com with SMTP id y20so362000ier.26 for ; Wed, 11 Jun 2014 14:56:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=LR/HQi1mGztpkbEFnreRR7T9D7seF6522p88gqDMZN4=; b=qRFXS8xmff9aO2ALosRM0hyx1WkMvKvMcoaMaYiEanDExZS3MhRB7Jg+cTEC19fuuO aVdEIPYO+XHsoxH4I17unHK+l9PpOtaLEl/pHzRXpvYBZRgVwgjQAPwBOc27DygfaOG6 U4wA4fZTOofnT86I0PQnS1I8++NsG4IGorXNHfL4+Rxvr9zc2qKwT0G67Yt+r5LEIBj7 PRINzueayI1o6FkXe1/iShVsAODmUOPL1BTkGt8F31Ma0zs/nBmGI+rXw+iFa2xikwTe CvPNl+EPJk1LSs1dubfgZE+EvxV11k298ckrClQsDCHlr5kVpp1wgk8HTuwZyACevtlV cF/Q== MIME-Version: 1.0 X-Received: by 10.50.138.105 with SMTP id qp9mr1149625igb.2.1402523776781; Wed, 11 Jun 2014 14:56:16 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Wed, 11 Jun 2014 14:56:16 -0700 (PDT) Date: Wed, 11 Jun 2014 14:56:16 -0700 Message-ID: Subject: Dealing with potential TEST_METADATA and atf_set duplication From: Garrett Cooper To: kyua-discuss@googlegroups.com Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 21:56:18 -0000 Hello again Julio! I'm adding testcases that require root and I was wondering: 1. Should I add TEST_METADATA.test_app+= require.user="root" to the Makefile, which puts the necessary lua magic in the Kyuafile? 2. Should I add atf_set "required.user" "root" -- or atf_tc_set_md_var("require.user", "root") -- in the testcase header? 3. Should I add both items? There are test engineers that run testcases without isolation, e.g. /path/to/test_app testcase_name -- should we support this workflow, or not? Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 22:25:15 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E80F7FC9; Wed, 11 Jun 2014 22:25:15 +0000 (UTC) Received: from mail-la0-x229.google.com (mail-la0-x229.google.com [IPv6:2a00:1450:4010:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 464692E47; Wed, 11 Jun 2014 22:25:15 +0000 (UTC) Received: by mail-la0-f41.google.com with SMTP id hz20so230076lab.14 for ; Wed, 11 Jun 2014 15:25:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=VTiLrjNm/MHD286Bk/TdwwhxKmAwShawgnEMiGnTqJs=; b=MV6TxyYUfFfRh6iwmhfWA35TkLfbDZAN7a16wL5DEEPHM8OlddMhF5X8T2p1tiJLME zpxaztS913PrtEvfcp/VpjoLj43ZPWf5DYj2BNpgAEDF6KyJFWUpWAdM2Zr5oNie4nI5 yE+KkAZM7XcZJ7VHj6ZmLwYT5zbRfBEjv7yByxc4lqzpwLpyAC2kAXzEJ262ZzIAthi2 a99H1Dj6bM/z0+xNzleMLZG3XxssG03kgbOyNeRYNuRa4e08hh1qpOmUB7a11g3PTfNK D3oNcZfDPh++VcSG9STT00a+JraNu4AIW8oTwM6D7cA4NG4UGXiab8wUTq9r+WaLBYWA odAg== MIME-Version: 1.0 X-Received: by 10.112.72.133 with SMTP id d5mr3651725lbv.58.1402525513076; Wed, 11 Jun 2014 15:25:13 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 15:25:13 -0700 (PDT) In-Reply-To: <5397B84E.302@freebsd.org> References: <5397B84E.302@freebsd.org> Date: Wed, 11 Jun 2014 15:25:13 -0700 X-Google-Sender-Auth: pxg_eFE7sN3LnpW9sfwBstQd-mM Message-ID: Subject: Re: [HEADSUP] Kyua + Jenkins integration: milestone 1 achieved From: Craig Rodrigues To: Allan Jude Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 22:25:16 -0000 On Tue, Jun 10, 2014 at 7:00 PM, Allan Jude wrote: > On 2014-06-10 21:53, Craig Rodrigues wrote: >> Hi, >> >> FYI: >> >> http://lists.freebsd.org/pipermail/freebsd-testing/2014-June/000409.html >> >> Please send followups to freebsd-testing@freebsd.org >> >> -- >> Craig >> > > Some of the test, like this one: > > https://jenkins.freebsd.org/jenkins/job/Kyua_Integration/5/testReport/usr.bin.truncate/truncate_test/bad_truncate/ > > Are failing because of ZFS. > > In this case, chflags uimmutable cannot be done on ZFS. > > Likely a problem with the kyua test, but, it explains that specific failure. Thanks for pointing that out. This test exists in src/usr.bin/truncate/tests/ I am not sure of the kyua syntax to disable this test from running on ZFS. I'm still learning about kyua. :) -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Jun 11 22:49:55 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DFE54C8; Wed, 11 Jun 2014 22:49:55 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B5482034; Wed, 11 Jun 2014 22:49:53 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id e16so238926lan.2 for ; Wed, 11 Jun 2014 15:49:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=srDrHyDOdSr4lgOvwHUesFw/nm/htOcTUD4zu1+na0M=; b=sv3LR2tww4V4mnWv8deo00sJjp0gmhBUWQItvQGplyRSf3FdFKinXBDWgaHzNiTjig Xc1xrcbKVXQgBXccCMfF3f4bc0wv4cwtiaezHMgCDztdcvvi3ZyuwXWJSx39ln9zFjs7 xicLRyDAU/53LN8o9Jlhb+9gT47oYkl32lD3L0rVG8DMoRfz3V086ufYR0aZPvS8Jrgs T9lByMqVH9e7dhkroauYDInKNI0UMD1QIuslz5tpy68w4af8wG1jBQtkfg6QITHl98fE 7WWwfxyEVV7Ttrb6niQXWM9ej82QBGKTibiBBC7BDIgssTrGJXCQKYxu0EHywwwFTFoN Gn5Q== MIME-Version: 1.0 X-Received: by 10.112.143.132 with SMTP id se4mr3847414lbb.53.1402526991435; Wed, 11 Jun 2014 15:49:51 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 15:49:51 -0700 (PDT) In-Reply-To: <20140611192514.GE2855@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> Date: Wed, 11 Jun 2014 15:49:51 -0700 X-Google-Sender-Auth: NivgK4Tr2vRCDeIfCncnV8vYRLw Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 22:49:55 -0000 On Wed, Jun 11, 2014 at 12:25 PM, Roman Bogorodskiy wrote: > I've attached a patch which should fix the segfault. Could you please > let me know if it fixes the problem? I have incorporated your patch into the libvirt port, and provided a tarball of the port here: http://people.freebsd.org/~rodrigc/libvirt_port.tgz I can confirm that your patch eliminates the coredump for me. So now I can do the following: (1) Start libvirtd: /usr/local/sbin/libvirtd (2) Start virsh: virsh -c "bhyve:///system" (3) Execute following inside virsh: define bhyve.xml start bhyve ( using bhyve.xml file from http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html ) In the libvirtd console, I see this: 2014-06-11 22:47:16.357+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-11 22:47:16.357+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-11 22:47:16.357+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-11 22:47:21.050+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-11 22:47:21.063+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-11 22:47:21.168+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-11 22:47:21.168+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-11 22:47:43.469+0000: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet18: Invalid argument 2014-06-11 22:47:43.573+0000: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 For the last line, it looks like it is trying to do bhyvectl --destroy --vm=bhyve on a VM which doesn't exist, i.e. there is no entry in /dev/vmm/bhyve It should ignore the error at this point and try to spawn the VM, but it fails. -- Craig From owner-freebsd-testing@FreeBSD.ORG Thu Jun 12 00:00:41 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDBABF6D; Thu, 12 Jun 2014 00:00:41 +0000 (UTC) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A32326CA; Thu, 12 Jun 2014 00:00:41 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id h3so1415388igd.14 for ; Wed, 11 Jun 2014 17:00:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=u/2x7j/fYPU4QTpTyFepMx0kR69IeMV7G0AhNofM3GQ=; b=KXFWUP5zvJjBGWZlXcB7PO1qun27ItbBs5A3TiEaoldAWWamVNlmXYfYlubYdtw99F O1r1BHl/PYja1xCLLG5ziQ1k3p9+1MoOq59mzCaVNH4WodiYy0enyc309QzlairxeA5P Ou/2brunOuOngci01dSHIx/bf8XQmP7XcUb+9AG5v3o2OpsqunpN7Kq9hAODkhWkbcd+ 8ewYq3yCR+w8xCJATOnpwOKUdYX9pgoPNIifu82aQ5TNPtoAbFC8+qUA6PWCiD6P+r5P odRRc+AhULrmZhaiq4o4icUwmIOP3H+mQXkMwQ6B8DZctmxLTOYojkPfyLF3MFEfPo9b dGew== MIME-Version: 1.0 X-Received: by 10.43.70.132 with SMTP id yg4mr7786303icb.7.1402531241010; Wed, 11 Jun 2014 17:00:41 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Wed, 11 Jun 2014 17:00:40 -0700 (PDT) In-Reply-To: References: <5397B84E.302@freebsd.org> Date: Wed, 11 Jun 2014 17:00:40 -0700 Message-ID: Subject: Re: [HEADSUP] Kyua + Jenkins integration: milestone 1 achieved From: Garrett Cooper To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , Allan Jude X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 00:00:41 -0000 On Wed, Jun 11, 2014 at 3:25 PM, Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 7:00 PM, Allan Jude wrote: >> On 2014-06-10 21:53, Craig Rodrigues wrote: >>> Hi, >>> >>> FYI: >>> >>> http://lists.freebsd.org/pipermail/freebsd-testing/2014-June/000409.html >>> >>> Please send followups to freebsd-testing@freebsd.org >>> >>> -- >>> Craig >>> >> >> Some of the test, like this one: >> >> https://jenkins.freebsd.org/jenkins/job/Kyua_Integration/5/testReport/usr.bin.truncate/truncate_test/bad_truncate/ >> >> Are failing because of ZFS. >> >> In this case, chflags uimmutable cannot be done on ZFS. >> >> Likely a problem with the kyua test, but, it explains that specific failure. > > > Thanks for pointing that out. This test exists > in src/usr.bin/truncate/tests/ > > I am not sure of the kyua syntax to disable this test from running on ZFS. > I'm still learning about kyua. :) Something like "require.filesystem" should probably be added to ATF/Kyua (it doesn't currently exist). Cheers! From owner-freebsd-testing@FreeBSD.ORG Thu Jun 12 04:37:08 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2286222 for ; Thu, 12 Jun 2014 04:37:08 +0000 (UTC) Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F9502D92 for ; Thu, 12 Jun 2014 04:37:08 +0000 (UTC) Received: by mail-ob0-f172.google.com with SMTP id uy5so771183obc.3 for ; Wed, 11 Jun 2014 21:37:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=jpDEAEi6jsIDxUSTcf+kbAIwW2Yt6JKWg5Hb/DBcCX0=; b=dVuiZkjTDehdrYsz1iWO4Ojk+DpcaMK7cy8IloA9ssTFvn4Fm6rroxx7xK2Yr1668g a0QaE5GIBVdoNx4070s8qcyv5MIi6ZWS8ujMBdDdMR3Pq5S8hA55W9eT+pOce2Mqp293 9aJayqRJ+qAcmoZPyU8LxkBUprBHPCo66/Dspys8g4qAO4tf70EGr6zzuFBmgF27MqYH cuX/VZKAKvFo/dBIMi6CT8P+0VJnDDBCgD7u84HIsKggD/x9ZxEcK/pvkwJ9CyzxSQ3b WMzonDgiDtqNAOPUv1Zb0ZPVbbN82gfAuNyOTDos67MYAzvRAoY+FN9XteG0eUfrMy0n osMQ== MIME-Version: 1.0 X-Received: by 10.60.103.78 with SMTP id fu14mr44010140oeb.21.1402547827705; Wed, 11 Jun 2014 21:37:07 -0700 (PDT) Received: by 10.60.98.137 with HTTP; Wed, 11 Jun 2014 21:37:07 -0700 (PDT) Date: Thu, 12 Jun 2014 00:37:07 -0400 Message-ID: Subject: Freebsd test areas or available test suites From: KS To: freebsd-testing@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 04:37:08 -0000 Hello people, I have been actively following freebsd mailing list since one week. You are doing a great job. I work on small term projects using FreeBSD. I would like to know if you guys can refer to any test areas/scenarios and/or any available test suites that are available. Some areas i have already covered are running operations like cd, del , chmod, chown and make use of posix test suite Can you please point me in right direction for any other information. Thanks, KSM From owner-freebsd-testing@FreeBSD.ORG Thu Jun 12 08:03:16 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC8FB4CC; Thu, 12 Jun 2014 08:03:16 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D39DB2CF6; Thu, 12 Jun 2014 08:03:15 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id q8so463816lbi.40 for ; Thu, 12 Jun 2014 01:03:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=0yq7A4cxJhqwVjEsUUsGX9VUhL2ycxxql7zvwxw8ji4=; b=ijvMpHd0io9j5o8QPYs19TlZmm8jsztvBqsCehN+yrz4SNb7V6K5fyJQY3jq32VdA4 iBEUTj73N3MgRfN+GefxMmcN3R7o/bDz/3PwTKHCzkjbdau8Hm9Wz3c+QeaWfmH7U2bQ Z0CxIFyJrSaSsJbEE8T9k/TxRQ1HA4gtTnPRKiZ27v6HQ75In3JL50D5ucnaCCDpns7z 6OMzqhTGv2I3bKpPdb3OdmqV4+lsdhIwgb2wQ/b8veazszYdbLOGzWi6cMctj8tzpyWs XpDmU6SRGT37LJAIjTrEv2ESPoT2D97+s2+WI1iQ+AWzxXAqPaM3yow5HcDUkB2RQaTc wDOg== X-Received: by 10.152.199.39 with SMTP id jh7mr31523168lac.18.1402560193797; Thu, 12 Jun 2014 01:03:13 -0700 (PDT) Received: from kloomba ([217.65.223.250]) by mx.google.com with ESMTPSA id 6sm1508783laz.6.2014.06.12.01.03.12 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jun 2014 01:03:12 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 12 Jun 2014 12:00:59 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140612080057.GA93486@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="x+6KMIRAuhnl3hBn" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 08:03:16 -0000 --x+6KMIRAuhnl3hBn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Wed, Jun 11, 2014 at 12:25 PM, Roman Bogorodskiy w= rote: > > I've attached a patch which should fix the segfault. Could you please > > let me know if it fixes the problem? >=20 > I have incorporated your patch into the libvirt port, and provided a tarb= all > of the port here: >=20 > http://people.freebsd.org/~rodrigc/libvirt_port.tgz >=20 > I can confirm that your patch eliminates the coredump for me. >=20 > So now I can do the following: >=20 > (1) Start libvirtd: > /usr/local/sbin/libvirtd >=20 > (2) Start virsh: > virsh -c "bhyve:///system" >=20 > (3) Execute following inside virsh: > define bhyve.xml > start bhyve >=20 > ( using bhyve.xml file from > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/00258= 8.html > ) >=20 > In the libvirtd console, I see this: >=20 > 2014-06-11 22:47:16.357+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-11 22:47:16.357+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-11 22:47:16.357+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-11 22:47:21.050+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-11 22:47:21.063+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-11 22:47:21.168+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-11 22:47:21.168+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-11 22:47:43.469+0000: 34485598208: error : > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet18: > Invalid argument > 2014-06-11 22:47:43.573+0000: 34485598208: error : virCommandWait:2426 > : internal error: Child process (/usr/sbin/bhyvectl --destroy > --vm=3Dbhyve) unexpected exit status 255 >=20 >=20 >=20 > For the last line, it looks like it is trying to do bhyvectl --destroy > --vm=3Dbhyve > on a VM which doesn't exist, i.e. there is no entry in /dev/vmm/bhyve >=20 > It should ignore the error at this point and try to spawn the VM, but it = fails. This actually happens because it fails to spawn a VM (because there's no bridge) and tries to make sure to unload previously loaded VM in order not to leave it in a stale state. I have adjusted the port: http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz With this setup, I'm able to get networking (e.g. virsh net-list works) and updated the fix for the previous problem. I was able to start a VM with that setup. Roman Bogorodskiy --x+6KMIRAuhnl3hBn Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmV45AAoJEMltX/4IwiJq9FgIAKHhXqh6lCsayHuBn3pWAFXL xqu9ND5I7CtQfKRtDFzMqGKPmca/mYUsvycw6GBiVPWiAWjKK6EPocZ6mr4joI3f v0Yt4w+d6HB1sFCVSg76sBKUSxIYW/wk7CO6MhqLcPvKi4XwBhXP59+2VdIP3hQJ Am8OysrgLOGhjJg0GjpY7hwHgJnKsgcvQztktg/4+dNtScs6cb498DY1WgnpJBdI 3CwefU1n8sHm8yxAQNHE8sD11kUeYAxDqn8YUaBUZH5GvKom3KsFLKYAkDdfdZoa 5FQFmmZtX1iwodIT6wHfK2aRJqH2sidQOf+xkakQOPEvaMUZPzrfScwjoLKj0Ho= =K62g -----END PGP SIGNATURE----- --x+6KMIRAuhnl3hBn-- From owner-freebsd-testing@FreeBSD.ORG Thu Jun 12 15:14:19 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 241162A3 for ; Thu, 12 Jun 2014 15:14:19 +0000 (UTC) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B28FF2755 for ; Thu, 12 Jun 2014 15:14:18 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id cc10so914531wib.5 for ; Thu, 12 Jun 2014 08:14:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=nDQGJFKL9uq85S7ejriTTWswhvcKs9VlYaGBfvR5lt8=; b=Zr2Y7Xj0nzYUZUU7JCZ3GvyDrAfy73IOaABSmymh27bLEaS0InGOxKXT6/SywjsBl2 snZ0+P2S5Cm70zwFILAeSaphTLkqqxwyaWwqsG7uR+a5pYkA0V5j9CtE1AI+YLav3rgf jE2ux4NnaIG78wjYTJuWdAHtN6OY3suZNUudD74uEPiW4U0q3H0IYXlOWRXrAz5kZ0BL SpCvUpms/0Vq0xxp1F+L1oQ8MQelKs7XymwA5zACRo3IzlJZLczkCTvPKPtGtfGBZTsh QxHqOXLkqMk3wSls5yDt/xDPsTib9J1loRbgXoBzY/QX1J0k97EfoAd9Hx9OZVSCryo2 oWKw== MIME-Version: 1.0 X-Received: by 10.180.228.39 with SMTP id sf7mr7496801wic.26.1402586056415; Thu, 12 Jun 2014 08:14:16 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.202 with HTTP; Thu, 12 Jun 2014 08:14:16 -0700 (PDT) In-Reply-To: References: Date: Thu, 12 Jun 2014 09:14:16 -0600 X-Google-Sender-Auth: Ue6MtdWlgiPFBTBX_A8zSrl3KTI Message-ID: Subject: Re: Freebsd test areas or available test suites From: Alan Somers To: KS Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 15:14:19 -0000 On Wed, Jun 11, 2014 at 10:37 PM, KS wrote: > Hello people, > > I have been actively following freebsd mailing list since one week. You are > doing a great job. Since you're a relative newcomer I'll start from the beginning and assume nothing about what you may already know. > > I work on small term projects using FreeBSD. I would like to know if you > guys can refer to any test areas/scenarios and/or any available test suites > that are available. Yes! This mailing list is all about discussing FreeBSD's builtin test suite. To install it, you'll have to rebuild world. On CURRENT or stable/10 (not stable/9), set WITH_TESTS="yes" in /etc/src.conf, then rebuild world according to the instructions here: http://www.freebsd.org/doc/handbook/makeworld.html You should now have the test suite installed at /usr/tests. To run it, you must install devel/kyua from ports, then do: $ cd /usr/tests $ kyua test $ kyua report See tests(7) for more details. In addition to the standard test suite, there are some tests in /usr/src/tools/regression. They aren't run automatically, so some of them may be broken. Converting them to the standard test suite is a work in progress. But if there's a specific feature that you want to to test, you may find a relevant test in there. -Alan > > Some areas i have already covered are running operations like cd, del , > chmod, chown and make use of posix test suite > > Can you please point me in right direction for any other information. > > Thanks, > KSM > _______________________________________________ > freebsd-testing@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-testing > To unsubscribe, send any mail to "freebsd-testing-unsubscribe@freebsd.org" From owner-freebsd-testing@FreeBSD.ORG Thu Jun 12 18:28:38 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 683B4B10; Thu, 12 Jun 2014 18:28:38 +0000 (UTC) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8264029A9; Thu, 12 Jun 2014 18:28:37 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id w7so943038lbi.29 for ; Thu, 12 Jun 2014 11:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=KT5Xy5y8HC+5Aa4yiFtbfuon91wVek4GWFN3gJLJ99M=; b=Fy0+BCfaUgK2QzuLi6EpftnSxWJmRcI4RxAg29sKVn7xvl6YT9OCo9rn2hUZHla80t JLJU/UQYB055WaF65bc6VHJXp/D+HFnB9vzFk0eOtAlfn5aL5Cf+h9QczUcMK3uSUL0I O3I5AOmIjdjZRZHB2f4ErLQ4NnhNM/HPSi2KdJuqK1/blZrN82Noev/Nt+dXVxxtZVur MIxLfbMahzIW44KocBAo0kE9HRy8A1I2Zo0QbIgB67LIlRzrb45bAS4dt+yFI6Q4g+kv e4Bde0b80n9XwN72BpMdMQcQErRLoFkqw/0Lf9gKpfjqm+YMbvXhodkIxXBJmC90W8Zj KKwg== MIME-Version: 1.0 X-Received: by 10.112.50.2 with SMTP id y2mr2420479lbn.66.1402597715344; Thu, 12 Jun 2014 11:28:35 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Thu, 12 Jun 2014 11:28:35 -0700 (PDT) In-Reply-To: <20140612080057.GA93486@kloomba> References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> Date: Thu, 12 Jun 2014 11:28:35 -0700 X-Google-Sender-Auth: JDwrMbCpR8oomfCGRbjjyGPF47c Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 18:28:38 -0000 On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy wrote: > > http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz > > With this setup, I'm able to get networking (e.g. virsh net-list works) > and updated the fix for the previous problem. I was able to start a VM > with that setup. I deleted the old port from my system and took your modified port, built it, and installed it. I followed my previous steps in: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html and got this: 2014-06-12 18:23:54.328+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-12 18:23:54.328+0000: 34485605376: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory 2014-06-12 18:23:54.328+0000: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-12 18:23:54.328+0000: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-12 18:23:54.438+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-12 18:23:54.439+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-12 18:23:54.460+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-12 18:23:54.461+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-12 18:23:54.560+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-12 18:23:54.560+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-12 18:24:17.940+0000: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: Invalid argument 2014-06-12 18:24:18.056+0000: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 -- Craig From owner-freebsd-testing@FreeBSD.ORG Thu Jun 12 18:33:58 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 969D2D6D; Thu, 12 Jun 2014 18:33:58 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA5442A54; Thu, 12 Jun 2014 18:33:57 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so945752lbj.3 for ; Thu, 12 Jun 2014 11:33:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=MG7LfJTB2HGlnQ9xXGx8xtCy+EFBUJulGI93O1NfYWU=; b=lIVzMDUVetbHNCK5c0dQFQ7w0d0vnGbiEw9Q99cVqsxqxxB72RS0/c0zB6o1RpHyhZ 1XhQj3cgYcHAbHDw8J60otIOe/869lJGb350yOqd8UYgqV/Q5kOXSIFmHWFid+GMkozj eX8mXsz4Q+7qYViZqbROpGlXT8nkO5Olpc9QzOW74aA5kMw9/vW3dQZfi/GTa/1rb19n iX3NVEm7vpRl0+AflfEkcvVFHU/hLjvxlUwJ2Pzb8as5wy6/mD+Opv0JqW8meSUnwLh0 t6sqgmrqmh196wJEK4Q8aB3ZXftPRRO1W/sC1IR1vxE621w9erH3renknR1cNCk6A0wd e13w== MIME-Version: 1.0 X-Received: by 10.152.43.43 with SMTP id t11mr41769lal.89.1402598035459; Thu, 12 Jun 2014 11:33:55 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Thu, 12 Jun 2014 11:33:55 -0700 (PDT) In-Reply-To: References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> Date: Thu, 12 Jun 2014 11:33:55 -0700 X-Google-Sender-Auth: 47XyXvW-qtAqbzqWzwGzqzb467Y Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 18:33:58 -0000 On Thu, Jun 12, 2014 at 11:28 AM, Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy wrote: >> >> http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz >> >> With this setup, I'm able to get networking (e.g. virsh net-list works) >> and updated the fix for the previous problem. I was able to start a VM >> with that setup. > > > I deleted the old port from my system and took your modified port, > built it, and installed it. > > I followed my previous steps in: > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html > and got this: > > 2014-06-12 18:23:54.328+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-12 18:23:54.328+0000: 34485605376: error : > dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: > No such file or directory > 2014-06-12 18:23:54.328+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-12 18:23:54.328+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-12 18:23:54.438+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-12 18:23:54.439+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-12 18:23:54.460+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-12 18:23:54.461+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-12 18:23:54.560+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-12 18:23:54.560+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-12 18:24:17.940+0000: 34485598208: error : > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: > Invalid argument > 2014-06-12 18:24:18.056+0000: 34485598208: error : virCommandWait:2426 > : internal error: Child process (/usr/sbin/bhyvectl --destroy > --vm=bhyve) unexpected exit status 255 > > > -- > Craig After the program fails, these are my tap and bridge devices, as shown by ifconfig: bridge0: flags=8843 metric 0 mtu 1500 ether 02:29:45:c7:8f:00 nd6 options=9 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: em0 flags=143 ifmaxaddr 0 port 1 priority 128 path cost 20000 member: tap2 flags=143 ifmaxaddr 0 port 7 priority 128 path cost 2000000 member: tap1 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 2000000 member: tap0 flags=143 ifmaxaddr 0 port 5 priority 128 path cost 2000000 tap0: flags=8902 metric 0 mtu 1500 options=80000 ether 00:bd:53:27:00:00 nd6 options=29 media: Ethernet autoselect status: no carrier tap1: flags=8902 metric 0 mtu 1500 options=80000 ether 00:bd:59:27:00:01 nd6 options=29 media: Ethernet autoselect status: no carrier tap2: flags=8943 metric 0 mtu 1500 options=80000 ether 00:bd:5e:27:00:02 nd6 options=29 media: Ethernet autoselect status: active Opened by PID 1506 vnet0: flags=8802 metric 0 mtu 1500 options=80000 ether fe:54:00:33:20:8c nd6 options=21 media: Ethernet autoselect status: no carrier This is in my /etc/rc.conf for creating bridge and tap devices on bootup: ##################################################### # Create tap devices, one tap interface per BHyve VM. # Add the tap interfaces to bridge0 #################################################### cloned_interfaces="bridge0 tap0 tap1 tap2" autobridge_interfaces="bridge0" autobridge_bridge0="tap* em0" -- Craig From owner-freebsd-testing@FreeBSD.ORG Thu Jun 12 19:03:02 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35247A7B; Thu, 12 Jun 2014 19:03:02 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 452B32DE8; Thu, 12 Jun 2014 19:03:01 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id u10so952375lbd.24 for ; Thu, 12 Jun 2014 12:02:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=kkn1bENRNXvAUpmFKYaAOigpj4IMxP8ifSDMNQd8a8I=; b=WFHazADP99dAYZQKwCEWfjLsCHWl9Pi4nuDZ9jnbnk9D2BV/c0taz7LQ0qjITUj4Rd TmBVi2caAqjOEZU+/JiOLRge4vUtMC4/YxZ/nFhy5JyfwkNAW9hFIPAAy5tcBi28thkR 0qHpwqhsdJyiY4RVTsB5tlboce2SsyFyI7ZctiW17uiqWRHavBbLB00VHzWa/pZsa9SG A90UPRpfGs2lfJphf4M3OEYs4QPTqP7zznKqUmzckrkK0MrbkIsCWloFTkBQ5lfYXYho SRkuJhRiukLd+zLZPbNyZV6H/cqEVWlz00hyV/ts663wOtXRiMMo9b3RZoz0cbET/6aq MAJA== X-Received: by 10.152.234.229 with SMTP id uh5mr2923590lac.56.1402599779011; Thu, 12 Jun 2014 12:02:59 -0700 (PDT) Received: from kloomba ([217.65.223.250]) by mx.google.com with ESMTPSA id d8sm25043689lah.12.2014.06.12.12.02.57 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jun 2014 12:02:58 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 12 Jun 2014 23:00:43 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140612190042.GA2858@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fdj2RfSjLxBAspz7" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2014 19:03:02 -0000 --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 11:28 AM, Craig Rodrigues w= rote: > > On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy = wrote: > >> > >> http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz > >> > >> With this setup, I'm able to get networking (e.g. virsh net-list works) > >> and updated the fix for the previous problem. I was able to start a VM > >> with that setup. > > > > > > I deleted the old port from my system and took your modified port, > > built it, and installed it. > > > > I followed my previous steps in: > > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002= 588.html > > and got this: > > > > 2014-06-12 18:23:54.328+0000: 34485605376: info : libvirt version: 1.2.5 > > 2014-06-12 18:23:54.328+0000: 34485605376: error : > > dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: > > No such file or directory > > 2014-06-12 18:23:54.328+0000: 34485605376: info : > > networkReloadFirewallRules:1750 : Reloading iptables rules > > 2014-06-12 18:23:54.328+0000: 34485605376: info : > > networkRefreshDaemons:1722 : Refreshing network daemons > > 2014-06-12 18:23:54.438+0000: 34485605376: error : virExec:417 : > > Cannot find 'pm-is-supported' in path: No such file or directory > > 2014-06-12 18:23:54.439+0000: 34485605376: warning : > > virQEMUCapsInit:948 : Failed to get host power management capabilities > > 2014-06-12 18:23:54.460+0000: 34485605376: info : > > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > > /usr/local/var/run/libvirt/qemu > > 2014-06-12 18:23:54.461+0000: 34485605376: info : > > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > > /usr/local/etc/libvirt/qemu > > 2014-06-12 18:23:54.560+0000: 34485605376: info : > > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > > /usr/local/etc/libvirt/bhyve > > 2014-06-12 18:23:54.560+0000: 34485605376: info : > > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > > 2014-06-12 18:24:17.940+0000: 34485598208: error : > > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: > > Invalid argument > > 2014-06-12 18:24:18.056+0000: 34485598208: error : virCommandWait:2426 > > : internal error: Child process (/usr/sbin/bhyvectl --destroy > > --vm=3Dbhyve) unexpected exit status 255 > > > > > > -- > > Craig >=20 >=20 > After the program fails, these are my tap and bridge devices, as shown > by ifconfig: >=20 > bridge0: flags=3D8843 metric 0 mt= u 1500 > ether 02:29:45:c7:8f:00 > nd6 options=3D9 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: em0 flags=3D143 > ifmaxaddr 0 port 1 priority 128 path cost 20000 > member: tap2 flags=3D143 > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > member: tap1 flags=3D143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > member: tap0 flags=3D143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > tap0: flags=3D8902 metric 0 mtu 1500 > options=3D80000 > ether 00:bd:53:27:00:00 > nd6 options=3D29 > media: Ethernet autoselect > status: no carrier > tap1: flags=3D8902 metric 0 mtu > 1500 > options=3D80000 > ether 00:bd:59:27:00:01 > nd6 options=3D29 > media: Ethernet autoselect > status: no carrier > tap2: flags=3D8943 > metric 0 mtu 1500 > options=3D80000 > ether 00:bd:5e:27:00:02 > nd6 options=3D29 > media: Ethernet autoselect > status: active > Opened by PID 1506 > vnet0: flags=3D8802 metric 0 mtu 1500 > options=3D80000 > ether fe:54:00:33:20:8c > nd6 options=3D21 > media: Ethernet autoselect > status: no carrier >=20 >=20 > This is in my /etc/rc.conf for creating bridge and tap devices on bootup: >=20 > ##################################################### > # Create tap devices, one tap interface per BHyve VM. > # Add the tap interfaces to bridge0 > #################################################### > cloned_interfaces=3D"bridge0 tap0 tap1 tap2" >=20 > autobridge_interfaces=3D"bridge0" > autobridge_bridge0=3D"tap* em0" Ah. Actually, libvirt manages bridges and taps on its own. So, the preferred flow is that libvirt starts its networks on startup automatically. By default it has a network with the 'virbr0' device which it creates on its own. Later on it creates vnet devices (which are tap devices) automatically as well on VM startup and plugs them to the bridge. The configuration you have probably confuses it, could you consider trying without that? Also, do you have working 'net-list' command in virsh with my version of port? PS If you're available for some real-time conversation such as on IRC, I think some kind of things would be faster to discuss there. Thanks, Roman Bogorodskiy --fdj2RfSjLxBAspz7 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmfjaAAoJEMltX/4IwiJq4K8H/1o5UX77Kw3FO+CN/cCOt4H7 Aa06bB9vCSJrRK8qlc2HJgiXvQgC2IeczkcCJ4EVkc6tLSnSdSTtvJt6u1PKaAz1 varEsL5KMHhP9KSXfqZ7VJC4/EY/mnJRkWtUErZDnFA51i4wDEFZa7KFm19TxZHI oP3DLkG5PwkaB+9WccuPWyNKYNwtvzYVAXsjyZKlfmC0bw5/ebWE9VWppXNoC2Ui guDLQGo8HYa+tpxSZg11xs4Obn9aQufn7idPs6/2FV+aTiPmI7UCYs2HA3WaO3fh BPGpQiEcG7rNQbjQ6xfsquQNv689zxm0RPK7ZeMvbtvn4dsDI9Xl7uANGYWv5Q0= =Bs7F -----END PGP SIGNATURE----- --fdj2RfSjLxBAspz7-- From owner-freebsd-testing@FreeBSD.ORG Fri Jun 13 00:53:48 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1954C71; Fri, 13 Jun 2014 00:53:48 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6368E2C9A; Fri, 13 Jun 2014 00:53:48 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id rl12so1848964iec.39 for ; Thu, 12 Jun 2014 17:53:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lRYFZ6toM12hksv1HWb8skXLgPNDXCySL7ui8yEHcM4=; b=eeXhenW6i66OD2quPWUBsx/rTymjtbECGf6GL5NLJXSkIYS0QN5LNRpscnVl+nSf7U v5r/RePf4GKx9BS30wJgv62D+yo2EwcjSs/MTGzxx4VVfxxRcLp6eTCMYWnx4/egyMSQ AiXyH9dw2av90D2EGxAtCbm0a97isGw4VWDLIenSJzcWHIjgdt0UMBC/bL1GihXCaMZp Dp8TooYiTrfcC2g6LT0ws1WAFJMrTCiq6ZiWG3Ud5Ma+qqgVMg9SI/ckx4bWh8ZOSu5F XqzjAmibwfDw/8VzhZlghMfeFnDESIDbyfM18uNHZbmcRwrkaIdchSVy2vkGjfbMwEJ4 mdCQ== MIME-Version: 1.0 X-Received: by 10.50.61.141 with SMTP id p13mr13955igr.38.1402620827771; Thu, 12 Jun 2014 17:53:47 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Thu, 12 Jun 2014 17:53:47 -0700 (PDT) In-Reply-To: <20140511153203.GA81002@stack.nl> References: <20140124162759.GB90996@stack.nl> <0A68C9FE-6A3A-4DCF-92A9-A506D7581D69@gmail.com> <96373EF8-77EA-4AA6-8422-63881C32E455@gmail.com> <20140511153203.GA81002@stack.nl> Date: Thu, 12 Jun 2014 17:53:47 -0700 Message-ID: Subject: Re: [PATCH v2] convert /bin/sh tests over to ATF From: Garrett Cooper To: Jilles Tjoelker Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 00:53:48 -0000 On Sun, May 11, 2014 at 8:32 AM, Jilles Tjoelker wrote: ... > This is OK with me as long as important use cases for me are not broken: > > 1. Allow running new tests on a potentially broken new sh without > installing anything. This used to be possible using 'make regress' in > src/bin/sh/ but this was removed. I now use > SH=$PWD/obj/sh sh tests/legacy_test.sh > (I have an obj/ subdirectory to make the built objects more easily > accessible). > This should also work on stable/9 although it is acceptable to need a > few local hacks for that (as is already the case for MK_TESTS and > src.opts.mk). On head, it should work without local hacks. I allow you to change the shell via TEST_SH -- is that acceptable for the first drop (I'll look at revising the code to permit runtime overrides via kyua)? > 2. Allow installing using a read-only /usr/obj (mounted over NFS). This > means that everything needs to be built during buildworld. > Installworld should not build anything. > (I currently have local hacks: Kyuafile.auto is not built so I added > all: Kyuafile.auto > to some tests/Makefiles; I guess this should be in .) That's another issue entirely that you brought up in another thread. I'll see if that's still an issue.. Thank you! -Garrett From owner-freebsd-testing@FreeBSD.ORG Fri Jun 13 01:01:47 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9F96DC9 for ; Fri, 13 Jun 2014 01:01:47 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9B5E62D41 for ; Fri, 13 Jun 2014 01:01:47 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id rl12so1854840iec.39 for ; Thu, 12 Jun 2014 18:01:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6Hntx9OwxLZ6HGxhacJr1b/kdOkplKOfvSoSiMylJM8=; b=Zf8/Pw36rYheh3XYuC6fak6GrBYkNbQqKs0P6HwabAL5+55Sy9MYEMVkJ9daGXaApe gaU0WVXzW6em9D6O0YC3yHLXrkjgHUr+PueauDDwOxQ5JWFCxkhM4m39q89WLz0Jx6Wo gGxBS3P5c1Wz0du0JZbg62G94FplCAaXm24d9+cpXMOq3+KaG1H/HrUoTeWZXA/sGRby AYSgB9V0AN1xnVXTBgZ1Xb1wnxk53udiOEBxHCK0H9MlElgjJIKmdrwxAumq/dZxDWoS bZvEWOzeLnFTBGRHJjhECROezzM1QBB1l3tyZ/mNeKNHs5qtf0yfZt2WqAkXR2W+CcsG 2kVQ== MIME-Version: 1.0 X-Received: by 10.42.185.73 with SMTP id cn9mr51075094icb.6.1402621307076; Thu, 12 Jun 2014 18:01:47 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Thu, 12 Jun 2014 18:01:47 -0700 (PDT) In-Reply-To: <20140517132614.GA85230@stack.nl> References: <20140517132614.GA85230@stack.nl> Date: Thu, 12 Jun 2014 18:01:47 -0700 Message-ID: Subject: Re: build Kyuafile.auto during buildworld, not installworld From: Garrett Cooper To: Jilles Tjoelker Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 01:01:47 -0000 On Sat, May 17, 2014 at 6:26 AM, Jilles Tjoelker wrote: > For some time, I have needed local hacks to make installworld work with > a read-only /usr/obj (over NFS). It is also philosophically wrong to > build things during installworld. > > I found two bugs that cause FILES not to be built during buildworld: > > * bsd.files.mk does not create its buildfiles target when a buildfiles > target already exists from bsd.subdir.mk. > > * bsd.progs.mk does not include bsd.files.mk if an install target > already exists (otherwise it does). > > The below patch appears to fix both problems in a somewhat ugly way. > > Index: share/mk/bsd.files.mk > =================================================================== > --- share/mk/bsd.files.mk (revision 266086) > +++ share/mk/bsd.files.mk (working copy) > @@ -6,11 +6,10 @@ > > FILESGROUPS?= FILES > > -.if !target(buildfiles) > +buildfiles: buildfiles0 > .for group in ${FILESGROUPS} > -buildfiles: ${${group}} > +buildfiles0: ${${group}} > .endfor > -.endif I would probably just remove the .if !target(buildfiles) part. I'd make sure to do an -exp run with this though. > all: buildfiles > > Index: share/mk/bsd.test.mk > =================================================================== > --- share/mk/bsd.test.mk (revision 266086) > +++ share/mk/bsd.test.mk (working copy) > @@ -82,7 +82,9 @@ > > .if !empty(PROGS) || !empty(PROGS_CXX) || !empty(SCRIPTS) > .include > -.elif !empty(FILES) > +.endif > +# may or may not include . > +.if !empty(FILES) && !target(buildfiles0) > .include > .endif I would just check for `!empty(FILES)` personally. I find it more annoying how with some versions of make/versions of share/mk it would install FILES multiple times (probably needs a sentinel somewhere, like bsd.progs.mk...). Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Fri Jun 13 16:21:09 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14D471AE; Fri, 13 Jun 2014 16:21:09 +0000 (UTC) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 382AC2C9D; Fri, 13 Jun 2014 16:21:08 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 65D5F1534D2; Fri, 13 Jun 2014 18:21:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nAHDWD0NCu0l; Fri, 13 Jun 2014 18:20:57 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:a15d:b174:138f:d372] (unknown [IPv6:2001:4cb8:3:1:a15d:b174:138f:d372]) by smtp.digiware.nl (Postfix) with ESMTP id E7C521534C0; Fri, 13 Jun 2014 18:20:57 +0200 (CEST) Message-ID: <539B24E1.7040006@digiware.nl> Date: Fri, 13 Jun 2014 18:20:49 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Craig Rodrigues , Roman Bogorodskiy Subject: Re: libvirt and bhyve problems References: <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 16:21:09 -0000 On 2014-06-12 20:33, Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 11:28 AM, Craig Rodrigues wrote: >> On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy wrote: >>> >>> http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz >>> >>> With this setup, I'm able to get networking (e.g. virsh net-list works) >>> and updated the fix for the previous problem. I was able to start a VM >>> with that setup. >> >> >> I deleted the old port from my system and took your modified port, >> built it, and installed it. >> >> I followed my previous steps in: >> http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html >> and got this: >> >> 2014-06-12 18:23:54.328+0000: 34485605376: info : libvirt version: 1.2.5 >> 2014-06-12 18:23:54.328+0000: 34485605376: error : >> dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: >> No such file or directory >> 2014-06-12 18:23:54.328+0000: 34485605376: info : >> networkReloadFirewallRules:1750 : Reloading iptables rules >> 2014-06-12 18:23:54.328+0000: 34485605376: info : >> networkRefreshDaemons:1722 : Refreshing network daemons >> 2014-06-12 18:23:54.438+0000: 34485605376: error : virExec:417 : >> Cannot find 'pm-is-supported' in path: No such file or directory >> 2014-06-12 18:23:54.439+0000: 34485605376: warning : >> virQEMUCapsInit:948 : Failed to get host power management capabilities >> 2014-06-12 18:23:54.460+0000: 34485605376: info : >> virDomainObjListLoadAllConfigs:18249 : Scanning for configs in >> /usr/local/var/run/libvirt/qemu >> 2014-06-12 18:23:54.461+0000: 34485605376: info : >> virDomainObjListLoadAllConfigs:18249 : Scanning for configs in >> /usr/local/etc/libvirt/qemu >> 2014-06-12 18:23:54.560+0000: 34485605376: info : >> virDomainObjListLoadAllConfigs:18249 : Scanning for configs in >> /usr/local/etc/libvirt/bhyve >> 2014-06-12 18:23:54.560+0000: 34485605376: info : >> virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' >> 2014-06-12 18:24:17.940+0000: 34485598208: error : >> virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: >> Invalid argument >> 2014-06-12 18:24:18.056+0000: 34485598208: error : virCommandWait:2426 >> : internal error: Child process (/usr/sbin/bhyvectl --destroy >> --vm=bhyve) unexpected exit status 255 >> >> >> -- >> Craig > > > After the program fails, these are my tap and bridge devices, as shown > by ifconfig: > > bridge0: flags=8843 metric 0 mtu 1500 > ether 02:29:45:c7:8f:00 > nd6 options=9 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: em0 flags=143 > ifmaxaddr 0 port 1 priority 128 path cost 20000 > member: tap2 flags=143 > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > member: tap1 flags=143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > member: tap0 flags=143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > tap0: flags=8902 metric 0 mtu 1500 > options=80000 > ether 00:bd:53:27:00:00 > nd6 options=29 > media: Ethernet autoselect > status: no carrier > tap1: flags=8902 metric 0 mtu > 1500 > options=80000 > ether 00:bd:59:27:00:01 > nd6 options=29 > media: Ethernet autoselect > status: no carrier > tap2: flags=8943 > metric 0 mtu 1500 > options=80000 > ether 00:bd:5e:27:00:02 > nd6 options=29 > media: Ethernet autoselect > status: active > Opened by PID 1506 > vnet0: flags=8802 metric 0 mtu 1500 > options=80000 > ether fe:54:00:33:20:8c > nd6 options=21 > media: Ethernet autoselect > status: no carrier > > > This is in my /etc/rc.conf for creating bridge and tap devices on bootup: > > ##################################################### > # Create tap devices, one tap interface per BHyve VM. > # Add the tap interfaces to bridge0 > #################################################### > cloned_interfaces="bridge0 tap0 tap1 tap2" > > autobridge_interfaces="bridge0" > autobridge_bridge0="tap* em0" I claim to know nothing about libvirt..... But in my scripts I need to delay using the tap?? interfaces for 1-2 secs with a sleep. Because otherwise my bhyve VM fails. Could be the same here? --WjW From owner-freebsd-testing@FreeBSD.ORG Fri Jun 13 19:18:39 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 56C1BD0B; Fri, 13 Jun 2014 19:18:39 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 75C422C8D; Fri, 13 Jun 2014 19:18:38 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id s18so1716539lam.4 for ; Fri, 13 Jun 2014 12:18:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=OlXaqaK53DBy4/q7wAQFn3f6qRPnikclnCpqFCq6oK4=; b=0/UVCzYwp7eAx7xSkM7McqNSMu0+fK8EufiOGYBt6jHaT7LV3CJPx6ex+nGTxhfV4a FZZnIvHvmTBagodNpNhIbQre9G4Fj7cGKEP6TXF0Hwa5FbV0TFYw3mQVkJUDt5ZlAvL4 K/WEpcnClrk1iHMU1bkE7uKIMf0+WIV/YgK0bsQNsxZBA1DeWxIrsEALk05jSStZLtUO 8PPg0tHmHk1tHFX05dk8RGTwuQ4/E+OyMU0ExqaGFtmOCvKbFbw57eX6APxrnisTCepp 6gn+yRb+ETWrfnCy5XvuQ7p/aSY6NabdettPvazmhO1KnI+XFrzyiej0GcjMLQNIVwVu j1EQ== MIME-Version: 1.0 X-Received: by 10.112.217.10 with SMTP id ou10mr2769615lbc.20.1402687116441; Fri, 13 Jun 2014 12:18:36 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Fri, 13 Jun 2014 12:18:36 -0700 (PDT) In-Reply-To: <20140612190042.GA2858@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> Date: Fri, 13 Jun 2014 12:18:36 -0700 X-Google-Sender-Auth: 8oqqzSOMKxju4urHTSRMYuXeI5A Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 19:18:39 -0000 On Thu, Jun 12, 2014 at 12:00 PM, Roman Bogorodskiy wrote: > > Ah. Actually, libvirt manages bridges and taps on its own. So, the > preferred flow is that libvirt starts its networks on startup > automatically. By default it has a network with the 'virbr0' device > which it creates on its own. > > Later on it creates vnet devices (which are tap devices) automatically > as well on VM startup and plugs them to the bridge. > > The configuration you have probably confuses it, could you consider > trying without that? OK, I removed all autobridge and cloned_interfaces entries from my rc.conf and rebooted. I am still using your version of the libvirt port. I tried the steps again and got this: 2014-06-13 19:09:10.482+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-13 19:09:10.482+0000: 34485605376: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory 2014-06-13 19:09:10.482+0000: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-13 19:09:10.482+0000: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-13 19:09:10.593+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-13 19:09:10.593+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-13 19:09:10.615+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-13 19:09:10.615+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-13 19:09:10.716+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-13 19:09:10.716+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-13 19:09:25.938+0000: 34485598208: error : virNetDevTapCreate:385 : Unable to create tap device: Invalid argument 2014-06-13 19:09:26.063+0000: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 If I do ifconfig, I have no bridge or tap devices. > > Also, do you have working 'net-list' command in virsh with my version of > port? I get this: virsh # net-list Name State Autostart Persistent ---------------------------------------------------------- > > PS If you're available for some real-time conversation such as on IRC, I > think some kind of things would be faster to discuss there. You can private mail me some times when you are available. I am busy the next few days, but might have a few hours on Sunday. -- Craig From owner-freebsd-testing@FreeBSD.ORG Fri Jun 13 21:00:18 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9827EA00; Fri, 13 Jun 2014 21:00:18 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE99125A9; Fri, 13 Jun 2014 21:00:17 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id gf5so1192384lab.22 for ; Fri, 13 Jun 2014 14:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5lghiJZqQCj8Re8+kBOATJPc/qwPXUv9BJJwLVYCU50=; b=ETtKA+79RXSOEjaVdMV8d6xxrPwM1qLfK1PEKzCAv6cTePg1WtAqqi9w3qU5Qd8LRL 7yJsncQT0dqNrRwDtb2L+8caHk64o17/dHYswCzkbwMxK3h0uonDtrcWU3kNZ3uN8VxP Oy/ZV50+UPDfhUZ5A5gTcIv4iL+5OPlCb/rVefyTIHndJznMUrHeTX8GSqe0zCZ6QNoo deTMtvtNwqfi1RhOwpRMKoq9BQVkO++luW4MItkWi622C/bgRYvfcQKOTJNlr1iYbI3r pIZnx4fd0/LR1syK14YHXTFi6n4j1MpI3oxgkH86mFotaJnCQotRDVARX5M48xWdbFXQ qQ8w== MIME-Version: 1.0 X-Received: by 10.152.19.162 with SMTP id g2mr76569lae.71.1402693214745; Fri, 13 Jun 2014 14:00:14 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Fri, 13 Jun 2014 14:00:14 -0700 (PDT) In-Reply-To: References: Date: Fri, 13 Jun 2014 14:00:14 -0700 X-Google-Sender-Auth: ifOhgw0W1BgfvM06bOSUmJCv4yw Message-ID: Subject: Re: Kyua + Jenkins integration From: Craig Rodrigues To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 21:00:18 -0000 On Tue, Jun 10, 2014 at 8:44 PM, Alan Somers wrote: > On Tue, Jun 10, 2014 at 7:49 PM, Craig Rodrigues wrote: > But I do see one problem: there's no message for why a > test was skipped. Plain "kyua report" will give an explanation like > "skipped: required configuration variable Foo was undefined". But > that doesn't show up in the Jenkins output. > > -Alan I took Julio's latest version of kyua, and regenerated the test result. Now it includes the reason why a test was skipped: https://jenkins.freebsd.org/jenkins/job/Kyua_Integration/10/ -- Craig From owner-freebsd-testing@FreeBSD.ORG Sat Jun 14 00:03:52 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F3D61D4E for ; Sat, 14 Jun 2014 00:03:51 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4B4A24C1 for ; Sat, 14 Jun 2014 00:03:51 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id rl12so3121378iec.11 for ; Fri, 13 Jun 2014 17:03:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sO/1ifytWdgziDFeZn9O9+irZbiGNqkUXG/fbkjNlCY=; b=R3F0U+OyXghgiJO80C/dQLMZurZWkTwg3XHB8opBibnO4yBh+l9jbDrzMMHzOzzckT GXIacz5MQYRFuXpbPd3iF+FJfO9U9TzQtKj5s/G13A2cL6JV+bs7vv8s8+wSEpI3wmpF jKbEngiCjYdBW4EaE1JYeYKaUD2QVEy37Bp4XQoAgJla4n/JBZatOLVfDU+A5QkPJgOc pE6QmZP+zf8mfqaMWrjyjfrfAirTDwaQzDswZ6z4t+gwnPj3nKcmrOwoVvNl7tBS//BO jDdDoMFF6y5aEEkSv3gFuKG4lJKzo0xfX2Jrh02Vj8flEGb1Pkd5iUv9Trzok0DepH4P 1cYw== MIME-Version: 1.0 X-Received: by 10.50.78.66 with SMTP id z2mr8722944igw.27.1402704231105; Fri, 13 Jun 2014 17:03:51 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Fri, 13 Jun 2014 17:03:51 -0700 (PDT) In-Reply-To: References: Date: Fri, 13 Jun 2014 17:03:51 -0700 Message-ID: Subject: Re: Failures on head From: Garrett Cooper To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jun 2014 00:03:52 -0000 On Mon, May 12, 2014 at 2:06 PM, Garrett Cooper wrote: > On Fri, Apr 25, 2014 at 2:03 PM, Garrett Cooper wrote: >> Hi, >> While looking at >> http://kyua3.nyi.freebsd.org/head/data/0-LATEST/results/index.html , I >> noticed that a handful of testcases are failing, in particular: >> >> Broken test cases >> >> usr.bin/sed/inplace_race_test:main >> >> Failed test cases >> >> bin/pkill/pgrep-j_test:main >> bin/pkill/pkill-j_test:main >> lib/atf/libatf-c/macros_test:use >> usr.bin/sed/multi_test:main >> usr.bin/yacc/legacy_test:main > > The yacc failures are because the tests in > /usr/src/tools/regression are super stale. The tests should be picked > up from contrib/byacc/test . I'll work on that. > Thanks, > -Garrett I did some investigation and some of the bugs I noted above were test bugs; others were test infrastructure bugs/missing enhancements. I've filed bugs for all of the issues in the "vanilla" 11-CURRENT kyua here: pgrep/pkill: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191019 libatf: https://github.com/jmmv/atf/issues/8 sed: https://github.com/jmmv/kyua/issues/85 yacc: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191020 Thank you! -Garrett From owner-freebsd-testing@FreeBSD.ORG Sun Jun 15 15:22:33 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71920AEF; Sun, 15 Jun 2014 15:22:33 +0000 (UTC) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CF552387; Sun, 15 Jun 2014 15:22:32 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id 10so1390934lbg.1 for ; Sun, 15 Jun 2014 08:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=7+0oghu3OKdcsfe2AkLRQsU4DA8kCON5PeA+yZy0RM8=; b=QjT+FiQz1Gk5cHLRJWSlqimyGYl2LQxvigwkGO7re2P1R0vDtMfIiYyf9z2E8p1WSm ZEg0+DUCSnMeQnxl48OS1oU3dAnT6Y1QyjcRGRkBU2F6E+erf/fN+QPxyxaLO6eXQbUg 8+RJNSRUxIAmNQ4I7i9ktBFPZMBES3se5Dejx4Mq3DMipDWpQJFUIfoD+jHMgmNFv9ta i/EacWdxw1gZ34/PxwXSJij7iZDwpMPc53UYfUmEUzVLOZErDaMThMnhzXfkOn4wV7O8 FcEOBksD0pda5qlFma5ASb5gn/MYhNIllH6ghNBQN8zhOJI9dWx2tUujXd3e+S1Ay3OW rY4Q== X-Received: by 10.152.36.134 with SMTP id q6mr10507893laj.29.1402845750233; Sun, 15 Jun 2014 08:22:30 -0700 (PDT) Received: from kloomba ([95.104.132.224]) by mx.google.com with ESMTPSA id bg4sm6414133lbb.6.2014.06.15.08.22.28 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jun 2014 08:22:29 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sun, 15 Jun 2014 19:20:09 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140615152008.GA19985@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wac7ysb48OaltWcw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 15:22:33 -0000 --wac7ysb48OaltWcw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 12:00 PM, Roman Bogorodskiy w= rote: > > > > Ah. Actually, libvirt manages bridges and taps on its own. So, the > > preferred flow is that libvirt starts its networks on startup > > automatically. By default it has a network with the 'virbr0' device > > which it creates on its own. > > > > Later on it creates vnet devices (which are tap devices) automatically > > as well on VM startup and plugs them to the bridge. > > > > The configuration you have probably confuses it, could you consider > > trying without that? >=20 > OK, I removed all autobridge and cloned_interfaces entries > from my rc.conf and rebooted. I am still using your version of the > libvirt port. >=20 > I tried the steps again and got this: >=20 > 2014-06-13 19:09:10.482+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-13 19:09:10.482+0000: 34485605376: error : > dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: > No such file or directory > 2014-06-13 19:09:10.482+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-13 19:09:10.482+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-13 19:09:10.593+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-13 19:09:10.593+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-13 19:09:10.615+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-13 19:09:10.615+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-13 19:09:10.716+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-13 19:09:10.716+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-13 19:09:25.938+0000: 34485598208: error : > virNetDevTapCreate:385 : Unable to create tap device: Invalid argument > 2014-06-13 19:09:26.063+0000: 34485598208: error : virCommandWait:2426 > : internal error: Child process (/usr/sbin/bhyvectl --destroy > --vm=3Dbhyve) unexpected exit status 255 >=20 >=20 > If I do ifconfig, I have no bridge or tap devices. >=20 >=20 > > > > Also, do you have working 'net-list' command in virsh with my version of > > port? >=20 > I get this: > virsh # net-list > Name State Autostart Persistent > ---------------------------------------------------------- >=20 I have updated the port: http://people.freebsd.org/~novel/misc/libvirt_port2.tgz Changes are: * added RUN_DEPENDS on dnsmasq (required for networking) * added installation on network definition files Hopefully this will allow to bring up network on start. When the daemon starts, virsh should show the following: virsh # net-list Name State Autostart Persistent ---------------------------------------------------------- default active yes yes virsh #=20 If that's not the case, then something is still going wrong. And yes, qemu support should be enabled (networks defs seem to be installed with qemu support). Even if it doesn't work, and 'net-list --all' is empty, we could try to do it manually (but still, if it's the case, then it's something wrong with the port that needs to be fixed): in virsh # net-define /usr/local/etc/libvirt/qemu/networks/default.xml # net-start default Roman Bogorodskiy --wac7ysb48OaltWcw Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTnbmoAAoJEMltX/4IwiJqJSYH/iB/li8gPsiD4tnzO7Wi7TAp hVxaStC3I2cMYfPxnKp/76dwDKzJDqveNTFSXr02gh6khQ4YilZ0h+Vi1jGcPfgH JKMWEKT7I/mcl6gkEOZr9Dwb9NKJQci+6RSaMMBkV7cjAweOByFlOz68tPJ/hV2O oiZbJj8uNA+KkbKEUYLxb/Digpo+mvdFlPWDXxtTMb02vW3J8eHnUUTy4EpvNo0r wWReT7ri57yG/LHcO/oGiK1QZ4OjeUBibD1iEKAXltX9kiN6XOK3+J/IZGKx0Ch5 hpBW9RK9g7adosdr4AZSVMqmsw6hi1mvkRJGcb+TnNym5pSEqwhQfx1OA1CWlj8= =TAba -----END PGP SIGNATURE----- --wac7ysb48OaltWcw-- From owner-freebsd-testing@FreeBSD.ORG Sun Jun 15 19:04:34 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6CE55488; Sun, 15 Jun 2014 19:04:34 +0000 (UTC) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 816512822; Sun, 15 Jun 2014 19:04:33 +0000 (UTC) Received: by mail-lb0-f180.google.com with SMTP id w7so645812lbi.11 for ; Sun, 15 Jun 2014 12:04:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=JoHVFedMMb5TE8byogy1mGIBDKWk30bv8MAw/jhTjNo=; b=hTkwuONQY9hzV3YFy5c8ygrdPv1HHeOr+f8w/pgon+bIludv9XrzXQJmrhueE8CUWz 8NVrghmTgPtg9WWsnC2+SSCdQslLQPNg4N7rTu6u2sUeLfYLWXduvBwbgQox2CHhv0el /1M4RcS4e0oqCPqN7Yya99rrlj+iF/QyQ+dQpGHaw4hbLmxY7h39Fj3u3iAB0B8dPECZ 22Zup23MAvU3AIeB3KCp7QIs5tSbhkh1+j3UaQ2csM3Rf2irbENps9H+8V6YQd7EsSV5 t59ZwsKbctKLE7MSw7Wyvl0UOm3ERwbsKCclsgDuIpbbgIx+IOe1JjD0k+qQNVzN8KFI RxGg== MIME-Version: 1.0 X-Received: by 10.112.132.233 with SMTP id ox9mr10274187lbb.35.1402859071356; Sun, 15 Jun 2014 12:04:31 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Sun, 15 Jun 2014 12:04:31 -0700 (PDT) In-Reply-To: <20140615152008.GA19985@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> <20140615152008.GA19985@kloomba> Date: Sun, 15 Jun 2014 12:04:31 -0700 X-Google-Sender-Auth: ilov8IrBjf_fJXAV9aETyXwMMbE Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 19:04:34 -0000 On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wrote: > > I have updated the port: > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz OK, I deinstall the old port, took your port, built it, and installed it. When I started libvirtd, I got this: 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-15 18:47:07.236+0000: 34485605376: info : dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic is present, SO_BINDTODEVICE is in use 2014-06-15 18:47:07.237+0000: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-15 18:47:07.237+0000: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 : internal error: Child process (/usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf) unexpected exit status 2: dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-15 18:47:07.484+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-15 18:47:07.495+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-15 18:47:07.495+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-15 18:47:07.583+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-15 18:47:07.584+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' Trying to start bhyve did not work. After some investigation, I did the following: service local_unbound stop Then I started libvirtd again: 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 2014-06-15 18:47:07.236+0000: 34485605376: info : dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic is present, SO_BINDTODEVICE is in use 2014-06-15 18:47:07.237+0000: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-15 18:47:07.237+0000: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 : internal error: Child process (/usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf) unexpected exit status 2: dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-15 18:47:07.484+0000: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-15 18:47:07.495+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-15 18:47:07.495+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-15 18:47:07.583+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-15 18:47:07.584+0000: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' At this point, I could see the virbr0 device: virbr0: flags=8843 metric 0 mtu 1500 ether 02:b8:18:5f:0f:00 inet 192.168.122.1 netmask 0xffffff00 broadcast 192.168.122.255 nd6 options=1 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 4 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 I was then able to do: virsh -c 'bhyve:///system' start bhyve and start a VM. This is good progress! Once thing I observed is that if I killed the libvirtd process, it did not clean up the dnsmasq process that it started: /usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf and a subsequent restart of libvirtd would fail. The devel/libvirt port is looking a lot better with your changes. Can you push these changes into the port? I filed bugs 190767 and 190783, but did not get any response from the port maintainer, jgh@. -- Craig From owner-freebsd-testing@FreeBSD.ORG Mon Jun 16 06:44:19 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26A20A34; Mon, 16 Jun 2014 06:44:19 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DEC12CD8; Mon, 16 Jun 2014 06:44:18 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id gf5so2124367lab.22 for ; Sun, 15 Jun 2014 23:44:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=60b/mZEKfhsOeWiaBPKnwOQRRossbb9/NKT4QBNRVbM=; b=lC1DlHTis9GiJ5Aqztei3sxY7ju6VwYJ13dBa8jhgJ0CqAEJc/7DmPEsQ48wMO4bEw xHUaRI8sKWBmaZly9Y1gJg321G6WcijlFndMZzfBPeV/ht5NWckrV3i3Zh3q30hrFXy3 6tMqejLkA7RMEtchwgvpDpil9Q3LTdOlwZqWfB10+ZCRvXrvfF4no5M/3Zm3kyEBkFxr DdH2THIK7CXJFQO89tKbgSe+hBGE3S8ARaKuaYWfB1rT8IHIhihjVQc2ny24KJlbHylt 4yb8Jqjhb6BP1EWeQGn/eEf++8J3LQmrx1DSIMsWbqy3922gDldB/zSWrLmD1SGt8g7c Y2DA== X-Received: by 10.112.17.102 with SMTP id n6mr12120948lbd.39.1402901056011; Sun, 15 Jun 2014 23:44:16 -0700 (PDT) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id s15sm7882189lbp.42.2014.06.15.23.44.14 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jun 2014 23:44:15 -0700 (PDT) Sender: Roman Bogorodskiy Date: Mon, 16 Jun 2014 10:44:07 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140616064406.GA41585@dev.san.ru> References: <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> <20140615152008.GA19985@kloomba> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 06:44:19 -0000 Craig Rodrigues wrote: > On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wr= ote: > > > > I have updated the port: > > > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz >=20 >=20 > OK, I deinstall the old port, took your port, built it, and installed it. >=20 > When I started libvirtd, I got this: >=20 > 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-15 18:47:07.236+0000: 34485605376: info : > dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic > is present, SO_BINDTODEVICE is in use > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 > : internal error: Child process (/usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf) > unexpected exit status 2: > dnsmasq: failed to create listening socket for 127.0.0.1: Address already= in use >=20 > 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-15 18:47:07.484+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-15 18:47:07.583+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-15 18:47:07.584+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' >=20 >=20 >=20 > Trying to start bhyve did not work. >=20 > After some investigation, I did the following: >=20 > service local_unbound stop >=20 >=20 > Then I started libvirtd again: >=20 > 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-15 18:47:07.236+0000: 34485605376: info : > dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic > is present, SO_BINDTODEVICE is in use > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 > : internal error: Child process (/usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf) > unexpected exit status 2: > dnsmasq: failed to create listening socket for 127.0.0.1: Address already= in use >=20 > 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-15 18:47:07.484+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-15 18:47:07.583+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-15 18:47:07.584+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' >=20 >=20 > At this point, I could see the virbr0 device: >=20 > virbr0: flags=3D8843 metric 0 mtu= 1500 > ether 02:b8:18:5f:0f:00 > inet 192.168.122.1 netmask 0xffffff00 broadcast 192.168.122.255 > nd6 options=3D1 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 4 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >=20 >=20 > I was then able to do: >=20 > virsh -c 'bhyve:///system' > start bhyve >=20 > and start a VM. > This is good progress! >=20 > Once thing I observed is that if I killed the libvirtd process, > it did not clean up the dnsmasq process that it started: >=20 > /usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf >=20 > and a subsequent restart of libvirtd would fail. I think it's intentional not to kill dnsmasq. There are persistent and transient objects in libvirt (such as domains and networks), and persistent objects live forever, while transient objects live until manually destroyed or host is restarted. So libvirt doesn't kill persistent VMs and, I guess, it doesn't stop dnsmasq to keep persistent networks functional. It should not affect restart. I'll fail to start the network, but as it's here already, we're fine. BTW, I'm still yet to write the code to catch up previously started bhyve VMs between libvirtd restarts... > The devel/libvirt port is looking a lot better with your changes. > Can you push these changes into the port? >=20 > I filed bugs 190767 and 190783, but did not get any response > from the port maintainer, jgh@. I'll send the patch, but I need to fix some issues first. I noticed that check-orphans complains about the orphaned libvirtd.conf file. Also, I need to have a poudriere run as well to make sure I didn't miss anything else. Roman Bogorodskiy From owner-freebsd-testing@FreeBSD.ORG Mon Jun 16 21:00:00 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8CEE0584; Mon, 16 Jun 2014 21:00:00 +0000 (UTC) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B07ED2D7E; Mon, 16 Jun 2014 20:59:59 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id 10so2344050lbg.1 for ; Mon, 16 Jun 2014 13:59:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=mzjfVPswdTYvK7qI7pOUOwUv0CuNA+i46K8OhtFTftQ=; b=CKJM9t+K2wjtP3A96o0gH0EB9gT6LW3aqSecqJRdDjSi4CFeoqJ9zMdeBIKYv2h7Zw LIeNvo1cad/fD6M8570pXqFVozWwWGXB1gTET/0UUhIaJ76B4MdNBVgn7Pw87+T7mTTO iHhkziUqHK2MDH+Vz6Lq8NpeJGjx6kUEp3dBM96p8337q242Daav5Ju7eydMJuYdoVl4 HS7jLBwLBMntqyB771msezkUBwR4HaWG3Au0vRfVuNO2CJCGfEUbaNY/o2aseE3Qc/yc hYCRbS2izlIqJABYKmofaLnwnmUi+RQ3NmUPGnLwPP2VZtjkv6hya1tYmpe/KBsS7cOH o4gg== MIME-Version: 1.0 X-Received: by 10.112.160.105 with SMTP id xj9mr14479473lbb.2.1402952397644; Mon, 16 Jun 2014 13:59:57 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Mon, 16 Jun 2014 13:59:57 -0700 (PDT) In-Reply-To: References: Date: Mon, 16 Jun 2014 13:59:57 -0700 X-Google-Sender-Auth: DX_zO0EepjV4aVM10dfkb-3Kdoo Message-ID: Subject: Re: Kyua + Jenkins integration From: Craig Rodrigues To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 21:00:00 -0000 On Wed, Jun 11, 2014 at 3:23 AM, Julio Merino wrote: > > https://github.com/jmmv/kyua/commit/7acd1a1fa05e600ac41094857ee1344f4ae7adb5 > By default, "kyua report-html" will write output files in the ./html directory, unless overridden by "--output [directory]". However, "kyua report-junit" seems to write output files to stdout instead by default. You might want to make the default behavior of "kyua report-junit" by closer to "kyua report-html", i.e. do not write the output files to stdout by default. -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Jun 17 08:40:19 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EECAD2; Tue, 17 Jun 2014 08:40:19 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D34C23C8; Tue, 17 Jun 2014 08:40:18 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id e16so3747014lan.30 for ; Tue, 17 Jun 2014 01:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Yg18n1LWiug4JZ6goXoPxkFrwkRNFaqMht8aFi4lHr4=; b=bn0XBcbiohMmE4HAcHny28hMDbc7zceHkwszONHwgQVYGs761C2ncRp5PReNodwVZD xA5D9oRRYtXVvH5XybUqQsNJRo1hLRjvM/mkc6X/bkErH/1bZKft495sIsf6zAH07KXA JdhJB1FtcO/UdZSqv/c/+oEkjwBVvPCwz5BY6Orj1I+3mZNrCMVQpVIiqQikPt7H929w MizjfkPLZ2DVVHa+Fo68gI5VSCdAXF4CUsl/a45mTSPcgm21eCHTWsRzJJfKIQA5vqCt JqiNkiiCqdtdS+pM39s9+VzW8KUam+I7lc6nYJVyRL++zlkmnXNyf9IJ5scmf9CNa7ol KuCw== MIME-Version: 1.0 X-Received: by 10.152.87.20 with SMTP id t20mr297582laz.79.1402994416383; Tue, 17 Jun 2014 01:40:16 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Tue, 17 Jun 2014 01:40:16 -0700 (PDT) In-Reply-To: <20140615152008.GA19985@kloomba> References: <20140611182101.GD2855@kloomba> <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> <20140615152008.GA19985@kloomba> Date: Tue, 17 Jun 2014 01:40:16 -0700 X-Google-Sender-Auth: LzSnJe6ArS6YQVDTBYwfNUGeucA Message-ID: Subject: Re: libvirt and bhyve problems From: Craig Rodrigues To: Roman Bogorodskiy Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 08:40:19 -0000 On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wrote: > > I have updated the port: > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz > > Changes are: > > * added RUN_DEPENDS on dnsmasq (required for networking) > * added installation on network definition files Hi, dnsmasq needs to be in BUILD_DEPENDS *and* RUN_DEPENDS for this port. I did a test on my system where I deleted the dnsmasq port, and then built the libvirt port. During the configure stage of this port, the #define DNSMASQ variable is expanded and written to a config.h. If the dnsmasq port is not around during the build stage of the libvirt port, then this variable is not expanded properly, and when the libvirt port is fully built, running libvirtd will not start dnsmasq, because the path to dnsmasq was not written properly to config.h. -- Craig From owner-freebsd-testing@FreeBSD.ORG Thu Jun 26 19:05:56 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4096BDD; Thu, 26 Jun 2014 19:05:56 +0000 (UTC) Received: from mail-ie0-x22b.google.com (mail-ie0-x22b.google.com [IPv6:2607:f8b0:4001:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8DF192CDD; Thu, 26 Jun 2014 19:05:56 +0000 (UTC) Received: by mail-ie0-f171.google.com with SMTP id x19so3462398ier.16 for ; Thu, 26 Jun 2014 12:05:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=c6v1tCX6A0FL1H7XuHkl6YpNPRDx7q4w3eyfKkHiOUw=; b=GCcKRWYRFx0HejPtz8swnYBIsV4siOcf2FrLGRtqWgU59Yuc+NoUi46vdLPgK4m0fh Xy+5a2SochmQjRXbPgr864SF7LTJYZNMMGtuL6Jv0mJkCGDR0zemjD7N4TVJ0B8z4F7A s6/uKTUHDdS/uHK26VWHovT59j1LyHijH6d0HIa/4y7k5js8+TtHFyYMFTbTt7j6+7MR a8ixYOwZ8f4cOtvt8fK8pnJfC+QCnjI7YGISsuvQ357RM+sVHqnA4gsQAwe/+M0D5DX5 JhqdqInJQ88xC7zxiNpGGVKBIInpEmtUrNUltjeSyrsZsPzH5NTz5BBNrPVF1Kcc0ui7 KVMg== MIME-Version: 1.0 X-Received: by 10.50.138.105 with SMTP id qp9mr7008745igb.2.1403809556005; Thu, 26 Jun 2014 12:05:56 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Thu, 26 Jun 2014 12:05:55 -0700 (PDT) Date: Thu, 26 Jun 2014 12:05:55 -0700 Message-ID: Subject: How do I extend Kyua requirements checking for FreeBSD? From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 19:05:56 -0000 Hello Julio! I'm looking at extending Kyua to create special FreeBSD-specific functionality, and I'm wondering how I should do it. In particular there are a couple items that could be checked as prerequisite requirements before running tests to avoid running tests: 1. Is a service running? 2. Is a driver loaded? 3. Is a driver loaded as a kernel module or is it statically compiled in to the kernel? 4. Am I running on a particular filesystem? Etc. If I was to add these "requirements functions" to kyua, how would I query the values in an interface agnostic way? I would usually set environment variables, but this makes things potentially messier (especially in C/C++ unless I create a standalone library to handle this)/harder to grep for than having a command which fetches this information, like atf-config does for the ATF tester interface. Thank you! -Garrett From owner-freebsd-testing@FreeBSD.ORG Thu Jun 26 19:39:57 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93FC35E4; Thu, 26 Jun 2014 19:39:57 +0000 (UTC) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 092DD203D; Thu, 26 Jun 2014 19:39:56 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id bs8so1693758wib.13 for ; Thu, 26 Jun 2014 12:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=roaN15d/ZJzosmYLaqqrrxDX9HehhdCZt0E8th8ecIs=; b=pbmdKOyXPgEnJON2e2eSPwdLd3MQag2dgW+wJDi0n3hvXdRq8STbFJEtaXxHX1rAf9 QswrfPB7TLHVjefQa7T9YKUJzC3t0SXX152Zh5dML0ikVbXGGD/oNKU1WSRHshgdKHLz W0Yp/8oxI2zk+Ydw/dLFdEgNf4H9dq4cJmVfmq/RAL52qgmaBWUYZHo216Pt5IeZiGqc sI/KHw4QDHJmSCqgVWghnWX9bBzBnk170I/X5ZV7MnEk6kKMzg+fotAnZE1rwVd5qg3C HkFPKNtuPynFcgG8t+nvSd5kqG2BBgQCVmLVAT1JfcItAKcur85XFH9/7Z2e+M7RPAoS RYiQ== MIME-Version: 1.0 X-Received: by 10.180.108.103 with SMTP id hj7mr6806848wib.82.1403811594701; Thu, 26 Jun 2014 12:39:54 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.168.202 with HTTP; Thu, 26 Jun 2014 12:39:54 -0700 (PDT) In-Reply-To: References: Date: Thu, 26 Jun 2014 13:39:54 -0600 X-Google-Sender-Auth: I2Hof9kEcyNq9-_ebJDjB7IPRG4 Message-ID: Subject: Re: How do I extend Kyua requirements checking for FreeBSD? From: Alan Somers To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 19:39:57 -0000 On Thu, Jun 26, 2014 at 1:05 PM, Garrett Cooper wrote: > Hello Julio! > I'm looking at extending Kyua to create special FreeBSD-specific > functionality, and I'm wondering how I should do it. In particular > there are a couple items that could be checked as prerequisite > requirements before running tests to avoid running tests: > 1. Is a service running? > 2. Is a driver loaded? > 3. Is a driver loaded as a kernel module or is it statically > compiled in to the kernel? > 4. Am I running on a particular filesystem? > Etc. > If I was to add these "requirements functions" to kyua, how would > I query the values in an interface agnostic way? I would usually set > environment variables, but this makes things potentially messier > (especially in C/C++ unless I create a standalone library to handle > this)/harder to grep for than having a command which fetches this > information, like atf-config does for the ATF tester interface. > Thank you! > -Garrett I did it just be creating a library of shell functions that I call from test bodies. For example, something like this function require_module() { kldstat $1 || atf_skip "Module $1 is not loaded" } But that's not useful for C and C++ tests. And it would be way cooler if it were more tightly integrated into Kyua. Would it be possible to load site-specific Lua code that would handle this sort of functionality, so the test could simply say "atf_set require.modules aio"? -Alan From owner-freebsd-testing@FreeBSD.ORG Thu Jun 26 19:54:43 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC9B2D91; Thu, 26 Jun 2014 19:54:43 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6924621E9; Thu, 26 Jun 2014 19:54:43 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id rl12so3393845iec.25 for ; Thu, 26 Jun 2014 12:54:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=iahlq07v5qRMU+g6t5i0aR7Q6pWlX/1Wz5nAgA3wSKM=; b=Q8YUhkP6Gb/fgUQ3wI6qP2qc2UVq2PMlrLXieHQdYAOfVImBx0rftJwfce90/nqUQ7 JSeT1Ti2eqHp1M5TR1kNa3WjqKBpJ6oMpvEp1brS3EtuDf1+0D+pHZXfQHXbD54eQpUC EhN84SwegQ06VIhQBEEATGZRGyD43h4ox4BHqytOmXIAldDmkpcV+Lz94FZCCVFvcrJ2 5VVrN4xlJMzAJ30v1j1m2NcHi07NRjFDNY2Aevh0gM795P4wiMfLiWhqPmeqz7c0sPC+ Aiyo1QbNxUq4LgUdETSWWlAN+KFxqx0NQLvxkviqxMWYbGDhbatJfK9mG6LNxklPOmR1 IgXA== MIME-Version: 1.0 X-Received: by 10.43.54.70 with SMTP id vt6mr4577221icb.91.1403812482887; Thu, 26 Jun 2014 12:54:42 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Thu, 26 Jun 2014 12:54:42 -0700 (PDT) In-Reply-To: References: Date: Thu, 26 Jun 2014 12:54:42 -0700 Message-ID: Subject: Re: How do I extend Kyua requirements checking for FreeBSD? From: Garrett Cooper To: Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 19:54:43 -0000 On Thu, Jun 26, 2014 at 12:39 PM, Alan Somers wrote: > On Thu, Jun 26, 2014 at 1:05 PM, Garrett Cooper wrote: >> Hello Julio! >> I'm looking at extending Kyua to create special FreeBSD-specific >> functionality, and I'm wondering how I should do it. In particular >> there are a couple items that could be checked as prerequisite >> requirements before running tests to avoid running tests: >> 1. Is a service running? >> 2. Is a driver loaded? >> 3. Is a driver loaded as a kernel module or is it statically >> compiled in to the kernel? >> 4. Am I running on a particular filesystem? >> Etc. >> If I was to add these "requirements functions" to kyua, how would >> I query the values in an interface agnostic way? I would usually set >> environment variables, but this makes things potentially messier >> (especially in C/C++ unless I create a standalone library to handle >> this)/harder to grep for than having a command which fetches this >> information, like atf-config does for the ATF tester interface. >> Thank you! >> -Garrett > > I did it just be creating a library of shell functions that I call > from test bodies. For example, something like this > function require_module() { > kldstat $1 || atf_skip "Module $1 is not loaded" > } This is sort of what I'm doing, but this only works for the ATF test interface :/. I used kldstat -m $1 by the way for detecting statically compiled kernel modules, like aio: # kldstat -q -m ada && echo ada is loaded ada is loaded # kldstat -q -m foobar && echo foobar is loaded # It's significantly trickier with drivers that don't have moduledata_t which matches the kernel module that's loaded, e.g. if_em: # kldstat -m pci/em Id Refs Name 117 1 pci/em # ifconfig em0 em0: flags=8843 metric 0 mtu 1500 options=9b ether 00:0c:29:a3:e9:9c inet 10.7.216.233 netmask 0xfffffe00 broadcast 10.7.217.255 nd6 options=29 media: Ethernet autoselect (1000baseT ) status: active # kldload if_em kldload: can't load if_em: module already loaded or in kernel > But that's not useful for C and C++ tests. And it would be way cooler > if it were more tightly integrated into Kyua. Would it be possible to > load site-specific Lua code that would handle this sort of > functionality, so the test could simply say "atf_set require.modules > aio"? Exactly! I'm going to write the C/shell libraries for ATF at least, add some testcases, then we can look at making them generic via Kyuafile/the testcase header/etc. Thanks :)! -Garrett From owner-freebsd-testing@FreeBSD.ORG Fri Jun 27 00:00:11 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2DC21ED; Fri, 27 Jun 2014 00:00:11 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E43E26EE; Fri, 27 Jun 2014 00:00:11 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id tr6so3822273ieb.1 for ; Thu, 26 Jun 2014 17:00:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8yQTqWxq9Snlayu15hPV9sFZvabPJMZWIGu6EQsMKFI=; b=YSSvZ37DbrG1e7Fo7k/R9hURk2pY7XesJ5S1pF8frWhintX0yMAsFk+vkq/S2FB7JQ A8FzAIXuTPF6KqvemWS0JYqhOwjnrhEvCgXXFiA1KeAXK1Ee1G3WJlCFa/2hNUPO91X0 tIceL4U8dVGjUULFH6L2LcT2NUp5zR7mi0KHmEGEzK/TDCn/aQ6fPEKJoPVD1njK9SE5 6Vds169kUPJipXxF9fdYDMt1jy/yPZopU8fxaUFNOJYUKTYBLCoUsOFRha3G59m9BvG7 yqobJbym5c19m06MDNB3Kyd6HtKDrdYYg1SvramT2lTCstRQjhODzXa8LtyOSdqoN+cX TBpw== MIME-Version: 1.0 X-Received: by 10.50.138.105 with SMTP id qp9mr8468032igb.2.1403827210513; Thu, 26 Jun 2014 17:00:10 -0700 (PDT) Received: by 10.50.28.198 with HTTP; Thu, 26 Jun 2014 17:00:10 -0700 (PDT) In-Reply-To: References: Date: Thu, 26 Jun 2014 17:00:10 -0700 Message-ID: Subject: Re: How do I extend Kyua requirements checking for FreeBSD? From: Garrett Cooper To: Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 00:00:11 -0000 On Thu, Jun 26, 2014 at 12:54 PM, Garrett Cooper wrote: > On Thu, Jun 26, 2014 at 12:39 PM, Alan Somers wrote: >> On Thu, Jun 26, 2014 at 1:05 PM, Garrett Cooper wrote: >>> Hello Julio! >>> I'm looking at extending Kyua to create special FreeBSD-specific >>> functionality, and I'm wondering how I should do it. In particular >>> there are a couple items that could be checked as prerequisite >>> requirements before running tests to avoid running tests: >>> 1. Is a service running? >>> 2. Is a driver loaded? >>> 3. Is a driver loaded as a kernel module or is it statically >>> compiled in to the kernel? >>> 4. Am I running on a particular filesystem? >>> Etc. >>> If I was to add these "requirements functions" to kyua, how would >>> I query the values in an interface agnostic way? I would usually set >>> environment variables, but this makes things potentially messier >>> (especially in C/C++ unless I create a standalone library to handle >>> this)/harder to grep for than having a command which fetches this >>> information, like atf-config does for the ATF tester interface. >>> Thank you! >>> -Garrett >> >> I did it just be creating a library of shell functions that I call >> from test bodies. For example, something like this >> function require_module() { >> kldstat $1 || atf_skip "Module $1 is not loaded" >> } > > This is sort of what I'm doing, but this only works for the ATF test > interface :/. I used kldstat -m $1 by the way for detecting statically > compiled kernel modules, like aio: > > # kldstat -q -m ada && echo ada is loaded > ada is loaded > # kldstat -q -m foobar && echo foobar is loaded > # > > It's significantly trickier with drivers that don't have moduledata_t > which matches the kernel module that's loaded, e.g. if_em: > > # kldstat -m pci/em > Id Refs Name > 117 1 pci/em > # ifconfig em0 > em0: flags=8843 metric 0 mtu 1500 > options=9b > ether 00:0c:29:a3:e9:9c > inet 10.7.216.233 netmask 0xfffffe00 broadcast 10.7.217.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > # kldload if_em > kldload: can't load if_em: module already loaded or in kernel > >> But that's not useful for C and C++ tests. And it would be way cooler >> if it were more tightly integrated into Kyua. Would it be possible to >> load site-specific Lua code that would handle this sort of >> functionality, so the test could simply say "atf_set require.modules >> aio"? > > Exactly! > > I'm going to write the C/shell libraries for ATF at least, add some > testcases, then we can look at making them generic via Kyuafile/the > testcase header/etc. I'm started work on it on github: https://github.com/yaneurabeya/freebsd_atf_kyua_test_extensions . Any review would be much appreciated! Testcases coming soon of course. Thanks! -Garrett From owner-freebsd-testing@FreeBSD.ORG Fri Jun 27 15:44:35 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23A775E1 for ; Fri, 27 Jun 2014 15:44:35 +0000 (UTC) Received: from mail-qg0-f42.google.com (mail-qg0-f42.google.com [209.85.192.42]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D50B32D21 for ; Fri, 27 Jun 2014 15:44:34 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id e89so4626656qgf.1 for ; Fri, 27 Jun 2014 08:44:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=HurkenV4lxkm+GXDvwuGGBTqrWpaFlbVl7gVV6mGA+I=; b=d0/tftNWbKkWt9sSyuzCOoHFwAjXtkLdYgmjGDds3oxMsYT/0bEFPUJYcKntxOQdPI xhhO6qozsfd1RMEbU6ov5qzlIcTK2ux07kI54gNDE/Z6C5ttV5fepuWYDXhVRMOcGBJj g8RmKyUoH5uZjt0j4E8+IPD4hZBhYd+XrnpZBQ4WEKc+lrakA2b2HCNOdOh3uMHsXNMF CnsJ7TbqHGOEMlgck9B/SZtVvPbH36tqo5djyhVpxYLIG2zDYU+lgxNTY/AkpIUxoRq9 D9x59gEZMyyQEU2V8dSCn5x8QgI7cTlcGkjEkZXbftQsJmMsjvJTy6ipfde1dmX6iVC9 7kKw== X-Gm-Message-State: ALoCoQlCgzDCfd9WC4d3Z+eLYYqlBTB1ryIJwPlt2jQrBxJ02QHU5v4H38jhqoJFTLc0bMIUYqmH X-Received: by 10.140.91.226 with SMTP id z89mr32294994qgd.65.1403883867188; Fri, 27 Jun 2014 08:44:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.83.99 with HTTP; Fri, 27 Jun 2014 08:44:07 -0700 (PDT) X-Originating-IP: [172.26.40.81] In-Reply-To: References: From: Julio Merino Date: Fri, 27 Jun 2014 11:44:07 -0400 Message-ID: Subject: Re: Kyua + Jenkins integration To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jun 2014 15:44:35 -0000 On Mon, Jun 16, 2014 at 4:59 PM, Craig Rodrigues wrote: > On Wed, Jun 11, 2014 at 3:23 AM, Julio Merino wrote: >> >> https://github.com/jmmv/kyua/commit/7acd1a1fa05e600ac41094857ee1344f4ae7adb5 >> > > By default, "kyua report-html" will write output files in the ./html directory, > unless overridden by "--output [directory]". > However, "kyua report-junit" seems to write output files to stdout > instead by default. > > You might want to make the default behavior of "kyua report-junit" by closer > to "kyua report-html", i.e. do not write the output files to stdout by default. This is intentional and is part of the reason why every output format has its own command: their behaviors are different and attempting to represent them with the same command-line interface would be more confusing. The HTML output generates more than one file so it cannot write to stdout. But that's the exception, not the norm: both report and report-junit generate single files and print them to stdout. The plan is to make report-html write a single file by default (with an option for chunked output) and write that to stdout. Other than for consistency reasons, this would resolve some of the concerns with the current HTML output (aka harder to manage than atf's due to the multi-file setup.) From owner-freebsd-testing@FreeBSD.ORG Wed Jul 2 00:27:24 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 961C5E91; Wed, 2 Jul 2014 00:27:24 +0000 (UTC) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F2A720B2; Wed, 2 Jul 2014 00:27:24 +0000 (UTC) Received: by mail-ig0-f177.google.com with SMTP id c1so6048829igq.16 for ; Tue, 01 Jul 2014 17:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=/7wl+DewZpdezpaOakjtI7TubW0zvOyi4okIiLFFutM=; b=oZj+c8L3GJ2AzIMgXAj3smAkWs7nUEFFNx+isHp3h9rIYo7W2cVOnZ9ekFwxdfm5z6 54y2zGlR8TJeP/Tu2YYKX+JGGe8K6tqexiDbjO+RAw8915I8kn3RddCjDHjfP5G3zRgU NATLPQP2O/qB4rgz/ICl04Gkk372tVmANQIuUNhgFBDtYLUtN2RaQyPXVphvt0OMzZ12 BRkgyBq3Q9vMuGVIIhlXaYdWsGWnOkOx72rKy5feRUdIQRfh5UuD6rzZAiWr0pKWk2nW qWFqHbi4FKDUDVXSMZ8JKt6T9TbO/4CVjZp9vhalH31F8CRqpkgb4CgUmp3vTAP1U7xj juzQ== MIME-Version: 1.0 X-Received: by 10.50.152.98 with SMTP id ux2mr45372922igb.27.1404260843852; Tue, 01 Jul 2014 17:27:23 -0700 (PDT) Received: by 10.50.155.195 with HTTP; Tue, 1 Jul 2014 17:27:23 -0700 (PDT) Date: Tue, 1 Jul 2014 17:27:23 -0700 Message-ID: Subject: Pull request for fixing TAP parser bugs in Kyua From: Garrett Cooper To: Julio Merino Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , Peter Holm X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2014 00:27:24 -0000 Hi Julio! I opened up a pull request yesterday to fix a number of bugs in Kyua related to the TAP parser interface: https://github.com/jmmv/kyua/pull/91 . Could you (or someone else) on the -testing list review my changes? Fixing this would resolve issues with integration with pjdfstest (some of the testcases use TODO/SKIP), and a few other bugs ( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191022 for instance). I ran all of the tests I've integrated in from tools/regression and they passed, with exception of some test bugs that I verified with prove afterwards (which I later fixed, then re-verified with both prove and kyua). Thanks! -Garrett PS The patched port can be found on my kyua-testers-tap-fixes branch ( https://github.com/yaneurabeya/freebsd-ports/tree/kyua-testers-tap-fixes ), under devel/kyua-testers . From owner-freebsd-testing@FreeBSD.ORG Thu Jul 3 00:39:29 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60DBE517; Thu, 3 Jul 2014 00:39:29 +0000 (UTC) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9B23725A8; Thu, 3 Jul 2014 00:39:28 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id s7so8634774lbd.30 for ; Wed, 02 Jul 2014 17:39:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=HNxRmsXMzm6GuOULk4Odl0ysi4i85WbHkSs47yobMXA=; b=euhPyuSQeTVN+yvpa3eHqNjMQZVZit5kthAotLwngVsgXoxYDSl1fvUeHfVu6UUeyD LnqKNE7qIxODVrI/C4NiMVfr5d7CHWpq3ao486Adb+e5f4P4O68mZymcnq5nOuwrF23n vBzyCwIk0wDTnrMBWgIgBbtRXBBN9jNLmNGQuouvxf8wvb70o5DZjHzFwTF1zsc7ttEd am93p0X6tQZ9rsueBnDwpcOvi7RROx76E+EEb9qNnHBR3i6q+F0O44OT8UaUN3ANgPkj 1z89HEwSFcV7PCagkrE15l8BolibcQvYybZGuosAoq9txNqZiXL0YWvWi70XRbixyG2a F45g== MIME-Version: 1.0 X-Received: by 10.112.97.202 with SMTP id ec10mr902439lbb.20.1404347966382; Wed, 02 Jul 2014 17:39:26 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Wed, 2 Jul 2014 17:39:26 -0700 (PDT) In-Reply-To: References: Date: Wed, 2 Jul 2014 17:39:26 -0700 X-Google-Sender-Auth: zt-zJSIKCTuuP7L-FhG_UDkJmyo Message-ID: Subject: Re: Kyua + Jenkins integration From: Craig Rodrigues To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 00:39:29 -0000 Hi, OK, thanks for the explanation. It would be useful if the reporting options had a flag to direct output to a file: kyua report-junit -o foo.xml instead of: kyua report-junit > foo.xml However, without the "-o" flag, I can still live with it. Can you do another release of kyua with this change, and maybe Garrett's TAP fixes? What you have done is very useful, and I want to deploy it on multiple machines via "pkg install" instead of downloading the source and building it myself. Thanks. -- Craig On Fri, Jun 27, 2014 at 8:44 AM, Julio Merino wrote: > On Mon, Jun 16, 2014 at 4:59 PM, Craig Rodrigues > wrote: > > On Wed, Jun 11, 2014 at 3:23 AM, Julio Merino wrote: > >> > >> > https://github.com/jmmv/kyua/commit/7acd1a1fa05e600ac41094857ee1344f4ae7adb5 > >> > > > > By default, "kyua report-html" will write output files in the ./html > directory, > > unless overridden by "--output [directory]". > > However, "kyua report-junit" seems to write output files to stdout > > instead by default. > > > > You might want to make the default behavior of "kyua report-junit" by > closer > > to "kyua report-html", i.e. do not write the output files to stdout by > default. > > This is intentional and is part of the reason why every output format > has its own command: their behaviors are different and attempting to > represent them with the same command-line interface would be more > confusing. The HTML output generates more than one file so it cannot > write to stdout. But that's the exception, not the norm: both report > and report-junit generate single files and print them to stdout. > > The plan is to make report-html write a single file by default (with > an option for chunked output) and write that to stdout. Other than for > consistency reasons, this would resolve some of the concerns with the > current HTML output (aka harder to manage than atf's due to the > multi-file setup.) > From owner-freebsd-testing@FreeBSD.ORG Thu Jul 3 19:47:57 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD0E972F; Thu, 3 Jul 2014 19:47:57 +0000 (UTC) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1CE0B2D62; Thu, 3 Jul 2014 19:47:56 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id s7so517253lbd.4 for ; Thu, 03 Jul 2014 12:47:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=S7pY/UTfTgOCmKZygoXlDnGmnrODecAtOvyxXUAZ4Pk=; b=G0LI0InmiwHf0Z+Q2B8UC/MG/zwl6dhAKYRYHSylN8qlwvYnbCusqdgqxz8mTkzlAT kCiREwwKNIiHk2vSRKkCj+ZNo+vV5oas7+2welgLRlojIiFiG5JcAwz5doHnZWOutJRw zrF+s5PVkwXcfu0l94bDlq4DyJF0tQYxicXdR5nIdz4NAme5SU8lXzD5taBj+hpuzpky f782nFunA0XtLXGbj+1AfX0yIVX2T4uXLK/8oI7VWRWc6M16OT62X2DO2J02X/ExW2xf /RHA3bb7sIu+zo5PMxS11enfPsko78ZyVgCCR9JHjcRo5/BDLZngadwgAoM55dpkZPPQ KN2Q== MIME-Version: 1.0 X-Received: by 10.152.29.168 with SMTP id l8mr4820445lah.35.1404416874919; Thu, 03 Jul 2014 12:47:54 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Thu, 3 Jul 2014 12:47:54 -0700 (PDT) Date: Thu, 3 Jul 2014 12:47:54 -0700 X-Google-Sender-Auth: N9qE20FZ3gfxl9s_ALbkWcXJt5k Message-ID: Subject: Jenkins interview on BSDNow.tv From: Craig Rodrigues To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 19:47:57 -0000 I was interviewed by BSDNow.tv to talk about the work that I have done to set up Jenkins in the FreeBSD cluster. The interview is here: http://www.bsdnow.tv/episodes/2014_07_02-base_iso_100 It starts at 00:12:59, and goes until about 01:00:00, almost 45 minutes!! Thanks to all the jenkins-admin@ team members for helping to set up and maintain Jenkins in the FreeBSD cluster! -- Craig From owner-freebsd-testing@FreeBSD.ORG Sun Jul 6 03:06:04 2014 Return-Path: Delivered-To: testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1B32BA7; Sun, 6 Jul 2014 03:06:04 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C9F7B288A; Sun, 6 Jul 2014 03:06:01 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:52945 helo=[192.168.1.10]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1X3cl6-00076R-BG; Sat, 05 Jul 2014 23:04:56 -0400 From: "George Neville-Neil" To: testing@freebsd.org, net@freebsd.org Subject: A new way to test systems in multiple machine scenarios... Date: Sat, 05 Jul 2014 23:04:55 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-Mailer: MailMate (1.7.2r3905) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 03:06:05 -0000 Hi, I've coded up a system to allow you to control multiple other systems for use in testing. https://github.com/gvnn3/conductor It's BSD licensed, of course, and is only alpha quality but I'm using it in the test lab to control hosts in forwarding tests. I'll be updating the system frequently over the coming months as I build out more test scenarios, add documentation and the like. There are two main scripts, player, and conductor. You run N players, one per machine, and a single conductor. The conductor controls the players by sending down phases which are encoded in INI style configs. There are a few, simple, samples in the config/ directory of the project. Best, George NOTE: Conductor MUST run as root to be useful. Do NOT run on the open Internet. It is meant for private test labs. From owner-freebsd-testing@FreeBSD.ORG Sun Jul 6 03:52:08 2014 Return-Path: Delivered-To: testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E28418D; Sun, 6 Jul 2014 03:52:08 +0000 (UTC) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F6D52BDD; Sun, 6 Jul 2014 03:52:08 +0000 (UTC) Received: by mail-pa0-f45.google.com with SMTP id rd3so3656631pab.18 for ; Sat, 05 Jul 2014 20:52:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=3cqFEQ/HxvRouvdmKBBL16okKV+iBeV03h2UtwkdR6U=; b=YGvmLqNE+DjG1x+OGNNvXpVYX1Lmxt2WZXaf1BMMaI/HEp+EQV8q4UGvMz7013S4/j stnkvOiYNXM1HgEPGjdHqCH+KVR6xXJ72Zvgj0fstH2u9OwqWn4XR7LLCCfvYPVFWBft Emb9F2y1CZfQOeYBizA/Ps/fUVMf82fsT+yQxDFpbI6T0bO1JbC7og8waT7FirnJqF/9 pegB7ZhyFDLrqsZuUO9an6A+aBcrwpaz4BQA7Ikh2FNbToaHWGnxpuszHE0l6EwOge70 +YfZuqr1XwIJgjB663cQmTqBNfoR+lklxFBCv5xaZEQL+dJN+zsOvVSwYocB03rOB0W2 p4IA== X-Received: by 10.70.102.10 with SMTP id fk10mr21886pdb.111.1404618727801; Sat, 05 Jul 2014 20:52:07 -0700 (PDT) Received: from [10.101.19.180] (mobile-166-137-212-195.mycingular.net. [166.137.212.195]) by mx.google.com with ESMTPSA id b3sm6331748pbu.8.2014.07.05.20.52.06 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 05 Jul 2014 20:52:06 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <19D0342C-3635-4DC1-ACB8-5697F1D579F0@gmail.com> X-Mailer: iPhone Mail (11D257) From: Garrett Cooper Subject: Re: A new way to test systems in multiple machine scenarios... Date: Sat, 5 Jul 2014 20:52:03 -0700 To: George Neville-Neil Cc: "testing@freebsd.org" , "net@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 03:52:08 -0000 > On Jul 5, 2014, at 20:04, "George Neville-Neil" wro= te: >=20 > Hi, >=20 > I've coded up a system to allow you to control multiple other systems for u= se in testing. >=20 > https://github.com/gvnn3/conductor >=20 > It's BSD licensed, of course, and is only alpha quality but I'm using it i= n the test lab > to control hosts in forwarding tests. >=20 > I'll be updating the system frequently over the coming months as I build o= ut more test scenarios, > add documentation and the like. >=20 > There are two main scripts, player, and conductor. You run N players, one= per machine, and > a single conductor. The conductor controls the players by sending down ph= ases which are > encoded in INI style configs. There are a few, simple, samples in the con= fig/ directory > of the project. >=20 > Best, > George >=20 > NOTE: Conductor MUST run as root to be useful. Do NOT run on the open Int= ernet. It is meant > for private test labs. I took a quick glance at the code -- have you considered using multiprocessi= ng managers instead? https://docs.python.org/2/library/multiprocessing.html#managers -Garrett= From owner-freebsd-testing@FreeBSD.ORG Mon Jul 7 04:06:49 2014 Return-Path: Delivered-To: testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01D1DFFD; Mon, 7 Jul 2014 04:06:49 +0000 (UTC) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 523F22519; Mon, 7 Jul 2014 04:06:48 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id s7so2448405lbd.18 for ; Sun, 06 Jul 2014 21:06:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UCrwhexUbtFEPxDcTnsSquDf8fS7I4/MURVzLxeZgTY=; b=hseFJXb+inyVMx8OXAP6k8XPXGogvVFdWI3Ui1a16n1JxqiS9jNQy42XO+xGEbze9U NaQI16i4VqElZRH0KC0+HYAnJGfz5yJDE45iWZfgwam9RPpCjTwmqcIGSNsAKBdWnsQS lBoDMEtKpDT/s/o64G5Iv7hO1eaWS9caF8jt+ABbvzgQAGZco3aEpQfApNHa3agzkmWa 5/kEPhhxDkJ1HLhb0z4654c+2+ah7N4eis+BPoXGUpwwpVJdVvFXcZLeyqeOXM+oD8lD SQysMIL2pi9k1U0gEfDzn+hOzpgkDmOBVsqb/FJAI0CBSjTR7NK5O2JJDSMchPeVDN6z CQsQ== MIME-Version: 1.0 X-Received: by 10.152.2.3 with SMTP id 3mr20962684laq.8.1404706006074; Sun, 06 Jul 2014 21:06:46 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Sun, 6 Jul 2014 21:06:46 -0700 (PDT) In-Reply-To: References: Date: Sun, 6 Jul 2014 21:06:46 -0700 X-Google-Sender-Auth: -lzvqJltz1khunkvyvAzA0-hnFQ Message-ID: Subject: Re: A new way to test systems in multiple machine scenarios... From: Craig Rodrigues To: George Neville-Neil Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 04:06:49 -0000 On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil wrote: > Hi, > > I've coded up a system to allow you to control multiple other systems for > use in testing. > > https://github.com/gvnn3/conductor > > Cool! The architecture you have is similar to that of the SPECsfs benchmark test ( http://www.spec.org/sfs2008/ ) which involves a "coordinator node" and multiple "client nodes" which direct NFS network traffic towards a System Under Test (SUT). Garrett Cooper actually set up the original testbed that I am using now at iXsystems. :) It would be cool to put together tools like Jenkins, Kyua, and conductor to do more advanced testing of FreeBSD before the project puts out releases. -- Craig From owner-freebsd-testing@FreeBSD.ORG Tue Jul 8 08:56:17 2014 Return-Path: Delivered-To: testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60FD1172; Tue, 8 Jul 2014 08:56:17 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28FCA2B14; Tue, 8 Jul 2014 08:56:17 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id fp1so6864087pdb.11 for ; Tue, 08 Jul 2014 01:56:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:mime-version:subject:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Q4k3NbEB1g6W2YFBRhljXgOqAWhtzsVqsntre7gfKJ0=; b=FjiX53SKVpKO/SBJ6b3CkRxoE86+M6LP/w50jVjVvUCia38CpJPgESbZJtkWgc6gvb 5MtUZuwbsWHcgz/Nj474py5fG6/nCJY5LzZ23Z6V33gXgziYyW2QZiWiR8KvDrVlj9Xd 56AOKeELEw5qL5hbpPtHH0U1aB57OpOS1cOP4H+xxTMmEMsEx+rAww5R+ToOavRumOVX mzGoMHhQ7Ww7SLeSnVyoMIVm5FGKThlXscm0pEwLpUsRWUznwbvzkM9cM79n5WviH0fm BMCsR3SAQfDw05s3K1YgenesRYdZ8vPKhdbxnp63xGcMTNgYIbH9mTsWEYSuQ32NTapf hbfg== X-Received: by 10.66.119.39 with SMTP id kr7mr1984020pab.131.1404809776724; Tue, 08 Jul 2014 01:56:16 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:8045:a94a:de:b331? ([2601:8:ab80:7d6:8045:a94a:de:b331]) by mx.google.com with ESMTPSA id w16sm23469180pdl.36.2014.07.08.01.56.15 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 01:56:15 -0700 (PDT) From: Garrett Cooper X-Google-Original-From: Garrett Cooper Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: A new way to test systems in multiple machine scenarios... In-Reply-To: Date: Tue, 8 Jul 2014 01:56:14 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> References: To: Craig Rodrigues X-Mailer: Apple Mail (2.1878.2) Cc: George Neville-Neil , "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 08:56:17 -0000 On Jul 6, 2014, at 9:06 PM, Craig Rodrigues wrote: > On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil = > wrote: >=20 >> Hi, >>=20 >> I've coded up a system to allow you to control multiple other systems = for >> use in testing. >>=20 >> https://github.com/gvnn3/conductor >>=20 >>=20 > Cool! The architecture you have is similar to that of the SPECsfs > benchmark test ( http://www.spec.org/sfs2008/ ) > which involves a "coordinator node" and multiple "client nodes" which > direct NFS network > traffic towards a System Under Test (SUT). Garrett Cooper actually = set up > the original testbed > that I am using now at iXsystems. :) >=20 > It would be cool to put together tools like Jenkins, Kyua, and = conductor to > do more advanced testing > of FreeBSD before the project puts out releases. Agreed. The only thing that I have some concern about is the reinventing = of the wheel in python. multiprocessing Managers are one viable option = that=92s existed since python 2.6; there=92s a learning curve though, = and you=92ll run into problems with pickling some objects because the = pickle protocol is far from complete (example: = http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemethod= -when-using-pythons-multiprocessing-pool-ma ); you might run into this = problems regardless because you=92re serializing objects using pickle = instead of using dill (or using a simpler serialization method like = JSON). Fabric has a framework that=92s nice to use if you have ssh = capability. There are other frameworks that use twisted conch I think = too (another library that implements ssh access). Isilon has a framework they use, but it=92s very customized to their = infrastructure and product assumptions and it=92s in need of an overhaul = :(. -Garrett= From owner-freebsd-testing@FreeBSD.ORG Tue Jul 8 11:34:17 2014 Return-Path: Delivered-To: testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AAA7C86; Tue, 8 Jul 2014 11:34:17 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23BF62838; Tue, 8 Jul 2014 11:34:17 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id ik5so5235448vcb.35 for ; Tue, 08 Jul 2014 04:34:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Cpgqfvy+eOW1xWLY6vvVz9fRr+x12KuzlqB9lucwcfM=; b=ejb/EZcYS/789HJPDdsyx/wdPEypEYdInI1hNNlrtf+ta0YEyKhiKDWqO3nq/3U+CS 5+ZaRkanOcM8Coexu7bBKRclBS3BXiJuP1HFIDRMtueJjoACPJlFQMGa8o3GSSSTzbn0 Wuoy4vJbuPT++pT58jcuOed7cc86KWP1iNiAUwZ7KOmcQscpPpCc7HpX6XeySLYt9lAq oXrW8KdWylFt2OqM0OUMcFHJ2r6IExQ6yW6ej1Uoqf2Xuxxwy8if2hGu7hksZVqvdqHU t3BlvfAK1rAtBqDihOiR+jJSK89La9xO2n4jeyAsyomTtDhlnCTL4LN86Ma1kL8HYDIQ HUyw== MIME-Version: 1.0 X-Received: by 10.53.7.204 with SMTP id de12mr963062vdd.41.1404819256010; Tue, 08 Jul 2014 04:34:16 -0700 (PDT) Sender: ndenev@gmail.com Received: by 10.220.136.79 with HTTP; Tue, 8 Jul 2014 04:34:15 -0700 (PDT) In-Reply-To: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> References: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> Date: Tue, 8 Jul 2014 12:34:15 +0100 X-Google-Sender-Auth: wcy4yYOB7tkSY-fVuFm9HWkVvhI Message-ID: Subject: Re: A new way to test systems in multiple machine scenarios... From: Nikolay Denev To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-testing@freebsd.org" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 11:34:17 -0000 On Tue, Jul 8, 2014 at 9:56 AM, Garrett Cooper wrote: > On Jul 6, 2014, at 9:06 PM, Craig Rodrigues wrote: > >> On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil >> wrote: >> >>> Hi, >>> >>> I've coded up a system to allow you to control multiple other systems f= or >>> use in testing. >>> >>> https://github.com/gvnn3/conductor >>> >>> >> Cool! The architecture you have is similar to that of the SPECsfs >> benchmark test ( http://www.spec.org/sfs2008/ ) >> which involves a "coordinator node" and multiple "client nodes" which >> direct NFS network >> traffic towards a System Under Test (SUT). Garrett Cooper actually set = up >> the original testbed >> that I am using now at iXsystems. :) >> >> It would be cool to put together tools like Jenkins, Kyua, and conductor= to >> do more advanced testing >> of FreeBSD before the project puts out releases. > > Agreed. The only thing that I have some concern about is the reinventing = of the wheel in python. multiprocessing Managers are one viable option that= =E2=80=99s existed since python 2.6; there=E2=80=99s a learning curve thoug= h, and you=E2=80=99ll run into problems with pickling some objects because = the pickle protocol is far from complete (example: http://stackoverflow.com= /questions/1816958/cant-pickle-type-instancemethod-when-using-pythons-multi= processing-pool-ma ); you might run into this problems regardless because y= ou=E2=80=99re serializing objects using pickle instead of using dill (or us= ing a simpler serialization method like JSON). Fabric has a framework that= =E2=80=99s nice to use if you have ssh capability. There are other framewor= ks that use twisted conch I think too (another library that implements ssh = access). > > Isilon has a framework they use, but it=E2=80=99s very customized to thei= r infrastructure and product assumptions and it=E2=80=99s in need of an ove= rhaul :(. > > -Garrett > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" This also looks as an interesting option : https://codespeak.net/execnet/ I haven't used personally, but judging from py-test (it's the same author) it should be good :) --Nikolay From owner-freebsd-testing@FreeBSD.ORG Wed Jul 9 15:33:45 2014 Return-Path: Delivered-To: freebsd-testing@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00F142AD; Wed, 9 Jul 2014 15:33:44 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 750F52B31; Wed, 9 Jul 2014 15:33:41 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s69FXVxF055255 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 9 Jul 2014 15:33:36 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s69FXVWT055254; Wed, 9 Jul 2014 15:33:31 GMT (envelope-from swills) Date: Wed, 9 Jul 2014 15:33:31 +0000 From: Steve Wills To: Craig Rodrigues Subject: Re: Jenkins interview on BSDNow.tv Message-ID: <20140709153331.GJ41010@mouf.net> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="yr/DzoowOgTDcSCF" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Wed, 09 Jul 2014 15:33:37 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: "freebsd-testing@freebsd.org" , "jenkins-admin@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 15:33:45 -0000 --yr/DzoowOgTDcSCF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 03, 2014 at 12:47:54PM -0700, Craig Rodrigues wrote: > I was interviewed by BSDNow.tv to talk about the work that I have done to > set up Jenkins in the FreeBSD cluster. The interview is here: >=20 > http://www.bsdnow.tv/episodes/2014_07_02-base_iso_100 > It starts at 00:12:59, and goes until about 01:00:00, almost 45 minutes!! >=20 > Thanks to all the jenkins-admin@ team members for helping > to set up and maintain Jenkins in the FreeBSD cluster! Finally watched this the other day, was very cool. I've mentioned the junit output to bdrewery with the idea of getting poudriere to support it as well, though that may not be the best idea. Would really mean setting up a build = in jenkins for every port and keeping that up to date, which would be difficul= t. Steve --yr/DzoowOgTDcSCF Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTvWDKAAoJEPXPYrMgexuhkwYH/23vg1f87tBjVpUCKiB0N+R2 tu5AZyaZ9e5hiKtxy++m0G/xwZmSFkmk/igF+6AF9TqnJyGC7YoFho7LRe9Gi0sA ojJDr900z23n9yFraDV4G0XOo3DQi6jp8JVuV5ZlcJL3zuFZ0xPUN2BKOzFV5tMg Gv6L5lytsr1zq+PrkEKobppcjmSVfkD73AIpYwNmOsNos55nk5xBjDkhWKt7s9xm kjzWmUGOoPQehlXMF8y82INs939RsWwxTwfDFZGWipL/8Kas2QKrr2mPIM1sJW5+ OZFaCkYGNwB03WgaXv9edpIzUHrKP4cl4GcSS5vbG8jWUVyJSPFXAAeeiacdVpo= =e5S+ -----END PGP SIGNATURE----- --yr/DzoowOgTDcSCF-- From owner-freebsd-testing@FreeBSD.ORG Thu Jul 10 15:22:00 2014 Return-Path: Delivered-To: testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48D9AF83; Thu, 10 Jul 2014 15:22:00 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1B2E325B2; Thu, 10 Jul 2014 15:21:59 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:52429 helo=[192.168.1.12]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1X5GAW-0006uM-Ky; Thu, 10 Jul 2014 11:21:57 -0400 From: "George Neville-Neil" To: "Garrett Cooper" Subject: Re: A new way to test systems in multiple machine scenarios... Date: Thu, 10 Jul 2014 11:21:54 -0400 Message-ID: In-Reply-To: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> References: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Mailer: MailMate (1.7.2r3905) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 15:22:00 -0000 On 8 Jul 2014, at 4:56, Garrett Cooper wrote: > On Jul 6, 2014, at 9:06 PM, Craig Rodrigues = > wrote: > >> On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil = >> >> wrote: >> >>> Hi, >>> >>> I've coded up a system to allow you to control multiple other = >>> systems for >>> use in testing. >>> >>> https://github.com/gvnn3/conductor >>> >>> >> Cool! The architecture you have is similar to that of the SPECsfs >> benchmark test ( http://www.spec.org/sfs2008/ ) >> which involves a "coordinator node" and multiple "client nodes" which >> direct NFS network >> traffic towards a System Under Test (SUT). Garrett Cooper actually = >> set up >> the original testbed >> that I am using now at iXsystems. :) >> >> It would be cool to put together tools like Jenkins, Kyua, and = >> conductor to >> do more advanced testing >> of FreeBSD before the project puts out releases. > > Agreed. The only thing that I have some concern about is the = > reinventing of the wheel in python. multiprocessing Managers are one = > viable option that=E2=80=99s existed since python 2.6; there=E2=80=99s = a learning = > curve though, and you=E2=80=99ll run into problems with pickling some = > objects because the pickle protocol is far from complete (example: = > http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemet= hod-when-using-pythons-multiprocessing-pool-ma = > ); you might run into this problems regardless because you=E2=80=99re = > serializing objects using pickle instead of using dill (or using a = > simpler serialization method like JSON). Fabric has a framework = > that=E2=80=99s nice to use if you have ssh capability. There are other = > frameworks that use twisted conch I think too (another library that = > implements ssh access). Yes, I learned quite a bit about pickling in writing this. Conductor = aims to be quite simple so I am hoping to avoid any crazy corner cases to do with pickling. > > Isilon has a framework they use, but it=E2=80=99s very customized to th= eir = > infrastructure and product assumptions and it=E2=80=99s in need of an = > overhaul :(. This, actually, is the problem I found. Lots of folks have partial = solutions that are either proprietary, internal, not read for prime time, not quite what we want, etc. etc. I = did get one private response of another system to look at as well. I basically did this as a stake in the ground around which to build = something we could possibly move forwards with. It's not a 100% solution but it's 80% of the solution to the = problem I run into 80% of the time. Best, George From owner-freebsd-testing@FreeBSD.ORG Thu Jul 10 17:07:40 2014 Return-Path: Delivered-To: testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1AA32163; Thu, 10 Jul 2014 17:07:40 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 682652F09; Thu, 10 Jul 2014 17:07:39 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id el20so6511543lab.35 for ; Thu, 10 Jul 2014 10:07:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UFBdicxDi6WH9o4QyTnTqrgUqm6bkxYXPvhmaJCUHVw=; b=FM7Vv7Ir27tqftU/QFsP0AI9n4mocOg9YegMYRhBMy5CQvCraMoPu74kqE0uHmf1pp qwxJHlhZGAjDqGwAAVM5p+OOWWpPJbpjHyqIuOhPVIKKx+6FxWEKQvRsC5FwBWH7Wtek 7LIa2+c2ZDz6dDhDI7I9Oay2sntpHK6PyGBccjV50/o3awq3mGxkme9E4HPpWJFMWbJs 6XV2AhPnYo4H0Y8rz3YKFAW7oN4TsWNna1hMKoYoWKJewbC0+GDCZ3EvkWWmNxXrIvt4 zLfPG1YuU2gwIxBVCSf6661fZcch0ZL4n0X5qAeQEW358p5bEzzDzjUeyqNe8dFOAFgx n42Q== MIME-Version: 1.0 X-Received: by 10.112.34.110 with SMTP id y14mr6566246lbi.36.1405012057304; Thu, 10 Jul 2014 10:07:37 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.71 with HTTP; Thu, 10 Jul 2014 10:07:37 -0700 (PDT) In-Reply-To: References: <154F5C90-B0A5-41AD-ABBC-C7ECE1281D7D@gmail.com> Date: Thu, 10 Jul 2014 10:07:37 -0700 X-Google-Sender-Auth: iMXwwAWS_9CLuTrjaRl5gM_G_-c Message-ID: Subject: Re: A new way to test systems in multiple machine scenarios... From: Craig Rodrigues To: George Neville-Neil Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Garrett Cooper , "freebsd-testing@freebsd.org" , net@freebsd.org X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 17:07:40 -0000 On Thu, Jul 10, 2014 at 8:21 AM, George Neville-Neil wrote: > This, actually, is the problem I found. Lots of folks have partial > solutions that are either proprietary, > internal, not read for prime time, not quite what we want, etc. etc. I > did get one private > response of another system to look at as well. > > I basically did this as a stake in the ground around which to build > something we could possibly move forwards > with. It's not a 100% solution but it's 80% of the solution to the > problem I run into 80% of the time. > You are absolutely right. Many companies have internal testing frameworks that are home-grown and proprietary. It's great for the individual company, but doesn't help people in the larger community. Since you put this stuff in github, I encourage folks to hack on the code and send pull requests to George to help improve it! -- Craig From owner-freebsd-testing@FreeBSD.ORG Wed Jul 16 01:47:51 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 73FB732A; Wed, 16 Jul 2014 01:47:51 +0000 (UTC) Received: from mail-ie0-x232.google.com (mail-ie0-x232.google.com [IPv6:2607:f8b0:4001:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39C472CC7; Wed, 16 Jul 2014 01:47:51 +0000 (UTC) Received: by mail-ie0-f178.google.com with SMTP id tp5so203201ieb.23 for ; Tue, 15 Jul 2014 18:47:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=mE3w1POD4ArUWNVPg6C77iem0J62t9JrkgajlDBf5eo=; b=IzgxbNRVL9YMxzeJb0BT1/URn78lQBgLTqlxTmknEHMI+iH0nkky9YoPBEADUBFOFm XQbtXU7d2Rm+3DMVprWt6mAVPAyyuGTEZgRXGK1Dsed0dcxmcD5/9NuPLr5bc8q82Ngn FFPclZBTI5hQUpv9uPM9CgqhL/kK7TIl6CkJiJYCaMb1m9vcSMt4s+H+VtUfUTjRndCs IlO1FV4caR/bydO+Mp3e8p0JVnUdFQJCARyDBnyUCNcdtpqqu1hSF5j56PTjj5Rs2yn6 L0kyVafHHNWihNrb/H201WfAL0goc8hpWYYrtJF4tCZ4GfHChlQV7o2lBar1bTEjPXEo MwsA== MIME-Version: 1.0 X-Received: by 10.50.112.136 with SMTP id iq8mr3977108igb.38.1405475270521; Tue, 15 Jul 2014 18:47:50 -0700 (PDT) Received: by 10.50.155.195 with HTTP; Tue, 15 Jul 2014 18:47:50 -0700 (PDT) In-Reply-To: References: Date: Tue, 15 Jul 2014 18:47:50 -0700 Message-ID: Subject: Re: Failures on head From: Garrett Cooper To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 01:47:51 -0000 On Fri, Jun 13, 2014 at 5:03 PM, Garrett Cooper wrote: > On Mon, May 12, 2014 at 2:06 PM, Garrett Cooper wrote: >> On Fri, Apr 25, 2014 at 2:03 PM, Garrett Cooper wrote: >>> Hi, >>> While looking at >>> http://kyua3.nyi.freebsd.org/head/data/0-LATEST/results/index.html , I >>> noticed that a handful of testcases are failing, in particular: >>> >>> Broken test cases >>> >>> usr.bin/sed/inplace_race_test:main >>> >>> Failed test cases >>> >>> bin/pkill/pgrep-j_test:main >>> bin/pkill/pkill-j_test:main >>> lib/atf/libatf-c/macros_test:use >>> usr.bin/sed/multi_test:main >>> usr.bin/yacc/legacy_test:main >> >> The yacc failures are because the tests in >> /usr/src/tools/regression are super stale. The tests should be picked >> up from contrib/byacc/test . I'll work on that. >> Thanks, >> -Garrett > > > I did some investigation and some of the bugs I noted above were test > bugs; others were test infrastructure bugs/missing enhancements. I've > filed bugs for all of the issues in the "vanilla" 11-CURRENT kyua > here: > > pgrep/pkill: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191019 I have a fix out for this which I'm requesting review for from Benno (if anyone else has the cycles to help though it would be much appreciated!). > libatf: https://github.com/jmmv/atf/issues/8 This is still outstanding. > sed: https://github.com/jmmv/kyua/issues/85 This is fixed as part of my TAP parser edgecase pull request that I mentioned earlier. > yacc: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191020 I've provided an initial patch to bapt@, but I would appreciate if someone else could review this as well. Thank you! -Garrett