From owner-freebsd-testing@freebsd.org Tue Jan 17 23:01:28 2017 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C395CB5DAE for ; Tue, 17 Jan 2017 23:01:28 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 180B41907 for ; Tue, 17 Jan 2017 23:01:28 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 175AFCB5DAD; Tue, 17 Jan 2017 23:01:28 +0000 (UTC) Delivered-To: testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16FDDCB5DAC for ; Tue, 17 Jan 2017 23:01:28 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pg0-x232.google.com (mail-pg0-x232.google.com [IPv6:2607:f8b0:400e:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E642A1905 for ; Tue, 17 Jan 2017 23:01:27 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pg0-x232.google.com with SMTP id 14so28095165pgg.1 for ; Tue, 17 Jan 2017 15:01:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:subject:date:message-id:to:mime-version; bh=6WziePpQi/8/5m8rY6SxWiVGYxc4y0ol/ZBvp/ucDnk=; b=aV66LobATNRXBrmCR4L+kJcgXnhJQpBJBJjutdbjTakcFagv47/3N+axbRqbSFLNin VpfNF/0NYPoA6a9Q2MemhQgK75gEwrYMU5oAqQrvpV2sEQFJTD84RhKwZWQwPlXyARd7 AmfAgjEPJGE5dqHHx76OUC6CsvoE7g1siqfvFfHDs4I+Wqtxo50flVYM71pCFgHAEYRN nby4vxXVYiAxBXY0dDl0ZqcuJi0oY3FFwoVg4JsKzBLXteFnRej0z1x/EEEDzqxSBaZ9 7/suCxQhDpZIBZ1EPB6zy0OE/cvPWg3gSj1dZaunqByNC1l5F3PwfANxPuX6DTcNdw7c Ljyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:date:message-id:to:mime-version; bh=6WziePpQi/8/5m8rY6SxWiVGYxc4y0ol/ZBvp/ucDnk=; b=I9FkPpS/OaInZ6DbTD2Zrsf5W8J/rkP1C7Xa27rq9p6CpbsJJR67/vQmv9AMvLlXPu HaHcQm9TyutvQlRfWdstPwQxSN4Bb5hkzu77Cl6ZcwCVkz3Hndnzj0Pmp426mK1ulB8e TOw/ymhUmJ5GIHqmyd9tWFhS9VvIZyVOFNsrBBic0gENhouhd4YADItRUzKvkdTZLX3T bu3GZZ9GisO2M3xrYbEAvGY1h9wViWqofAE3QH8vIJdf8ugz2WZTtYUFAAhk1+pkABRK E262lMzxQyk5c5ErFiMh8Wp/98Vq20DGBGwNmpBE8/slEzMRgeeTqdseP7cgpRCfpR/Q Sgew== X-Gm-Message-State: AIkVDXKDnpgNjjI+j+TZegarSceezuc9Qhldsiw3SjV3v7p7mZgWBTAYAK5w80Q5oaOnqQ== X-Received: by 10.99.222.85 with SMTP id y21mr91986pgi.119.1484694086941; Tue, 17 Jan 2017 15:01:26 -0800 (PST) Received: from pinklady.local (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id p67sm58210640pfb.2.2017.01.17.15.01.26 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 17 Jan 2017 15:01:26 -0800 (PST) From: "Ngie Cooper (yaneurabeya)" X-Pgp-Agent: GPGMail Content-Type: multipart/signed; boundary="Apple-Mail=_742A2872-7A6F-49B5-B87B-648383D50897"; protocol="application/pgp-signature"; micalg=pgp-sha512 Subject: Looking at replacing ATF/Kyua (in a limited fashion) with Google Test/shunit2 Date: Tue, 17 Jan 2017 15:01:25 -0800 Message-Id: <45D23581-C780-4C55-80CF-19A81813D672@gmail.com> To: "freebsd-testing@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 23:01:28 -0000 --Apple-Mail=_742A2872-7A6F-49B5-B87B-648383D50897 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hello all, I had an initial discussion with a handful of other test = stakeholders and due to the number of caveats with ATF/Kyua and a = variety of issues in contributing back to the ATF/Kyua projects (time on = maintainer=E2=80=99s end, legal issues, technical issues, etc), I'm = seriously considering replacing parts of ATF/Kyua with GoogleTest and = shunit2. In particular, I want to do these things [better]: - Detecting kernels features and modules. It is ad hoc and done = on a per-testcase basis or in ad hoc common routines, which bloats tests = unnecessarily. - Detecting network interfaces. My employer desires that =E2=80=94= we had something in use internally for detecting interfaces in tests, = but it was really messy; it would be nice if there was a clean, = generalized way to do this. - Adding test fixtures to ease testing on a per-suite basis. For = example, setting up a service like bsnmpd for testing, running pjdfstest = in different configurations, (a temporarily setup UFS or ZFS filesystem, = etc), collecting core files after tests for analysis purposes, etc. Why these frameworks? GoogleTest: - It is easier to extend upon. - It is better documented than ATF/Kyua. - It has more brevity in terms of writing testcases. - It has features that I (and others who write tests) really = want. - It is very similar to atf-c-api in core ways, in terms of = macros. - It has been accepted by multiple open source projects, like = clang/llvm, so including them in the FreeBSD Test Suite is more trivial = after the porting and integrating work. - It is lightweight and a good candidate for inclusion in the = base system. - Google and other groups are actively maintaining the project. shunit2: - It has more brevity in terms of writing testcases. - It has features that I (and others who write tests) really = want. - It has syntactic sugar/API construction patterned after = JUnit/XUnit in other ways, so it=E2=80=99s familiar to testers who have = written JUnit/XUnit tests. Some concerns that have been brought up: Q: Why those frameworks? Have you considered other options? A: Yes. A number of users have provided a matrix on Wikipedia = and I=E2=80=99ve looked at the list to try and determine : a. C: = https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C b. C++: = https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B c. sh: = https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#Shell Q: Google sometimes tinkers with projects, then abandons them = later. Is GoogleTest one of those projects? A: As far as I can tell, no. This is widely used internally and = developed on by their engineers. It is used external to Google by larger = projects like LLVM, so even if Google disappeared tomorrow I believe = there=E2=80=99s enough inertia and interest to keep things going here. Q: What about C test integration? A: I=E2=80=99m trying to see how far I can get with GoogleTest = integration on the C API side, and if replacing atf-c-api is = possible/doable using C integration in libc++ (the syntax/API = construction for atf-c-api and GoogleTest is similar in some regards). I=E2=80=99m really early on in the evaluation phase, so if = anyone has any input to provide with the direction I=E2=80=99m going, I = would really love the input. Writing tests for bsnmp will be my first = proof of concept (I hope to use both GoogleTest and shunit2). Thank you, -Ngie --Apple-Mail=_742A2872-7A6F-49B5-B87B-648383D50897 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJYfqJFAAoJEPWDqSZpMIYV7osP/j9ymv75+SXLwsIs+K2wXAuj TTR64vU0LtW30FLT3Y2iNOsXPLxoUlTeG1IQY/yTjGiVYTdtLsRmfYW1DbMmpMF1 OOoe2yUHZrx4NcbR8dU9CHSqhFyDI+WBmAxoZrvF+VylhukqlrZJtfwKGhH4quK9 pdHQM8b8pX/kh9I8NZcZHfYeKZf3qbO/e7Nid2dIvx0D7v2ws7Y+8Zjq/PP0beuf Sw3CM4YZkKtiOif6/gQ1JTBh7sSwcE4soWeCuuEOEmdGJqd1yeEb/MO5fGbMAD6c dcRkyDcmm03P63JKbW0qjlnFED2DPGuhmKjBue4vaHtjTCoFhWE8yPti3lBwHqBU Yby6KI0hlh0skECSRYm1njc8oNviyMpv5wpSIOtMqSVqn5lcSBnK4a8XK70K1oPC aiuV7eFkUQNU1jfnebm6q4pIX4NSWuhiSchvPrIxpommGftriNXwovf2BohkNSxT hfq4oLe/SDZ9+kN2HRdhoM1xUKC35m0jIrXZDg30MlCiR9+wI1yAIvfJukfUxHld kj4zrk5XeznSHgO2aaSozTNdwkgH/1Cplhj6RGtN1B7FO+xUTqkT8UKeJ/8RCd00 gtBnventU9as9Deb9mjS4kS13lh9eQrqrXQlebVn3ZgNeTOaBu27ONdNaJqA5KSj 59btJvmEZ7uMu+P4ifnk =1ThW -----END PGP SIGNATURE----- --Apple-Mail=_742A2872-7A6F-49B5-B87B-648383D50897-- From owner-freebsd-testing@freebsd.org Tue Jan 17 23:11:19 2017 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9308CB54CD for ; Tue, 17 Jan 2017 23:11:19 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 876F612B1 for ; Tue, 17 Jan 2017 23:11:19 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 86CDECB54CC; Tue, 17 Jan 2017 23:11:19 +0000 (UTC) Delivered-To: testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86748CB54CB for ; Tue, 17 Jan 2017 23:11:19 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-yw0-x232.google.com (mail-yw0-x232.google.com [IPv6:2607:f8b0:4002:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4574A12AC for ; Tue, 17 Jan 2017 23:11:19 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-yw0-x232.google.com with SMTP id l75so101122698ywb.0 for ; Tue, 17 Jan 2017 15:11:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=o724LwPDQwWu3RiwaPjofqiB+2x5N5rncarnW5Qp2G4=; b=CVKBePQNMD7g6/8pq7gzv8DyNwyjgCPlyopoE/CPu/ibPtLV6f8FvJCMW0K9NPztwO D1i/QeK1aFJvZIeshaCNkVZZlcOnWH14ak5yttB9EIjPdGqGhgQVIv1sFJE3HgmZeklX kbd/ciJm2KvWJ8V/yI8eOaozh1Md3BFPRCq25lKzEgHdZe+xW2isYhtgNLZMAH80utzG Iimn9ltYw91iN2QqkS+IKlWXom47VGjw6xVoyo1a1y4x019by55D6BHeLRQ6/j5pbVES G1NBdhPAtVa0sZUKo7GDITAlFThr54vG4VXsDxUughNIOYNKV+wRxdAkA3IgjTvYya36 DfhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=o724LwPDQwWu3RiwaPjofqiB+2x5N5rncarnW5Qp2G4=; b=MVBLK9h8t1HcZmjaUd9GEZwb0tbIA8wg1M+qmXcJ8DOR3D4BhXpRFSZYQmTOLVVQbQ BwZWDWhajnZ8KCQVz5skMxUV8pi756RIE8vu0M7gZKhZGYGVSCNGIysBtHyY1YdJohWk qRA+5piqaBrSe2dJ/5EqtlUVJmFVPinJglU6Cu8Pwus1Bqc8aBR+t8I0eQT7AM6+h3rs d+dKeBA3mzAe3elE6g9BJZp341zVJgPHjfaPjBo2rnkRpP2ojOzd/ogHrK/yrs5LMIdC Eas5dZ+8WA1MFR8GRub4wB2s/rrGZlw58rj34nSioNnBrhTJ0tdMTVn8xD/pCPitYz/r qO2Q== X-Gm-Message-State: AIkVDXKOt6pLtgwPW/XXf01+Fpjags+8Tc7MXRqxINOA+Bn9ySKIXz4KJsCc5czIUpe2T5XkymNWWHlNXgO7Qg== X-Received: by 10.129.76.74 with SMTP id z71mr81161ywa.307.1484694678487; Tue, 17 Jan 2017 15:11:18 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.129.38.133 with HTTP; Tue, 17 Jan 2017 15:11:18 -0800 (PST) In-Reply-To: <45D23581-C780-4C55-80CF-19A81813D672@gmail.com> References: <45D23581-C780-4C55-80CF-19A81813D672@gmail.com> From: Alan Somers Date: Tue, 17 Jan 2017 16:11:18 -0700 X-Google-Sender-Auth: BDDyML5cH_G7hSA-xqWoUnY9fmU Message-ID: Subject: Re: Looking at replacing ATF/Kyua (in a limited fashion) with Google Test/shunit2 To: "Ngie Cooper (yaneurabeya)" Cc: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 23:11:19 -0000 On Tue, Jan 17, 2017 at 4:01 PM, Ngie Cooper (yaneurabeya) wrote: > Hello all, > > I had an initial discussion with a handful of other test stakehol= ders and due to the number of caveats with ATF/Kyua and a variety of issues= in contributing back to the ATF/Kyua projects (time on maintainer=E2=80=99= s end, legal issues, technical issues, etc), I'm seriously considering repl= acing parts of ATF/Kyua with GoogleTest and shunit2. In particular, I want = to do these things [better]: > > - Detecting kernels features and modules. It is ad hoc and done o= n a per-testcase basis or in ad hoc common routines, which bloats tests unn= ecessarily. > - Detecting network interfaces. My employer desires that =E2=80= =94 we had something in use internally for detecting interfaces in tests, b= ut it was really messy; it would be nice if there was a clean, generalized = way to do this. > - Adding test fixtures to ease testing on a per-suite basis. For = example, setting up a service like bsnmpd for testing, running pjdfstest in= different configurations, (a temporarily setup UFS or ZFS filesystem, etc)= , collecting core files after tests for analysis purposes, etc. > Why these frameworks? > > GoogleTest: > - It is easier to extend upon. > - It is better documented than ATF/Kyua. > - It has more brevity in terms of writing testcases. > - It has features that I (and others who write tests) really want= . > - It is very similar to atf-c-api in core ways, in terms of macro= s. > - It has been accepted by multiple open source projects, like cla= ng/llvm, so including them in the FreeBSD Test Suite is more trivial after = the porting and integrating work. > - It is lightweight and a good candidate for inclusion in the bas= e system. > - Google and other groups are actively maintaining the project. > > shunit2: > - It has more brevity in terms of writing testcases. > - It has features that I (and others who write tests) really want= . > - It has syntactic sugar/API construction patterned after JUnit/X= Unit in other ways, so it=E2=80=99s familiar to testers who have written JU= nit/XUnit tests. > > Some concerns that have been brought up: > > Q: Why those frameworks? Have you considered other options? > A: Yes. A number of users have provided a matrix on Wikipedia and= I=E2=80=99ve looked at the list to try and determine : > a. C: https://en.wikipedia.org/wiki/List_of_unit_testing_= frameworks#C > b. C++: https://en.wikipedia.org/wiki/List_of_unit_testin= g_frameworks#C.2B.2B > c. sh: https://en.wikipedia.org/wiki/List_of_unit_testing= _frameworks#Shell > > Q: Google sometimes tinkers with projects, then abandons them lat= er. Is GoogleTest one of those projects? > A: As far as I can tell, no. This is widely used internally and d= eveloped on by their engineers. It is used external to Google by larger pro= jects like LLVM, so even if Google disappeared tomorrow I believe there=E2= =80=99s enough inertia and interest to keep things going here. > > Q: What about C test integration? > A: I=E2=80=99m trying to see how far I can get with GoogleTest in= tegration on the C API side, and if replacing atf-c-api is possible/doable = using C integration in libc++ (the syntax/API construction for atf-c-api an= d GoogleTest is similar in some regards). > > I=E2=80=99m really early on in the evaluation phase, so if anyone= has any input to provide with the direction I=E2=80=99m going, I would rea= lly love the input. Writing tests for bsnmp will be my first proof of conce= pt (I hope to use both GoogleTest and shunit2). > > Thank you, > -Ngie Before you get to far, you should ask yourself how much test case isolation you desire. Googletest is great, but I think it'll be hard to get the strong isolation between testcases provided by ATF. Perhaps it'll be find if GoogleTest tests provide no per-testcase isolation; most C/C++ programmers don't expect it anyway. But many of FreeBSD's atf-sh tests depend on it. Also, how are you planning to hook up GoogleTest and shunit2 to Kyua? -Alan From owner-freebsd-testing@freebsd.org Wed Jan 18 00:09:24 2017 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A957CB5F54 for ; Wed, 18 Jan 2017 00:09:24 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 384631E85 for ; Wed, 18 Jan 2017 00:09:24 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 37A2DCB5F53; Wed, 18 Jan 2017 00:09:24 +0000 (UTC) Delivered-To: testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3744ACB5F52 for ; Wed, 18 Jan 2017 00:09:24 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA76F1E84 for ; Wed, 18 Jan 2017 00:09:23 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: by mail-wm0-x22a.google.com with SMTP id f73so42925080wmf.1 for ; Tue, 17 Jan 2017 16:09:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=Zy4z5rlJueHb5EFmuJo8CghbtKmkqjiFE/D8l5jRI0k=; b=bZNzuewuZPPf2B+HYHP6TyN4ZF1/imGteA6LxnwgV8SHT0F8wBSagls3XUjLqVNT26 gIPzPmyPLWNoYJclWD0BtAHx0ccv7YCjc4KZoONExE0rKb2rjbM2RnqCw5G31ELQwgYf B4udlvaGR9WdR9fInxNARmkaNYy0nCloKZvv9BjdS8PVVc/pqapWDzuKzX7Nu7Nbci6x XSL9WdkW4kHcTW5SxZss2/B0D6xqCNc9miJpVURE6Svcs5OWTGmu4NKgKUjhGI5b90V5 kpp3jt4+p64QyCllTzcD3LYHzWLVUjaWvwpf+L87FVLXkF2H5MWaPT8QmvHIN93jxDx/ wTmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=Zy4z5rlJueHb5EFmuJo8CghbtKmkqjiFE/D8l5jRI0k=; b=LwtrirnLA/Y6ESvzlwTH0+8dHrivXRqwW9msUCR6DzOhLo9cFtDXoJ8VIX8K1tp3Ne prU+e3pMfB0xxzsn0x9PdL3cUK+Uz1gT/RE4aW2rWYtV8lga9wH5cSuFw4No8xJ73tHn oaj+aRsEPWWM6X9tIBNs5hM5QBBq88aZrOtf3bRq8h6YvCh9gijbnOg+Dtb96ipEkeaM hqwMDwlWuKXG+1zFcT1M3EqN9dGfCvGia8E0OkTahimV+3vudBJglyaqAv1rWnWhVGEi JhaF5+rKFcmmL44NrBRbCzj7UW+7DBReRo/DVk0Nnnmm4eKEUZaSfsF6JgwSVYiRqoej /yag== X-Gm-Message-State: AIkVDXLfdlJ4sfNnNeAr03f462qmwGSWzf6LbKZ48wFOxu4KHnyB86Kvxsm+tcr/2QBjwNCU0aLv6vnXYe/p6Q== X-Received: by 10.223.164.203 with SMTP id h11mr201724wrb.153.1484698162058; Tue, 17 Jan 2017 16:09:22 -0800 (PST) MIME-Version: 1.0 Sender: crodr001@gmail.com Received: by 10.28.163.195 with HTTP; Tue, 17 Jan 2017 16:09:21 -0800 (PST) In-Reply-To: <45D23581-C780-4C55-80CF-19A81813D672@gmail.com> References: <45D23581-C780-4C55-80CF-19A81813D672@gmail.com> From: Craig Rodrigues Date: Tue, 17 Jan 2017 16:09:21 -0800 X-Google-Sender-Auth: n7SdTBcdf0JLbwVUe5x0XcKZBzI Message-ID: Subject: Re: Looking at replacing ATF/Kyua (in a limited fashion) with Google Test/shunit2 To: "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 00:09:24 -0000 On Tue, Jan 17, 2017 at 3:01 PM, Ngie Cooper (yaneurabeya) < yaneurabeya@gmail.com> wrote: > Hello all, > > I had an initial discussion with a handful of other test > stakeholders and due to the number of caveats with ATF/Kyua and a variety > of issues in contributing back to the ATF/Kyua projects (time on > maintainer=E2=80=99s end, legal issues, technical issues, etc), I'm serio= usly > considering replacing parts of ATF/Kyua with GoogleTest and shunit2. In > particular, I want to do these things [better]: I'll provide my perspective on your proposal. For me, the nice thing about ATF/Kyua was that Julio pushed unit tests to live along-side the code in FreeBSD, and provided a consistent structure by which these tests could run and provide test output. The fact that he was able to modify the Makefile system to get this working in FreeBSD *and* NetBSD is amazing. I never would have had the energy to do that!! I was hoping that ATF/Kyua would attract interest from projects other than FreeBSD and NetBSD, so that there would be an ecosystem of projects that would use this stuff and drive this forward. I updated the Mac Homebrew port of Kyua a few times, and also tried to submit a Debian package for ATF and Kyua but got stalled there. Unfortunately, ATF/Kyua never really took off outside of FreeBSD and NetBSD= . Also, the implementation details of modern C++ and Lua for Kyua, and shell script and C for ATF is a high barrier of entry for people wanting to work on the code and extend it. There are enough developers in FreeBSD who could probably pick up the C and ATF parts. For the C++ and Lua parts, I think the barrier is quite high, and there are not that many developers in the FreeBSD community with the skills or interest to work on that. I have a selfish perspective in that I like to invest time in libraries and technologies that help me get jobs, even in companies that are not FreeBSD. In my last job at a company making a FreeBSD appliance, I tried to push adoption of ATF/Kyua but wasn't able to convince folks to pick it up. The developers gave me feedback that the ATF/Kyua documentation didn't give them confidence. It didn't help that I couldn't point to other projects outside of FreeBSD/NetBSD using this stuff. At least for me, I cannot say that ATF/Kyua is a marketable skillset which opens doors for jobs. :( Moving forward, if you try to steer towards libraries and technologies that have active user communities *even outside of FreeBSD*, that would be nice. Testing software isn't unique to FreeBSD, and there is a lot of good work going on in these other communities. You mentioned that GoogleTest is actively worked on by Google, but there at last one non-Google community (LLVM) is using it, and has a selfish interest to keep it going. That is one plus in my opinion. GoogleTest uses a 2-clause BSD license, so that is OK. GoogleTest requires signing a Contributor License Agreement for people who want to submit patches upstream to them. That's no different from Kyua. That's a bit annoying, but maybe something we can live with. So I have no problem with GoogleTest on those points. My questions for you are: (1) GoogleTest is very C++-focused. ATF has API's for C, C++, and shell. How successful will you be at testing a pile of C code with a C++-focused library? It is technically possible ( https://meekrosoft.wordpress.com/2009/11/09/unit-testing-c-code-with-the-go= ogletest-framework/ ) but what are the gotchas? (2) What do we do with existing tests? Do they need to be rewritten to the GoogleTest API? Or do we have infrastructure that allows existing ATF tests to run with Google Test? (3) If this is adopted, are new tests written in GoogleTest API, or in the ATF API? (4) Kyua is the tool currently used for running the tests, and providing test result output. Kyua has test-case isolation, and the potential to parallelize the execution of test-cases. Are the Kyua features of test-case isolation and parallelization of test-cases must-haves, or can we live without them? Do you want to move away from using Kyua, or modify it to work with GoogleTest? If you want to move away from Kyua, how do you want people to run the GoogleTests and get the test results? I have no objection to your proposal. Using a more mainstream testing framework with a larger user community, and some momentum to keep things going would be nice. -- Craig From owner-freebsd-testing@freebsd.org Wed Jan 18 06:47:34 2017 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64117CB45D0 for ; Wed, 18 Jan 2017 06:47:34 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 3BBB2162F for ; Wed, 18 Jan 2017 06:47:34 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 36AAECB45CE; Wed, 18 Jan 2017 06:47:34 +0000 (UTC) Delivered-To: testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34A65CB45CC for ; Wed, 18 Jan 2017 06:47:34 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pf0-x232.google.com (mail-pf0-x232.google.com [IPv6:2607:f8b0:400e:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EAC75162E; Wed, 18 Jan 2017 06:47:33 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pf0-x232.google.com with SMTP id e4so1933658pfg.1; Tue, 17 Jan 2017 22:47:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:mime-version:from:in-reply-to:date:cc:message-id:references :to; bh=Vu2hEzJdF1f0zEhh8p7mGRxavXf5no9NMbW6pA47PXk=; b=aFNNg/cvgCbt2ymLca/Z4DWAkvGfq6KHmOTvPgB14uF+IS8MPwJKqBr4dV7UYD+xpD dZ/3UJNDZXW7V9o2NT2v/iqp5GFf1bod69JZHO6U103gJcGwpyDo8G6+K4MYblbvMSfT KVDsa8ZoORER0We6mHsbHpk5J2NYOrRbXdM8fJb8x5YSbnq9TwSNvoqGWEptIfoyyPSL EB8o/LwWWVao4ISpnj7Rd9NFrk/PzhcYseKDVJddggA7jhDhMOohqrgWTmrZzxs5ZSiW vOv/FGfZThTKx1KMUPdPg+V9jn8YhjXpb8Y7W4mfdCB0EQYeK6Zk7AfV4sSN9PwaOgEi 4SoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :message-id:references:to; bh=Vu2hEzJdF1f0zEhh8p7mGRxavXf5no9NMbW6pA47PXk=; b=tzps87HNFu6faD30XmgkVH8KfCn2kqTftfpGuJ2Dl9GkV3nhDLp/XQqWHfT9UFlAL1 kzJ6EWeeMbOOXti3fWrdGGsoyh6/eVshwpi304LJl4mwHIlAoQmcw12iGhWOSF3FzUon zBJe9qsq4wNa/0jLu7sdIDLwJvEdahs/kp01qAeFXfmxIQD0SC20ShmPuq/XSW4HNqwU QRATr/yy4s/4gWaDGWxoKwldFj4MyqTCVWOGGcO5GeUK5uWX3fs6QgtV37+31qQstRh1 MtnI31izgkqsH63tPikQ1IhyqpBvQDZelxg9MmZ4KAotiZWqUW8thxAtpMwAOaxcoOI9 veMw== X-Gm-Message-State: AIkVDXL+ZRKfIZH/Fw86huE0FyhuhiZx3RZqkl/fG2rCJdG+Erv6QrP3Kj/SOQZl5OvwMA== X-Received: by 10.98.15.143 with SMTP id 15mr2059652pfp.100.1484722053345; Tue, 17 Jan 2017 22:47:33 -0800 (PST) Received: from pinklady.local (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id h4sm46050296pfk.96.2017.01.17.22.47.32 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 17 Jan 2017 22:47:32 -0800 (PST) Subject: Re: Looking at replacing ATF/Kyua (in a limited fashion) with Google Test/shunit2 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_2FE8D42A-9796-4BB0-BA51-C0384D8B84D5"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail From: "Ngie Cooper (yaneurabeya)" In-Reply-To: Date: Tue, 17 Jan 2017 22:47:31 -0800 Cc: "freebsd-testing@freebsd.org" Message-Id: <901D465A-9F32-4BBC-9FF6-7EB5AFAF6858@gmail.com> References: <45D23581-C780-4C55-80CF-19A81813D672@gmail.com> To: Alan Somers X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 06:47:34 -0000 --Apple-Mail=_2FE8D42A-9796-4BB0-BA51-C0384D8B84D5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jan 17, 2017, at 15:11, Alan Somers wrote: =E2=80=A6 > Before you get to far, you should ask yourself how much test case > isolation you desire. Googletest is great, but I think it'll be hard > to get the strong isolation between testcases provided by ATF. > Perhaps it'll be find if GoogleTest tests provide no per-testcase > isolation; most C/C++ programmers don't expect it anyway. But many of > FreeBSD's atf-sh tests depend on it. A lot of programmers don=E2=80=99t desire this at all, until they = realize how their tests might be affected by system state if/when there = are files/directories/other inputs infecting their results on the test = system. > Also, how are you planning to hook up GoogleTest and shunit2 to Kyua? It=E2=80=99s early enough on in the process of trying to figure out how = things work that I cannot definitively say, other than it may end up = being something like: kyua <-> googletest <-> atf or something more like: kyua <-> googletest_atf_compat and just mimic the output of ATF, while hiding the implementation = details of the test suites. If that turns out to be a pain, then I=E2=80=99= ll just implement another hook into kyua for googletest which will = implement the features we need, potentially as a patch to the port, = because =E2=80=9Clegal reasons=E2=80=9D (contributor agreements = shouldn=E2=80=99t be impediments, but because I signed an individual CLA = vs corporate CLA and they aren=E2=80=99t equivalent) are blocking me = from giving code back to Google as a Dell EMC employee (things may = change, but when I fought for a corporate CLA, it got stuck in the = bureaucratic quagmire that is my company=E2=80=99s legal/business dept). Thanks, -Ngie --Apple-Mail=_2FE8D42A-9796-4BB0-BA51-C0384D8B84D5 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJYfw+DAAoJEPWDqSZpMIYVDbIP+QGKN2zl18xSgWsohE4rkbfS uRHPddu+qhkOEMn5SSuroZbWTdzMC9fqPq03TKAvafS2gkvWGVduwU/sG8v1uR5O simKs8uj65bDKj9tOSPqvLPlE7+lHt7gve8C3YTmSjp86hdk1sMCFmCa0pdUJmhg EuWG/cUUgKV/EsDvDFIasOw1jgqMMVjMV53rzWhxo6mSPVyq0Q1I+swZUkhhACa1 JDCBeYDcF2vZdCUNCo/e9K1s47ZlRjGPtnKU60i7p2LK2KZ5ekqVS//0PD5+0cfi lTazIIVJQdQy3/OqS3FYO0QXlWmTZOet/csS9qZaADF3SXZKoQ9U6AbZ5Ptp4lym yHlow8casju1T158ghfvSflzFS9o6zx4MJwRgKc+Q0dmRuA7EDGMMewuWdBy5M/H 2iI4ZkxS2Xc2Y918krIiWHZcpdkapSUI5nFW1xm1WN42FZ+qaIOWgnFmcH3qB500 EPJxrhVqb5tCoKaw40gv6rATgzGk6JI2cqKUV1TZekXnsEDb2jzfpxpny5dGZh0J a0yI7WvrgUhGDkzwiygTRh8SP7FttZ/l4d5svWXk4lkbOkHKR96BtPr4RSm2O3Xx Je8QYb8ir97Levltn9aFN3cjth9AFnLCAUo0prC1d2Nitme9UxML7X0rBJE9Jhqa Syc2NiuRYPLM8lb74hTi =eyRn -----END PGP SIGNATURE----- --Apple-Mail=_2FE8D42A-9796-4BB0-BA51-C0384D8B84D5-- From owner-freebsd-testing@freebsd.org Sat Jan 21 01:30:57 2017 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C580CB9107 for ; Sat, 21 Jan 2017 01:30:57 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6BCAB18E8 for ; Sat, 21 Jan 2017 01:30:57 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 6B2FDCB9103; Sat, 21 Jan 2017 01:30:57 +0000 (UTC) Delivered-To: testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6ADF2CB9102 for ; Sat, 21 Jan 2017 01:30:57 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pg0-x231.google.com (mail-pg0-x231.google.com [IPv6:2607:f8b0:400e:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 40A2918E6; Sat, 21 Jan 2017 01:30:57 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pg0-x231.google.com with SMTP id 204so27832616pge.0; Fri, 20 Jan 2017 17:30:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:mime-version:subject:message-id:date :cc:to; bh=QXM985hWCVQUXO0VxVJRWq9lWnEkX/yV3Ue5Sg9Ybe0=; b=IO5wBDJexVpPl9mYwVJb7QSoXQ5+OwnImX6KKpcaWarYh4XtTOWT6svD+crJuioU5v i9zm3+wC4FGBxcrkJAcpKdX6QscJlN3q4JbDsL2CQJTcy27KinN6f6lLhAJtAWGNk1AD SzszfujRUTsVEygxHlzh55u+rRFZNYgqf0c7E6WjKdq4mvohKRqnnVRSiYErtNAba1zP kx81VNOcjdKxP1wYYOqQKIJor5XWM4Mdpdr9heK5vniULDRpO8Jm5A0mzUBCpEyLWj/6 DKdKySqgsd1SMXzUs+TzRiCdrlhCu6T8STCyo9lHI/7TymfQdoWqu5eHFCoSXQ/3N8Vb 1E4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:cc:to; bh=QXM985hWCVQUXO0VxVJRWq9lWnEkX/yV3Ue5Sg9Ybe0=; b=LoXo7FRt77U353UsGvMVIrzqggLJLuzb7HMV2u3efal0I0cQSrA2aT3KmPnb8oSoCf PruRVXcRRnzmSXFk52MdMyLIg/f47gfsvbXxHkEFyOCHuHbZCXLmbQRwQrFYgqy1iOfU RcHRB1g46u1pyfLsuacZ9hNSFpdA+PQQTSNmKqnFH7jkcca00APznp3CeZuuSxRPNR8L 0yzSzUw3ZPrsWFe3APK2z4b89SFUP1VcraWNfvRDswUFYb2oWedttx8qFy8NrDfZIGX0 LcX/qNXM1Sk+n+5WMZSUrajiO2V0MHOZYJ0aRpFRoqyBt0zUWnrQYUnQcmwDM5FYT7T+ lqng== X-Gm-Message-State: AIkVDXJMLJ+RUcLZ1eaAK5Kr54xtdgURJevca6UavgKM5z1NhDOMT2LC5iGW6lkepovntQ== X-Received: by 10.99.131.65 with SMTP id h62mr20108855pge.41.1484962256649; Fri, 20 Jan 2017 17:30:56 -0800 (PST) Received: from fuji-wireless.local (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id q19sm19608032pfl.21.2017.01.20.17.30.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Jan 2017 17:30:55 -0800 (PST) From: "Ngie Cooper (yaneurabeya)" Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Jenkins FreeBSD-HEAD job is stuck again... Message-Id: Date: Fri, 20 Jan 2017 17:30:54 -0800 Cc: "freebsd-testing@freebsd.org" To: jenkins-admin@freebsd.org X-Mailer: Apple Mail (2.3259) X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 01:30:57 -0000 =E2=80=A6 and has been since 12/03/2016. Thanks, -Ngie PS I really think that a set of Nagios or Zabbix alerts should be setup.= From owner-freebsd-testing@freebsd.org Sat Jan 21 04:19:08 2017 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CAE1ECBA9C2 for ; Sat, 21 Jan 2017 04:19:08 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id A8F371B47 for ; Sat, 21 Jan 2017 04:19:08 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id A8562CBA9C1; Sat, 21 Jan 2017 04:19:08 +0000 (UTC) Delivered-To: testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7F96CBA9C0 for ; Sat, 21 Jan 2017 04:19:08 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: from mail-wm0-x244.google.com (mail-wm0-x244.google.com [IPv6:2a00:1450:400c:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 424DC1B46; Sat, 21 Jan 2017 04:19:08 +0000 (UTC) (envelope-from crodr001@gmail.com) Received: by mail-wm0-x244.google.com with SMTP id c85so11152107wmi.1; Fri, 20 Jan 2017 20:19:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Xpabj7FsMOkE35MtCY6N91J7BvjWA8vgZ3h/MvGQ604=; b=uLQHeBFNKe2PCXri83cKenjGdiMjdi+rIJ4mFsk80lMEwSndy2jyej8CcJKHIcdrDE CJInuZOQ3L5lAoS6exXpv4rue7jfBc5xQpf1CHj6q44Y3yS/RCNSbiTmwzpEGpu4c3sS wdqSSSegMKKHL7U48bGhcX6LPjY9YUjpr0b4KjrYeT5Jmsw0vLAM+ayIPz2ABRDKvVm9 i6bCnZx/iIIWmlLoOgzQNlE2DX+Iylj9fqrNUA6PlpsEsLL3hJLzJL3m2M8wD0TkUnMJ PhD6b/UP1UEwM8jQRZrMMhcnLhvEsSvGTee+L2OY97ifrjg4qI2NSlCVJNlbqky9vJBo lZHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Xpabj7FsMOkE35MtCY6N91J7BvjWA8vgZ3h/MvGQ604=; b=tdpvYiYtCVOSAmnyFIkjqqrcH1S7jlRJm5o2wfhxMjTx4ysIDVB3LRGjvlJWekPHlG Xa2GslRWyV05EuZpOFs9FyYNZRdyRjKQqUHmp+DMr5PA9pn7Z0dqxj2ZRSlG/XSmis6f H+/malEKueCObsllhlEnXzLs7oB4hR5CvtpT7uXdpLte4QYfCLeOCusZ9QIa9BSagIdH U9PxgV6ugBONV2IXkYCkcwesoFGyT4JoKT1g252i3ZZRszSAarLqOjIXYYfnXc+hglgE cAgEs0A0phgp/3FGZ847U6wtacEfVgP0PjfsibTpPeQimeuhnqsey+lVko87Yv5uUWjO 1keA== X-Gm-Message-State: AIkVDXLTjgFBLqj2hj0ei/gmEeudCG6g2kLz82t2JZsr49LF11YAvrE8vzeZr3giHXK1/J7emIRDLx9ZqOWXdg== X-Received: by 10.223.164.203 with SMTP id h11mr14525898wrb.153.1484972346176; Fri, 20 Jan 2017 20:19:06 -0800 (PST) MIME-Version: 1.0 Sender: crodr001@gmail.com Received: by 10.28.163.195 with HTTP; Fri, 20 Jan 2017 20:19:05 -0800 (PST) In-Reply-To: References: From: Craig Rodrigues Date: Fri, 20 Jan 2017 20:19:05 -0800 X-Google-Sender-Auth: QWX35nAZIekUJ0u5OZYOWrdAaDs Message-ID: Subject: Re: Jenkins FreeBSD-HEAD job is stuck again... To: "Ngie Cooper (yaneurabeya)" Cc: "jenkins-admin@freebsd.org" , "freebsd-testing@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 04:19:08 -0000 Yes, I've known about this. This is due to hardware problems. lwhsu is setting up some new hardware, and will hopefully done in the next few months. -- Craig On Fri, Jan 20, 2017 at 5:30 PM, Ngie Cooper (yaneurabeya) < yaneurabeya@gmail.com> wrote: > =E2=80=A6 and has been since 12/03/2016. > Thanks, > -Ngie > > PS I really think that a set of Nagios or Zabbix alerts should be setup. From owner-freebsd-testing@freebsd.org Sat Jan 21 04:59:18 2017 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7469FCBAD71 for ; Sat, 21 Jan 2017 04:59:18 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 5139C1208 for ; Sat, 21 Jan 2017 04:59:18 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 50961CBAD70; Sat, 21 Jan 2017 04:59:18 +0000 (UTC) Delivered-To: testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 503B1CBAD6E for ; Sat, 21 Jan 2017 04:59:18 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pg0-x244.google.com (mail-pg0-x244.google.com [IPv6:2607:f8b0:400e:c05::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FFE21207; Sat, 21 Jan 2017 04:59:18 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pg0-x244.google.com with SMTP id 204so8465244pge.2; Fri, 20 Jan 2017 20:59:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=55v6fHWuMgzf66HwXMmCFvlw4x0YTHkiUFb0raTtPxg=; b=ffsy087jcJ6Horr+lXOvy+BW3rBTqdSYWM5WTV3/ZvAWqrx8Hk2FdOEHUI1ITKsZKX jAFUYe7VbRZGAjCHcYA6KDWym4FIRbwntricxze3wV7NsGEz/PrPSTGBqb+DSyQ2+LyS vm6B/ssCsrcUK7PUop6OYfOEsNNHDUqE3aKt5Bjft9YFKrqHB9Gn7yJUwkriinxOusEv d/2CexyR/GsYA1rcO+Eg/Jl/xUhwTzG87juXBtefdTeRnDGMf9h9UdyD0ho9FGjgepzh rpuXYL/tpgS9FE37yD3QQBWM8IWtGocd41EmFjSrQzGxfnT9PdPWPRDsXYKtPkY+tfhc axFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=55v6fHWuMgzf66HwXMmCFvlw4x0YTHkiUFb0raTtPxg=; b=M8PIGQV0BIeuPwXc8qDB43bixz5q2km8Mnwn9qMIVs/UpN+WiMYcWYT/nWTFCu0wdu 3ZUt12/ocoalrANHS+Sd4pFgsTIjqKwA3rbF3EcEaAmKucE7Yg4hox84qEr5bFE4fWMP gi9c5b+e3BaqRfQfNlIINzPakTpsUiMAr5whNWy0dCsOSgavpslBmEAWz49jdhOR5Dyw VrTHWadx2aaMUzzC9Mj3RpaQhTgavrBfKEGqxqdzlV721tdBbyYEN2ySSH9FP25iRkMs veUYalDkB1T9K7ku+YgTFEEgkecB/p3JmUn5CZVk+0/MaTerenkqY08cZDQ3pWzP4Hpx tc1A== X-Gm-Message-State: AIkVDXJWBRfO7PpUrWDfV29VUtQkFIUlf0ZauYlZbUjCKil+AieZebW8Im6GKnd6ueEIWA== X-Received: by 10.84.175.234 with SMTP id t97mr26872568plb.145.1484974757522; Fri, 20 Jan 2017 20:59:17 -0800 (PST) Received: from ?IPv6:2607:fb90:f24:eefe:3131:9173:666d:2b34? ([2607:fb90:f24:eefe:3131:9173:666d:2b34]) by smtp.gmail.com with ESMTPSA id w2sm20305965pfi.65.2017.01.20.20.59.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Jan 2017 20:59:16 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Jenkins FreeBSD-HEAD job is stuck again... From: Ngie Cooper X-Mailer: iPhone Mail (14C92) In-Reply-To: Date: Fri, 20 Jan 2017 20:59:15 -0800 Cc: "jenkins-admin@freebsd.org" , "freebsd-testing@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <6964F5E7-CEA0-4E5F-82C5-A9AC41B5309B@gmail.com> References: To: Craig Rodrigues X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jan 2017 04:59:18 -0000 > On Jan 20, 2017, at 20:19, Craig Rodrigues wrote:= >=20 > Yes, I've known about this. This is due to hardware problems. > lwhsu is setting up some new hardware, and will hopefully > done in the next few months. Could the pipeline be broken down so the build portion continues to exec= ute until the new hardware comes available? Thanks, -Ngie=