From owner-freebsd-testing@freebsd.org Wed Feb 27 20:53:50 2019 Return-Path: Delivered-To: freebsd-testing@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8C931504F25 for ; Wed, 27 Feb 2019 20:53:49 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 898CC8F725; Wed, 27 Feb 2019 20:53:48 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pl1-x62e.google.com with SMTP id q3so8588610pll.4; Wed, 27 Feb 2019 12:53:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=ppuWvmlsoZ6FyBdGGtHKNeOV1hGnHyvLUPIPIaM44PA=; b=Sqszd7v7xvcs7IDyeelLDljrb9MASNI84CI1Z0YDPjHsAodgeuLcLkC3dhO+KlmlCs sc12z1D5r8QNkl8jyGahCq/G6lNx5Qh9ZJ6Y0NKtusoRsmEfe6Qv3DDrdMcVcaMthJIM u+boQkc+hxvsoiEPrSMsbAtEsSDoRxsc/bG7k2A7HbWyTGkvXjZegkz9z+P5uq3cy2aB N6F9m8nBfpL7Ua5B+gO5GwQeGhbRiC4AsBbncydGJzAXpQBxLd9vahBU8V+3M6pfplEA 4KbZhWEFLCVmgS3vJGj85ai9z1834bex1fbqgzbQkOpNeYCyVLdZkw6LLc4itDFm2TG6 0GGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=ppuWvmlsoZ6FyBdGGtHKNeOV1hGnHyvLUPIPIaM44PA=; b=DsHYjJsORFPzFd6b2lGc+b70q2FjqfXTU0HqvEJwyHZoxHY1DYlezb+9GNc2MX4tQR vLQ0FT0X3w/Yepg3wwwX4IJbRyXTbXbNhaFvnWNvWQR+UpZzGI0EI0c3PlbeO0VFjRDA b4uYCBat+i7vPHZXFNTMb0N2BiypENGjFamNPxvJfUgveoUTkbxJQoZOpwJBVgwmVQNF O52/JUL9d7G04ANtR3G0b2yWZrYNwrAbQgGXvUyah4ZfnioY6KbXQKht87mWvcIrf6XT +rtmFxbDDp7daJ0ReQ7jNdn2xyNK/TDL9im+Or8ThZ0XVFgFSr3AK+jQPrSPAFM7xeOp V3Xg== X-Gm-Message-State: AHQUAuaE5xKuYt2ozNzdhiiL9ahgn2tLM31vOpZSJL+ntXI6ciAxJT9C IhcuX2t6H2eOr8K9n5VmjF3Nr2QS X-Google-Smtp-Source: AHgI3IYh434RbtBaom6yBwaeEdHzx5QToTBpq9ICS3K0x+T+Cde6RFz7DK//ctDoYOKOS1vwYHTVeQ== X-Received: by 2002:a17:902:6949:: with SMTP id k9mr4268666plt.188.1551300827339; Wed, 27 Feb 2019 12:53:47 -0800 (PST) Received: from [172.17.7.154] (163-64-212-66.spl.org. [66.212.64.163]) by smtp.gmail.com with ESMTPSA id l2sm24434243pgj.64.2019.02.27.12.53.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Feb 2019 12:53:46 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: atf-c++ vs GoogleTest vs share/mk From: Enji Cooper In-Reply-To: Date: Wed, 27 Feb 2019 12:53:45 -0800 Cc: "freebsd-testing@freebsd.org" , Ed Maste , Julio Merino , "Jonathan T. Looney" Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Alan Somers X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 898CC8F725 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Sqszd7v7; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of yaneurabeya@gmail.com designates 2607:f8b0:4864:20::62e as permitted sender) smtp.mailfrom=yaneurabeya@gmail.com X-Spamd-Result: default: False [-6.33 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MV_CASE(0.50)[]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-2.84)[ip: (-9.43), ipnet: 2607:f8b0::/32(-2.69), asn: 15169(-2.02), country: US(-0.07)]; RCVD_IN_DNSWL_NONE(0.00)[e.2.6.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2019 20:53:50 -0000 > On Feb 27, 2019, at 12:42 PM, Alan Somers wrote: >=20 > On Wed, Feb 27, 2019 at 12:46 PM Enji Cooper = wrote: >>=20 >>=20 >>> On Feb 27, 2019, at 9:59 AM, Alan Somers = wrote: >>>=20 >>> So it turns out to be impossible to use GoogleMock with atf-c++. = The >>> problem is that atf-c++'s only way to report a test failure is >>> ATF_FAIL, which immediately terminates the program on failure. That >>> conflicts with the way that GoogleMock uses pthreads, which is to >>> report a failure while a pthread mutex is locked. atf-c++ has many >>> other shortcomings, too. It lacks the ATF_CHECK_* macros, and its >>> syntax is surprisingly inconsistent with atf-c=E2=80=99s. >>=20 >> Yes, I=E2=80=99ve brought that up before in the past. This is part of = the reason why I want to abandon/kill off atf-c++ in favor of something = more functionally complete, i.e., googletest >>=20 >>> So I tried writing a C++ program that uses atf-c instead. But the >>> Makefiles in share/mk make that a frustrating proposition. They = don't >>> want C++ programs to link to atf-c, and they don't want atf-c = programs >>> to be built in C++ mode. >>=20 >> Well=E2=80=A6 yeah. This makes sense. >>=20 >>> Googletest would probably work fine, but I would sorely miss ATF's >>> test case isolation features. >>=20 >> In what ways? Using plain tests builds in some level of isolation, = but it=E2=80=99s not perfect, and this is the direction I=E2=80=99m = taking Googletest for a first iteration. >=20 > Having $PWD be a disposable tmpdir is convenient, This is already handled via kyua, just like it is with plain tests (and = ATF and TAP tests for that matter). > but I'm even more concerned about what happens if a googletest = segfaults. In that case, > its cleanup routines won't run. That could potentially be > catastrophic for the entire test run if the test program does > something like alter network configuration. Or do you have a plan for > this? I will look into how Googletest currently handles segfaults, etc, and = raise that as a point of concern with the upstream project if need be, = as I don=E2=80=99t know how exactly this is handled in exceptional = cases. Cheers, -Enji=