From nobody Sat May 2 15:31:46 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4g7BjG0HCfz6c7gK for ; Sat, 02 May 2026 15:32:02 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4g7BjF0lR7z3GW2 for ; Sat, 02 May 2026 15:32:01 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-f41.google.com with SMTP id 4fb4d7f45d1cf-67b6a6bd7b8so3849255a12.0 for ; Sat, 02 May 2026 08:32:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1777735919; cv=none; d=google.com; s=arc-20240605; b=HyZBnrnQkkO8WmbBGoCXgFgRJSlfC07Zop/AeTY/yiiJr/EJtxNst1oLHQorYPnGeM HO/ENfVP8O+vScCNi76Fh9VWV+PN/jj1zciSsUHhcsA2cuUXsyYQM8rBXxmcHvH9wTnL wLFdV2WzqzEHnoUoW7bB24Hy5xi4Tf/tWavi0jG/GA8Axu75W/khdEd6RA6OG+urMtvl z1U7AQvOZ8VvWfKeTdj4lVIsFilrltpVFCzQZFQF49ZUQTU0fTDBnaDHWDNK9mopT1yl pE6esnjXFqmtXDQBOsYIjC4VXfUgKItNfcFFFW76/kI2VD2PodaV0g5wY1GiBl74qn0w 9Zgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version; bh=u2H4NHdpLl4a/XZuIN4Tnkf7Unxhwl07hVsLUSLyeD4=; fh=C7iERwoyd78B0WZCxYqTEnjw76x2IKV3Lyco6U9U2XY=; b=JkVnJsfjwL/3ohQlhYcfNSUMbVd1yFG7ydbu68uaIpI03tQjMip078h65Az4iEDLFd CJ0pdquWcXct+7RKeWcMrw2QXPMc2fHAShvwcKbwnR/NZBUJpmrKW03ci2dLXNVLiUzr 8TP09mYSBcpDqgIn0PB4wr04Qp/4u3nOV6vebjJPTVhdb5tpTK1Y9dwRi9KR/tAcYsq3 59+QTz3Yzdfg2gIY+XL9dMfi0JGSv4LBgXXveh9y1km6dj4jUhJFR3TCYl1zbg4nJxRg 4HD4jlX1ounSy4JqxTbJ55AN5AahqMCvTHw5Qclngm7rOf5ii2jhM0Ts/sx8UeAZ/ZVG /6mg==; darn=freebsd.org ARC-Authentication-Results: i=1; mx.google.com; arc=none X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777735919; x=1778340719; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=u2H4NHdpLl4a/XZuIN4Tnkf7Unxhwl07hVsLUSLyeD4=; b=MBYPxgop07fxo9z+eiSy+IZZApHTehq8VYFWU3uVlb+iY7uJmeWXDnKszJ8CMh497B uUWHUxY+OVzSmn5Xd3wxDv4mDh/gVxREZ227CLhLnix8PLnS7r+m2l8N73M4AjC7IqYf VKba+I/jDSDGql1EZSsBk5YRfihZVIW6c0YcZPACxj6dBdpRFz2AE8++kZW8nQFgG+6n 9dvx25uw0ebTGuLZCKaQDaqOnlaSjzpjSzRTd1UxMfnWDag3icUgAnx4Z8rj+x/vLGT7 YMqk4ixP3Ln5SbE7K/IA1gLiI20pbvMk5qnQPakL4Lv8N+oqjxexOiqBkn6fY62/18V6 QT0Q== X-Forwarded-Encrypted: i=1; AFNElJ/c1kVGNdcLDxgwVjRWqCmvOcpPTeZTJ4FrblFRTqHrMXIB6Pe3CqmR/V83Mng4dsJbUCrwrNFufKuDfmL4921E+Pl/@freebsd.org X-Gm-Message-State: AOJu0YxADjoOpjY+WLM5f+7ywhSIC4AXCAVq8Ns+cbH5r8jW9zvelhfw m4ufCi2Qoetei/6tLs3uQQUeySRr3lADiat7NesGCMG2gJRPAEi2Enkk548+2/vah6hF+pPCT48 PS4K/ZRScah9PS6ACEHSsJdgtU8Uv6yU= X-Gm-Gg: AeBDiesFDbQav/p+I+UbQTnTVT1AxDyogB6On+Tg+/XY7HaQIA8jWFqmiIbILlzl960 1Wh7XRLjxqMhC1UTe7lMSe39/6gJRnczpqbZggvACjS56oST1cxxqNQmGXP6OBdIFWcAzTHss4F epGaSHxSm8V2UdB97Ri8/DuMr8+t5QYNyEnPEE1++Lp++6YgPaisq2th43B65mFyB8fIQ0D+srL bRWQfBsaD74UcZwBWweLXslD+r5L7AOT8Dz+Dc47SSafwAOdvBVIPRwVa4QU200T56ikb2lG0f9 5VDUhl3Osmxg/zeyY82n5dZtajcPtT0u2YJYJgEVNYYirONamGjLl+Cqxo+3jBUwpgOtuVFbZyT zypY= X-Received: by 2002:a05:6402:a605:b0:67c:34bb:1bd1 with SMTP id 4fb4d7f45d1cf-67c34bb1ee8mr376094a12.1.1777735918863; Sat, 02 May 2026 08:31:58 -0700 (PDT) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 References: <69f61649.1c022.73b2a773@gitrepo.freebsd.org> In-Reply-To: From: Alan Somers Date: Sat, 2 May 2026 09:31:46 -0600 X-Gm-Features: AVHnY4IJb9g52Q4ZlDv1g5Z9rIY5FmzkvYLInIt80kh21leox3DGa0hY79W-qTk Message-ID: Subject: Re: git: 3e845b109056 - main - ping: fix listing test cases when scapy is not installed To: Gleb Smirnoff Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4g7BjF0lR7z3GW2 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated On Sat, May 2, 2026 at 9:24=E2=80=AFAM Gleb Smirnoff = wrote: > > On Sat, May 02, 2026 at 03:20:41PM +0000, Alan Somers wrote: > A> commit 3e845b1090565912375c5578cf0399d27b7fa70c > A> Author: Alan Somers > A> AuthorDate: 2026-05-01 23:06:35 +0000 > A> Commit: Alan Somers > A> CommitDate: 2026-05-02 15:20:16 +0000 > A> > A> ping: fix listing test cases when scapy is not installed > A> > A> The ATF-python test program was attempting to list test cases that > A> require scapy. But it attempted to import the scapy module before= the > A> test cases had been listed, resulting in an ImportError that kyua > A> interpreted as a test program crash. > A> > A> Fix this behavior by handling that ImportError well enough to list= test > A> cases, but not run them. If scapy isn't present, Kyua will refuse= to > A> run the test cases. But it needs to be able to list them in order= to > A> know to skip them. > A> > A> Sponsored by: ConnectWise > A> MFC after: 2 weeks > A> Reviewed by: maxim > A> Differential Revision: https://reviews.freebsd.org/D56765 > > Sorry for not reviewing in time. I don't think Kyua expects a test progr= am to > be able to list the test cases, but then fail to run them. > > Isn't there a standard way of dealing with the problem? Many tests have > @pytest.mark.require_progs(["scapy"]) in them. > > -- > Gleb Smirnoff Yes, these tests already include @pytest.mark.require_progs(["scapy"]) correctly. That's how Kyua is supposed to learn that it can't run those test cases. You can tell by running the program manually, bypassing kyua: $ ./test_ping.py - Content-Type: application/X-atf-tp; version=3D"1" ident: TestPing::test_ping[_4_c1_s56_t1_localhost] descr: Test ping has.cleanup: true require.config: unprivileged_user require.user: root ... ident: TestPing::test_pinger[_0_0] descr: Test ping using pinger(), a reply faker has.cleanup: true require.user: root require.progs: scapy ... Notice that the first test case does not require scapy, but the second one does. This is how Kyua learns that it can't execute the second test if scapy is not installed. However, prior to my commit, if scapy was not installed, running test_ping.py -l would fail with exit status 2 and produce no output, and Kyua would interpret that as a broken test program, like this: test_ping.py:__test_cases_list__