From nobody Sat May 2 15:31:46 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4g7BjK5lQrz6c7tg for ; Sat, 02 May 2026 15:32:05 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4g7BjK4q5yz3GrC for ; Sat, 02 May 2026 15:32:05 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-678a16429c6so2967712a12.1 for ; Sat, 02 May 2026 08:32:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1777735919; cv=none; d=google.com; s=arc-20240605; b=f6n/zS54T0iJC2rQUXhlznuYJGFLOILNY006wZtSsYm3ythzW9UgTpiPf1w2qjN5pp G/7JLH6Hhp/T0gFo6PHOAUeQ/+uvmhhEB4FOOLl6G3m5eXMrtqjF9+d4OHwkDQY7bGqR PmSYOsepljmn3iWnbuRF8T2ch4MlTXxeO2GGPCpEhs+n/tWcnelUV1Vi4AyzfmHm4PU5 Guhll2n1cK/x/Tq68Mx/iS/qkELN8tbGpJg31XhfIojza7VXYOKOdqie8mnm8XT1ym7z 0gkf1mcqBsvHefeCPXz1BgAof4PqyWTUtAqaJ3QV82SGOfHV98eNwaZqC0xlufAaQoxC ck2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version; bh=u2H4NHdpLl4a/XZuIN4Tnkf7Unxhwl07hVsLUSLyeD4=; fh=EyPK9oLIez4e0GV0H5wRzmmSrbJTJv3I6Yjya0R94Og=; b=dc2scdpTtYuGvGlufwpSecssFs7T3fn5HkpYS2DUhl9cmq4M9MZfE3rfJR1lAt8ItY 3KVD4KqBYBVDsuQXcfQOk3STNItciCKyCufGJvIQBm2XtULwMeMnah0MgHcn8V70y6fW bvS9453XdrmmkjSysZeH6jYlS2m5oX4udwpLA5fvczqTtZUdGSQGhP3gJY0neuca0N6X xoEZHo8MrbnvxtG0Li+LUhThvABIclGXnmoJBPp5quJPD3n0KKBT0YT2ucpbbflbke4F a7IAYnZX1MyGszuo9gjd4n0mswasAfbfyDZN7jNHi9rOIZkhsbxo5mlsiVpcB9FVRuu8 gW+w==; darn=freebsd.org ARC-Authentication-Results: i=1; mx.google.com; arc=none X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777735919; x=1778340719; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=u2H4NHdpLl4a/XZuIN4Tnkf7Unxhwl07hVsLUSLyeD4=; b=MrV8UrBfGt1PsK6756djmAxJ+GkmygLtwrKJ2A/WJRuVmTD8yWTN4hmT4c24nlPPH/ hSfqGttZU7QUI0Icc2mwFWRGKmmHK+L7tnDe75KbVzseP95m90NY7AEGgh4Gqrs9O2Wu TMvbaxyh3AdvWnDPquRYyqkokav+scpzwP6oQ+u1hjaKPbZmPGbQ0zoJ+zj7vbvJ+DiX g2JAKvdeIeN2C5e5fVUrvAOClK7blLJVjjeO78uyciTuDTZFGznlhadnTxRc++TV/83K QnSdYrrgfJDnyHEf7XoJkStDQWIJojtRt+L5geK80uARDS32SmQ0uX3E0bi9x/6NrTgK Zu6A== X-Forwarded-Encrypted: i=1; AFNElJ+pw7wfun4cHq5du0ozamfFIyDIMhNZHbphtzd7YX0UziHZIXWI4KPdoheesvX+MnMmUTcMKpqGowXE6tzPrvx3jykwWQ==@freebsd.org X-Gm-Message-State: AOJu0Yxrr2YuAIYKr0YeA8kTZihH05NdXSC6SGkX+C4TKbH10oUmnOjB 506NwCs69Dc/uWUOO6hJ4AclXWJ1UafJInkeTt0HRmtP6SZBSAX7GNxwVzneq4UbIP9VuKZ+85E 1kRh0xBGAMrr01blFlluu/bIiLM8+S2o= X-Gm-Gg: AeBDievnf+DEP16a8q6C/rxvo28T9xBqfrlfqqIkKGcNVZOmaApdU+HZncDhkcfiO8G eIih46EdFtyqs9YMiI2isl7funSIs4va0jyFZkFInu1AM1CFCTYJ2OdcdowdorzMcVKXpMRHrE/ W3PobTlNreIFVElhVl0x0W/ka9tdosadPbRIkX8WguPYSAmAsEP2hhFKHl4NrgkuOJ7H0xjD5su qn3yEj1z+CzaZET1eEsQRLj8dImA58iIBs61+2VUyB/2JlE6qYzCjgxduwYu6JfEX7ME+G8byrM MNNjdeSlQoQ84Pk2NZ9NNWT1XPv0HA1izD2NQb4MaqxdhDQBMqH63jS0568wGdxitBLnccY6W7E idUQ= X-Received: by 2002:a05:6402:a605:b0:67c:34bb:1bd1 with SMTP id 4fb4d7f45d1cf-67c34bb1ee8mr376094a12.1.1777735918863; Sat, 02 May 2026 08:31:58 -0700 (PDT) List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 References: <69f61649.1c022.73b2a773@gitrepo.freebsd.org> In-Reply-To: From: Alan Somers Date: Sat, 2 May 2026 09:31:46 -0600 X-Gm-Features: AVHnY4IJb9g52Q4ZlDv1g5Z9rIY5FmzkvYLInIt80kh21leox3DGa0hY79W-qTk Message-ID: Subject: Re: git: 3e845b109056 - main - ping: fix listing test cases when scapy is not installed To: Gleb Smirnoff Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4g7BjK4q5yz3GrC X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated On Sat, May 2, 2026 at 9:24=E2=80=AFAM Gleb Smirnoff = wrote: > > On Sat, May 02, 2026 at 03:20:41PM +0000, Alan Somers wrote: > A> commit 3e845b1090565912375c5578cf0399d27b7fa70c > A> Author: Alan Somers > A> AuthorDate: 2026-05-01 23:06:35 +0000 > A> Commit: Alan Somers > A> CommitDate: 2026-05-02 15:20:16 +0000 > A> > A> ping: fix listing test cases when scapy is not installed > A> > A> The ATF-python test program was attempting to list test cases that > A> require scapy. But it attempted to import the scapy module before= the > A> test cases had been listed, resulting in an ImportError that kyua > A> interpreted as a test program crash. > A> > A> Fix this behavior by handling that ImportError well enough to list= test > A> cases, but not run them. If scapy isn't present, Kyua will refuse= to > A> run the test cases. But it needs to be able to list them in order= to > A> know to skip them. > A> > A> Sponsored by: ConnectWise > A> MFC after: 2 weeks > A> Reviewed by: maxim > A> Differential Revision: https://reviews.freebsd.org/D56765 > > Sorry for not reviewing in time. I don't think Kyua expects a test progr= am to > be able to list the test cases, but then fail to run them. > > Isn't there a standard way of dealing with the problem? Many tests have > @pytest.mark.require_progs(["scapy"]) in them. > > -- > Gleb Smirnoff Yes, these tests already include @pytest.mark.require_progs(["scapy"]) correctly. That's how Kyua is supposed to learn that it can't run those test cases. You can tell by running the program manually, bypassing kyua: $ ./test_ping.py - Content-Type: application/X-atf-tp; version=3D"1" ident: TestPing::test_ping[_4_c1_s56_t1_localhost] descr: Test ping has.cleanup: true require.config: unprivileged_user require.user: root ... ident: TestPing::test_pinger[_0_0] descr: Test ping using pinger(), a reply faker has.cleanup: true require.user: root require.progs: scapy ... Notice that the first test case does not require scapy, but the second one does. This is how Kyua learns that it can't execute the second test if scapy is not installed. However, prior to my commit, if scapy was not installed, running test_ping.py -l would fail with exit status 2 and produce no output, and Kyua would interpret that as a broken test program, like this: test_ping.py:__test_cases_list__