Date: Thu, 28 Aug 2025 14:18:22 +0000 From: bugzilla-noreply@freebsd.org To: testing@FreeBSD.org Subject: [Bug 289146] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_action tests failing in CI Message-ID: <bug-289146-32464@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=289146 Bug ID: 289146 Summary: sbin/ipfw/test_add_rule.py:TestAddRule::test_add_actio n tests failing in CI Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Keywords: regression Severity: Affects Only Me Priority: --- Component: tests Assignee: melifaro@FreeBSD.org Reporter: me@svmhdvn.name CC: testing@FreeBSD.org Created attachment 263289 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=263289&action=edit kyua test report After https://cgit.freebsd.org/src/commit/?id=d879f1c8961d99bdbe6727004bfc6c23924526f8 added the HAS_TESTS= variable to sbin/ipfw/Makefile (to reveal the hidden testcases that were never being run), the following parameterized testcases inside the 'sbin/ipfw/test_add_rule.py:TestAddRule::test_add_action' testsuite are now showing up to fail in CI. sbin/ipfw/test_add_rule.py:TestAddRule::test_add_action[call_420] -> failed: /usr/tests/atf_python/sys/netpfil/ipfw/insns.py:105: ValueError [0.231s] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_action[return] -> failed: /usr/tests/atf_python/sys/netpfil/ipfw/insns.py:105: ValueError [0.272s] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_action[skipto_42] -> failed: /usr/tests/atf_python/sys/netpfil/ipfw/insns.py:105: ValueError [0.278s] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_rule[test_check_state] -> failed: /usr/tests/atf_python/sys/netpfil/ipfw/insns.py:105: ValueError [0.223s] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_rule[test_eaction_ntp] -> failed: /usr/tests/sbin/ipfw/test_add_rule.py:119: AssertionError [0.189s] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_rule[test_eaction_tcp-setmss] -> failed: /usr/tests/sbin/ipfw/test_add_rule.py:119: AssertionError [0.192s] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_rule[test_keep_state] -> failed: /usr/tests/atf_python/sys/netpfil/ipfw/insns.py:105: ValueError [0.222s] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_rule[test_record_state] -> failed: /usr/tests/atf_python/sys/netpfil/ipfw/insns.py:105: ValueError [0.218s] sbin/ipfw/test_add_rule.py:TestAddRule::test_add_rule[test_tables] -> failed: /usr/tests/sbin/ipfw/test_add_rule.py:119: AssertionError [0.202s] I've attached the full Kyua test report from my test run in this bug, which contains the stderr from these failing cases. I have reproduced this error with two methods and attached a sample test-report.txt: 1. using src in-tree local CI: # make -C tests/ci CITYPE=full PARALLEL_JOBS=2 VM_MEM_SIZE=4g KYUA_TEST_FILTERS='sbin/ipfw/test_add_rule.py' 2. using bricoler[0]: $ bricoler run freebsd-src-regression-suite --param freebsd-src:url=/usr/src --param freebsd-src:branch= --param freebsd-src-regression-suite:hypervisor=bhyve --param freebsd-src-regression-suite:memory=4096 --param freebsd-src-regression-suite:ncpus=2 --param freebsd-src-regression-suite:parallelism=1 --param freebsd-src-regression-suite:tests='sbin/ipfw/test_add_rule.py' @melifaro I see that you wrote these test cases. Please triage as necessary. [0] https://github.com/markjdb/bricoler -- You are receiving this mail because: You are on the CC list for the bug.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-289146-32464>