From nobody Mon Jan 23 22:12:16 2023
X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4P148d2czjz2t14y;
Mon, 23 Jan 2023 22:12:17 +0000 (UTC)
(envelope-from git@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
client-signature RSA-PSS (4096 bits) client-digest SHA256)
(Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4P148d1P9tz3MLW;
Mon, 23 Jan 2023 22:12:17 +0000 (UTC)
(envelope-from git@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
t=1674511937;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding;
bh=Hr4ctser7LBBNSahpZesnlm5sqtrt8bW/92rs4IHNZg=;
b=wY6r8NLI8niY7LPk1VaE0zY7KsKLHYf5+SsHAWhK/4vJza7sWlfVcIrKyvH5t+f1i85qux
R5kwTeB5Gv5ZSwwOex276o4/2vcMwTL22708pTGW4I+RhK8/4VKvDf/XcnSuulBNaLFZ1c
//KjoRj9611HgxFkXRGaXmCl12vBO6hni1CPq10XKr7G8KbQpKJ9uNvT8ijwDfs6/7c3SB
jIh4K5LK21fBLnHhLSiPRXsJJiB00Sa64xRwAUakctRQenr9fvTEouc0WFJCM9JGdclzsI
TcnwYj/Bjipde8j1e63cw7Z75CqSELtjS9iq55l/L0m2W6VSwGkG4PR5ZWKRlw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
s=dkim; t=1674511937;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding;
bh=Hr4ctser7LBBNSahpZesnlm5sqtrt8bW/92rs4IHNZg=;
b=s4rj3yGJ0f8kMdh0O3av/Jn4VH99HTEJppm/SW0CdNww3Ec6hWJzlLOIRcEE3QrJQggu/Y
bl1ZUubF52TtQ1FDCRPvRKFLq+1nXaKzIHHN7CfPN9G9UEuxVdH2+mENwOCsvBi47XG7BF
pHhHwIa2MRv23voDsIiHaHcHw58WzN1iCte9ajz6PtM03684JBM+wri/DRfIf6JL+ey23b
rlCium/E43Gp1UbuU04fyMwdUy8XpLQUmjEX5uUBTBHR158Uyx8Ii+UJhQCrqcLMlRoucD
bo4HQMVMarhIkjz4HA6UYkqnRKf/cR9S6eHJwAHwCHx1AeWI/bSWTH5sGPm6SA==
ARC-Authentication-Results: i=1;
mx1.freebsd.org;
none
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1674511937; a=rsa-sha256; cv=none;
b=FcdMDUNcP6T01YuekzpVpCbvg2yifND0q6bqBX7Cd55wh+3T9oJqW01qWSiNfoC+qaKt3T
v5YSn4Uo6y92thPTtktZ6K2ICu560bAFe0vEMXb+4oGk/jVxUnG4QtENgo7H7TqCE+NY0R
ut7AGudtG1TNbGsdSDoccfBg0bg+RK4Lq/Jaix/sCZJtYgbdFGk24Q2oh7GqxALEG8jA62
BqTuJuOvdHGHOYY6yp3YTYJUMZfwnNzPeM+RLCctZ9kXzmWRW5DEnuqImbgE6nkW+M6spP
LW45gk1VqSqEvYrw6OJws2257NYEkPZ6HGoYYU8y+diz7Sn9SyjtzkVmVHXssw==
Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4P148d088szlrJ;
Mon, 23 Jan 2023 22:12:17 +0000 (UTC)
(envelope-from git@FreeBSD.org)
Received: from gitrepo.freebsd.org ([127.0.1.44])
by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 30NMCGoS017227;
Mon, 23 Jan 2023 22:12:16 GMT
(envelope-from git@gitrepo.freebsd.org)
Received: (from git@localhost)
by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 30NMCGXa017226;
Mon, 23 Jan 2023 22:12:16 GMT
(envelope-from git)
Date: Mon, 23 Jan 2023 22:12:16 GMT
Message-Id: <202301232212.30NMCGXa017226@gitrepo.freebsd.org>
To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org,
dev-commits-src-branches@FreeBSD.org
From: "Alexander V. Chernikov" <melifaro@FreeBSD.org>
Subject: git: 176e0427b208 - stable/13 - testing: add python test examples
List-Id: Commits to the stable branches of the FreeBSD src repository <dev-commits-src-branches.freebsd.org>
List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches
List-Help: <mailto:dev-commits-src-branches+help@freebsd.org>
List-Post: <mailto:dev-commits-src-branches@freebsd.org>
List-Subscribe: <mailto:dev-commits-src-branches+subscribe@freebsd.org>
List-Unsubscribe: <mailto:dev-commits-src-branches+unsubscribe@freebsd.org>
Sender: owner-dev-commits-src-branches@freebsd.org
X-BeenThere: dev-commits-src-branches@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Git-Committer: melifaro
X-Git-Repository: src
X-Git-Refname: refs/heads/stable/13
X-Git-Reftype: branch
X-Git-Commit: 176e0427b208a326ef8256c53c79edeea43e13dc
Auto-Submitted: auto-generated
X-ThisMailContainsUnwantedMimeParts: N
The branch stable/13 has been updated by melifaro:
URL: https://cgit.FreeBSD.org/src/commit/?id=176e0427b208a326ef8256c53c79edeea43e13dc
commit 176e0427b208a326ef8256c53c79edeea43e13dc
Author: Alexander V. Chernikov <melifaro@FreeBSD.org>
AuthorDate: 2023-01-01 14:35:41 +0000
Commit: Alexander V. Chernikov <melifaro@FreeBSD.org>
CommitDate: 2023-01-23 22:09:04 +0000
testing: add python test examples
Simplify the adoption of python tests by proving some examples,
utilising commonly-used patterns.
Differential Revision: https://reviews.freebsd.org/D37902
Reviewed by: asomers
MFC after: 2 weeks
(cherry picked from commit 8161b823d77f9d89ffabd47444a83d693f74c515)
---
etc/mtree/BSD.tests.dist | 2 +
tests/Makefile | 1 +
tests/examples/Makefile | 10 ++
tests/examples/test_examples.py | 198 ++++++++++++++++++++++++++++++++++++++++
4 files changed, 211 insertions(+)
diff --git a/etc/mtree/BSD.tests.dist b/etc/mtree/BSD.tests.dist
index 0dabe96110d8..136877db00d6 100644
--- a/etc/mtree/BSD.tests.dist
+++ b/etc/mtree/BSD.tests.dist
@@ -263,6 +263,8 @@
rc.d
..
..
+ examples
+ ..
games
..
gnu
diff --git a/tests/Makefile b/tests/Makefile
index b406b8dc6c17..47fc9488f772 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -9,6 +9,7 @@ ${PACKAGE}FILES+= README __init__.py conftest.py
KYUAFILE= yes
SUBDIR+= etc
+SUBDIR+= examples
SUBDIR+= sys
SUBDIR+= atf_python
diff --git a/tests/examples/Makefile b/tests/examples/Makefile
new file mode 100644
index 000000000000..7a5d84a98dfe
--- /dev/null
+++ b/tests/examples/Makefile
@@ -0,0 +1,10 @@
+# $FreeBSD$
+
+PACKAGE= tests
+
+TESTSDIR= ${TESTSBASE}/examples
+
+ATF_TESTS_PYTEST += test_examples.py
+
+.include <bsd.test.mk>
+
diff --git a/tests/examples/test_examples.py b/tests/examples/test_examples.py
new file mode 100644
index 000000000000..13fdcc420f0e
--- /dev/null
+++ b/tests/examples/test_examples.py
@@ -0,0 +1,198 @@
+import pytest
+from atf_python.utils import BaseTest
+from atf_python.sys.net.tools import ToolsHelper
+from atf_python.sys.net.vnet import SingleVnetTestTemplate
+from atf_python.sys.net.vnet import VnetTestTemplate
+from atf_python.sys.net.vnet import VnetInstance
+
+import errno
+import socket
+import subprocess
+import json
+
+from typing import List
+
+
+# Test classes should be inherited
+# from the BaseTest
+
+
+class TestExampleSimplest(BaseTest):
+ @pytest.mark.skip(reason="comment me to run the test")
+ def test_one(self):
+ assert ToolsHelper.get_output("uname -s").strip() == "FreeBSD"
+
+
+class TestExampleSimple(BaseTest):
+ # List of required kernel modules (kldstat -v)
+ # that needs to be present for the tests to run
+ REQUIRED_MODULES = ["null"]
+
+ @pytest.mark.skip(reason="comment me to run the test")
+ def test_one(self):
+ """Optional test description
+ This and the following lines are not propagated
+ to the ATF test description.
+ """
+ pass
+
+ @pytest.mark.skip(reason="comment me to run the test")
+ # List of all requirements supported by an atf runner
+ # See atf-test-case(4) for the detailed description
+ @pytest.mark.require_user("root")
+ @pytest.mark.require_arch(["amd64", "i386"])
+ @pytest.mark.require_files(["/path/file1", "/path/file2"])
+ @pytest.mark.require_machine(["amd64", "i386"])
+ @pytest.mark.require_memory("200M")
+ @pytest.mark.require_progs(["prog1", "prog2"])
+ @pytest.mark.timeout(300)
+ def test_two(self):
+ pass
+
+ @pytest.mark.skip(reason="comment me to run the test")
+ @pytest.mark.require_user("unprivileged")
+ def test_syscall_failure(self):
+ s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
+ with pytest.raises(OSError) as exc_info:
+ s.bind(("::1", 42))
+ assert exc_info.value.errno == errno.EACCES
+
+ @pytest.mark.skip(reason="comment me to run the test")
+ @pytest.mark.parametrize(
+ "family_tuple",
+ [
+ pytest.param([socket.AF_INET, None], id="AF_INET"),
+ pytest.param([socket.AF_INET6, None], id="AF_INET6"),
+ pytest.param([39, errno.EAFNOSUPPORT], id="FAMILY_39"),
+ ],
+ )
+ def test_parametrize(self, family_tuple):
+ family, error = family_tuple
+ try:
+ s = socket.socket(family, socket.SOCK_STREAM)
+ s.close()
+ except OSError as e:
+ if error is None or error != e.errno:
+ raise
+
+ # @pytest.mark.skip(reason="comment me to run the test")
+ def test_with_cleanup(self):
+ print("TEST BODY")
+
+ def cleanup_test_with_cleanup(self, test_id):
+ print("CLEANUP HANDLER")
+
+
+class TestVnetSimple(SingleVnetTestTemplate):
+ """
+ SingleVnetTestTemplate creates a topology with a single
+ vnet and a single epair between this vnet and the host system.
+ Additionally, lo0 interface is created inside the vnet.
+
+ Both vnets and interfaces are aliased as vnetX and ifY.
+ They can be accessed via maps:
+ vnet: VnetInstance = self.vnet_map["vnet1"]
+ iface: VnetInterface = vnet.iface_alias_map["if1"]
+
+ All prefixes from IPV4_PREFIXES and IPV6_PREFIXES are
+ assigned to the single epair interface inside the jail.
+
+ One can rely on the fact that there are no IPv6 prefixes
+ in the tentative state when the test method is called.
+ """
+
+ IPV6_PREFIXES: List[str] = ["2001:db8::1/64"]
+ IPV4_PREFIXES: List[str] = ["192.0.2.1/24"]
+
+ def setup_method(self, method):
+ """
+ Optional pre-setup for all of the tests inside the class
+ """
+ # Code to run before vnet setup
+ #
+ super().setup_method(method)
+ #
+ # Code to run after vnet setup
+ # Executed inside the vnet
+
+ @pytest.mark.skip(reason="comment me to run the test")
+ @pytest.mark.require_user("root")
+ def test_ping(self):
+ assert subprocess.run("ping -c1 192.0.2.1".split()).returncode == 0
+ assert subprocess.run("ping -c1 2001:db8::1".split()).returncode == 0
+
+ @pytest.mark.skip(reason="comment me to run the test")
+ def test_topology(self):
+ vnet = self.vnet_map["vnet1"]
+ iface = vnet.iface_alias_map["if1"]
+ print("Iface {} inside vnet {}".format(iface.name, vnet.name))
+
+
+class TestVnetDual1(VnetTestTemplate):
+ """
+ VnetTestTemplate creates topology described in the self.TOPOLOGY
+
+ Each vnet (except vnet1) can have a handler function, named
+ vnetX_handler. This function will be run in a separate process
+ inside vnetX jail. The framework automatically creates a pipe
+ to allow communication between the main test and the vnet handler.
+
+ This topology contains 2 VNETs connected with 2 epairs:
+
+ [ VNET1 ] [ VNET2 ]
+ if1(epair) 2001:db8:a::1/64 <-> 2001:db8:a::2/64 if1(epair)
+ if2(epair) 2001:db8:b::1/64 <-> 2001:db8:b::2/64 if2(epair)
+ lo0 lo0
+
+ """
+
+ TOPOLOGY = {
+ "vnet1": {"ifaces": ["if1", "if2"]},
+ "vnet2": {"ifaces": ["if1", "if2"]},
+ "if1": {"prefixes6": [("2001:db8:a::1/64", "2001:db8:a::2/64")]},
+ "if2": {"prefixes6": [("2001:db8:b::1/64", "2001:db8:b::2/64")]},
+ }
+
+ def _get_iface_stat(self, os_ifname: str):
+ out = ToolsHelper.get_output(
+ "{} -I {} --libxo json".format(ToolsHelper.NETSTAT_PATH, os_ifname)
+ )
+ js = json.loads(out)
+ return js["statistics"]["interface"][0]
+
+ def vnet2_handler(self, vnet: VnetInstance):
+ """
+ Test handler that runs in the vnet2 as a separate process.
+
+ This handler receives an interface name, fetches received/sent packets
+ and returns this data back to the parent process.
+ """
+ while True:
+ # receives 'ifX' with an infinite timeout
+ iface_alias = self.wait_object(vnet.pipe, None)
+ # Translates topology interface name to the actual OS-assigned name
+ os_ifname = vnet.iface_alias_map[iface_alias].name
+ self.send_object(vnet.pipe, self._get_iface_stat(os_ifname))
+
+ @pytest.mark.skip(reason="comment me to run the test")
+ @pytest.mark.require_user("root")
+ def test_ifstat(self):
+ """Checks that RX interface packets are properly accounted for"""
+ second_vnet = self.vnet_map["vnet2"]
+ pipe = second_vnet.pipe
+
+ # Ping neighbor IP on if1 and verify that the counter was incremented
+ self.send_object(pipe, "if1")
+ old_stat = self.wait_object(pipe)
+ assert subprocess.run("ping -c5 2001:db8:a::2".split()).returncode == 0
+ self.send_object(pipe, "if1")
+ new_stat = self.wait_object(pipe)
+ assert new_stat["received-packets"] - old_stat["received-packets"] >= 5
+
+ # Ping neighbor IP on if2 and verify that the counter was incremented
+ self.send_object(pipe, "if2")
+ old_stat = self.wait_object(pipe)
+ assert subprocess.run("ping -c5 2001:db8:b::2".split()).returncode == 0
+ self.send_object(pipe, "if2")
+ new_stat = self.wait_object(pipe)
+ assert new_stat["received-packets"] - old_stat["received-packets"] >= 5