From nobody Fri Mar 31 00:26:30 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pnh163P9Lz42bWt; Fri, 31 Mar 2023 00:26:34 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pnh161TYHz3HYQ; Fri, 31 Mar 2023 00:26:34 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qv1-xf2a.google.com with SMTP id kr22so1410129qvb.5; Thu, 30 Mar 2023 17:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680222393; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=5slujqQJG/GptGzXid1sqlcXiZc+OJxF8OYdCLtqw/w=; b=PMX35DyR2nk/E254SA7RaLthfJN/1tZf1YhKDoH7cnS4N4RZnZih4O2jqjq/vjJjWO oBldUNanG8W7/dAgPQYUxsW3qKPVjuHU+K/A+qsi1jwIczdfY7xyuONP6b1XJaG0RAQy 30ninohCjFqbcKmg3CZeq0qbV2r+R542GfjeU0L8uMn2Cpme7TnkIjWluVoAhkZOTNuq zMYsRE42vCIUa8QU4u9bC0OMm6Q8NDVUQIl3Qci3hBWvpIW6DB6h9//utkDL1N6VMw6D Wl5i1BUNJ3x266AxU+BAO8HOwbBeWwdhMR6NHx0o2IC0qj+L01NXyrwt1yPGe1pbAT/d 4HyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680222393; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5slujqQJG/GptGzXid1sqlcXiZc+OJxF8OYdCLtqw/w=; b=FekYQ+KaflX4tUpP0BVgywNlByCTORqBKKzXG55jkiOgI/VaUmC4OIKb2iIw6x5wp+ 0bV71h8lfm6Qlp7HPy5Z/xvHsr7bWwT35ssG82nf4AOkz2qDCTe3LRzFKF26V14nGQwd JSXnXZPrVaANbsuL8ysR5RzIB62uJmUdjIskSRLtl6lYCkEW1lt34SBRuJCU4jzbdr4M Uf14408IFX0w4sMOcDPkPW+PWJW+CgWkc4rmkgJIKkq4Q3LTij+xS3i15CSvsR4h2x7R mnh6W7mcKVGdmzu55kH81B4uBmfQ4YNX6Rhm5aSsvtmQABfcfdzHCJ35/7l/ogVq6w7i atPg== X-Gm-Message-State: AAQBX9dZVwHqhj11IVrW86fCC7T60vPk1a5HpeF+0Q7S46BK6hfo6qds UQJq2uNPTsH5gPIMtkjsGcU790CjnC8= X-Google-Smtp-Source: AKy350ZPrQZyFlNVa++PqKVoD1qTEGP3RKsnX/J7GTymur/2wGkxhdD04ZwdKgI9lvTkeGzsUPCvBw== X-Received: by 2002:a05:6214:d05:b0:56e:9c11:651e with SMTP id 5-20020a0562140d0500b0056e9c11651emr42121511qvh.27.1680222392899; Thu, 30 Mar 2023 17:26:32 -0700 (PDT) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id s11-20020a05621412cb00b005df44d4d953sm203301qvv.127.2023.03.30.17.26.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Mar 2023 17:26:32 -0700 (PDT) Date: Thu, 30 Mar 2023 20:26:30 -0400 From: Mark Johnston To: Kristof Provost Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: b60600ceeb68 - main - pf tests: Serialize Message-ID: References: <202303302336.32UNatba067594@gitrepo.freebsd.org> <3A26AB82-357F-421F-853E-07320387ACBE@FreeBSD.org> List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 4Pnh161TYHz3HYQ X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Fri, Mar 31, 2023 at 09:15:27AM +0900, Kristof Provost wrote: > On 31 Mar 2023, at 9:05, Mark Johnston wrote: > > On Fri, Mar 31, 2023 at 08:56:56AM +0900, Kristof Provost wrote: > >> On 31 Mar 2023, at 8:36, Mark Johnston wrote: > >>> The branch main has been updated by markj: > >>> > >>> URL: https://cgit.FreeBSD.org/src/commit/?id=b60600ceeb68d1001d61222830e0be3441ef0885 > >>> > >>> commit b60600ceeb68d1001d61222830e0be3441ef0885 > >>> Author: Mark Johnston > >>> AuthorDate: 2023-03-25 12:55:41 +0000 > >>> Commit: Mark Johnston > >>> CommitDate: 2023-03-30 23:35:59 +0000 > >>> > >>> pf tests: Serialize > >>> > >>> These tests reuse jail names and cannot run in parallel. Until this is > >>> fixed - which is desirable since these takes take a while to run - tell > >>> kyua to serialize them. > >>> > >> > >> The tests also recycle IP ranges, so merely changing the jail names is insufficient. > > > > Could you please give an example? I looked at some of the tests but can > > only see cases where addresses are assigned within the vnet jail(s) > > created by the tests, in which case I'd expect no problems. > > > Altq:hfsq assigns 192.0.2.1/24 on an epair on the host, so does altq:match, altq:cbq_vlan, altq:codel_bridge, dup:dup_to, ether:mac, ether:proto and .. then I got bored looking. > > It’s not just the pf tests either. For example the netinet/forward:fwd_ip_icmp_iface_fast_success test does that too (well, 192.0.2.1/29, but anyway). There are going to be more, this is just from a very quick look. I see now, thanks. > >> Realistically the easiest way to get these to run in parallel would be to run each test in its own vnet so both overlapping IP ranges and name conflicts don’t matter. > > > > Yeah, I was wondering whether it'd be possible to have kyua handle the > > creation and teardown of a per-test jail, if only to avoid having to go > > through all of the tests which use static jail names. > > That’s what I was thinking as well, yes. > > We might be able to get to a point where all the tests have unique jail names, but we’re never going to manage to de-conflict all of the IP assignments. Even if we could, it’d make writing tests harder and that’s counterproductive. As it happens I'm looking at netinet6/forward6 tests, which have exactly the same problem. Each of those tests creates an epair, puts one end in a vnet jail, and configures the other on the host. It was pretty easy to simply create a second jail for the other end of the epair, and to my eye doesn't make the test much more complicated. Some more commands have to be prefixed with jexec is all. I'm sure that converting the pf tests would be much more tedious, if not more complicated. There is some downside to having kyua manage even more context, since that could make tests harder to debug, and kyua already makes that somewhat unpleasant. > The only issue I can think of with having the tests run in their own vnet is that there may be some tests that play with non-vneted sysctls, so we will have to make that vnet feature optional. Yeah, I think this would have to be opt-in.