From nobody Thu Dec 19 16:47:29 2024 X-Original-To: testing@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YDc0g6KSnz5hRs4 for ; Thu, 19 Dec 2024 16:47:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YDc0g3c0mz3xlc for ; Thu, 19 Dec 2024 16:47:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1734626851; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DCxM/OE40unQyXDDfZDK5xnEC0dtikRVlI/MQA6oNxM=; b=hyYuLEkGzcm7ClYud/bGi6WnwIz3n/xgtgsdI9Pycflg8l25HVc6bWV6mSDh6c7O+OHzSy l59+I2l6scyah4YyiNwJHVoIDqbxCLuAV3z9OPbfxNYMtdkqPe2XGctdlsrLUmZpXQtQc3 wvqBYFX2KbBOMzKn+Jmf2ROI+Ee29THSY/jn9L8p9HH1rBKuJGak3bEqsXzUlxEnccInJs mF7KggYoesgn3C8BdEpAWE7JHbCGZpXJEYOJPu/fvggaSQyuP6bfkFubggU5xNyQAFDrAO ojnDnf/sjjI7mlU7bHvdI9Os2aqYQATJtXlndSDiVECeap1KlQzi9WUxLP3tDQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1734626851; a=rsa-sha256; cv=none; b=O1GfAvedb8EpoTf+0KaYFcv5aK3wrSrfJOJAtl1bZujwQlj2XfPsOuJDMonB2vJK3Z1olQ RzI6y9NgpfUs1BGBNs++Blsw0OvOwHZY4YsN6leCrrYkNNjAMC5zbbrcKBTBFAnUim2bqr 6bCmA9BpQgc66rKt+WCFMHPpGj/tPkOT7rzbTwyOeTwlBF1Ll8idi7kFxbIsebhkYbn9BA v0SH06Ldx0pUEWFSsvc+n2cpDXI7wn/zleKguQpNerv9J4hwntK0k3IxqaqzPX3fdILAdv TfFXJej1OZlwHcXhozICuACf8kzPXKoH+zr2mcUI8sC2PSN70bmjJ3r0hYvxTA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4YDc0d5Y6fz122Q for ; Thu, 19 Dec 2024 16:47:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 4BJGlTHb004796 for ; Thu, 19 Dec 2024 16:47:29 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 4BJGlTrs004795 for testing@FreeBSD.org; Thu, 19 Dec 2024 16:47:29 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: testing@FreeBSD.org Subject: [Bug 267006] Make clean tests fail on FreeBSD 12.3-STABLE machine Date: Thu, 19 Dec 2024 16:47:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: tests X-Bugzilla-Version: 12.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: admin@support.od.ua X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: testing@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Testing List-Archive: https://lists.freebsd.org/archives/freebsd-testing List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-testing@freebsd.org Sender: owner-freebsd-testing@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D267006 --- Comment #2 from Vladyslav V. Prodan --- This is no longer observed in the stable/13 and stable/14 branches. The bug report can be closed. # Host: git.freebsd.org # Port: 443 # Repository Path: /src.git # Target Directory: /usr/src # Low memory mode: Yes # Want: 11c7eb30cc9b38d199c4686ad00071a678a1fc58 # Branch: stable/13 # Action: clone ... and=20 # Host: git.freebsd.org # Port: 443 # Repository Path: /src.git # Target Directory: /usr/src # Low memory mode: Yes # Want: 705337455f0622223fc58a78a77ea1a536d746a5 # Branch: stable/14 ... --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Thu Dec 19 21:05:42 2024 X-Original-To: testing@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YDjkb1KRCz5hlMM for ; Thu, 19 Dec 2024 21:05:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YDjkY6V5Wz4VFv for ; Thu, 19 Dec 2024 21:05:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1734642341; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KI1/gmNLfn3RTipq7y9Z/UaJFRdBw+Lt8+aBviH00WE=; b=n7PuemYRVxR/fvKhAamh/jwqVXVG6dsJdzVP7hvO/bfhO7nwo3ElehqcdSnYaeUNmFURJS YZtP7c1k/l4B4XZfEDZ534nXBzlqI8Uj3ooMatRBScwo4HpXOPqhgA1yqoXfVHQyv3j5vP uVaNwYFKfzxg78E3a62JcNHlWQwAq/+aPD2exAZo5mbOsqVnRNtmOBnlmDrc9fZSEmZqPV Ro0iv/YHU2uKydfLHU5MvmiX8n69SJt4k6flt1AVtkWTiH3bv/FnKGao1eKADe8IZTHyWF u99XMli7KOZD1ImT6ZOnpzQW1EHcAHSE9RfFIZPjQJM4WrX4Ptbld0uELIJiIQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1734642341; a=rsa-sha256; cv=none; b=eXfGXsC2+Z6iejQdoTX7+95dsrj2LfPTMTDicRwKSqJKG3zQPkzvjJ/LHgYDhMNlA4osNw XVXMQ1iC0cToUjoGDl0D6GMPo9j8AB3eRTXyRHNIkDByXCqsRHhTo7ZV7h5eMxLO9j1fCE MgSSMvxNt6KpGRXd1CrGNR4IyVUZZcmdiaoDuyQFMymddHYZWkDkerHGtmVdpCRBjuhmPa 8RA2OqtifUSJd5vKDPeW/5IPUtX1Ap28Vjo/7LsL0H5aOdk5IB59gLzelcbp/Pl0in6Rsf ZsqlS0myRulBv6ivihxid5hztn+I1hil5xb9D4DnAUo2/BwLtVemwXvsYi19tA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4YDjkY5xhXz18nB for ; Thu, 19 Dec 2024 21:05:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 4BJL5fJM039078 for ; Thu, 19 Dec 2024 21:05:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 4BJL5fSR039077 for testing@FreeBSD.org; Thu, 19 Dec 2024 21:05:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: testing@FreeBSD.org Subject: [Bug 267006] Make clean tests fail on FreeBSD 12.3-STABLE machine Date: Thu, 19 Dec 2024 21:05:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: tests X-Bugzilla-Version: 12.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: emaste@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Unable to Reproduce X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: testing@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Testing List-Archive: https://lists.freebsd.org/archives/freebsd-testing List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-testing@freebsd.org Sender: owner-freebsd-testing@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D267006 Ed Maste changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |Unable to Reproduce Status|New |Closed --- Comment #3 from Ed Maste --- Thanks for the followup, closing as not reproducible. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sun Dec 22 16:08:27 2024 X-Original-To: freebsd-testing@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YGR0Z4vzDz5YwPf; Sun, 22 Dec 2024 16:08:46 +0000 (UTC) (envelope-from igoro@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YGR0Z2sBlz4Z8k; Sun, 22 Dec 2024 16:08:46 +0000 (UTC) (envelope-from igoro@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1734883726; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nk8KCtozKyxT1acNcx8Vfg+RZrZzkZIJUvqKODgLub0=; b=RHPaCoP1fhggMBx4B4qo2a5l9nhf0Slpqsau/FLFAWuzsr7NRWmKjRPOHaF9Qld7K+GoQc VlbVSOfVVBrcUMKlC/iBUHNhWSoF62X6WezWQJvRUFvuuVCuwCfv9oIInHHCsRtgWxP8VT WVECMeRGYiWhq5uWTkJofYWyd0z2UqsEMlkm8fV4FZlkNgXLQ8b5YW1pB1GK6Xfbw/5KPe KKG4RELO+Z+eOzmxFhPYXxPLztv4lE0IlgHuGchs7O3tibsLtTuAGfx2spJWaNxSUcPsDX 7pG8YDDZuiXvq6s3KEk3IUtzz9glRqGF2ZTiX+Fdeio9VCz+sNENi/on9Z5MHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1734883726; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nk8KCtozKyxT1acNcx8Vfg+RZrZzkZIJUvqKODgLub0=; b=aGFry0+LT0iaNNMd2j9Mye727I7McIVUn+btwBeVd2UIX8UB6r01eKdPiopQndCmuNG/zn vZvnZPPBYDLsg8jUASQq/U+I/2zRPq4OE0T+v4uJvzbZM39dIb9zE6kAyl4vTtXe4OQY0F v9UlAl2Pv/ZBXHnbL5EfsnGTfizuRxySElyZLNhnRqi797mkk77wjYgSrFEMCr3NbpLDm/ 46mwY8av+qTDeCc4lzarnz7utN65WhADh/0/ju3oUFMbNjH1VW/WP4t0X/Gy8Mve3WfLgn rG0uxOGRJRQjkZIUPPI+gMqvPt77om5g+qc4fHlF5pURulE2mNcrUd3D8FvLTw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1734883726; a=rsa-sha256; cv=none; b=icQMlkUjExpADFEI+bOB4l78GBiWRo07A/UEgNsUgQM1zARQqhLKtpNi/WNARhAg0gQi9O X5gx1vvl3RPPv25gV9mZgicMOZSSFJp+8pT2h4fLIzEoOGK9oSkprUaGUa3+rqL+UGiakX 3FyTJvQDv/Jr7qCq0h4dsT1SGoPtfY197ESFEUvM42oEwzRxz8mntZa8k0IO2gW5aB8jME ohDYtAMkqg/CvInREN9b3dGh25luTTv5dc3NQZc5jXojxUrymbzK8y3dOVvbZxObNAlR+A tUtIrbUPYJu1lDTlycAWWE1Ake7AilG9YwkdG0xNjUpAjORash8p5gwHtu2SIg== Received: from [192.168.0.8] (unknown [78.83.216.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: igoro/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4YGR0Y6fxXz1MGl; Sun, 22 Dec 2024 16:08:45 +0000 (UTC) (envelope-from igoro@FreeBSD.org) Message-ID: <67e3f4b3-ef83-4f65-88e4-e1228caee830@FreeBSD.org> Date: Sun, 22 Dec 2024 18:08:27 +0200 List-Id: Testing List-Archive: https://lists.freebsd.org/archives/freebsd-testing List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-testing@freebsd.org Sender: owner-freebsd-testing@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Igor Ostapenko Subject: Re: RFC: Add required_klds metadata to Kyua To: freebsd-testing@freebsd.org, freebsd-hackers@freebsd.org References: <9fc6fc72-4c2c-4b09-bdb5-122a49b45295@FreeBSD.org> Content-Language: en-US In-Reply-To: <9fc6fc72-4c2c-4b09-bdb5-122a49b45295@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, The Project B patch is open for testing and discussion: https://reviews.freebsd.org/D48087 The outline of the currently proposed UI/UX is as follows: # kyua prepare all Run all requirement resolvers kmods Resolve required_kmods for FreeBSD # grep -r kmods * Kyuafile:atf_test_program{name="divert-to", ... required_kmods="if_epair if_bridge pf ipdivert"} mbuf: atf_set require.kmods if_epair if_bridge pf dummymbuf # kyua prepare --dry-run all kldload dummymbuf if_bridge if_epair ipdivert pf # kyua prepare kmods kldload dummymbuf if_bridge if_epair ipdivert pf Testing and sharing your thoughts are welcome. Best regards, igoro Igor Ostapenko wrote on 11/6/24 8:12 PM: > Hi FreeBSD developers, > > During Kyua's execenv=jail feature implementation there were discussions to > get some help from Kyua side with kernel modules required by specific tests. > The thing got "required_klds" codename. > > The problem to solve here is a well-known one: typically we invoke "kyua > test" and get a lot of "Skipped" due to many kernel module requirements. And > the pain is to know which ones need loading, especially if we target a > sub-tree of /usr/tests. Also, the FreeBSD CI needs periodic sync if a new > test is added with new module requirements. And other related issues. > > I would like to share the existing ideas and vision before the > implementation itself. I propose to split it onto several subsequent > projects to ease reasoning, review, etc. > > > Project A: declaration & skipping > --------------------------------- > > Add a new metadata property to Kyua which is used by a test to define its > requirements on specific kernel modules expected being loaded. The following > interface is expected: > > - Define it on ATF level, e.g. for atf-sh: > > atf_set require.klds pf pflog > > - Define it on Kyuafile level: > > test_program{name="testbinary1", required_klds="pf pflog"} > > - Get it listed per test case: > > > kyua list -v testbinary1 | grep required_klds > required_klds = pf pflog if_epair > > Having it declared we could simplify tests and avoid manual checks for > modules with the respective "skip test" lib calls. Thus, this new metadata > would work the same way as the existing required_programs and required_files > ones -- if some of the required modules are not loaded Kyua will skip such > test with a respective message including names of the missing modules. > > An extra benefit is that we do not need to iterate it like: 1) kyua test, 2) > kldload a-missing-module, 3) repeat. Such iteration is usually needed due to > our tests typically do kldstat sequentially with "skip fast" logic. Skipping > reason information still is not aggregated to make it easy to run a single > kldload for all missing modules, but that's another story which is covered > by the following Project B. > > Q1: I have doubts regarding the name of this new metadata. There are several > other options in my mind, it would be appreciated to hear opinions regarding > this: > a) required_klds -- the original idea based on the vibe that FreeBSD Kyua > fork is mainly for FreeBSD > > b) required_mods -- to keep it OS agnostic while Kyua still is treated > as an OS agnostic tool, and freebsd/kyua GitHub repo actually is the > main Kyua repo in the world today > > c) required_kmods -- a little improvement of the previous option to make > sure it talks about "kernel modules" and leave space for other meanings > in the future, just in case > > d) required_klds | required_kmods -- to have some official aliases > supported, but it's expected to bring confusion instead > > e) your variant > > > Project B: automatic kldload > ---------------------------- > > Having Project A implemented and the existing tests augmented with the > respective metadata we could ask Kyua to automatically load the required > kernel modules. > > From the implementation perspective, Kyua is expected to have a generic > concept of a requirement resolver. A resolver may read all metadata and > decide on its actions. If we decide on "klds" name in Project A then the > very first resolver implemented would look for required_klds metadata, if > it's present, then the mentioned kernel modules will be loaded. Such > resolver could be named "klds", and we can think of future additions like > "pkgs" resolver and so on. > > Currently, I see it as a new kyua command to make it separate from the > testing process itself. The interface could be as follows: > > - List all available requirement resolvers (name and short description): > > > kyua rr list > > - Run all resolvers or only specified ones. It would deal with the current > dir, and later we can think of "-k" option like "kyua test" command has. > > > kyua rr run [resolver1-name resolver2-name ...] > > - It's proposed to be a sub-set of commands in case if in the future we want > additions, e.g. dry-run or verify command without actual resolving. > > Having this interface the FreeBSD CI could have a separate step to run "kyua > rr run" before the existing step which runs "kyua test". But from > development perspective it may quickly end up with a need to have a single > command for both. The following interface addition could be there: > > - Run all or specified requirement resolvers before the actual testing: > > > kyua test --rr ["resolver1-name resolver2-name ..."] > > Another option is to use the existing Kyua mechanisms and add it as a > configuration variable, i.e. it can be configured once in the local > kyua.conf not to specify it with every test command invocation: > > # without configuration: > > kyua -v rr="klds pkgs" test > > # with configuration: > > echo 'rr = "klds pkgs"' >> /etc/kyua/kyua.conf > > kyua test # now it does resolving first > > > Q2: Project B has the same set of questions about naming and style. For > example, "rr" name for kyua.conf feels non-obvious and probably it could be > named like "requirement_resolvers" to keep it self-explainable. > > > What do you think? Any idea, comment, or suggestion are welcome. > > > Best regards, > igoro