From nobody Tue May 24 17:35:51 2022 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6D9AD1B499F0 for ; Tue, 24 May 2022 17:35:56 +0000 (UTC) (envelope-from se@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4L71ZN2JSSz3hcy; Tue, 24 May 2022 17:35:56 +0000 (UTC) (envelope-from se@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653413756; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yNHD5sw7khgzzX/A49KIZ31HM5g4mVNeHXVzbtS0cBY=; b=Jl1gQBKu4mzt5MfBWxXNQuXhP5ShiZX04FVNtCqtkyeytHf1cIxrC+g77aRQNhxXgSsAdb 9rnDJkWISPlgk3cnJbJdll2wD6HFU4iZBE2sWva6NsEMKydbn+XdJNMdegjyh1dLdwV5jX T35xCdu0mDpTjFAa0cTyikqAVmUbmOr/kXJA3pF13GLAZ1ltelsXEhFtYNssyA2xtnOfTC 3Iaejdl7i6BtKTXC8V/JMqVy6WuSiybGIZtbOI8VkDz7wEaT/phI1YBPBEEYtCxRbMWpwz 9/GTHRPEGAFBUxSz1T28FiP/2JIIb1Rp/hFo5EOtLwWfTnxg0AnC7+Fy97oFCw== Received: from [IPV6:2003:cd:5f12:400:688d:b306:d02:dad9] (p200300cd5f120400688db3060d02dad9.dip0.t-ipconnect.de [IPv6:2003:cd:5f12:400:688d:b306:d02:dad9]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id BF9C01AC4; Tue, 24 May 2022 17:35:55 +0000 (UTC) (envelope-from se@FreeBSD.org) Message-ID: <06672670-a08c-7e83-d484-dd52f07443bb@FreeBSD.org> Date: Tue, 24 May 2022 19:35:51 +0200 List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: NO_TEST, do-test: and TEST_DEPENDS? Content-Language: en-US To: "Bjoern A. Zeeb" Cc: freebsd-ports@freebsd.org References: <848ba292-d306-dcd7-2456-383f1a3652f6@FreeBSD.org> From: Stefan Esser In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------G0M1EAFcqTydvDmeQycdsGAC" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653413756; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yNHD5sw7khgzzX/A49KIZ31HM5g4mVNeHXVzbtS0cBY=; b=unHX87sUlDj/YuJj1I8uNT7w5ier0BnIydOQSVgp0LEeG0DDpNBY/fqgyCybDkhNsMdzCk Un3Ra5FzX7cae8oENBtlVkd3zAnJXBN1myiau/TWZTL5gstHFChqLztJQD1M+h/R8BoG4D OTUGH8enrFNQumq5brA7zELXu5GijB841lyXrihb5R/qVcN5kUAz3J4E9my45SH92yvViP xWJeDr4w6OeD5QNtMBF46hM1ZfzSkxRZFjdkV2GV1zTCD5yhOOOUY23ZasnUD7CJRad2YE fo0heG9w1pTYBB4zqzTWyLP/Xi0KZJrqgld5JP/wF/OKEU5E1Aj2hnC+bApqug== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1653413756; a=rsa-sha256; cv=none; b=DpPDVMQkusuDHMf6qnjRguIc6ZApvLtkF3iJdC4DfQDEKqjARyoYwgdp5OnJOlFwtQeydE PSvViezF0tzvZ29z+F2XBt74vJL7aB1ljyWqxH7icy0b1RMemP5cZDHvlgb0fdTovMKnGB fnLajvlRoSqeC2DYft6a5ZcIhGMs7FMvVzYsPxfL5a5lg3RBeJHwLvlKyO37iCF2Kzzd8J EqmPz97rZy5+orM0Bf44SGGb3HEAJRrZpZ1v5NePNfaIjkzSWAUTj6bXVPMb/hmkH57I8G hkjAaAt2tWIbC96owMLnBpANJQj0vCJJfRyMFpuIO6KZimXXlkAs+pWqYGE+aQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------G0M1EAFcqTydvDmeQycdsGAC Content-Type: multipart/mixed; boundary="------------zbixgBzyUFBqxw5FCIoracOg"; protected-headers="v1" From: Stefan Esser To: "Bjoern A. Zeeb" Cc: freebsd-ports@freebsd.org Message-ID: <06672670-a08c-7e83-d484-dd52f07443bb@FreeBSD.org> Subject: Re: NO_TEST, do-test: and TEST_DEPENDS? References: <848ba292-d306-dcd7-2456-383f1a3652f6@FreeBSD.org> In-Reply-To: --------------zbixgBzyUFBqxw5FCIoracOg Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Am 24.05.22 um 17:27 schrieb Bjoern A. Zeeb:> On Sun, 22 May 2022, Stefan= Esser wrote: >=20 >> I'm not sure I understand the goal of your request. >=20 > the goal is to cut down dependencies and build time. As long as you do not invoke the tests (with "make test"), no dependencies will be built that are only required to run tests. >> Do you want to have a switch that prevents the execution of tests when= >> "make test" explicitly asks for the tests to be run??? >> >> This could easily be added to bsd.port.mk, but what for??? >> >>> What is the proper way to >>> =C2=A0=C2=A0=C2=A0=C2=A0(1) disable tests >> >> That's what I do not understand ... Tests are not run by default, and >> thus I never saw the need to disable them in any of my ports. >> >> Quite the opposite: If "make test" is invoked for a port that supports= >> tests, I definitely want them to be executed. >=20 > Right, so maybe solving (2) will solve (1) implicitly as I probably > don't have to disable them then using .ifdef bits. I still do not see the issue. TEST_DEPENDS are only relevant if you specifically require them with "make test". And no, not building the TEST_DEPENDS will only cause "make test" to fail early due to unsatisfied build and/or run requirements, if "make test" is invoked. >>> and more important >>> =C2=A0=C2=A0=C2=A0=C2=A0(2) to amke sure the TEST_DEPENDS and not bui= ld in these cases? >> >> You can set "DO_MAKE_TEST=3Dtrue" to execute the command "true" instea= d >> of "make" when bsd.port.mk wants to execute the tests. >> >> But this is an undocumented internal variable of the ports system, and= >> it could be removed or renamed at any time, when changes are applied >> to bsd.port.mk. >> >> And you could override the TEST_DEPENDS macro instead of wrapping it >> into an .if block. >=20 > how?=C2=A0 putting > .if defined(TEST_DEPENDS) > .undef TEST_DEPENDS > .endif > or something like that in make.conf? No, this has to be done in such a way that it can not later be overridden by a definition in the port's Makefile. >>> While I can iterate on port after port and apply a change like the be= low it >>> feels utterly wrong and it seems I am doing something silly here whic= h has a >>> proper knob I cannot find? >> >> Please explain what you want to achieve. There may be other ways to >> achieve your goal, but I think I do not understand what your goal is .= =2E. >=20 > Okay, for I have no idea anymore how long I've built the ports I need > in the classical way; I have a "meta port" depending on the 25 > packages I need to build. How is this meta-port implemented? Do you just invoke "make all deinstall install" for each origin from which a port has been installed? Or do you collect all dependencies of each of the 25 ports in the list and re-build and re-install all of them?? > The problem has bcome that rather than simple builds the amounts of > ports build exploded.=C2=A0 With that the addition of conflicts has als= o > exploded.=C2=A0 something people do probably not see (anymore these day= s) > if their build depends conflict. I see it all the time. The "poudriere" builds tend to not see nested build conflicts, since build dependencies are installed from packages. Therefore, a lot of them are not detected by maintainers. And there are quite a number of ports that have "CONFLICTS" where "CONFLICTS_INSTALL" would do. This is a non-issue when using poudriere, since it does only care about direct conflicts (i.e. install conflicts of build tools with each other and with the port being built). Building from a port may cause indirect build dependencies to be in conflict with each other and since they are not de-installed after use (which would be possible for pure build dependencies), the conflicts cause build failures. > One larger contributor to the explosion is TEST_DEPENDS.=C2=A0 I'll giv= e > you an example. >=20 > cd dns/bind9 && env BATCH=3Dyes NO_TEST=3D make all-depends-list This origin does not exist, only bind9-devel, bind916, and bind918. But I get your point, the generated list for bind9-devel has 472 entries.= But what I do not understand: why do you think that this list is of relevance, if you do not run "make test"? If you just count the dependencies that are actually built if you do not run any tests, the list has 311 entries. Still quite a large number, about 2/3 of the list including test dependencies. > Feel free to disable DNSTAP DOCS and MANPAGES (manually) to avoid > python for any of that (but there are different stories for another > time). >=20 > Crazy, right?=C2=A0 It comes from TEST_DEPENDS mostly.=C2=A0 I still ha= ven't all > tracked down which include stuff which is never even used. No, while TEST_DEPENDS accounts for 161 additional ports, there are 311 other dependencies. You can remove TEST_DEPENDS from the definition of _UNIFIED_DEPENDS on line 4024 of bsd.port.mk to exclude them from the output of "make all-depends-list", if you never want to have them in this list. > Sorry, the world is warm enough.. did we stop thinking again? I really do not understand what you are doing ... Are you forcefully rebuilding all ports in the all-depends-list each time you decide to upgrade bind? As I said: none of the TEST_DEPENDS will be built and installed, unless you cause them to be built for "make test". This is the purpose of TEST_DEPENDS: to only build those ports if they will be required to execute the tests. But if you actually forcefully rebuild all ports in all-depends-list, then you are really wasting lots of cycles and energy. Just use "portmaster -a" or "portupgrade" (with whatever option it takes to update all outdated ports), they exist to streamline updating ports on the base system. These tools can install build dependencies from packages and remove them again, when they are no longer required. They can build packages with your set of options for deployment in jails or on other systems. And they do of course only build or re-build ports that are actually required, not tests unless you request tests to be run (-T option of portmaster). They are much easier to use than poudriere, but even poudriere can be set up in a few minutes, and it builds ports in a clean environment (and thus with less risk of side-effects of already installed ports). It is of course possible to add a NO_TEST condition to bsd.port.mk to make anything related to "make test" a NOP. But not invoking "make test" if it is not desired has always been the simpler solution for me. Regards, STefan --------------zbixgBzyUFBqxw5FCIoracOg-- --------------G0M1EAFcqTydvDmeQycdsGAC Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAmKNF3gFAwAAAAAACgkQR+u171r99USs BAgAqX95iHDiNwnhORbWN/+pi6KRnng0r8plSQV+PWSCiKdwRy47+gvFLh5ee6lcy+GhWIPYq3Yl 7znpAF7SNpLbbruFfvMze7vCOfketd4gJYObY05goWtq4PF41f2kJEGXJeiFw2WLyoey0wFHJqs4 5ENKp4aeMYZDJVOawkAizcROTwL0o7JdfftGfmyLCokXb4Q46ne9pOus3q2TTeDjPoR0URADsz6b s0nFX5KMqR4PSapMjUgIvrN6uoQ19lutFVV8fzrjTWCuEU9eayNDiVcIYkVjo6mCXMWcULSbVwoC hACpUnitnbMB9rYae8nKNkl0aeYMWJQf6s5KmTXGLw== =vOqu -----END PGP SIGNATURE----- --------------G0M1EAFcqTydvDmeQycdsGAC--