Date: Thu, 26 Jan 2012 19:19:36 +0000 From: Matthew Seaman <m.seaman@infracaninophile.co.uk> To: freebsd-ports@freebsd.org Subject: Re: NOT_FOR_ARCHS considered harmful [was: with the cvs history? trying to help INDEX builds.] Message-ID: <4F21A748.6010307@infracaninophile.co.uk> In-Reply-To: <4F1BE7B4.6090702@infracaninophile.co.uk> References: <4F177264.3090708@freebsd.org> <4F17DB1C.6080503@infracaninophile.co.uk> <CADLo83-WtVmyGHM=O4FbTNbDy9h=A1t111bP6eYc%2BTL8-RGmuA@mail.gmail.com> <4F193FD5.8070208@infracaninophile.co.uk> <20120121204614.GH4729@lonesome.com> <4F1BE7B4.6090702@infracaninophile.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig9CDC4A00F9A907A2F8254622
Content-Type: multipart/mixed; boundary="------------030703060201030600050905"
This is a multi-part message in MIME format.
--------------030703060201030600050905
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
On 22/01/2012 10:40, Matthew Seaman wrote:
> On 21/01/2012 20:46, Mark Linimon wrote:
>> tl;dr: I want to switch the default assumption we're making.
>>
>> IMHO when new ports come into the tree, we should make our default
>> assumption that we will try to build them on amd64 and i386. For case=
s
>> that this does not hold, we consider this Bad and committer-must-fix.
>> For the tier-2s, we shift the default assumption to "only set it to
>> buildable once it has been shown to be so". So, the burden of proof
>> shifts the other way: to a user of a tier-2 to claim "I tried this and=
>> it works", rather than portmgr saying "we tried this and it doesn't wo=
rk".
>=20
> Doesn't your proposed change in semantics of the 'FOR_ARCHS' stuff mean=
> that over time, as other architectures become more popular, most ports
> will have to have an explicit 'ONLY_FOR_ARCHS' setting? If the default=
> effectively becomes 'ONLY_FOR_ARCHS=3D i386 amd64' then as ports are sh=
own
> to work on different platforms they will need an ONLY_FOR_ARCHS line in=
> their Makefiles listing where they are known to work? Or else the port=
s
> becomes effectively i386 / amd64 only?
>=20
>> (Of course, for things like p5-* it doesn't really matter; if perl
>> builds, to a first approximation they'll build as well. I'm talking
>> about the things like biology/, deskutils/, games/, math/, science,
>> x11*/, and so forth.)
>>
>> What do people think?
>=20
> There are a lot of ports where the distinction between CPU architecture=
s
> is pretty much irrelevant. I can't see portmaster(8) (for example)
> failing to work anywhere the base system works.
>=20
> I was thinking about this a while back. Test the contents of packages
> to see if they install any object code -- ports/129210 -- and mark the
> ones that don't as arch-independent in some way (CATEGORIES+=3D arch-in=
dep
> perhaps?)
So, apropos Mark's comments earlier, and after a few false starts and a
lot of head-scratching here is a patch to address many of his points.
It does basically three things:
* Removes the NOT_FOR_ARCHS variable and associate functionality.
* Creates a new BROKEN_OSARCHS variable. This is largely syntactic
sugar, meaning that it is no longer necessary to use
.include <bsd.ports.pre.mk> in order to test ${ARCH} or ${OSVERSION}
* Creates a new SUPPORTED_OSARCHS variable. This allows maintainers
to indicate which architectures they will support, and whether the
package build cluster should routinely build packages for that port.
A full patch removing NOT_FOR_ARCHS from the ports is at:
http://www.infracaninophile.co.uk/not_for_archs.diff
Comments? Critiques?
Now for the more long-winded explanation bit....
NOT_FOR_ARCHS is frequently misused to indicate temporary build or
installation failures on particular architectures. The _FOR_ARCHS stuff
should be really only be used to indicate ports designed to work on
specific CPU architectures. ONLY_FOR_ARCHS exists for this purpose.
The problem with extending it to cover temporary failures is that the
package build cluster will never retry building that port, so there's no
way to tell if changes to the toolchain or updates to the port have
subsequently fixed the problem.
This is where the BROKEN_OSARCHS variable is useful. This is a list of
achitecture-OS version pairs corresponding to the different environments
used in the package build cluster. Thus you can easily mark a port as
broken on sparc64-7, rather than having to use a construction like this:
=2Einclude <bsd.port.pre.mk>
=2Eif ${ARCH} =3D=3D "sparc64" && ${OSVERSION} < 800000
BROKEN=3D compiler dumps core
=2Eendif
=2Einclude <bsd.port.post.mk>
(Not using bsd.port.{pre,post}.mk so much should help a bit with index
building times too.)
It's also rather easier to grep for BROKEN_OSARCHS than it is for a
construct like the above.
As BROKEN_OSARCHS ultimately uses the pre-existing BROKEN variable,
setting TRY_BROKEN will allow these ports to be retested.
Finally there is SUPPORTED_OSARCHS, which can be used to indicate policy
about where a port is known to usable. For regular users it doesn't do
anything except print a warning message if an attempt is made to compile
a port on an unsupported architecture/OS combination.
For the ports build cluster, conditional on PACKAGE_BUILDING being set,
it turns off building on unsupported environments by setting IGNORE.
As the patch stands at the moment, every port will be using the default,
which is to support these architecture/OS combinations:
i386-7 i386-8 i386-9 amd64-7 amd64-8 amd64-9
but the defaults can be modified easily or port-specific settings can be
added. There is also a TRY_UNSUPPORTED knob to turn on building
packages on all architectures / OS versions.
Finally, all this testing the CPU architecture and the OS version makes
no sense at all for ports that don't install any compiled binaries, such
as shell scripts. Those ports can define a variable ARCH_INDEP to opt
out of the SUPPORTED_OSARCHS thing altogether.
Cheers,
Matthew
--=20
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matthew@infracaninophile.co.uk Kent, CT11 9PW
--------------030703060201030600050905
Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0";
name="bsd.port.mk.diff"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="bsd.port.mk.diff"
Index: bsd.port.mk
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /home/ncvs/ports/Mk/bsd.port.mk,v
retrieving revision 1.703
diff -u -u -r1.703 bsd.port.mk
--- bsd.port.mk 22 Jan 2012 15:41:33 -0000 1.703
+++ bsd.port.mk 26 Jan 2012 18:20:31 -0000
@@ -207,17 +207,69 @@
#
# Set these if your port only makes sense to certain architectures.
# They are lists containing names for them (e.g., "alpha i386").
-# (Defaults: not set.)
+# (Defaults: unset)
#
# ONLY_FOR_ARCHS
# - Only build ports if ${ARCH} matches one of these.
-# NOT_FOR_ARCHS - Only build ports if ${ARCH} doesn't match one of these=
=2E
# ONLY_FOR_ARCHS_REASON
# ONLY_FOR_ARCHS_REASON_${ARCH}
# - Reason why it's only for ${ONLY_FOR_ARCHS}s
-# NOT_FOR_ARCHS_REASON
-# NOT_FOR_ARCHS_REASON_${ARCH}
-# - Reason why it's not for ${NOT_FOR_ARCHS}s
+#
+# Show architecture and OS major version combinations where ported
+# software is expected to operate correctly and for which packages
+# should be built. Port maintainers are expected to address problems
+# with ports on supported OS versions and architectures. Where
+# physically feasible, all Tier-1 architectures and all OS versions
+# under active support by the FreeBSD project must be supported.
+# Architecture and OS combinations not listed as supported may or may
+# not work, and maintainers need not feel obliged to address reported
+# problems.
+#
+# OS version and architecture combinations are expressed as
+# ${ARCH}-${OSMAJOR} (eg. amd64-9, i386-10). The wildcard 'ALL' may
+# be used to indicate support for the default set of either
+# component. Thus i386-ALL means supported on all active supported
+# major OS versions for i386 architecture, whereas ALL-9 means
+# supported on all Tier-1 architectures but only for FreeBSD 9.x. The
+# default setting is ALL-ALL
+#
+# Ports that install no architecture dependent files (eg. shell
+# scripts) can define ARCH_INDEP to opt-out of all supported OS
+# version and architecure tests, implicitly declaring that they are
+# supported on all platforms.
+#
+# ARCH_INDEP - Define this to declare that this port installs identic=
al
+# content irrespective of OS version and system architec=
ture
+# SUPPORTED_OSARCHS
+# - List of supported architecture-OS combinations this
+# port is supported for. Only build packages if
+# ${ARCH}-${OSMAJOR} matches one of these. (Default:
+# ALL-ALL)
+# SUPPORTED_REASONS
+# - List of reasons why architecture is or is not supporte=
d
+# on various platforms. The format is <osarch>:"<reason=
>"
+# The first match is selected, so order from most to lea=
st
+# specific.
+# Example:
+# ALL-7:"supported on standard architectures for FreeB=
SD 7.x"
+# ALL-8:"supported on standard architectures for FreeB=
SD 8.x"
+# ALL-ALL:"part of base system"
+# (Default: unset)
+#
+# Set these if your port temporarily does not work on certain
+# architectures and OS combinations due to programming errors or other
+# unintentional problems. Values are lists of architecture-OS
+# combinations as above. (Defaults: unset)
+#=20
+# BROKEN_OSARCHS
+# - List of architecture-OS combinatins where port is
+# known not to work. See BROKEN above.
+# BROKEN_REASONS
+# - List of reasons why port is broken on
+# ${ARCH}-${OSMAJOR}. The format is the same as for
+# SUPPORTED_REASONS above.
+# (Default: unset)
+#
# IA32_BINARY_PORT
# - Set this instead of ONLY_FOR_ARCHS if the given port
# fetches and installs compiled i386 binaries.
@@ -1195,6 +1247,11 @@
.endif
.endif
=20
+# Get the operating system major version
+.if !defined(OSMAJOR)
+OSMAJOR=3D ${OSVERSION:C,.....$,,}
+.endif
+
MASTERDIR?=3D ${.CURDIR}
=20
.if ${MASTERDIR} !=3D ${.CURDIR}
@@ -3126,21 +3183,8 @@
__ARCH_OK?=3D 1
.endif
=20
-.if defined(NOT_FOR_ARCHS)
-.for __NARCH in ${NOT_FOR_ARCHS}
-.if ${ARCH:M${__NARCH}} !=3D ""
-.undef __ARCH_OK
-.endif
-.endfor
-.endif
-
.if !defined(__ARCH_OK)
-.if defined(ONLY_FOR_ARCHS)
-IGNORE=3D is only for ${ONLY_FOR_ARCHS},
-.else # defined(NOT_FOR_ARCHS)
-IGNORE=3D does not run on ${NOT_FOR_ARCHS},
-.endif
-IGNORE+=3D while you are running ${ARCH}
+IGNORE=3D is only for ${ONLY_FOR_ARCHS}, while you are running ${ARCH}
=20
.if defined(ONLY_FOR_ARCHS_REASON_${ARCH})
IGNORE+=3D (reason: ${ONLY_FOR_ARCHS_REASON_${ARCH}})
@@ -3148,12 +3192,143 @@
IGNORE+=3D (reason: ${ONLY_FOR_ARCHS_REASON})
.endif
=20
-.if defined(NOT_FOR_ARCHS_REASON_${ARCH})
-IGNORE+=3D (reason: ${NOT_FOR_ARCHS_REASON_${ARCH}})
-.elif defined(NOT_FOR_ARCHS_REASON)
-IGNORE+=3D (reason: ${NOT_FOR_ARCHS_REASON})
.endif
=20
+# Check the machine architectures. Assume ports are only supported on
+# Tier-1 architectures unless maintainers commit to supporting other
+# achitectures and they have been tested and shown to work there. Skip a=
ll
+# this for ports that install entirely architecture independent files.
+
+.if !defined(ARCH_INDEP)
+SUPPORTED_OSARCHS?=3D ALL-ALL
+SUPPORTED_REASONS?=3D ALL-ALL:"standard Tier-1 architecture and supporte=
d OS"
+
+SUPPORTED_OSARCHS_DEFAULT=3D amd64-7 amd64-8 amd64-9 \
+ i386-7 i386-8 i386-9
+
+_ARCH_OS=3D ${ARCH}-${OSMAJOR}
+_ARCH_WILD=3D ALL-${OSMAJOR}
+_WILD_OS=3D ${ARCH}-ALL
+_WILD_WILD=3D ALL-ALL
+
+# Id the current environment one that would be supported by default?
+. for _osarch_def in ${SUPPORTED_OSARCHS_DEFAULT}
+. if ${_ARCH_OS} =3D=3D ${_osarch_def}
+__DEFAULT_OSARCH=3D 1
+. endif
+. endfor
+
+. for _osarch in ${SUPPORTED_OSARCHS}
+. if !defined(__SUPPORTED)
+. if ${_ARCH_OS} =3D=3D ${_osarch}
+__SUPPORTED=3D ${_ARCH_OS}
+. elif defined(__DEFAULT_OSARCH)
+. if ${_WILD_WILD} =3D=3D ${_osarch}=20
+__SUPPORTED=3D ${_WILD_WILD}
+. elif ${_ARCH_WILD} =3D=3D ${_osarch}
+__SUPPORTED=3D ${_ARCH_WILD}
+. elif ${_WILD_OS} =3D=3D ${_osarch}
+__SUPPORTED=3D ${_WILD_OS}
+. endif
+. endif
+. endif
+. endfor
+
+. for _supported_reason in ${SUPPORTED_REASONS}
+. if !defined(__SUPPORTED_REASON)
+_osarch=3D ${_supported_reason:C,:.*$,,}
+_reason=3D ${_supported_reason:C,^[^:]+:",,:C,"$,,}
+
+. if ${_ARCH_OS} =3D=3D ${_osarch}
+__SUPPORTED_REASON=3D ${_reason}
+. elif defined(__DEFAULT_OSARCH)
+. if ${_WILD_WILD} =3D=3D ${_osarch}=20
+__SUPPORTED_REASON=3D ${_reason}
+. elif ${_ARCH_WILD} =3D=3D ${_osarch}
+__SUPPORTED_REASON=3D ${_reason}
+. elif ${_WILD_OS} =3D=3D ${_osarch}
+__SUPPORTED_REASON=3D ${_reason}
+. endif
+. endif
+. endif
+. endfor
+
+# Don't build packages for unsupported architectures
+. if defined(PACKAGE_BUILDING) && !defined(TRY_UNSUPPORTED)
+. if !defined(__SUPPORTED)
+IGNORE=3D unsupported: ${SUPPORTED_OSARCHS} does not match ${_ARCH_OS}
+. if defined(__SUPPORTED_REASON)
+IGNORE+=3D (reason: ${__SUPPORTED_REASON})
+. endif
+. endif
+. endif
+.endif # !defined(ARCH_INDEP)
+
+
+.if defined(BROKEN_OSARCHS)
+
+. for _osarch in ${BROKEN_OSARCHS}
+. if !defined(__BROKEN)
+. if ${_ARCH_OS} =3D=3D ${_osarch}
+__BROKEN=3D ${_ARCH_OS}
+. elif defined(__DEFAULT_OSARCH)
+. if ${_WILD_WILD} =3D=3D ${_osarch}=20
+__BROKEN=3D ${_WILD_WILD}
+. elif ${_ARCH_WILD} =3D=3D ${_osarch}
+__BROKEN=3D ${_ARCH_WILD}
+. elif ${_WILD_OS} =3D=3D ${_osarch}
+__BROKEN=3D ${_WILD_OS}
+. endif
+. endif
+. endif
+. endfor
+
+. for _broken_reason in ${BROKEN_REASONS}
+. if !defined(__BROKEN_REASON)
+_osarch=3D ${_broken_reason:C,:.*$,,}
+_reason=3D ${_broken_reason:C,^[^:]+:",,:C,"$,,}
+
+. if ${_ARCH_OS} =3D=3D ${_osarch}
+__BROKEN_REASON=3D ${_reason}
+. elif defined(__DEFAULT_OSARCH)
+. if ${_WILD_WILD} =3D=3D ${_osarch}=20
+__BROKEN_REASON=3D ${_reason}
+. elif ${_ARCH_WILD} =3D=3D ${_osarch}
+__BROKEN_REASON=3D ${_reason}
+. elif ${_WILD_OS} =3D=3D ${_osarch}
+__BROKEN_REASON=3D ${_reason}
+. endif
+. endif
+. endif
+. endfor
+
+. if defined(__BROKEN)
+BROKEN=3D on ${BROKEN_OSARCHS}; ${__BROKEN} matches ${_ARCH_OS}
+. if defined(__BROKEN_REASON)
+BROKEN+=3D (reason: ${__BROKEN_REASON})
+. endif
+. endif
+
+.endif # defined(BROKEN_OSARCHS)
+
+.if !target(check-support)
+check-support:
+. if defined(ARCH_INDEP)
+ @${ECHO_MSG} "=3D=3D=3D> Supported anywhere: architecture-independent =
content"
+. else
+. if defined(__SUPPORTED)
+. if defined(__SUPPORTED_REASON)
+ @${ECHO_MSG} "=3D=3D=3D> Supported on ${_ARCH_OS}: ${__SUPPORTED_REASO=
N}"
+. endif
+. else
+ @${ECHO_MSG} "=3D=3D=3D> WARNING: this port is not supported on ${_ARC=
H_OS}"
+. if defined(__SUPPORTED_REASON)
+ @${ECHO_MSG} "=3D=3D=3D> (reason: ${__SUPPORTED_REASON})"
+. else
+ @${ECHO_MSG} "=3D=3D=3D> (reason: not properly tested and may or may n=
ot work)"
+. endif
+. endif
+. endif
.endif
=20
# Check the user interaction and legal issues
@@ -4266,8 +4441,8 @@
_SANITY_SEQ=3D ${_CHROOT_SEQ} pre-everything check-makefile \
check-categories check-makevars check-desktop-entries \
check-depends identify-install-conflicts check-deprecated \
- check-vulnerable check-license buildanyway-message \
- options-message
+ check-vulnerable check-license check-support \
+ buildanyway-message options-message
_FETCH_DEP=3D check-sanity
_FETCH_SEQ=3D fetch-depends pre-fetch pre-fetch-script \
do-fetch post-fetch post-fetch-script
--------------030703060201030600050905--
--------------enig9CDC4A00F9A907A2F8254622
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8hp08ACgkQ8Mjk52CukIwvPQCdFjx6F6pbSzm7SJogw3h3Ra1L
ZxkAn2PspxQh0YPoKRNrmPUgFW3Ds4CH
=ThUJ
-----END PGP SIGNATURE-----
--------------enig9CDC4A00F9A907A2F8254622--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F21A748.6010307>