Date: Sat, 21 Jan 2012 14:46:14 -0600 From: Mark Linimon <linimon@lonesome.com> To: Matthew Seaman <m.seaman@infracaninophile.co.uk> Cc: freebsd-ports@freebsd.org, Chris Rees <utisoft@gmail.com> Subject: NOT_FOR_ARCHS considered harmful [was: with the cvs history? trying to help INDEX builds.] Message-ID: <20120121204614.GH4729@lonesome.com> In-Reply-To: <4F193FD5.8070208@infracaninophile.co.uk> References: <4F177264.3090708@freebsd.org> <4F17DB1C.6080503@infracaninophile.co.uk> <CADLo83-WtVmyGHM=O4FbTNbDy9h=A1t111bP6eYc%2BTL8-RGmuA@mail.gmail.com> <4F193FD5.8070208@infracaninophile.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Jan 20, 2012 at 10:20:05AM +0000, Matthew Seaman wrote: > Actually I take your point, that it should be possible to distinguish > between ports that permanently won't work on some architectures by > design, and ports that temporarily don't work because of mistakes or > broken dependencies or so forth, and that are expected to be fixed > sooner rather than later. There's a secondary problem which I keep meaning to write up a rant on. This thread seems as good a place as any. Warning: the following is only my own opinion, not portmgr's. We made a design mistake by allowing in NOT_FOR_ARCHS as an alternative to ONLY_FOR_ARCHS. This has always been a shortcut to say "doesn't build on !(amd64!i386)". As long as the archs were (alpha|amd64|i386| sparc64) this was merely annoying. The problem comes when we start up package builds on new archs. The primary utility of package builds for tier-2s has arguably been to QA whether the ports build at all. The secondary utility, in order: - test that the arch's srcbase has't regressed to the point where it's too unstable to build packages - flag ports (and infrastructure) with bad assumptions about wordlength and endianness - create usable packages (really, only the most fundamental ports will have packages that are timely, due to the > 1 month cycle times) In general, I claim if a port already has some kind of NOT_FOR_ARCH entry, it's unlikely to build on a new arch. This was clearly demonstrated a couple of years ago when I first started powerpc builds once we had a machine donated (note: powerpc = 32-bit). The errors were correlated to sparc64, but only roughly. sparc64 builds tend to trip up on the following, in order of which I think we should care: - 64-bit issues (which have a high correlation to amd64 build failures), - lack of arch-specific build stanzas (which have a high correlation to powerpc), - endianness issues. The powerpc build also pointed out that many ports assumed that 32bit <> i386. On occasion, I fix notable occurences of this (e.g. python). So, why does this matter? Surely our sparc64 and powerpc/Mac userbases are tiny. The reason to do this work is that there is demand for arm and mips builds for embedded systems[1], and once these are turned on, we're going to have a bazillion build errors to sort through. To the extent that our first attempts include only ports that don't have the NOT_FOR stanzas, IMHO we're going to make more progress more quickly. Now, for mips, only the "fundamental" ports are ever going to matter, since there are no viable mips desktops to worry about. But, for embedded, getting a subset of things in net/ and sysutils/ (and to some extent things like lang/perl) is going to be useful -- again, not so much for uploadable packages, but to ensure buildability when vendors try to use these in their products. (I'm told that people are speculating about running desktop stuff on native arm, so that's why I picked mips for the use-case. I have no idea if that's anything but blue-sky.) So IMHO we should do a sweep: - move all the true cases of NOT_FOR_ARCHS to ONLY_FOR_ARCHS - move all the false cases of NOT_FOR_ARCHS to BROKEN and then drop support for NOT_FOR_ARCHS. If anyone is interested in coming up with patches, I'll do the -exp run (on amd64 :-) ) to prove that there are no regressions on that arch at least. Finally, for those willing to investigate how messed-up the metadata currently are, pull up the following page: http://pointyhat.freebsd.org/errorlogs/packagestats.html You'll find a column marked "skipped". That contains URLs to files called "duds.verbose" for each buildenv. This file is the output of 'make ignorelist-verbose' called from the top of the tree. The reason that we decided to archive these per-pointyhat-run is so that you can tell _exactly_ why pointyhat believed that it did not have to try to build that package at that exact point in time. In the past, you had to impute it from whatever the state of the tree was. Since the tree evolves so quickly, it was impossible to tell. (There are also a very small handful of degenerate cases where things don't build on pointyhat due to its build environment. I've worked on getting these down to < 10 over the past few years.) tl;dr: I want to switch the default assumption we're making. IMHO when new ports come into the tree, we should make our default assumption that we will try to build them on amd64 and i386. For cases that this does not hold, we consider this Bad and committer-must-fix. For the tier-2s, we shift the default assumption to "only set it to buildable once it has been shown to be so". So, the burden of proof shifts the other way: to a user of a tier-2 to claim "I tried this and it works", rather than portmgr saying "we tried this and it doesn't work". (Of course, for things like p5-* it doesn't really matter; if perl builds, to a first approximation they'll build as well. I'm talking about the things like biology/, deskutils/, games/, math/, science, x11*/, and so forth.) What do people think? mcl [1] most likely in some kind of emulated instances. Cross-builds are left as an exercise to the reader.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120121204614.GH4729>