Date: Thu, 12 Nov 2015 15:42:10 -0800 From: "Simon J. Gerraty" <sjg@juniper.net> To: Bryan Drewery <bdrewery@FreeBSD.org> Cc: <brooks@FreeBSD.org>, <imp@FreeBSD.org>, <emaste@FreeBSD.org>, <toolchain@FreeBSD.org>, <sjg@juniper.net> Subject: Re: Meta mode toolchain bootstrapping [was Re: FreeBSD targets/ out-of-date] Message-ID: <13427.1447371730@chaos> In-Reply-To: <56450AB8.90402@FreeBSD.org> References: <55E769EF.7090908@FreeBSD.org> <4924.1441306006@chaos> <56450AB8.90402@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Bryan Drewery <bdrewery@FreeBSD.org> wrote: > On 9/3/2015 11:46 AM, Simon J. Gerraty wrote: > > Anyway, bootstrap-toolchain leverages src/Makefile.inc1 to build an > > initial set of tools. It then attempts to use that to build toolchain > > for MACHINE=host which is currently failing because in-tree libelf and > > libdwarf are needed and libelf needs sys/sys/elf_common.h but but that > > doesn't currently get staged for host (we try to minimize what we put in > > host stage tree). > > Using the special hack you came up with for lib/libelf, and a lot of > other fixes for replacing binutils with elftoolchain, I've managed to > get the targets/pseduo/bootstrap-tools build working again. I will > commit this likely today or tomorrow. Cool - thanks > However... > > It may be better to simply skip targets/pseudo/toolchain.host > > that will work so long as we set TOOLSDIR to point to the stuff built by > > Makefile.inc1 > > > > Depending on where we are with external toolchian support that would > > make sense - might be able to skip bootstrap-toolchain completely > > which would be nice. > > I think this is more right because there's many more people thinking > about, testing daily, and maintaining the bootstrap logic in > Makefile.inc1. The way targets/pseudo/bootstrap-tools works currently > lead to bitrot and breakage with the elftoolchain replacement. It will > very easily happen again. I'd be favor of removing the need completely. All that is required is *some* means of producing a viable toolchain in such a way that you can point a variable at it. > What I plan to do is make pseudo/targets/bootstrap-tools always build > all of cross-tools,etc, from Makefile.inc1 respecting its own internal > logic for when to bootstrap the compiler (without us telling it to skip > the bootstrap compiler) and then add pseudo/targets/bootstrap-tools as a > global DIRDEP. Given the use of cookies, this will only be done once per > meta build, but it at least won't be a manual step anymore. For the That can still add a lot of time to build, and IIRC building clang alone poses problems for smaller machines (though using a mutex for linking clang bits could at least serialize the huge linker steps so as to avoid exhausting memory). I quite like the way NetBSD allow separating the tools from the rest of the build. I do the equivalent of 'make tools' very rarely - and so do not care how [in]efficient it is. IIRC it will throw an error if your tools are incompatible - prompting an update. I would suggest if you want to be able to hook bootstrap-tools in always, that you do it via a knob so it can be disabled. > record, I do plan to extend .MAKE.MODE=meta to buildworld and its > bootstrapping as well, which will give a lot of incremental build > improvements to it. WITH_META_FILES should give you improvements already in that regard. > This means that the meta build will then default to bootstrapping the > compiler, which we really must do to build the source tree reliably. > There's really no reason the meta build should default to not > bootstrapping the compiler. Setting WITHOUT_CROSS_COMPILER or > WITHOUT_CLANG_BOOTSTRAP will avoid this and use just the host compiler > as the meta build does now. So there's no real problem for those people > who want to skip the clang bootstrap. Ok, so long as there is a way to do so. Otherwise we'd need to add it locally for our own builds - where we need to use the toolchains provided by our compiler team. > Then, I will work on the project of "building clang less" which will > avoid building the bootstrap compiler for both the meta mode build and > the buildworld build (and universe eventually) if /usr/bin/cc satisfies > the needs (can cross compile the TARGET and is "new enough" compared to > the version we want). This is not trivial but I think it is possible and > it will make everyone happy! Indeed. As I say, NetBSD have this reasonably sorted. But of course they have 2k line shell script driving a lot of it ;-) > Related tidbit, using WITHOUT_CROSS_COMPILER (or WITHOUT_*_BOOTSTRAP) > with buildworld leads to a broken build currently since it is the user > basically asking to use their default external toolchain of /usr/bin/*, > but the logic does not kick in to use --sysroot against WORLDTMP. I plan > to fix this soon as well. Assuming that you have previously built the correct toolchain it should be valid to have something like -DWITH_TOOLSDIR -DWITHOUT_BOOTSTRAP_TOOLSDIR (or -DWITHOUT_TOOLSDIR_BOOTSTRAP) Such that the build logic is identical - all that is being skipped is the expense of re-generating the toolchain
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?13427.1447371730>