Date: Wed, 8 Sep 2021 06:47:11 -0600 From: Warner Losh <imp@bsdimp.com> To: David Chisnall <theraven@freebsd.org> Cc: Stefan Esser <se@freebsd.org>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: -CURRENT compilation time Message-ID: <CANCZdfrChE3x1qiY_940hD8ZDUjckjHePbynTOvuudDQEAYYLQ@mail.gmail.com> In-Reply-To: <2cfb912a-618b-9f06-9cef-d2fe1d78fe97@FreeBSD.org> References: <CAGSa5y0FMwt8g4%2BmqFegvrgBzxJCwupB_RJqb52ZUQy9EFB__Q@mail.gmail.com> <b0dadbd1-3f99-ef8e-0a35-f35d3dab828c@FreeBSD.org> <CAMWY7CCUXJmUT8g1irOUNAxgihhoqpcTcV1nF7MR1NHCVY-%2BEQ@mail.gmail.com> <a16ae438-cef8-d21f-cf5c-891a2a328a26@FreeBSD.org> <e9476190-e098-ae0c-8c40-5445516fa06d@freebsd.org> <2cfb912a-618b-9f06-9cef-d2fe1d78fe97@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000098786c05cb7b4836 Content-Type: text/plain; charset="UTF-8" On Wed, Sep 8, 2021 at 2:59 AM David Chisnall <theraven@freebsd.org> wrote: > On 07/09/2021 18:02, Stefan Esser wrote: > > Wouldn't this break META_MODE? > > I have never managed to get META_MODE to work but my understanding is > that META_MODE is addressing a problem that doesn't really exist in any > other build system that I've used: that dependencies are not properly > tracked. > It does track the dependencies. It uses filemon(4) to do this so that every dependency is tracked. This is better than what other build systems do because no dependencies are missed. > When I do a build of LLVM with the upstream build system with no > changes, it takes Ninja approximately a tenth of a second to stat all of > the relevant files and tell me that I have no work to do. META_MODE > apparently lets the FreeBSD build system extract these dependencies and > do something similar, but it's not enabled by default and it's difficult > to make work. > META_MODE does the same thing, and it's just as fast as ninja make. And it has the advantage that, unlike meson, it isn't rebuilding the makefiles all the time. And apart from loading filmon, it was quite easy to enable last time I was using it (though I did have trouble finding the right docs). > > I'd rather be able to continue building the world within a few minutes > > (generally much less than 10 minutes, as long as there is no major LLVM > > upgrade) than have a faster LLVM build and then a slower build of the > world ... > > The rest of this thread has determined that building LLVM accounts for > half of the build time in a clean FreeBSD build. LLVM's CMake is not a > great example: it has been incrementally improved since CMake 2.8 and > doesn't yet use any of the modern CMake features that allow > encapsulating targets and providing import / export configurations. > > In spite of that, it generates a ninja file that compiles > *significantly* faster than the bmake-based system in FreeBSD. In other > projects that I've worked on with a similar-sized codebase to FreeBSD > that use CMake + Ninja, I've never had the same problems with build > speed that I have with FreeBSD. > The speed is comparable to META_MODE, but much faster than the default. It suffers a bit because our buildworld stuff is a kinda sorts not quite good enough dependency tracker and it's layered on top of that. > Working on LLVM, I generally spend well under 10% of my time either > waiting for builds or fighting the build system. Working on FreeBSD, I > generally spend over 90% of my time waiting for builds or fighting the > build system. This means that my productivity contributing to FreeBSD > is almost zero. > > For reference, changes to LLVM typically build for me in under 30 > seconds with Ninja, unless I've changed a header that everything > > In particular, building FreeBSD on a 10-24 core machine has very long > periods where a number of the cores are completely idle. > > Ninja also has a few other nice features that improve performance > relative to bmake: > > - It lets you put jobs in different pools. In LLVM this is used to > put link and compile jobs in different pools because linking with LLD > uses multiple threads and a lot more memory than compilation, so a > 10-core machine may want to do 12 compile jobs in parallel but only 2 > link jobs. This makes it much easier to completely saturate the machine. > This is nice. > - Ninja provides each parallel build task with a separate pipe for > stdout and stderr, and does not print their output unless a build step > fails (or unless you build with -v). With bmake, if a parallel build > fails I have to rerun the build without -j, because the output is > interleaved with succeeding jobs and it's difficult to see what actually > failed. With ninja, the output is from each failed job, with no > interleaving. > META_MODE does this too. It's the primary reason I use it, along with the speed. The big downside of ninja make is that it requires you to have some other, higher-level build system to generate the Makefiles. And my experience to date with meson is a mixed bag: it's quite a bit better than normal bmake for some things, and quite a bit worse for others in terms of the code you need to write to get things done, or the efforts required to debug mistakes. I've not used cmake as the ninja makefile generator, so I have no comments on it. Doing the conversion to at least meson would be a quite large job, at least by my initial estimates. Hacking bmake to generate ninja makefiles looks to be quite a bit simpler... But looks can be deceiving. Warner --00000000000098786c05cb7b4836--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrChE3x1qiY_940hD8ZDUjckjHePbynTOvuudDQEAYYLQ>