Date: Tue, 7 Sep 2021 15:28:56 -0700 From: Mark Millard via freebsd-current <freebsd-current@freebsd.org> To: David Chisnall <theraven@FreeBSD.org>, freebsd-current <freebsd-current@freebsd.org> Subject: Re: -CURRENT compilation time Message-ID: <C968681B-F5FB-4266-A19A-A20B813A6731@yahoo.com> References: <C968681B-F5FB-4266-A19A-A20B813A6731.ref@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> From: David Chisnall <theraven_at_FreeBSD.org>=20
> Date: Tue, 7 Sep 2021 14:51:21 +0100
> On 06/09/2021 20:34, Wolfram Schneider wrote:
> > With the option WITHOUT_TOOLCHAIN=3Dyes the world build time is 2.5
> > times faster (real or user+sys), down from 48 min to 19.5 min real
> > time.
>=20
> Note that building LLVM with the upstream CMake + Ninja build system =
is=20
> *significantly* faster on a decent multicore machine than the FreeBSD=20=
> bmake-based in-tree version.
Just examples of llvm12 builds from my checking this . . .
poudriere-devel llvm12 buil times, nothing else being built, all
prerequisites already built (not that such are trivial):
amd64 ThreadRipper 1950X with 128 GiByte of RAM, optane 1.4T media, 32 =
FreeBSD cpus:
[00:00:07] [01] [00:00:00] Building devel/llvm12 | llvm12-12.0.1_4
[00:22:54] [01] [00:22:47] Finished devel/llvm12 | llvm12-12.0.1_4: =
Success
for:
---Begin OPTIONS List---
=3D=3D=3D> The following configuration options are available for =
llvm12-12.0.1_4:
BE_AMDGPU=3Don: AMD GPU backend (required by mesa)
CLANG=3Don: Build clang
COMPILER_RT=3Don: Sanitizer libraries
DOCS=3Don: Build and/or install documentation
EXTRAS=3Don: Extra clang tools
GOLD=3Don: Build the LLVM Gold plugin for LTO
LIT=3Don: Install lit and FileCheck test tools
LLD=3Don: Install lld, the LLVM linker
LLDB=3Don: Install lldb, the LLVM debugger
OPENMP=3Don: Install libomp, the LLVM OpenMP runtime library
PYCLANG=3Doff: Install python bindings to libclang
=3D=3D=3D=3D> Options available for the single BACKENDS: you have to =
select exactly one of them
BE_FREEBSD=3Doff: Backends for FreeBSD architectures
BE_NATIVE=3Don: Backend(s) for this architecture (X86)
BE_STANDARD=3Doff: All non-experimental backends
=3D=3D=3D> Use 'make config' to modify these settings
---End OPTIONS List---
Note the BE_NATIVE use. Also poudriere.conf has:
ZPOOL=3Dzoptb
USE_TMPFS=3Dyes
ALLOW_MAKE_JOBS=3Dyes
For reference:
# uname -apKU
FreeBSD amd64_ZFS 13.0-RELEASE-p4 FreeBSD 13.0-RELEASE-p4 #1 =
releng/13.0-n244760-940681634ee1-dirty: Mon Aug 30 11:28:31 PDT 2021 =
root@amd64_ZFS:/usr/obj/BUILDs/13_0R-amd64-nodbg-clang/usr/13_0R-src/amd64=
.amd64/sys/GENERIC-NODBG amd64 amd64 1300139 1300139
The swap/paging space stayed unused.
aarch64 Honycomb with 64 GiByte of RAM, optane 480 GiByte media, 16 =
FreeBSD cpus:
[00:00:15] [01] [00:00:00] Building devel/llvm12 | llvm12-12.0.1_4
[00:58:07] [01] [00:57:52] Finished devel/llvm12 | llvm12-12.0.1_4: =
Success
for:
---Begin OPTIONS List---
=3D=3D=3D> The following configuration options are available for =
llvm12-12.0.1_4:
BE_AMDGPU=3Don: AMD GPU backend (required by mesa)
CLANG=3Don: Build clang
DOCS=3Don: Build and/or install documentation
EXTRAS=3Don: Extra clang tools
LIT=3Don: Install lit and FileCheck test tools
LLD=3Don: Install lld, the LLVM linker
OPENMP=3Don: Install libomp, the LLVM OpenMP runtime library
PYCLANG=3Doff: Install python bindings to libclang
=3D=3D=3D=3D> Options available for the single BACKENDS: you have to =
select exactly one of them
BE_FREEBSD=3Doff: Backends for FreeBSD architectures
BE_NATIVE=3Don: Backend(s) for this architecture (AArch64)
BE_STANDARD=3Doff: All non-experimental backends
=3D=3D=3D> Use 'make config' to modify these settings
---End OPTIONS List---
Note the BE_NATIVE use. Also poudriere.conf has:
ZPOOL=3Dzopt0
USE_TMPFS=3D"data"
ALLOW_MAKE_JOBS=3Dyes
For reference:
# uname -apKU
FreeBSD CA72_16Gp_ZFS 13.0-RELEASE-p4 FreeBSD 13.0-RELEASE-p4 #4 =
releng/13.0-n244760-940681634ee1-dirty: Mon Aug 30 11:35:45 PDT 2021 =
root@CA72_16Gp_ZFS:/usr/obj/BUILDs/13_0R-CA72-nodbg-clang/usr/13_0R-src/ar=
m64.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300139 1300139
The swap/paging space stayed unused.
I've not yet figured out a good way to time just an analogous
subset of buildworld . It would need to be stable/13 or main
[so: 14] targetted to involve building an llvm12 based set of
material. releng/13.0 is llvm11 based.
One point is that likely the system builds libc++ and such
but devel/llvm12 does not: devel/llvm12 likely is bound to the
system libc++ and such. This may make "analogous subset of
buildworld" harder to isolate.
So I've not made it to well-measured differences in time-used
yet.
> One of the things I'd love to prototype if I had time is a CMake-based=20=
> build system for FreeBSD so that we could get all of the tooling=20
> integration from the compile_commands.json, reuse LLVM's (and any =
other=20
> contrib things that use CMake) build system without having to recreate=20=
> it, and be able to use ninja, to build.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C968681B-F5FB-4266-A19A-A20B813A6731>