Date: Sun, 27 Oct 2024 20:47:55 -0500 From: "Maku Bex" <zagazaw2004@gmail.com> To: "'David Wolfskill'" <david@catwhisker.org> Cc: <freebsd-current@freebsd.org> Subject: RE: speedup build time Message-ID: <015001db28db$690b03c0$3b210b40$@gmail.com> In-Reply-To: <Zx7fqFzbjuCvZgaw@albert.catwhisker.org> References: <012d01db28cc$5fc66ac0$1f534040$@gmail.com> <Zx7fqFzbjuCvZgaw@albert.catwhisker.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multipart message in MIME format.
--=-=cRby8em2/lhT9t=-=
Content-Type: text/plain;
charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Hi David,
* What (if any) "max_jobs" (make's "-j" parameter) are you using? 16
* For a 16T system, I expect I would start with a value of around 28. Shoul=
dn't this number be equal to $(nproc --all)?
* Are you using make's "meta" mode? I do. Yes.
* Can you get more memory for that system? Yes, I can bump it to 32GB. This=
is a Dell OptiPlex 7080 Micro, and not sure if it is worth upgrading the o=
ther components.
If your laptop can complete the builds in such a small amount of time, then=
there's either something wrong with my machine or the config files. Oh! Th=
e build just finished as I was typing this reply and...(not for the weak he=
arts):
--------------------------------------------------------------
>>> World built in 32437 seconds, ncpu: 16
-------------------------------------------------------------- =20
Here are the contents of:
make.conf:
CPUTYPE?=3Dicelake-client
CFLAGS=3D -O2 -pipe -march=3D${CPUTYPE}
#CXXFLAGS=3D${CFLAGS}
MACHINE_CPU+=3D avx512 avx2 avx sse42 sse41 ssse3 sse3 f16c
MAKE_JOBS_NUMBER=3D 16
BATCH=3Dyes
WRKDIRPREFIX=3D/var/ports/tmp
CCACHE_DIR=3D/var/ccache/tmp
OPTIONS_UNSET+=3D QT5 QT6 CUPS VESA SCFB TWM XCONSOLE XCLOCK GNOME KDE EXAM=
PLES \
DEBUG ASPELL NLS DOCS
---------------------------------------------------------------------------=
----
src-env.conf:
WITH_META_MODE=3Dyes
WITH_CCACHE_BUILD=3Dyes
#WITH_DIRDEPS_BUILD=3Dyes <<fails
#WITH_DIRDEPS_CACHE=3Dyes <<fails=20
WITHOUT_TALK=3Dyes
WITHOUT_TELNET=3Dyes
WITHOUT_TFTP=3Dyes
WITHOUT_IPFILTER=3Dyes
WITHOUT_IPFW=3Dyes
WITHOUT_FLOPPY=3Dyes
WITHOUT_GAMES=3Dyes
WITHOUT_BLUETOOTH=3Dyes
WITHOUT_EE=3Dyes
WITHOUT_FREEBSD_UPDATE=3Dyes
WITHOUT_GPIO=3Dyes
WITHOUT_MAIL=3Dyes
WITHOUT_TESTS=3Dyes
WITHOUT_AMD=3Dyes
WITHOUT_CLANG=3Dyes
WITHOUT_DEBUG_FILES=3Dyes
WITHOUT_EXAMPLES=3Dyes
WITHOUT_HYPERV=3Dyes
Happy Hacking,
-----BEGIN PGP PUBLIC KEY BLOCK-----
mJMEZlPMfxMFK4EEACMEIwQA/hAHZ4KNJLw5eRl6DAOyzkuHQ7PaK2hTYLVIPoxC
sCe8lB/hzET5KxMW9GXgFgPaSP7Es+ul6ajyq8pr9DeGnXUAFymi7GoT1kLIqgrn
X+rDAwMk9JNEElTmVNvgKWv/G+pSg2rAQ8sIw6smgckA0CaX1JdcNavrHDgKMO4u
Duo44Te0BkpPTUlTTYjbBBMTCgBBFiEENyUvu5bQVBXM019e8anbed+sQ8YFAmZT
zH8CGwMFCQHnaREFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQ8anbed+s
Q8Y5BgIHcX19jz0KP89uTmqvLGZcKsITDJKweAnccCZRS8hvXT8JBwR1LyxsBBKF
++eN2JJasZLo+s6sy7pDMN+9z4Gkdy0CCOj+arfLdvajfzdK4QeKpINfpa2SkQD1
CP98lvfI/luIbetUVW+qNDkzT1jZphYuzgsCaofTGCIQpFk19q22ZcZquJcEZlPM
fxIFK4EEACMEIwQANy4VvpnTHneZipCPwzVJnMN82coCudRAiL2i2m3GPA06lRtU
njcn9r9Sm26A0sajwc3kQw/ekWrHXpXV0aL076cAUW9GKYDljIHVlr76wgHbLt6q
BX5VkA6xS0cq3skbMEI0QpIqIK81Yf0z8wfyF5uqAgNPUpY4nHMV9S856JB2VDQD
AQoJiMEEGBMKACYWIQQ3JS+7ltBUFczTX17xqdt536xDxgUCZlPMfwIbDAUJAedp
EQAKCRDxqdt536xDxjIuAgjacZCttPWpKGfMbnNWePz6t9rcMUb496tSWfKRActr
Rco8lSaDNTVohT/6hLZ5wUX5NFUqTb+kOXJcUGHGbnw2KQIJAQu1m9zEP5XdWmFi
SvGg1NHW2kzqAvFsG37flbwrGRu5fmTnS/LZ/oPzOCuwU6F+o1q0E7gLwFwnzD93
riKeabdd
=3DiCV+
-----END PGP PUBLIC KEY BLOCK-----
-----Original Message-----
From: David Wolfskill <david@catwhisker.org>=20
Sent: Sunday, October 27, 2024 19:50
To: Maku Bex <zagazaw2004@gmail.com>
Cc: freebsd-current@freebsd.org
Subject: Re: speedup build time
On Sun, Oct 27, 2024 at 07:00:17PM -0500, Maku Bex wrote:
> ...
> I'm trying to figure out a way to decrease build time of -CURRENT=20
> and/or -STABLE but it is not working out that well.
> I've tried the following:
> * Tuning /etc/make.conf and /etc/src-env.conf based on their manpages
> (make.conf(5), make(1), src.conf(5)).
> * Removed certain options from GENERIC, which reduced the kernel build
> time significantly.=20
>=20
> Hardware specs:
> * I7-10700T 8C/16T @2.0GHz.
> * 16GB RAM.
> * 256GB NVMe.
>=20
> Currently, 'make buildworld' has been running for 7hrs and counting.=20
> Is there anything else I can do to speed up world build?
> ....
First, some questions -- whether or not you wish to share the answers, I be=
lieve that answering the questions (if only for yourself) may be
helpful:
* What (if any) "max_jobs" (make's "-j" parameter) are you using? For a
16T system, I expect I would start with a value of around 28.
* Are you using make's "meta" mode? I do.
* Can you get more memory for that system?
Some context: I am in the (definitely peculiar; perhaps unfortunate) habit=
of "tracking" FreeBSD head and latest stable branch on a small collection =
of machines ... daily. One of those is the laptop that I use for normal "d=
esktop" stuff, as well as accessing all other machines on the planet that I=
access. (Oh: I also update ports daily, too -- though I let my dedicated =
"build machine" handle chromium.)
That laptop has 4C/8T & 32 GB RAM. This morning, it took:
* 17:46 for stable/14-n269310-bbd018d0aaaf -> stable/14-n269315-b21c677ed28a
(0:02 to update /usr/src; 17:44 to build; 0:00 to delete old libraries)
* 24:03 for main-n273250-9d585fc395c3 -> main-n273270-cadb71e4b013
(0:03 to update /usr/src; 23:59 to build; 0:01 to delete old libraries)
* Updating ports (other than www/chromium) took 1:52.
(Those times are "mm:ss" in case it wasn't obvious.)
The above is fairly classic "sample size of 1" -- it's not necessarily repr=
esentative. Folks who are "sufficiently interested" are welcome to trawl t=
hrough the logs at https://www.catwhisker.org/~david/FreeBSD/history/ to ge=
t a rough idea of how long it is before the "uname" output from a successfu=
l build is logged; there is probably some correlation to build times.
I also recently acquired a replacement "build machine": 64C/128T; 256 GB RA=
M; a fistful of SATA SSDs (build/boot on one; the rest is a poudriere play =
area in a zraid). That's a wee bit faster than the laptop. :-)
Peace,
david
--=20
David H. Wolfskill david@catwhisker.org
It has been said that history repeats itself. This is perhaps not quite cor=
rect; it merely rhymes. -- Theodor Reik
See https://www.catwhisker.org/~david/publickey.gpg for my public key.
--=-=cRby8em2/lhT9t=-=
Content-Type: application/pgp-signature;
name="openpgp-digital-signature.asc"
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
iLgEARMKAB0WIQQ3JS+7ltBUFczTX17xqdt536xDxgUCZx7tSgAKCRDxqdt536xD
xr6eAgkBRPpdhJ/BhScL8hAI9ytFNXWl9hxw6PFaRY9yZ+Ap8e2Bn2EacYdT4mEF
MUpZowUjxk2oqVYBNiLQmRj885VKoYYCCPlmNNOvUw0t3JC4oKntgjEbFVOSlaLx
NGhFKgIAk4rX+F03hJY+Mra4ZA0A47L27ZryKFcCQaQf05rRHJi7GxDY
=qoPq
-----END PGP SIGNATURE-----
--=-=cRby8em2/lhT9t=-=--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?015001db28db$690b03c0$3b210b40$>