Date: Mon, 28 Oct 2024 11:00:54 +0100 From: Alexander Leidinger <Alexander@Leidinger.net> To: Maku Bex <zagazaw2004@gmail.com>, "'David Wolfskill'" <david@catwhisker.org> Cc: <freebsd-current@freebsd.org> Subject: RE: speedup build time Message-ID: <192d2702590.2805.fa4b1493b064008fe79f0f905b8e5741@Leidinger.net> In-Reply-To: <015001db28db$690b03c0$3b210b40$@gmail.com> References: <012d01db28cc$5fc66ac0$1f534040$@gmail.com> <Zx7fqFzbjuCvZgaw@albert.catwhisker.org> <015001db28db$690b03c0$3b210b40$@gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
Hi,
Try (add)
src.conf:
WITHOUT_CLEAN=YES
WITH_MALLOC_PRODUCTION=YES
WITHOUT_LLVM_ASSERTIONS=yes
This may only give a result in subsequent builds, not on the first build or
when a lot or something essential has changed.
Note (just in case...) the make jobs number you use is not used for the
world, only for ports. For the world/kernel build it's really the -j option.
Bye,
Alexander.
--
Send from a mobile device, please forgive brevity and misspellings.
Am 28. Oktober 2024 02:50:38 schrieb "Maku Bex" <zagazaw2004@gmail.com>:
> Hi David,
>
> * What (if any) "max_jobs" (make's "-j" parameter) are you using? 16
>
> * For a 16T system, I expect I would start with a value of around 28.
> Shouldn't this number be equal to $(nproc --all)?
>
> * Are you using make's "meta" mode? I do. Yes.
>
> * Can you get more memory for that system? Yes, I can bump it to 32GB. This
> is a Dell OptiPlex 7080 Micro, and not sure if it is worth upgrading the
> other components.
>
> If your laptop can complete the builds in such a small amount of time, then
> there's either something wrong with my machine or the config files. Oh! The
> build just finished as I was typing this reply and...(not for the weak hearts):
>
> --------------------------------------------------------------
>>>> World built in 32437 seconds, ncpu: 16
> --------------------------------------------------------------
>
> Here are the contents of:
>
> make.conf:
>
> CPUTYPE?=icelake-client
> CFLAGS= -O2 -pipe -march=${CPUTYPE}
> #CXXFLAGS=${CFLAGS}
> MACHINE_CPU+= avx512 avx2 avx sse42 sse41 ssse3 sse3 f16c
> MAKE_JOBS_NUMBER= 16
> BATCH=yes
> WRKDIRPREFIX=/var/ports/tmp
> CCACHE_DIR=/var/ccache/tmp
>
> OPTIONS_UNSET+= QT5 QT6 CUPS VESA SCFB TWM XCONSOLE XCLOCK GNOME KDE EXAMPLES \
> DEBUG ASPELL NLS DOCS
> -------------------------------------------------------------------------------
> src-env.conf:
>
> WITH_META_MODE=yes
> WITH_CCACHE_BUILD=yes
> #WITH_DIRDEPS_BUILD=yes <<fails
> #WITH_DIRDEPS_CACHE=yes <<fails
>
> WITHOUT_TALK=yes
> WITHOUT_TELNET=yes
> WITHOUT_TFTP=yes
> WITHOUT_IPFILTER=yes
> WITHOUT_IPFW=yes
> WITHOUT_FLOPPY=yes
> WITHOUT_GAMES=yes
> WITHOUT_BLUETOOTH=yes
> WITHOUT_EE=yes
> WITHOUT_FREEBSD_UPDATE=yes
> WITHOUT_GPIO=yes
> WITHOUT_MAIL=yes
> WITHOUT_TESTS=yes
> WITHOUT_AMD=yes
> WITHOUT_CLANG=yes
> WITHOUT_DEBUG_FILES=yes
> WITHOUT_EXAMPLES=yes
> WITHOUT_HYPERV=yes
>
> Happy Hacking,
>
> -----BEGIN PGP PUBLIC KEY BLOCK-----
>
> mJMEZlPMfxMFK4EEACMEIwQA/hAHZ4KNJLw5eRl6DAOyzkuHQ7PaK2hTYLVIPoxC
> sCe8lB/hzET5KxMW9GXgFgPaSP7Es+ul6ajyq8pr9DeGnXUAFymi7GoT1kLIqgrn
> X+rDAwMk9JNEElTmVNvgKWv/G+pSg2rAQ8sIw6smgckA0CaX1JdcNavrHDgKMO4u
> Duo44Te0BkpPTUlTTYjbBBMTCgBBFiEENyUvu5bQVBXM019e8anbed+sQ8YFAmZT
> zH8CGwMFCQHnaREFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQ8anbed+s
> Q8Y5BgIHcX19jz0KP89uTmqvLGZcKsITDJKweAnccCZRS8hvXT8JBwR1LyxsBBKF
> ++eN2JJasZLo+s6sy7pDMN+9z4Gkdy0CCOj+arfLdvajfzdK4QeKpINfpa2SkQD1
> CP98lvfI/luIbetUVW+qNDkzT1jZphYuzgsCaofTGCIQpFk19q22ZcZquJcEZlPM
> fxIFK4EEACMEIwQANy4VvpnTHneZipCPwzVJnMN82coCudRAiL2i2m3GPA06lRtU
> njcn9r9Sm26A0sajwc3kQw/ekWrHXpXV0aL076cAUW9GKYDljIHVlr76wgHbLt6q
> BX5VkA6xS0cq3skbMEI0QpIqIK81Yf0z8wfyF5uqAgNPUpY4nHMV9S856JB2VDQD
> AQoJiMEEGBMKACYWIQQ3JS+7ltBUFczTX17xqdt536xDxgUCZlPMfwIbDAUJAedp
> EQAKCRDxqdt536xDxjIuAgjacZCttPWpKGfMbnNWePz6t9rcMUb496tSWfKRActr
> Rco8lSaDNTVohT/6hLZ5wUX5NFUqTb+kOXJcUGHGbnw2KQIJAQu1m9zEP5XdWmFi
> SvGg1NHW2kzqAvFsG37flbwrGRu5fmTnS/LZ/oPzOCuwU6F+o1q0E7gLwFwnzD93
> riKeabdd
> =iCV+
> -----END PGP PUBLIC KEY BLOCK-----
>
> -----Original Message-----
> From: David Wolfskill <david@catwhisker.org>
> Sent: Sunday, October 27, 2024 19:50
> To: Maku Bex <zagazaw2004@gmail.com>
> Cc: freebsd-current@freebsd.org
> Subject: Re: speedup build time
>
> On Sun, Oct 27, 2024 at 07:00:17PM -0500, Maku Bex wrote:
>> ...
>> I'm trying to figure out a way to decrease build time of -CURRENT
>> and/or -STABLE but it is not working out that well.
>> I've tried the following:
>> * Tuning /etc/make.conf and /etc/src-env.conf based on their manpages
>> (make.conf(5), make(1), src.conf(5)).
>> * Removed certain options from GENERIC, which reduced the kernel build
>> time significantly.
>>
>> Hardware specs:
>> * I7-10700T 8C/16T @2.0GHz.
>> * 16GB RAM.
>> * 256GB NVMe.
>>
>> Currently, 'make buildworld' has been running for 7hrs and counting.
>> Is there anything else I can do to speed up world build?
>> ....
>
> First, some questions -- whether or not you wish to share the answers, I
> believe that answering the questions (if only for yourself) may be
> helpful:
>
> * What (if any) "max_jobs" (make's "-j" parameter) are you using? For a
> 16T system, I expect I would start with a value of around 28.
>
> * Are you using make's "meta" mode? I do.
>
> * Can you get more memory for that system?
>
> Some context: I am in the (definitely peculiar; perhaps unfortunate) habit
> of "tracking" FreeBSD head and latest stable branch on a small collection
> of machines ... daily. One of those is the laptop that I use for normal
> "desktop" stuff, as well as accessing all other machines on the planet that
> I access. (Oh: I also update ports daily, too -- though I let my dedicated
> "build machine" handle chromium.)
>
> That laptop has 4C/8T & 32 GB RAM. This morning, it took:
>
> * 17:46 for stable/14-n269310-bbd018d0aaaf -> stable/14-n269315-b21c677ed28a
> (0:02 to update /usr/src; 17:44 to build; 0:00 to delete old libraries)
>
> * 24:03 for main-n273250-9d585fc395c3 -> main-n273270-cadb71e4b013
> (0:03 to update /usr/src; 23:59 to build; 0:01 to delete old libraries)
>
> * Updating ports (other than www/chromium) took 1:52.
>
> (Those times are "mm:ss" in case it wasn't obvious.)
>
> The above is fairly classic "sample size of 1" -- it's not necessarily
> representative. Folks who are "sufficiently interested" are welcome to
> trawl through the logs at
> https://www.catwhisker.org/~david/FreeBSD/history/ to get a rough idea of
> how long it is before the "uname" output from a successful build is logged;
> there is probably some correlation to build times.
>
> I also recently acquired a replacement "build machine": 64C/128T; 256 GB
> RAM; a fistful of SATA SSDs (build/boot on one; the rest is a poudriere
> play area in a zraid). That's a wee bit faster than the laptop. :-)
>
> Peace,
> david
> --
> David H. Wolfskill david@catwhisker.org
> It has been said that history repeats itself. This is perhaps not quite
> correct; it merely rhymes. -- Theodor Reik
>
> See https://www.catwhisker.org/~david/publickey.gpg for my public key.
[-- Attachment #2 --]
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">;
<html>
<body>
<div dir="auto">
<div dir="auto">Hi, </div><div dir="auto"><br></div><div dir="auto">Try (add)</div><div dir="auto">src.conf:</div><div dir="auto">WITHOUT_CLEAN=YES</div><div dir="auto">WITH_MALLOC_PRODUCTION=YES</div><div dir="auto">WITHOUT_LLVM_ASSERTIONS=yes</div><div dir="auto"><br></div><div dir="auto">This may only give a result in subsequent builds, not on the first build or when a lot or something essential has changed. </div><div dir="auto"><br></div><div dir="auto">Note (just in case...) the make jobs number you use is not used for the world, only for ports. For the world/kernel build it's really the -j option. </div><div dir="auto"><br></div><div dir="auto">Bye, </div><div dir="auto">Alexander. </div><div dir="auto"><br></div><div id="aqm-signature" dir="auto" style="color: black;"><div dir="auto">-- </div><div dir="auto">Send from a mobile device, please forgive brevity and misspellings.</div></div><div dir="auto"><br></div>
<div id="aqm-original" style="color: black;">
<div dir="auto">Am 28. Oktober 2024 02:50:38 schrieb "Maku Bex" <zagazaw2004@gmail.com>:</div>
<div><br></div>
<blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;">
<div dir="auto">Hi David,</div>
<div dir="auto"><br></div>
<div dir="auto">* What (if any) "max_jobs" (make's "-j" parameter) are you using? 16</div>
<div dir="auto"><br></div>
<div dir="auto">* For a 16T system, I expect I would start with a value of around 28. Shouldn't this number be equal to $(nproc --all)?</div>
<div dir="auto"><br></div>
<div dir="auto">* Are you using make's "meta" mode? I do. Yes.</div>
<div dir="auto"><br></div>
<div dir="auto">* Can you get more memory for that system? Yes, I can bump it to 32GB. This is a Dell OptiPlex 7080 Micro, and not sure if it is worth upgrading the other components.</div>
<div dir="auto"><br></div>
<div dir="auto">If your laptop can complete the builds in such a small amount of time, then there's either something wrong with my machine or the config files. Oh! The build just finished as I was typing this reply and...(not for the weak hearts):</div>
<div dir="auto"><br></div>
<div dir="auto">--------------------------------------------------------------</div>
<blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #0099CC; padding-left: 0.75ex;">
<blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #9933CC; padding-left: 0.75ex;">
<blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #669900; padding-left: 0.75ex;">
<div dir="auto">World built in 32437 seconds, ncpu: 16</div>
</blockquote>
</blockquote>
</blockquote>
<div dir="auto">-------------------------------------------------------------- </div>
<div dir="auto"><br></div>
<div dir="auto">Here are the contents of:</div>
<div dir="auto"><br></div>
<div dir="auto">make.conf:</div>
<div dir="auto"><br></div>
<div dir="auto">CPUTYPE?=icelake-client</div>
<div dir="auto">CFLAGS= -O2 -pipe -march=${CPUTYPE}</div>
<div dir="auto">#CXXFLAGS=${CFLAGS}</div>
<div dir="auto">MACHINE_CPU+= avx512 avx2 avx sse42 sse41 ssse3 sse3 f16c</div>
<div dir="auto">MAKE_JOBS_NUMBER= 16</div>
<div dir="auto">BATCH=yes</div>
<div dir="auto">WRKDIRPREFIX=/var/ports/tmp</div>
<div dir="auto">CCACHE_DIR=/var/ccache/tmp</div>
<div dir="auto"><br></div>
<div dir="auto">OPTIONS_UNSET+= QT5 QT6 CUPS VESA SCFB TWM XCONSOLE XCLOCK GNOME KDE EXAMPLES \</div>
<div dir="auto"> DEBUG ASPELL NLS DOCS</div>
<div dir="auto">-------------------------------------------------------------------------------</div>
<div dir="auto">src-env.conf:</div>
<div dir="auto"><br></div>
<div dir="auto">WITH_META_MODE=yes</div>
<div dir="auto">WITH_CCACHE_BUILD=yes</div>
<div dir="auto">#WITH_DIRDEPS_BUILD=yes <<fails</div>
<div dir="auto">#WITH_DIRDEPS_CACHE=yes <<fails </div>
<div dir="auto"><br></div>
<div dir="auto">WITHOUT_TALK=yes</div>
<div dir="auto">WITHOUT_TELNET=yes</div>
<div dir="auto">WITHOUT_TFTP=yes</div>
<div dir="auto">WITHOUT_IPFILTER=yes</div>
<div dir="auto">WITHOUT_IPFW=yes</div>
<div dir="auto">WITHOUT_FLOPPY=yes</div>
<div dir="auto">WITHOUT_GAMES=yes</div>
<div dir="auto">WITHOUT_BLUETOOTH=yes</div>
<div dir="auto">WITHOUT_EE=yes</div>
<div dir="auto">WITHOUT_FREEBSD_UPDATE=yes</div>
<div dir="auto">WITHOUT_GPIO=yes</div>
<div dir="auto">WITHOUT_MAIL=yes</div>
<div dir="auto">WITHOUT_TESTS=yes</div>
<div dir="auto">WITHOUT_AMD=yes</div>
<div dir="auto">WITHOUT_CLANG=yes</div>
<div dir="auto">WITHOUT_DEBUG_FILES=yes</div>
<div dir="auto">WITHOUT_EXAMPLES=yes</div>
<div dir="auto">WITHOUT_HYPERV=yes</div>
<div dir="auto"><br></div>
<div dir="auto">Happy Hacking,</div>
<div dir="auto"><br></div>
<div dir="auto">-----BEGIN PGP PUBLIC KEY BLOCK-----</div>
<div dir="auto"><br></div>
<div dir="auto">mJMEZlPMfxMFK4EEACMEIwQA/hAHZ4KNJLw5eRl6DAOyzkuHQ7PaK2hTYLVIPoxC</div>
<div dir="auto">sCe8lB/hzET5KxMW9GXgFgPaSP7Es+ul6ajyq8pr9DeGnXUAFymi7GoT1kLIqgrn</div>
<div dir="auto">X+rDAwMk9JNEElTmVNvgKWv/G+pSg2rAQ8sIw6smgckA0CaX1JdcNavrHDgKMO4u</div>
<div dir="auto">Duo44Te0BkpPTUlTTYjbBBMTCgBBFiEENyUvu5bQVBXM019e8anbed+sQ8YFAmZT</div>
<div dir="auto">zH8CGwMFCQHnaREFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQ8anbed+s</div>
<div dir="auto">Q8Y5BgIHcX19jz0KP89uTmqvLGZcKsITDJKweAnccCZRS8hvXT8JBwR1LyxsBBKF</div>
<div dir="auto">++eN2JJasZLo+s6sy7pDMN+9z4Gkdy0CCOj+arfLdvajfzdK4QeKpINfpa2SkQD1</div>
<div dir="auto">CP98lvfI/luIbetUVW+qNDkzT1jZphYuzgsCaofTGCIQpFk19q22ZcZquJcEZlPM</div>
<div dir="auto">fxIFK4EEACMEIwQANy4VvpnTHneZipCPwzVJnMN82coCudRAiL2i2m3GPA06lRtU</div>
<div dir="auto">njcn9r9Sm26A0sajwc3kQw/ekWrHXpXV0aL076cAUW9GKYDljIHVlr76wgHbLt6q</div>
<div dir="auto">BX5VkA6xS0cq3skbMEI0QpIqIK81Yf0z8wfyF5uqAgNPUpY4nHMV9S856JB2VDQD</div>
<div dir="auto">AQoJiMEEGBMKACYWIQQ3JS+7ltBUFczTX17xqdt536xDxgUCZlPMfwIbDAUJAedp</div>
<div dir="auto">EQAKCRDxqdt536xDxjIuAgjacZCttPWpKGfMbnNWePz6t9rcMUb496tSWfKRActr</div>
<div dir="auto">Rco8lSaDNTVohT/6hLZ5wUX5NFUqTb+kOXJcUGHGbnw2KQIJAQu1m9zEP5XdWmFi</div>
<div dir="auto">SvGg1NHW2kzqAvFsG37flbwrGRu5fmTnS/LZ/oPzOCuwU6F+o1q0E7gLwFwnzD93</div>
<div dir="auto">riKeabdd</div>
<div dir="auto">=iCV+</div>
<div dir="auto">-----END PGP PUBLIC KEY BLOCK-----</div>
<div dir="auto"><br></div>
<div dir="auto">-----Original Message-----</div>
<div dir="auto">From: David Wolfskill <david@catwhisker.org> </div>
<div dir="auto">Sent: Sunday, October 27, 2024 19:50</div>
<div dir="auto">To: Maku Bex <zagazaw2004@gmail.com></div>
<div dir="auto">Cc: freebsd-current@freebsd.org</div>
<div dir="auto">Subject: Re: speedup build time</div>
<div dir="auto"><br></div>
<div dir="auto">On Sun, Oct 27, 2024 at 07:00:17PM -0500, Maku Bex wrote:</div>
<blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #0099CC; padding-left: 0.75ex;">
<div dir="auto">...</div>
<div dir="auto">I'm trying to figure out a way to decrease build time of -CURRENT </div>
<div dir="auto">and/or -STABLE but it is not working out that well.</div>
<div dir="auto">I've tried the following:</div>
<div dir="auto">* Tuning /etc/make.conf and /etc/src-env.conf based on their manpages</div>
<div dir="auto">(make.conf(5), make(1), src.conf(5)).</div>
<div dir="auto">* Removed certain options from GENERIC, which reduced the kernel build</div>
<div dir="auto">time significantly. </div>
<div dir="auto"><br></div>
<div dir="auto">Hardware specs:</div>
<div dir="auto">* I7-10700T 8C/16T @2.0GHz.</div>
<div dir="auto">* 16GB RAM.</div>
<div dir="auto">* 256GB NVMe.</div>
<div dir="auto"><br></div>
<div dir="auto">Currently, 'make buildworld' has been running for 7hrs and counting. </div>
<div dir="auto">Is there anything else I can do to speed up world build?</div>
<div dir="auto">....</div>
</blockquote>
<div dir="auto"><br></div>
<div dir="auto">First, some questions -- whether or not you wish to share the answers, I believe that answering the questions (if only for yourself) may be</div>
<div dir="auto">helpful:</div>
<div dir="auto"><br></div>
<div dir="auto">* What (if any) "max_jobs" (make's "-j" parameter) are you using? For a</div>
<div dir="auto"> 16T system, I expect I would start with a value of around 28.</div>
<div dir="auto"><br></div>
<div dir="auto">* Are you using make's "meta" mode? I do.</div>
<div dir="auto"><br></div>
<div dir="auto">* Can you get more memory for that system?</div>
<div dir="auto"><br></div>
<div dir="auto">Some context: I am in the (definitely peculiar; perhaps unfortunate) habit of "tracking" FreeBSD head and latest stable branch on a small collection of machines ... daily. One of those is the laptop that I use for normal "desktop" stuff, as well as accessing all other machines on the planet that I access. (Oh: I also update ports daily, too -- though I let my dedicated "build machine" handle chromium.)</div>
<div dir="auto"><br></div>
<div dir="auto">That laptop has 4C/8T & 32 GB RAM. This morning, it took:</div>
<div dir="auto"><br></div>
<div dir="auto">* 17:46 for stable/14-n269310-bbd018d0aaaf -> stable/14-n269315-b21c677ed28a</div>
<div dir="auto"> (0:02 to update /usr/src; 17:44 to build; 0:00 to delete old libraries)</div>
<div dir="auto"><br></div>
<div dir="auto">* 24:03 for main-n273250-9d585fc395c3 -> main-n273270-cadb71e4b013</div>
<div dir="auto"> (0:03 to update /usr/src; 23:59 to build; 0:01 to delete old libraries)</div>
<div dir="auto"><br></div>
<div dir="auto">* Updating ports (other than www/chromium) took 1:52.</div>
<div dir="auto"><br></div>
<div dir="auto">(Those times are "mm:ss" in case it wasn't obvious.)</div>
<div dir="auto"><br></div>
<div dir="auto">The above is fairly classic "sample size of 1" -- it's not necessarily representative. Folks who are "sufficiently interested" are welcome to trawl through the logs at https://www.catwhisker.org/~david/FreeBSD/history/ to get a rough idea of how long it is before the "uname" output from a successful build is logged; there is probably some correlation to build times.</div>
<div dir="auto"><br></div>
<div dir="auto">I also recently acquired a replacement "build machine": 64C/128T; 256 GB RAM; a fistful of SATA SSDs (build/boot on one; the rest is a poudriere play area in a zraid). That's a wee bit faster than the laptop. :-)</div>
<div dir="auto"><br></div>
<div dir="auto">Peace,</div>
<div dir="auto">david</div>
<div dir="auto">-- </div>
<div dir="auto">David H. Wolfskill david@catwhisker.org</div>
<div dir="auto">It has been said that history repeats itself. This is perhaps not quite correct; it merely rhymes. -- Theodor Reik</div>
<div dir="auto"><br></div>
<div dir="auto">See https://www.catwhisker.org/~david/publickey.gpg for my public key.</div>
</blockquote>
</div><div dir="auto"><br></div>
</div></body>
</html>
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?192d2702590.2805.fa4b1493b064008fe79f0f905b8e5741>