Date: Tue, 30 May 2023 11:53:24 +0100 From: Nuno Teixeira <eduardo@freebsd.org> To: Mark Millard <marklmi@yahoo.com> Cc: Ronald Klop <ronald-lists@klop.ws>, FreeBSD Mailing List <freebsd-ports@freebsd.org>, freebsd-arm@freebsd.org Subject: Re: Improving www/chromium build time on arm64 Message-ID: <CAFDf7ULrFcULXqQOHJK_jgvs2JxeVrDb5%2BdnPRuUkfix1PuJDQ@mail.gmail.com> In-Reply-To: <CAFDf7U%2BxOMkpLOUYHipHyOp-vw96--JKV4PxfXE8DkKCZCSMGw@mail.gmail.com> References: <CAFDf7U%2B1p5HSEmXzSKU62ULgAevGK-g93nBqix2hzafEDs4F-Q@mail.gmail.com> <CAFDf7ULoK-P7h8E2PGz1S3-8Dr8EZLD_o7dWku3eZn7FEXV2Cg@mail.gmail.com> <1623315797.5.1684837400228@mailrelay> <CAFDf7UK=UQhJt5wi7zk%2BoP%2BCmob5W6Zt-=jkV0rV-kd%2Bm6zMvQ@mail.gmail.com> <EDC26FF9-0D8D-4412-85F8-21BE157BAF6F@yahoo.com> <CAFDf7U%2BxOMkpLOUYHipHyOp-vw96--JKV4PxfXE8DkKCZCSMGw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000d25d3f05fce70368 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable (...) (I've missed OPTIONS_EXCLUDE_aarch64=3DLTO so forgeting it.) Conlusion: Thats a big difference on compilation times: --- build of www/firefox | firefox-113.0.2_1,2 build time: 05:35:57 --- build of www/chromium | chromium-113.0.5672.126 build time: 56:59:23 --- But I'm enjoying the fast and smooth chrome run. Really nice. Nuno Teixeira <eduardo@freebsd.org> escreveu no dia ter=C3=A7a, 30/05/2023 = =C3=A0(s) 11:07: > Hello Mark, > > Wrong poudriere.conf. Uploaded correct one. > > Summary: > --- > USE_TMPFS=3Dno > PARALLEL_JOBS=3D1 > ALLOW_MAKE_JOBS=3Dyes > MAX_EXECUTION_TIME=3D259200 > NOHANG_TIME=3D259200 > --- > > htop showed me 4<load<5 (around 4,70) with temperature at ~60/65 degrees > Celsius > > Mark Millard <marklmi@yahoo.com> escreveu no dia ter=C3=A7a, 30/05/2023 = =C3=A0(s) > 10:47: > >> On May 30, 2023, at 01:39, Nuno Teixeira <eduardo@freebsd.org> wrote: >> >> > Hello Ronald and all! >> > >> > Finally I build it after 57 hours @ 2000MHz !!! >> >> So the prior "[88:53:28] Failed ports: www/chromium:build/timeout" >> was not with the overclocking? Other differences that could >> contribute to the time difference? >> >> MAX_EXECUTION_TIME=3D86400 (the default) only allows for 24 hours >> before it starts analyzing the log for failure information (while >> letting the build continue while that scanning is done). >> >> > Configs, photos and log at: >> https://people.freebsd.org/~eduardo/logs/chromium/ >> >> https://people.freebsd.org/~eduardo/logs/chromium/poudriere.conf >> shows no assignments to MAX_EXECUTION_TIME or NOHANG_TIME or >> other such (outside comments). In the comments are the default >> values, not increases. >> >> Similarly for PARALLEL_JOBS , PREPARE_PARALLEL_JOBS , ALLOW_MAKE_JOBS , >> and so on. (It is not clear what the load average behavior/ idle time >> behavior might have been.) >> >> The file does show explicit assignments to: ZPOOL, FREEBSD_HOST , >> RESOLV_CONF , BASEFS , USE_PORTLINT , USE_TMPFS , DISTFILES_CACHE , >> and PACKAGE_FETCH_URL . >> >> > I'm running it right now and it runs very smooth! >> > >> > Next step will be compiling with LTO to check compile time and do some >> comparisons on binary size and run performance. >> > >> > Should I bother with LTO? Will it add performance? >> > >> > Thanks all! >> > >> > >> > Ronald Klop <ronald-lists@klop.ws> escreveu no dia ter=C3=A7a, 23/05/2= 023 >> =C3=A0(s) 11:23: >> >> =C3=82 >> >>> Van: Nuno Teixeira <eduardo@freebsd.org> >> >>> Datum: dinsdag, 23 mei 2023 09:48 >> >>> Aan: FreeBSD Mailing List <freebsd-ports@freebsd.org>, >> freebsd-arm@freebsd.org >> >>> Onderwerp: Re: Improving www/chromium build time on arm64 >> >>> (...) >> >>> =C3=82 >> >>> But I also found that poudriere: >> >>> [88:53:28] Failed ports: www/chromium:build/timeout >> >>> =C3=82 >> >>> So I will search how do I increase build time to avoid timeout: >> >>> --- >> >>> # This defines the max time (in seconds) that a command may run for = a >> build >> >>> # before it is killed for taking too long. Default: 86400 >> >>> #MAX_EXECUTION_TIME=3D86400 >> >>> --- >> >>> =C3=82 >> >>> I suspect it can be the cause of build failure because build >24h. >> >>> =C3=82 >> >>> Nuno Teixeira <eduardo@freebsd.org> escreveu no dia ter=C3=83=C2=A7a= , >> 23/05/2023 =C3=83 (s) 08:31: >> >>> Hello all, >> >>> =C3=82 >> >>> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is constantl= y >> tab crashing and seg faulting I decided to give chromium a try. >> >>> chromium pkg isn't available for arm64 and I didn't found error log >> at https://pkg-status.freebsd.org/ so I decided to build it on poudriere= . >> >>> =C3=82 >> >>> It fails to build around 70% but my biggest concern is build time: 2= 7 >> hours for 70% compilation! >> >>> =C3=82 >> >>> Just for comparison: >> >>> llvm15: 12h >> >>> rust: 9h >> >>> firefox: 6h >> >>> =C3=82 >> >>> rip4 v1.5 8GB @ 2000Hz ~62 Celsius >> >>> zfs >> >>> poudriere: >> >>> USE_TMPFS=3Dno >> >>> PARALLEL_JOBS=3D1 >> >>> ALLOW_MAKE_JOBS=3Dyes >> >>> =C3=82 >> >>> Any hints on how to speed build? ccache? >> >>> Does cflags "-O -pipe" not present in compilation is related to buil= d >> time? >> >>> =C3=82 >> >>> Full log: >> >>> >> https://people.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.12= 6.log >> >>> =C3=82 >> >>> Thanks! >> >>> =C3=82 >> >>> --- >> >>> <SNIP> >> >>> In file included from ../../base/check.h:11: >> >>> ../../base/compiler_specific.h:8:10: fatal error: >> 'build/build_config.h' file not found >> >>> #include "build/build_config.h" >> >>> ^~~~~~~~~~~~~~~~~~~~~~ >> >>> 1 error generated. >> >>> ninja: build stopped: subcommand failed. >> >>> =3D=3D=3D> Compilation failed unexpectedly. >> >>> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting the >> failure to >> >>> the maintainer. >> >>> *** Error code 1 >> >>> >> >>> Stop. >> >>> make: stopped in /usr/ports/www/chromium >> >>> build of www/chromium | chromium-113.0.5672.126 ended at Tue May 23 >> 02:22:56 WEST 2023 >> >>> build time: 27:15:37 >> >>> !!! build failure encountered !!! >> >>> --- >> >>> -- Nuno Teixeira >> >>> FreeBSD Committer (ports) >> >>> =C3=82 >> >>> -- Nuno Teixeira >> >>> FreeBSD Committer (ports) >> >>> >> >> >> >> Hi, >> >> >> >> My poudriere.conf for building ports on rpi4 contains these two >> changes: >> >> >> >> # This defines the max time (in seconds) that a command may run for a >> build >> >> # before it is killed for taking too long. Default: 86400 >> >> #MAX_EXECUTION_TIME=3D86400 >> >> MAX_EXECUTION_TIME=3D172800 >> >> >> >> # This defines the time (in seconds) before a command is considered t= o >> >> # be in a runaway state for having no output on stdout. Default: 7200 >> >> #NOHANG_TIME=3D7200 >> >> NOHANG_TIME=3D172800 >> >> >> >> >> >> I'm maintaining some mongodb ports which also take a long long time t= o >> build on rpi4. >> >> But I don't remember which of these settings has the preferred impact >> on your situation. >> >> >> >> Oh... now I'm reading your log better. It contains this: >> >> "=3D>> Killing timed out build after 86400 seconds" >> >> >> >> And some time later: >> >> "=3D>> Cleaning up wrkdir >> >> =3D=3D=3D> Cleaning for chromium-113.0.5672.126" >> >> >> >> So the errors about missing files happen because poudriere is removin= g >> files while some processes of the build are still running. >> >> >> >> I'm curious how far it gets after you increase MAX_EXECUTION_TIME. >> > >> >> =3D=3D=3D >> Mark Millard >> marklmi at yahoo.com >> >> > > -- > Nuno Teixeira > FreeBSD Committer (ports) > --=20 Nuno Teixeira FreeBSD Committer (ports) --000000000000d25d3f05fce70368 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>(...)</div><div><br></div><div>(I've missed OPTIO= NS_EXCLUDE_aarch64=3DLTO so forgeting it.)<br></div><div><br></div><div>Con= lusion:</div><div><br></div><div>Thats a big difference on compilation time= s:</div><div><br></div><div>---<br></div><div>build of www/firefox | firefo= x-113.0.2_1,2<br>build time: 05:35:57</div><div>---<br></div><div>build of = www/chromium | chromium-113.0.5672.126<br></div><div>build time: 56:59:23</= div><div>---<br></div><div>But I'm enjoying the fast and smooth chrome = run.</div><div>Really nice.<br></div></div><br><div class=3D"gmail_quote"><= div dir=3D"ltr" class=3D"gmail_attr">Nuno Teixeira <<a href=3D"mailto:ed= uardo@freebsd.org">eduardo@freebsd.org</a>> escreveu no dia ter=C3=A7a, = 30/05/2023 =C3=A0(s) 11:07:<br></div><blockquote class=3D"gmail_quote" styl= e=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddin= g-left:1ex"><div dir=3D"ltr"><div>Hello Mark,</div><div><br></div><div>Wron= g poudriere.conf. Uploaded correct one.</div><div><br></div><div>Summary:</= div><div>---<br></div>USE_TMPFS=3Dno<div>PARALLEL_JOBS=3D1</div><div>ALLOW_= MAKE_JOBS=3Dyes</div><div><div>MAX_EXECUTION_TIME=3D259200<br>NOHANG_TIME= =3D259200</div><div>---</div><div><br></div><div>htop showed me 4<load&l= t;5 (around 4,70) with temperature at ~60/65 degrees Celsius</div></div></d= iv><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">Mar= k Millard <<a href=3D"mailto:marklmi@yahoo.com" target=3D"_blank">marklm= i@yahoo.com</a>> escreveu no dia ter=C3=A7a, 30/05/2023 =C3=A0(s) 10:47:= <br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8= ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On May 30, 2023= , at 01:39, Nuno Teixeira <<a href=3D"mailto:eduardo@freebsd.org" target= =3D"_blank">eduardo@freebsd.org</a>> wrote:<br> <br> > Hello Ronald and all!<br> > <br> > Finally I build it after 57 hours @ 2000MHz !!!<br> <br> So the prior "[88:53:28] Failed ports: www/chromium:build/timeout"= ;<br> was not with the overclocking? Other differences that could<br> contribute to the time difference?<br> <br> MAX_EXECUTION_TIME=3D86400 (the default) only allows for 24 hours<br> before it starts analyzing the log for failure information (while<br> letting the build continue while that scanning is done).<br> <br> > Configs, photos and log at: <a href=3D"https://people.freebsd.org/~edu= ardo/logs/chromium/" rel=3D"noreferrer" target=3D"_blank">https://people.fr= eebsd.org/~eduardo/logs/chromium/</a><br> <br> <a href=3D"https://people.freebsd.org/~eduardo/logs/chromium/poudriere.conf= " rel=3D"noreferrer" target=3D"_blank">https://people.freebsd.org/~eduardo/= logs/chromium/poudriere.conf</a><br> shows no assignments to MAX_EXECUTION_TIME or NOHANG_TIME or<br> other such (outside comments). In the comments are the default<br> values, not increases.<br> <br> Similarly for PARALLEL_JOBS , PREPARE_PARALLEL_JOBS , ALLOW_MAKE_JOBS ,<br> and so on. (It is not clear what the load average behavior/ idle time<br> behavior might have been.)<br> <br> The file does show explicit assignments to: ZPOOL, FREEBSD_HOST ,<br> RESOLV_CONF , BASEFS , USE_PORTLINT , USE_TMPFS , DISTFILES_CACHE ,<br> and PACKAGE_FETCH_URL .<br> <br> > I'm running it right now and it runs very smooth!<br> > <br> > Next step will be compiling with LTO to check compile time and do some= comparisons on binary size and run performance.<br> > <br> > Should I bother with LTO? Will it add performance?<br> > <br> > Thanks all!<br> > <br> > <br> > Ronald Klop <<a href=3D"mailto:ronald-lists@klop.ws" target=3D"_bla= nk">ronald-lists@klop.ws</a>> escreveu no dia ter=C3=A7a, 23/05/2023 =C3= =A0(s) 11:23:<br> >> =C3=82=C2=A0 <br> >>> Van: Nuno Teixeira <<a href=3D"mailto:eduardo@freebsd.org" = target=3D"_blank">eduardo@freebsd.org</a>><br> >>> Datum: dinsdag, 23 mei 2023 09:48<br> >>> Aan: FreeBSD Mailing List <<a href=3D"mailto:freebsd-ports@= freebsd.org" target=3D"_blank">freebsd-ports@freebsd.org</a>>, <a href= =3D"mailto:freebsd-arm@freebsd.org" target=3D"_blank">freebsd-arm@freebsd.o= rg</a><br> >>> Onderwerp: Re: Improving www/chromium build time on arm64<br> >>> (...) <br> >>> =C3=82 <br> >>> But I also found that poudriere:<br> >>> [88:53:28] Failed ports: www/chromium:build/timeout<br> >>> =C3=82 <br> >>> So I will search how do I increase build time to avoid timeout= :<br> >>> ---<br> >>> # This defines the max time (in seconds) that a command may ru= n for a build<br> >>> # before it is killed for taking too long. Default: 86400<br> >>> #MAX_EXECUTION_TIME=3D86400<br> >>> ---<br> >>> =C3=82 <br> >>> I suspect it can be the cause of build failure because build &= gt;24h.<br> >>> =C3=82=C2=A0 <br> >>> Nuno Teixeira <<a href=3D"mailto:eduardo@freebsd.org" targe= t=3D"_blank">eduardo@freebsd.org</a>> escreveu no dia ter=C3=83=C2=A7a, = 23/05/2023 =C3=83 (s) 08:31:<br> >>> Hello all, <br> >>> =C3=82 <br> >>> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is= constantly tab crashing and seg faulting I decided to give chromium a try.= <br> >>> chromium pkg isn't available for arm64 and I didn't fo= und error log at <a href=3D"https://pkg-status.freebsd.org/" rel=3D"norefer= rer" target=3D"_blank">https://pkg-status.freebsd.org/</a>; so I decided to = build it on poudriere.<br> >>> =C3=82 <br> >>> It fails to build around 70% but my biggest concern is build t= ime: 27 hours for 70% compilation!<br> >>> =C3=82 <br> >>> Just for comparison:<br> >>> llvm15: 12h<br> >>> rust: 9h<br> >>> firefox: 6h<br> >>> =C3=82 <br> >>> rip4 v1.5 8GB @ 2000Hz ~62 Celsius<br> >>> zfs<br> >>> poudriere:<br> >>> USE_TMPFS=3Dno<br> >>> PARALLEL_JOBS=3D1<br> >>> ALLOW_MAKE_JOBS=3Dyes<br> >>> =C3=82 <br> >>> Any hints on how to speed build? ccache?<br> >>> Does cflags "-O -pipe" not present in compilation is= related to build time? <br> >>> =C3=82 <br> >>> Full log:<br> >>> <a href=3D"https://people.freebsd.org/~eduardo/logs/chromium/c= hromium-113.0.5672.126.log" rel=3D"noreferrer" target=3D"_blank">https://pe= ople.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.126.log</a><br> >>> =C3=82 <br> >>> Thanks!<br> >>> =C3=82 <br> >>> ---<br> >>> <SNIP><br> >>> In file included from ../../base/check.h:11:<br> >>> ../../base/compiler_specific.h:8:10: fatal error: 'build/b= uild_config.h' file not found<br> >>> #include "build/build_config.h"<br> >>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ^~~~~~~~~~~~~~~~~~~~~~<br> >>> 1 error generated.<br> >>> ninja: build stopped: subcommand failed.<br> >>> =3D=3D=3D> Compilation failed unexpectedly.<br> >>> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting= the failure to<br> >>> the maintainer.<br> >>> *** Error code 1<br> >>> <br> >>> Stop.<br> >>> make: stopped in /usr/ports/www/chromium<br> >>> build of www/chromium | chromium-113.0.5672.126 ended at Tue M= ay 23 02:22:56 WEST 2023<br> >>> build time: 27:15:37<br> >>> !!! build failure encountered !!!<br> >>> ---<br> >>> -- Nuno Teixeira<br> >>> FreeBSD Committer (ports)<br> >>>=C2=A0 =C2=A0=C3=82 <br> >>> -- Nuno Teixeira<br> >>> FreeBSD Committer (ports)<br> >>> <br> >> <br> >> Hi,<br> >> <br> >> My poudriere.conf for building ports on rpi4 contains these two ch= anges:<br> >> <br> >> # This defines the max time (in seconds) that a command may run fo= r a build<br> >> # before it is killed for taking too long. Default: 86400<br> >> #MAX_EXECUTION_TIME=3D86400<br> >> MAX_EXECUTION_TIME=3D172800<br> >> <br> >> # This defines the time (in seconds) before a command is considere= d to<br> >> # be in a runaway state for having no output on stdout. Default: 7= 200<br> >> #NOHANG_TIME=3D7200<br> >> NOHANG_TIME=3D172800<br> >> <br> >> <br> >> I'm maintaining some mongodb ports which also take a long long= time to build on rpi4.<br> >> But I don't remember which of these settings has the preferred= impact on your situation.<br> >> <br> >> Oh... now I'm reading your log better. It contains this:<br> >> "=3D>> Killing timed out build after 86400 seconds"= ;<br> >> <br> >> And some time later:<br> >> "=3D>> Cleaning up wrkdir<br> >> =3D=3D=3D> Cleaning for chromium-113.0.5672.126"<br> >> <br> >> So the errors about missing files happen because poudriere is remo= ving files while some processes of the build are still running.<br> >> <br> >> I'm curious how far it gets after you increase MAX_EXECUTION_T= IME.<br> > <br> <br> =3D=3D=3D<br> Mark Millard<br> marklmi at <a href=3D"http://yahoo.com" rel=3D"noreferrer" target=3D"_blank= ">yahoo.com</a><br> <br> </blockquote></div><br clear=3D"all"><br><span class=3D"gmail_signature_pre= fix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"l= tr"><span style=3D"color:rgb(102,102,102)">Nuno Teixeira<br>FreeBSD Committ= er (ports)</span></div></div> </blockquote></div><br clear=3D"all"><br><span class=3D"gmail_signature_pre= fix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"l= tr"><span style=3D"color:rgb(102,102,102)">Nuno Teixeira<br>FreeBSD Committ= er (ports)</span></div></div> --000000000000d25d3f05fce70368--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFDf7ULrFcULXqQOHJK_jgvs2JxeVrDb5%2BdnPRuUkfix1PuJDQ>