Date: Tue, 30 May 2023 11:07:21 +0100 From: Nuno Teixeira <eduardo@freebsd.org> To: Mark Millard <marklmi@yahoo.com> Cc: Ronald Klop <ronald-lists@klop.ws>, FreeBSD Mailing List <freebsd-ports@freebsd.org>, freebsd-arm@freebsd.org Subject: Re: Improving www/chromium build time on arm64 Message-ID: <CAFDf7U%2BxOMkpLOUYHipHyOp-vw96--JKV4PxfXE8DkKCZCSMGw@mail.gmail.com> In-Reply-To: <EDC26FF9-0D8D-4412-85F8-21BE157BAF6F@yahoo.com> References: <CAFDf7U%2B1p5HSEmXzSKU62ULgAevGK-g93nBqix2hzafEDs4F-Q@mail.gmail.com> <CAFDf7ULoK-P7h8E2PGz1S3-8Dr8EZLD_o7dWku3eZn7FEXV2Cg@mail.gmail.com> <1623315797.5.1684837400228@mailrelay> <CAFDf7UK=UQhJt5wi7zk%2BoP%2BCmob5W6Zt-=jkV0rV-kd%2Bm6zMvQ@mail.gmail.com> <EDC26FF9-0D8D-4412-85F8-21BE157BAF6F@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000002d87fc05fce65f39 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello Mark, Wrong poudriere.conf. Uploaded correct one. Summary: --- USE_TMPFS=3Dno PARALLEL_JOBS=3D1 ALLOW_MAKE_JOBS=3Dyes MAX_EXECUTION_TIME=3D259200 NOHANG_TIME=3D259200 --- htop showed me 4<load<5 (around 4,70) with temperature at ~60/65 degrees Celsius Mark Millard <marklmi@yahoo.com> escreveu no dia ter=C3=A7a, 30/05/2023 =C3= =A0(s) 10:47: > On May 30, 2023, at 01:39, Nuno Teixeira <eduardo@freebsd.org> wrote: > > > Hello Ronald and all! > > > > Finally I build it after 57 hours @ 2000MHz !!! > > So the prior "[88:53:28] Failed ports: www/chromium:build/timeout" > was not with the overclocking? Other differences that could > contribute to the time difference? > > MAX_EXECUTION_TIME=3D86400 (the default) only allows for 24 hours > before it starts analyzing the log for failure information (while > letting the build continue while that scanning is done). > > > Configs, photos and log at: > https://people.freebsd.org/~eduardo/logs/chromium/ > > https://people.freebsd.org/~eduardo/logs/chromium/poudriere.conf > shows no assignments to MAX_EXECUTION_TIME or NOHANG_TIME or > other such (outside comments). In the comments are the default > values, not increases. > > Similarly for PARALLEL_JOBS , PREPARE_PARALLEL_JOBS , ALLOW_MAKE_JOBS , > and so on. (It is not clear what the load average behavior/ idle time > behavior might have been.) > > The file does show explicit assignments to: ZPOOL, FREEBSD_HOST , > RESOLV_CONF , BASEFS , USE_PORTLINT , USE_TMPFS , DISTFILES_CACHE , > and PACKAGE_FETCH_URL . > > > I'm running it right now and it runs very smooth! > > > > Next step will be compiling with LTO to check compile time and do some > comparisons on binary size and run performance. > > > > Should I bother with LTO? Will it add performance? > > > > Thanks all! > > > > > > Ronald Klop <ronald-lists@klop.ws> escreveu no dia ter=C3=A7a, 23/05/20= 23 > =C3=A0(s) 11:23: > >> =C3=82 > >>> Van: Nuno Teixeira <eduardo@freebsd.org> > >>> Datum: dinsdag, 23 mei 2023 09:48 > >>> Aan: FreeBSD Mailing List <freebsd-ports@freebsd.org>, > freebsd-arm@freebsd.org > >>> Onderwerp: Re: Improving www/chromium build time on arm64 > >>> (...) > >>> =C3=82 > >>> But I also found that poudriere: > >>> [88:53:28] Failed ports: www/chromium:build/timeout > >>> =C3=82 > >>> So I will search how do I increase build time to avoid timeout: > >>> --- > >>> # This defines the max time (in seconds) that a command may run for a > build > >>> # before it is killed for taking too long. Default: 86400 > >>> #MAX_EXECUTION_TIME=3D86400 > >>> --- > >>> =C3=82 > >>> I suspect it can be the cause of build failure because build >24h. > >>> =C3=82 > >>> Nuno Teixeira <eduardo@freebsd.org> escreveu no dia ter=C3=83=C2=A7a, > 23/05/2023 =C3=83 (s) 08:31: > >>> Hello all, > >>> =C3=82 > >>> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is constantly > tab crashing and seg faulting I decided to give chromium a try. > >>> chromium pkg isn't available for arm64 and I didn't found error log a= t > https://pkg-status.freebsd.org/ so I decided to build it on poudriere. > >>> =C3=82 > >>> It fails to build around 70% but my biggest concern is build time: 27 > hours for 70% compilation! > >>> =C3=82 > >>> Just for comparison: > >>> llvm15: 12h > >>> rust: 9h > >>> firefox: 6h > >>> =C3=82 > >>> rip4 v1.5 8GB @ 2000Hz ~62 Celsius > >>> zfs > >>> poudriere: > >>> USE_TMPFS=3Dno > >>> PARALLEL_JOBS=3D1 > >>> ALLOW_MAKE_JOBS=3Dyes > >>> =C3=82 > >>> Any hints on how to speed build? ccache? > >>> Does cflags "-O -pipe" not present in compilation is related to build > time? > >>> =C3=82 > >>> Full log: > >>> > https://people.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.126= .log > >>> =C3=82 > >>> Thanks! > >>> =C3=82 > >>> --- > >>> <SNIP> > >>> In file included from ../../base/check.h:11: > >>> ../../base/compiler_specific.h:8:10: fatal error: > 'build/build_config.h' file not found > >>> #include "build/build_config.h" > >>> ^~~~~~~~~~~~~~~~~~~~~~ > >>> 1 error generated. > >>> ninja: build stopped: subcommand failed. > >>> =3D=3D=3D> Compilation failed unexpectedly. > >>> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting the > failure to > >>> the maintainer. > >>> *** Error code 1 > >>> > >>> Stop. > >>> make: stopped in /usr/ports/www/chromium > >>> build of www/chromium | chromium-113.0.5672.126 ended at Tue May 23 > 02:22:56 WEST 2023 > >>> build time: 27:15:37 > >>> !!! build failure encountered !!! > >>> --- > >>> -- Nuno Teixeira > >>> FreeBSD Committer (ports) > >>> =C3=82 > >>> -- Nuno Teixeira > >>> FreeBSD Committer (ports) > >>> > >> > >> Hi, > >> > >> My poudriere.conf for building ports on rpi4 contains these two change= s: > >> > >> # This defines the max time (in seconds) that a command may run for a > build > >> # before it is killed for taking too long. Default: 86400 > >> #MAX_EXECUTION_TIME=3D86400 > >> MAX_EXECUTION_TIME=3D172800 > >> > >> # This defines the time (in seconds) before a command is considered to > >> # be in a runaway state for having no output on stdout. Default: 7200 > >> #NOHANG_TIME=3D7200 > >> NOHANG_TIME=3D172800 > >> > >> > >> I'm maintaining some mongodb ports which also take a long long time to > build on rpi4. > >> But I don't remember which of these settings has the preferred impact > on your situation. > >> > >> Oh... now I'm reading your log better. It contains this: > >> "=3D>> Killing timed out build after 86400 seconds" > >> > >> And some time later: > >> "=3D>> Cleaning up wrkdir > >> =3D=3D=3D> Cleaning for chromium-113.0.5672.126" > >> > >> So the errors about missing files happen because poudriere is removing > files while some processes of the build are still running. > >> > >> I'm curious how far it gets after you increase MAX_EXECUTION_TIME. > > > > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > > --=20 Nuno Teixeira FreeBSD Committer (ports) --0000000000002d87fc05fce65f39 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Hello Mark,</div><div><br></div><div>Wrong poudriere.= conf. Uploaded correct one.</div><div><br></div><div>Summary:</div><div>---= <br></div>USE_TMPFS=3Dno<div>PARALLEL_JOBS=3D1</div><div>ALLOW_MAKE_JOBS=3D= yes</div><div><div>MAX_EXECUTION_TIME=3D259200<br>NOHANG_TIME=3D259200</div= ><div>---</div><div><br></div><div>htop showed me 4<load<5 (around 4,= 70) with temperature at ~60/65 degrees Celsius</div></div></div><br><div cl= ass=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">Mark Millard <= <a href=3D"mailto:marklmi@yahoo.com">marklmi@yahoo.com</a>> escreveu no = dia ter=C3=A7a, 30/05/2023 =C3=A0(s) 10:47:<br></div><blockquote class=3D"g= mail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204= ,204,204);padding-left:1ex">On May 30, 2023, at 01:39, Nuno Teixeira <<a= href=3D"mailto:eduardo@freebsd.org" target=3D"_blank">eduardo@freebsd.org<= /a>> wrote:<br> <br> > Hello Ronald and all!<br> > <br> > Finally I build it after 57 hours @ 2000MHz !!!<br> <br> So the prior "[88:53:28] Failed ports: www/chromium:build/timeout"= ;<br> was not with the overclocking? Other differences that could<br> contribute to the time difference?<br> <br> MAX_EXECUTION_TIME=3D86400 (the default) only allows for 24 hours<br> before it starts analyzing the log for failure information (while<br> letting the build continue while that scanning is done).<br> <br> > Configs, photos and log at: <a href=3D"https://people.freebsd.org/~edu= ardo/logs/chromium/" rel=3D"noreferrer" target=3D"_blank">https://people.fr= eebsd.org/~eduardo/logs/chromium/</a><br> <br> <a href=3D"https://people.freebsd.org/~eduardo/logs/chromium/poudriere.conf= " rel=3D"noreferrer" target=3D"_blank">https://people.freebsd.org/~eduardo/= logs/chromium/poudriere.conf</a><br> shows no assignments to MAX_EXECUTION_TIME or NOHANG_TIME or<br> other such (outside comments). In the comments are the default<br> values, not increases.<br> <br> Similarly for PARALLEL_JOBS , PREPARE_PARALLEL_JOBS , ALLOW_MAKE_JOBS ,<br> and so on. (It is not clear what the load average behavior/ idle time<br> behavior might have been.)<br> <br> The file does show explicit assignments to: ZPOOL, FREEBSD_HOST ,<br> RESOLV_CONF , BASEFS , USE_PORTLINT , USE_TMPFS , DISTFILES_CACHE ,<br> and PACKAGE_FETCH_URL .<br> <br> > I'm running it right now and it runs very smooth!<br> > <br> > Next step will be compiling with LTO to check compile time and do some= comparisons on binary size and run performance.<br> > <br> > Should I bother with LTO? Will it add performance?<br> > <br> > Thanks all!<br> > <br> > <br> > Ronald Klop <<a href=3D"mailto:ronald-lists@klop.ws" target=3D"_bla= nk">ronald-lists@klop.ws</a>> escreveu no dia ter=C3=A7a, 23/05/2023 =C3= =A0(s) 11:23:<br> >> =C3=82=C2=A0 <br> >>> Van: Nuno Teixeira <<a href=3D"mailto:eduardo@freebsd.org" = target=3D"_blank">eduardo@freebsd.org</a>><br> >>> Datum: dinsdag, 23 mei 2023 09:48<br> >>> Aan: FreeBSD Mailing List <<a href=3D"mailto:freebsd-ports@= freebsd.org" target=3D"_blank">freebsd-ports@freebsd.org</a>>, <a href= =3D"mailto:freebsd-arm@freebsd.org" target=3D"_blank">freebsd-arm@freebsd.o= rg</a><br> >>> Onderwerp: Re: Improving www/chromium build time on arm64<br> >>> (...) <br> >>> =C3=82 <br> >>> But I also found that poudriere:<br> >>> [88:53:28] Failed ports: www/chromium:build/timeout<br> >>> =C3=82 <br> >>> So I will search how do I increase build time to avoid timeout= :<br> >>> ---<br> >>> # This defines the max time (in seconds) that a command may ru= n for a build<br> >>> # before it is killed for taking too long. Default: 86400<br> >>> #MAX_EXECUTION_TIME=3D86400<br> >>> ---<br> >>> =C3=82 <br> >>> I suspect it can be the cause of build failure because build &= gt;24h.<br> >>> =C3=82=C2=A0 <br> >>> Nuno Teixeira <<a href=3D"mailto:eduardo@freebsd.org" targe= t=3D"_blank">eduardo@freebsd.org</a>> escreveu no dia ter=C3=83=C2=A7a, = 23/05/2023 =C3=83 (s) 08:31:<br> >>> Hello all, <br> >>> =C3=82 <br> >>> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is= constantly tab crashing and seg faulting I decided to give chromium a try.= <br> >>> chromium pkg isn't available for arm64 and I didn't fo= und error log at <a href=3D"https://pkg-status.freebsd.org/" rel=3D"norefer= rer" target=3D"_blank">https://pkg-status.freebsd.org/</a>; so I decided to = build it on poudriere.<br> >>> =C3=82 <br> >>> It fails to build around 70% but my biggest concern is build t= ime: 27 hours for 70% compilation!<br> >>> =C3=82 <br> >>> Just for comparison:<br> >>> llvm15: 12h<br> >>> rust: 9h<br> >>> firefox: 6h<br> >>> =C3=82 <br> >>> rip4 v1.5 8GB @ 2000Hz ~62 Celsius<br> >>> zfs<br> >>> poudriere:<br> >>> USE_TMPFS=3Dno<br> >>> PARALLEL_JOBS=3D1<br> >>> ALLOW_MAKE_JOBS=3Dyes<br> >>> =C3=82 <br> >>> Any hints on how to speed build? ccache?<br> >>> Does cflags "-O -pipe" not present in compilation is= related to build time? <br> >>> =C3=82 <br> >>> Full log:<br> >>> <a href=3D"https://people.freebsd.org/~eduardo/logs/chromium/c= hromium-113.0.5672.126.log" rel=3D"noreferrer" target=3D"_blank">https://pe= ople.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.126.log</a><br> >>> =C3=82 <br> >>> Thanks!<br> >>> =C3=82 <br> >>> ---<br> >>> <SNIP><br> >>> In file included from ../../base/check.h:11:<br> >>> ../../base/compiler_specific.h:8:10: fatal error: 'build/b= uild_config.h' file not found<br> >>> #include "build/build_config.h"<br> >>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ^~~~~~~~~~~~~~~~~~~~~~<br> >>> 1 error generated.<br> >>> ninja: build stopped: subcommand failed.<br> >>> =3D=3D=3D> Compilation failed unexpectedly.<br> >>> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting= the failure to<br> >>> the maintainer.<br> >>> *** Error code 1<br> >>> <br> >>> Stop.<br> >>> make: stopped in /usr/ports/www/chromium<br> >>> build of www/chromium | chromium-113.0.5672.126 ended at Tue M= ay 23 02:22:56 WEST 2023<br> >>> build time: 27:15:37<br> >>> !!! build failure encountered !!!<br> >>> ---<br> >>> -- Nuno Teixeira<br> >>> FreeBSD Committer (ports)<br> >>>=C2=A0 =C2=A0=C3=82 <br> >>> -- Nuno Teixeira<br> >>> FreeBSD Committer (ports)<br> >>> <br> >> <br> >> Hi,<br> >> <br> >> My poudriere.conf for building ports on rpi4 contains these two ch= anges:<br> >> <br> >> # This defines the max time (in seconds) that a command may run fo= r a build<br> >> # before it is killed for taking too long. Default: 86400<br> >> #MAX_EXECUTION_TIME=3D86400<br> >> MAX_EXECUTION_TIME=3D172800<br> >> <br> >> # This defines the time (in seconds) before a command is considere= d to<br> >> # be in a runaway state for having no output on stdout. Default: 7= 200<br> >> #NOHANG_TIME=3D7200<br> >> NOHANG_TIME=3D172800<br> >> <br> >> <br> >> I'm maintaining some mongodb ports which also take a long long= time to build on rpi4.<br> >> But I don't remember which of these settings has the preferred= impact on your situation.<br> >> <br> >> Oh... now I'm reading your log better. It contains this:<br> >> "=3D>> Killing timed out build after 86400 seconds"= ;<br> >> <br> >> And some time later:<br> >> "=3D>> Cleaning up wrkdir<br> >> =3D=3D=3D> Cleaning for chromium-113.0.5672.126"<br> >> <br> >> So the errors about missing files happen because poudriere is remo= ving files while some processes of the build are still running.<br> >> <br> >> I'm curious how far it gets after you increase MAX_EXECUTION_T= IME.<br> > <br> <br> =3D=3D=3D<br> Mark Millard<br> marklmi at <a href=3D"http://yahoo.com" rel=3D"noreferrer" target=3D"_blank= ">yahoo.com</a><br> <br> </blockquote></div><br clear=3D"all"><br><span class=3D"gmail_signature_pre= fix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"l= tr"><span style=3D"color:rgb(102,102,102)">Nuno Teixeira<br>FreeBSD Committ= er (ports)</span></div></div> --0000000000002d87fc05fce65f39--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFDf7U%2BxOMkpLOUYHipHyOp-vw96--JKV4PxfXE8DkKCZCSMGw>