Date: Thu, 24 Aug 2017 20:15:34 +0200 From: Dimitry Andric <dim@FreeBSD.org> To: =?utf-8?Q?Fernando_Apestegu=C3=ADa?= <fernando.apesteguia@gmail.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org>, Ed Maste <emaste@freebsd.org>, Benjamin Kramer <benny.kra@gmail.com>, Hans Wennborg <hans@chromium.org> Subject: Re: Runaway process in -CURRENT Message-ID: <EF12ACA3-3E86-4DCA-AC0A-E7D5E6D27438@FreeBSD.org> In-Reply-To: <CAGwOe2Y1wFZSr0zEsugzPDoTA2b6oznG=ggZwfSTWoi5h173WA@mail.gmail.com> References: <CAGwOe2aCU7HigZJdx2YJCrDZsWUtRVApFzzvPTx%2BLE-AP_B4qA@mail.gmail.com> <71697463-4E52-4B46-B60B-00BDD87D8A2C@FreeBSD.org> <CAGwOe2aODZ3skAEVn9kF3ruKFQtQjFfUk_sa2_hJp99YYgnKsQ@mail.gmail.com> <CAGwOe2bkkNye_B4OjG9e0p4Dnp3EbRyP0=vh6yhfHBQ_6r-7yQ@mail.gmail.com> <CAGwOe2awSX9Svh9Y-3Ew_PMY1=srHLD01CKfqDzeuXNMA3KS5g@mail.gmail.com> <CAGwOe2Y1wFZSr0zEsugzPDoTA2b6oznG=ggZwfSTWoi5h173WA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_CA0A6D5A-0480-4BCC-8553-52B4A5ED8945 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 On 24 Aug 2017, at 16:42, Fernando Apestegu=C3=ADa = <fernando.apesteguia@gmail.com> wrote: >=20 > On Thu, Aug 24, 2017 at 11:14 AM, Fernando Apestegu=C3=ADa > <fernando.apesteguia@gmail.com> wrote: ... > Just for the record, I got another email. Log URL: >=20 > = http://beefy11.nyi.freebsd.org/data/head-i386-default/p448640_s322824/logs= /stepcode-0.8_1.log ... > schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o -MF > schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o.d -o > schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o -c > schemas/sdai_wip210e3/SdaiAll.cc > =3D=3D=3D=3D>> Killing runaway build after 7200 seconds with no output Hm, I had not noticed two facts, one being that there are a great many copies of SdaiAll.cc files, which all seem to be generated, and the other that this is occurring on i386. There are also other generated files, and specifically the compstructs.cc files can take extremely long to compile. On amd64, the longest compile time in the whole port is ~185 seconds, for a generated file of ~28000 lines: time lines filename =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 184.90 28345 schemas/sdai_wip210e3/compstructs.cc 115.08 103 = schemas/sdai_ap239/SdaiAP239_PRODUCT_LIFE_CYCLE_SUPPORT_ARM_LF_unity_types= .cc 110.00 440 = schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_AND_DESIGN_MIM_LF_= unity_types.cc 106.70 292 = schemas/sdai_wip210e3/SdaiAP210_ELECTRONIC_ASSEMBLY_INTERCONNECT_AND_PACKA= GING_DESIGN_MIM_LF_unity_types.cc 90.73 290 = schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGINEERING_MIM_LF_uni= ty_types.cc 88.77 2232 = schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_AND_DESIGN_MIM_LF_= unity_entities.cc 81.87 2174 = schemas/sdai_wip210e3/SdaiAP210_ELECTRONIC_ASSEMBLY_INTERCONNECT_AND_PACKA= GING_DESIGN_MIM_LF_unity_entities.cc 73.92 19658 schemas/sdai_cd209/compstructs.cc 64.82 1734 = schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGINEERING_MIM_LF_uni= ty_entities.cc 61.42 189 = schemas/sdai_ap210e2/SdaiAP210_ELECTRONIC_ASSEMBLY_INTERCONNECT_AND_PACKAG= ING_DESIGN_MIM_LF_unity_types.cc On i386, this time is enormously increased, especially for the files with lots of lines: time lines filename =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 4809.23 28345 schemas/sdai_wip210e3/compstructs.cc 2164.37 19658 schemas/sdai_cd209/compstructs.cc 2056.54 15806 schemas/sdai_ap210e2/compstructs.cc 1545.19 16679 schemas/sdai_cd242/compstructs.cc 446.49 7667 schemas/sdai_ap203e2/compstructs.cc 273.96 6297 schemas/sdai_ap214e3/compstructs.cc 187.16 103 = schemas/sdai_ap239/SdaiAP239_PRODUCT_LIFE_CYCLE_SUPPORT_ARM_LF_unity_types= .cc 163.39 440 = schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_AND_DESIGN_MIM_LF_= unity_types.cc 130.02 2232 = schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_AND_DESIGN_MIM_LF_= unity_entities.cc 122.15 290 = schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGINEERING_MIM_LF_uni= ty_types.cc The compstructs.cc and SdaiAll.cc files contain *extremely* large generated functions, in most cases embodying the whole file, even. It seems that this is the cause for the long compile times. A test case tarball with the preprocessed version of the top compstructs.cc file, and a shell script to run it, is here: http://www.andric.com/freebsd/clang/stepcode.tar.xz With clang 4.0.0, the compile times are more at the level of the amd64 builds, even on i386. So this regressed somewhere along the way to 5.0.0. I will have to try to find out where, to see if there is an existing bug report or workaround. Note that in r321664, about three weeks ago, I already committed a couple of upstream fixes for some situations where compile times could get excessive (see also https://bugs.llvm.org/show_bug.cgi?id=3D33900), but it doesn't seem to have helped in this case. As a quick workaround, the port can be modified to compile these files using a lower optimization level. If -O1 is used for the longest case, e.g. schemas/sdai_wip210e3/compstructs.cc, the compile time drops down to ~30s. I don't think it will matter very much for the speed or size of the output. -Dimitry --Apple-Mail=_CA0A6D5A-0480-4BCC-8553-52B4A5ED8945 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.1 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCWZ8XxgAKCRCwXqMKLiCW o5poAJ0cJ1pF6j7vSKpYl9y3dsyrKg6qEgCgtwYrJQRKKuZlsahlStSK121oOFs= =wlbL -----END PGP SIGNATURE----- --Apple-Mail=_CA0A6D5A-0480-4BCC-8553-52B4A5ED8945--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?EF12ACA3-3E86-4DCA-AC0A-E7D5E6D27438>