Date: Thu, 24 Aug 2017 20:27:46 +0200 From: =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= <fernando.apesteguia@gmail.com> To: Dimitry Andric <dim@freebsd.org> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org>, Ed Maste <emaste@freebsd.org>, Benjamin Kramer <benny.kra@gmail.com>, Hans Wennborg <hans@chromium.org> Subject: Re: Runaway process in -CURRENT Message-ID: <CAGwOe2ZBUDqSXzofkPESkv0q=cCqp%2BCGRMLXERUq7PYBA0znrg@mail.gmail.com> In-Reply-To: <EF12ACA3-3E86-4DCA-AC0A-E7D5E6D27438@FreeBSD.org> References: <CAGwOe2aCU7HigZJdx2YJCrDZsWUtRVApFzzvPTx%2BLE-AP_B4qA@mail.gmail.com> <71697463-4E52-4B46-B60B-00BDD87D8A2C@FreeBSD.org> <CAGwOe2aODZ3skAEVn9kF3ruKFQtQjFfUk_sa2_hJp99YYgnKsQ@mail.gmail.com> <CAGwOe2bkkNye_B4OjG9e0p4Dnp3EbRyP0=vh6yhfHBQ_6r-7yQ@mail.gmail.com> <CAGwOe2awSX9Svh9Y-3Ew_PMY1=srHLD01CKfqDzeuXNMA3KS5g@mail.gmail.com> <CAGwOe2Y1wFZSr0zEsugzPDoTA2b6oznG=ggZwfSTWoi5h173WA@mail.gmail.com> <EF12ACA3-3E86-4DCA-AC0A-E7D5E6D27438@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Aug 24, 2017 at 8:15 PM, Dimitry Andric <dim@freebsd.org> wrote: > On 24 Aug 2017, at 16:42, Fernando Apestegu=C3=ADa <fernando.apesteguia@g= mail.com> wrote: >> >> On Thu, Aug 24, 2017 at 11:14 AM, Fernando Apestegu=C3=ADa >> <fernando.apesteguia@gmail.com> wrote: > ... >> Just for the record, I got another email. Log URL: >> >> http://beefy11.nyi.freebsd.org/data/head-i386-default/p448640_s322824/lo= gs/stepcode-0.8_1.log > ... >> schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o -MF >> schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o.d -o >> schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o -c >> schemas/sdai_wip210e3/SdaiAll.cc >> =3D=3D=3D=3D>> Killing runaway build after 7200 seconds with no output > > Hm, I had not noticed two facts, one being that there are a great many > copies of SdaiAll.cc files, which all seem to be generated, and the > other that this is occurring on i386. There are also other generated > files, and specifically the compstructs.cc files can take extremely long > to compile. > > On amd64, the longest compile time in the whole port is ~185 seconds, > for a generated file of ~28000 lines: > > time lines filename > =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D > 184.90 28345 schemas/sdai_wip210e3/compstructs.cc > 115.08 103 schemas/sdai_ap239/SdaiAP239_PRODUCT_LIFE_CYCLE_SUPPORT_A= RM_LF_unity_types.cc > 110.00 440 schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_A= ND_DESIGN_MIM_LF_unity_types.cc > 106.70 292 schemas/sdai_wip210e3/SdaiAP210_ELECTRONIC_ASSEMBLY_INTER= CONNECT_AND_PACKAGING_DESIGN_MIM_LF_unity_types.cc > 90.73 290 schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGIN= EERING_MIM_LF_unity_types.cc > 88.77 2232 schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_A= ND_DESIGN_MIM_LF_unity_entities.cc > 81.87 2174 schemas/sdai_wip210e3/SdaiAP210_ELECTRONIC_ASSEMBLY_INTER= CONNECT_AND_PACKAGING_DESIGN_MIM_LF_unity_entities.cc > 73.92 19658 schemas/sdai_cd209/compstructs.cc > 64.82 1734 schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGIN= EERING_MIM_LF_unity_entities.cc > 61.42 189 schemas/sdai_ap210e2/SdaiAP210_ELECTRONIC_ASSEMBLY_INTERC= ONNECT_AND_PACKAGING_DESIGN_MIM_LF_unity_types.cc > > On i386, this time is enormously increased, especially for the files > with lots of lines: > > time lines filename > =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D > 4809.23 28345 schemas/sdai_wip210e3/compstructs.cc > 2164.37 19658 schemas/sdai_cd209/compstructs.cc > 2056.54 15806 schemas/sdai_ap210e2/compstructs.cc > 1545.19 16679 schemas/sdai_cd242/compstructs.cc > 446.49 7667 schemas/sdai_ap203e2/compstructs.cc > 273.96 6297 schemas/sdai_ap214e3/compstructs.cc > 187.16 103 schemas/sdai_ap239/SdaiAP239_PRODUCT_LIFE_CYCLE_SUPPORT_A= RM_LF_unity_types.cc > 163.39 440 schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_A= ND_DESIGN_MIM_LF_unity_types.cc > 130.02 2232 schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_A= ND_DESIGN_MIM_LF_unity_entities.cc > 122.15 290 schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGIN= EERING_MIM_LF_unity_types.cc > I did some tests: -CURRENT (i326) in VirtualBox, 1 CPU, 8Gb RAM cd /usr/ports/cad/stepcode && time make * gcc 6.4.0 real 27m17.867s user 50m.0.428s sys 2m47.619 * clang38 real 18m28.996s user 33m22.790s sys 1m22.711s * clang 5.0.0 It takes a lot of time compiling schemas/sdai_{wip210e3,ap210e2}/SdaiAll.cc and other SdaAll.cc files. It's still compiling after several hours. > The compstructs.cc and SdaiAll.cc files contain *extremely* large > generated functions, in most cases embodying the whole file, even. It > seems that this is the cause for the long compile times. A test case > tarball with the preprocessed version of the top compstructs.cc file, > and a shell script to run it, is here: > http://www.andric.com/freebsd/clang/stepcode.tar.xz > > With clang 4.0.0, the compile times are more at the level of the amd64 > builds, even on i386. So this regressed somewhere along the way to > 5.0.0. I will have to try to find out where, to see if there is an > existing bug report or workaround. > > Note that in r321664, about three weeks ago, I already committed a > couple of upstream fixes for some situations where compile times could > get excessive (see also https://bugs.llvm.org/show_bug.cgi?id=3D33900), > but it doesn't seem to have helped in this case. > > As a quick workaround, the port can be modified to compile these files > using a lower optimization level. If -O1 is used for the longest case, > e.g. schemas/sdai_wip210e3/compstructs.cc, the compile time drops down > to ~30s. I don't think it will matter very much for the speed or size > of the output. I'll have a look at this. Thanks! > > -Dimitry >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGwOe2ZBUDqSXzofkPESkv0q=cCqp%2BCGRMLXERUq7PYBA0znrg>