Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 24 Aug 2017 20:15:34 +0200
From:      Dimitry Andric <dim@FreeBSD.org>
To:        =?utf-8?Q?Fernando_Apestegu=C3=ADa?= <fernando.apesteguia@gmail.com>
Cc:        FreeBSD Hackers <freebsd-hackers@freebsd.org>, Ed Maste <emaste@freebsd.org>, Benjamin Kramer <benny.kra@gmail.com>, Hans Wennborg <hans@chromium.org>
Subject:   Re: Runaway process in -CURRENT
Message-ID:  <EF12ACA3-3E86-4DCA-AC0A-E7D5E6D27438@FreeBSD.org>
In-Reply-To: <CAGwOe2Y1wFZSr0zEsugzPDoTA2b6oznG=ggZwfSTWoi5h173WA@mail.gmail.com>
References:  <CAGwOe2aCU7HigZJdx2YJCrDZsWUtRVApFzzvPTx%2BLE-AP_B4qA@mail.gmail.com> <71697463-4E52-4B46-B60B-00BDD87D8A2C@FreeBSD.org> <CAGwOe2aODZ3skAEVn9kF3ruKFQtQjFfUk_sa2_hJp99YYgnKsQ@mail.gmail.com> <CAGwOe2bkkNye_B4OjG9e0p4Dnp3EbRyP0=vh6yhfHBQ_6r-7yQ@mail.gmail.com> <CAGwOe2awSX9Svh9Y-3Ew_PMY1=srHLD01CKfqDzeuXNMA3KS5g@mail.gmail.com> <CAGwOe2Y1wFZSr0zEsugzPDoTA2b6oznG=ggZwfSTWoi5h173WA@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help

--Apple-Mail=_CA0A6D5A-0480-4BCC-8553-52B4A5ED8945
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

On 24 Aug 2017, at 16:42, Fernando Apestegu=C3=ADa =
<fernando.apesteguia@gmail.com> wrote:
>=20
> On Thu, Aug 24, 2017 at 11:14 AM, Fernando Apestegu=C3=ADa
> <fernando.apesteguia@gmail.com> wrote:
...
> Just for the record, I got another email. Log URL:
>=20
> =
http://beefy11.nyi.freebsd.org/data/head-i386-default/p448640_s322824/logs=
/stepcode-0.8_1.log
...
> schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o -MF
> schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o.d -o
> schemas/sdai_wip210e3/CMakeFiles/sdai_wip210e3.dir/SdaiAll.cc.o -c
> schemas/sdai_wip210e3/SdaiAll.cc
> =3D=3D=3D=3D>> Killing runaway build after 7200 seconds with no output

Hm, I had not noticed two facts, one being that there are a great many
copies of SdaiAll.cc files, which all seem to be generated, and the
other that this is occurring on i386.  There are also other generated
files, and specifically the compstructs.cc files can take extremely long
to compile.

On amd64, the longest compile time in the whole port is ~185 seconds,
for a generated file of ~28000 lines:

time     lines  filename
=3D=3D=3D=3D=3D=3D=3D  =3D=3D=3D=3D=3D  =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
 184.90  28345  schemas/sdai_wip210e3/compstructs.cc
 115.08    103  =
schemas/sdai_ap239/SdaiAP239_PRODUCT_LIFE_CYCLE_SUPPORT_ARM_LF_unity_types=
.cc
 110.00    440  =
schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_AND_DESIGN_MIM_LF_=
unity_types.cc
 106.70    292  =
schemas/sdai_wip210e3/SdaiAP210_ELECTRONIC_ASSEMBLY_INTERCONNECT_AND_PACKA=
GING_DESIGN_MIM_LF_unity_types.cc
  90.73    290  =
schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGINEERING_MIM_LF_uni=
ty_types.cc
  88.77   2232  =
schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_AND_DESIGN_MIM_LF_=
unity_entities.cc
  81.87   2174  =
schemas/sdai_wip210e3/SdaiAP210_ELECTRONIC_ASSEMBLY_INTERCONNECT_AND_PACKA=
GING_DESIGN_MIM_LF_unity_entities.cc
  73.92  19658  schemas/sdai_cd209/compstructs.cc
  64.82   1734  =
schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGINEERING_MIM_LF_uni=
ty_entities.cc
  61.42    189  =
schemas/sdai_ap210e2/SdaiAP210_ELECTRONIC_ASSEMBLY_INTERCONNECT_AND_PACKAG=
ING_DESIGN_MIM_LF_unity_types.cc

On i386, this time is enormously increased, especially for the files
with lots of lines:

time     lines  filename
=3D=3D=3D=3D=3D=3D=3D  =3D=3D=3D=3D=3D  =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
4809.23  28345  schemas/sdai_wip210e3/compstructs.cc
2164.37  19658  schemas/sdai_cd209/compstructs.cc
2056.54  15806  schemas/sdai_ap210e2/compstructs.cc
1545.19  16679  schemas/sdai_cd242/compstructs.cc
 446.49   7667  schemas/sdai_ap203e2/compstructs.cc
 273.96   6297  schemas/sdai_ap214e3/compstructs.cc
 187.16    103  =
schemas/sdai_ap239/SdaiAP239_PRODUCT_LIFE_CYCLE_SUPPORT_ARM_LF_unity_types=
.cc
 163.39    440  =
schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_AND_DESIGN_MIM_LF_=
unity_types.cc
 130.02   2232  =
schemas/sdai_cd209/SdaiAP209_MULTIDISCIPLINARY_ANALYSIS_AND_DESIGN_MIM_LF_=
unity_entities.cc
 122.15    290  =
schemas/sdai_cd242/SdaiAP242_MANAGED_MODEL_BASED_3D_ENGINEERING_MIM_LF_uni=
ty_types.cc

The compstructs.cc and SdaiAll.cc files contain *extremely* large
generated functions, in most cases embodying the whole file, even.  It
seems that this is the cause for the long compile times.  A test case
tarball with the preprocessed version of the top compstructs.cc file,
and a shell script to run it, is here:
http://www.andric.com/freebsd/clang/stepcode.tar.xz

With clang 4.0.0, the compile times are more at the level of the amd64
builds, even on i386.  So this regressed somewhere along the way to
5.0.0.  I will have to try to find out where, to see if there is an
existing bug report or workaround.

Note that in r321664, about three weeks ago, I already committed a
couple of upstream fixes for some situations where compile times could
get excessive (see also https://bugs.llvm.org/show_bug.cgi?id=3D33900),
but it doesn't seem to have helped in this case.

As a quick workaround, the port can be modified to compile these files
using a lower optimization level.  If -O1 is used for the longest case,
e.g. schemas/sdai_wip210e3/compstructs.cc, the compile time drops down
to ~30s.  I don't think it will matter very much for the speed or size
of the output.

-Dimitry


--Apple-Mail=_CA0A6D5A-0480-4BCC-8553-52B4A5ED8945
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.1

iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCWZ8XxgAKCRCwXqMKLiCW
o5poAJ0cJ1pF6j7vSKpYl9y3dsyrKg6qEgCgtwYrJQRKKuZlsahlStSK121oOFs=
=wlbL
-----END PGP SIGNATURE-----

--Apple-Mail=_CA0A6D5A-0480-4BCC-8553-52B4A5ED8945--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?EF12ACA3-3E86-4DCA-AC0A-E7D5E6D27438>