Date: Sat, 1 Apr 2017 03:51:10 -0700 From: Mark Millard <markmi@dsl-only.net> To: Brooks Davis <brooks@freebsd.org>, Dimitry Andric <dim@FreeBSD.org>, FreeBSD Ports <freebsd-ports@freebsd.org> Cc: FreeBSD Current <freebsd-current@freebsd.org>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Subject: Re: FYI: what it takes for RAM+swap to build devel/llvm40 with 4 processors or cores and WITH__DEBUG= (powerpc64 example) Message-ID: <FD405CB7-CF1C-4F81-A5A8-46FACF886576@dsl-only.net> In-Reply-To: <F315A080-0989-4521-A899-A0A06721934B@dsl-only.net> References: <3EDEF0B7-59C5-4648-9737-6682E18645BC@dsl-only.net> <39C60316-F905-490D-B0AB-BC24D7F351A2@dsl-only.net> <7F94CE59-D2CC-4D6F-B1CD-FF3D1F8EDCE7@FreeBSD.org> <45E32F4F-A238-47AA-8E1E-7AD4D9E857D9@dsl-only.net> <20170329155316.GK59667@spindle.one-eyed-alien.net> <F6F4B285-5D72-4B14-A12E-3D789AED063C@dsl-only.net> <FDFF229D-B9DB-4798-8743-01CAF218BD09@dsl-only.net> <F315A080-0989-4521-A899-A0A06721934B@dsl-only.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2017-Mar-31, at 4:51 PM, Mark Millard <markmi at dsl-only.net> wrote:
> On 2017-Mar-30, at 7:51 PM, Mark Millard <markmi at dsl-only.net> =
wrote:
>=20
>> On 2017-Mar-30, at 1:22 PM, Mark Millard <markmi at dsl-only.net> =
wrote:
>>=20
>>> Sounds like the ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG technique
>>> would not change the "WITNESS and INVARIANTS"-like part of the
>>> issue. In fact if WITH_DEBUG=3D causes the cmake debug-style
>>> llvm40 build ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG might not
>>> make any difference: separate enforcing of lack of optimization.
>>>=20
>>> But just to see what results I've done "pkg delete llvm40"
>>> and am doing another build with ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG=3D
>>> and its supporting code in place in addition to using WITH_DEBUG=3D
>>> as the type of build fro FreeBSD's viewpoint.
>>>=20
>>> If you know that the test is a waste of machine cycles, you can
>>> let me know if you want.
>>=20
>> The experiment showed that ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG
>> use made no difference for devel/llvm40 so devel/llvm40 itself
>> has to change such as what Dimitry Andric reported separately
>> as a working change to the Makefile .
>>=20
>> (ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG would still have its uses
>> for various other ports.)
>=20
> I've now tried with both ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG and:
I may have had a textual error that prevented
ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG from even potentially
contributing. So I'll re-run this test.
For now I presume that what I reported was okay and so
I continue to refer to these figures later below.
> # svnlite diff /usr/ports/devel/llvm40/
> Index: /usr/ports/devel/llvm40/Makefile
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> --- /usr/ports/devel/llvm40/Makefile (revision 436747)
> +++ /usr/ports/devel/llvm40/Makefile (working copy)
> @@ -236,6 +236,11 @@
>=20
> .include <bsd.port.pre.mk>
>=20
> +.if defined(WITH_DEBUG)
> +CMAKE_BUILD_TYPE=3D RelWithDebInfo
> +STRIP=3D
> +.endif
> +
> _CRTLIBDIR=3D =
${LLVM_PREFIX:S|${PREFIX}/||}/lib/clang/${LLVM_RELEASE}/lib/freebsd
> .if ${ARCH} =3D=3D "amd64"
> _COMPILER_RT_LIBS=3D \
>=20
>=20
>=20
> pkg delete after the build reports:
>=20
> Installed packages to be REMOVED:
> llvm40-4.0.0
>=20
> Number of packages to be removed: 1
>=20
> The operation will free 42 GiB.
>=20
> So down by 7 GiBytes from 49 GiBytes.
>=20
> (I did not actually delete it.)
>=20
> Also:
>=20
> # du -sg /usr/obj/portswork/usr/ports/devel/llvm40
> 102 /usr/obj/portswork/usr/ports/devel/llvm40
>=20
> which is down by 16 GiBytes from 118 GiBytes.
>=20
> Reminder: These are from portmaster -DK so no
> cleanup after the build, which is what leaves
> the source code and such around in case of
> needing to look at a problem.
>=20
> (102+42) GiBytes =3D=3D 146 GiBytes.
> vs.
> (118+49) GiBytes =3D=3D 167 GiBytes.
>=20
> So a difference of 21 GiBytes (or so).
>=20
> But that is for everything in each case (and
> WITH_DEBUG=3D in use):
>=20
> # more /var/db/ports/devel_llvm40/options
> # This file is auto-generated by 'make config'.
> # Options for llvm40-4.0.0.r4
> _OPTIONS_READ=3Dllvm40-4.0.0.r4
> _FILE_COMPLETE_OPTIONS_LIST=3DCLANG DOCS EXTRAS LIT LLD LLDB
> OPTIONS_FILE_SET+=3DCLANG
> OPTIONS_FILE_SET+=3DDOCS
> OPTIONS_FILE_SET+=3DEXTRAS
> OPTIONS_FILE_SET+=3DLIT
> OPTIONS_FILE_SET+=3DLLD
> OPTIONS_FILE_SET+=3DLLDB
>=20
> So avoiding WITH_DEBUG=3D and/or various build options
> is still the major way of avoiding use of lots of space
> if it is an issue.
>=20
>=20
>=20
> Why no RAM+SWAP total report this time:
>=20
> As far as I know FreeBSD does not track or report peak
> swap-space usage since the last boot. And, unfortunately
> I was not around to just sit and watch a top display this
> time and I did not set up any periodic recording into a
> file.
>=20
> That is why I've not reported on the RAM+SWAP total
> this time. It will have to be another experiment
> some other time.
>=20
> [I do wish FreeBSD had a way of reporting peak swap-space
> usage.]
I've also tried without WITH_DEBUG=3D and now. . .
# pkg delete llvm40
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0 =
packages in the universe):
Installed packages to be REMOVED:
llvm40-4.0.0
Number of packages to be removed: 1
The operation will free 1 GiB.
Proceed with deinstalling packages? [y/N]: n
# du -sg /usr/obj/portswork/usr/ports/devel/llvm40/
5 /usr/obj/portswork/usr/ports/devel/llvm40/
So the alternatives (with everything built each time):
(5+1) GiBytes =3D=3D 6 GiBytes. (without WITH_DEBUG=3D)
vs.
(102+42) GiBytes =3D=3D 146 GiBytes. (WITH_DEBUG=3D but the adjusted =
llvm40/Makefiele)
vs.
(118+49) GiBytes =3D=3D 167 GiBytes. (WITH_DEBUG=3D with Makefile =
adjustment)
I'll likely end up having /etc/make.conf contain
something like the following for most or all of
my FreeBSD environments:
#
# =46rom a local /usr/ports/Mk/bsd.port.mk extension:
ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG=3D
#
.if ${.CURDIR:M*/devel/llvm*}
#WITH_DEBUG=3D
.else
WITH_DEBUG=3D
.endif
WITH_DEBUG_FILES=3D
Along with using:
# svnlite diff /usr/ports/Mk/
Index: /usr/ports/Mk/bsd.port.mk
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- /usr/ports/Mk/bsd.port.mk (revision 436747)
+++ /usr/ports/Mk/bsd.port.mk (working copy)
@@ -1646,7 +1646,11 @@
STRIP_CMD=3D ${TRUE}
.endif
DEBUG_FLAGS?=3D -g
+.if defined(ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG)
+CFLAGS:=3D ${CFLAGS} ${DEBUG_FLAGS}
+.else
CFLAGS:=3D ${CFLAGS:N-O*:N-fno-strict*} ${DEBUG_FLAGS}
+.endif
.if defined(INSTALL_TARGET)
INSTALL_TARGET:=3D ${INSTALL_TARGET:S/^install-strip$/install/g}
.endif
[Although apparently the ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG
use has no effect for devel/llvm40 .]
=3D=3D=3D
Mark Millard
markmi at dsl-only.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?FD405CB7-CF1C-4F81-A5A8-46FACF886576>