From owner-freebsd-ppc@freebsd.org Sat Apr 1 10:51:14 2017 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F842D2915A for ; Sat, 1 Apr 2017 10:51:14 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-79.reflexion.net [208.70.210.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3539531D for ; Sat, 1 Apr 2017 10:51:12 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 19734 invoked from network); 1 Apr 2017 10:51:11 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 1 Apr 2017 10:51:11 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Sat, 01 Apr 2017 06:51:11 -0400 (EDT) Received: (qmail 23165 invoked from network); 1 Apr 2017 10:51:11 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 1 Apr 2017 10:51:11 -0000 Received: from [192.168.1.119] (c-67-170-167-181.hsd1.or.comcast.net [67.170.167.181]) by iron2.pdx.net (Postfix) with ESMTPSA id 97D6FEC8630; Sat, 1 Apr 2017 03:51:10 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: FYI: what it takes for RAM+swap to build devel/llvm40 with 4 processors or cores and WITH__DEBUG= (powerpc64 example) From: Mark Millard In-Reply-To: Date: Sat, 1 Apr 2017 03:51:10 -0700 Cc: FreeBSD Current , FreeBSD PowerPC ML , FreeBSD Toolchain Content-Transfer-Encoding: quoted-printable Message-Id: References: <3EDEF0B7-59C5-4648-9737-6682E18645BC@dsl-only.net> <39C60316-F905-490D-B0AB-BC24D7F351A2@dsl-only.net> <7F94CE59-D2CC-4D6F-B1CD-FF3D1F8EDCE7@FreeBSD.org> <45E32F4F-A238-47AA-8E1E-7AD4D9E857D9@dsl-only.net> <20170329155316.GK59667@spindle.one-eyed-alien.net> To: Brooks Davis , Dimitry Andric , FreeBSD Ports X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Apr 2017 10:51:14 -0000 On 2017-Mar-31, at 4:51 PM, Mark Millard wrote: > On 2017-Mar-30, at 7:51 PM, Mark Millard = wrote: >=20 >> On 2017-Mar-30, at 1:22 PM, Mark Millard = wrote: >>=20 >>> Sounds like the ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG technique >>> would not change the "WITNESS and INVARIANTS"-like part of the >>> issue. In fact if WITH_DEBUG=3D causes the cmake debug-style >>> llvm40 build ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG might not >>> make any difference: separate enforcing of lack of optimization. >>>=20 >>> But just to see what results I've done "pkg delete llvm40" >>> and am doing another build with ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG=3D >>> and its supporting code in place in addition to using WITH_DEBUG=3D >>> as the type of build fro FreeBSD's viewpoint. >>>=20 >>> If you know that the test is a waste of machine cycles, you can >>> let me know if you want. >>=20 >> The experiment showed that ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG >> use made no difference for devel/llvm40 so devel/llvm40 itself >> has to change such as what Dimitry Andric reported separately >> as a working change to the Makefile . >>=20 >> (ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG would still have its uses >> for various other ports.) >=20 > I've now tried with both ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG and: I may have had a textual error that prevented ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG from even potentially contributing. So I'll re-run this test. For now I presume that what I reported was okay and so I continue to refer to these figures later below. > # svnlite diff /usr/ports/devel/llvm40/ > Index: /usr/ports/devel/llvm40/Makefile > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- /usr/ports/devel/llvm40/Makefile (revision 436747) > +++ /usr/ports/devel/llvm40/Makefile (working copy) > @@ -236,6 +236,11 @@ >=20 > .include >=20 > +.if defined(WITH_DEBUG) > +CMAKE_BUILD_TYPE=3D RelWithDebInfo > +STRIP=3D > +.endif > + > _CRTLIBDIR=3D = ${LLVM_PREFIX:S|${PREFIX}/||}/lib/clang/${LLVM_RELEASE}/lib/freebsd > .if ${ARCH} =3D=3D "amd64" > _COMPILER_RT_LIBS=3D \ >=20 >=20 >=20 > pkg delete after the build reports: >=20 > Installed packages to be REMOVED: > llvm40-4.0.0 >=20 > Number of packages to be removed: 1 >=20 > The operation will free 42 GiB. >=20 > So down by 7 GiBytes from 49 GiBytes. >=20 > (I did not actually delete it.) >=20 > Also: >=20 > # du -sg /usr/obj/portswork/usr/ports/devel/llvm40 > 102 /usr/obj/portswork/usr/ports/devel/llvm40 >=20 > which is down by 16 GiBytes from 118 GiBytes. >=20 > Reminder: These are from portmaster -DK so no > cleanup after the build, which is what leaves > the source code and such around in case of > needing to look at a problem. >=20 > (102+42) GiBytes =3D=3D 146 GiBytes. > vs. > (118+49) GiBytes =3D=3D 167 GiBytes. >=20 > So a difference of 21 GiBytes (or so). >=20 > But that is for everything in each case (and > WITH_DEBUG=3D in use): >=20 > # more /var/db/ports/devel_llvm40/options > # This file is auto-generated by 'make config'. > # Options for llvm40-4.0.0.r4 > _OPTIONS_READ=3Dllvm40-4.0.0.r4 > _FILE_COMPLETE_OPTIONS_LIST=3DCLANG DOCS EXTRAS LIT LLD LLDB > OPTIONS_FILE_SET+=3DCLANG > OPTIONS_FILE_SET+=3DDOCS > OPTIONS_FILE_SET+=3DEXTRAS > OPTIONS_FILE_SET+=3DLIT > OPTIONS_FILE_SET+=3DLLD > OPTIONS_FILE_SET+=3DLLDB >=20 > So avoiding WITH_DEBUG=3D and/or various build options > is still the major way of avoiding use of lots of space > if it is an issue. >=20 >=20 >=20 > Why no RAM+SWAP total report this time: >=20 > As far as I know FreeBSD does not track or report peak > swap-space usage since the last boot. And, unfortunately > I was not around to just sit and watch a top display this > time and I did not set up any periodic recording into a > file. >=20 > That is why I've not reported on the RAM+SWAP total > this time. It will have to be another experiment > some other time. >=20 > [I do wish FreeBSD had a way of reporting peak swap-space > usage.] I've also tried without WITH_DEBUG=3D and now. . . # pkg delete llvm40 Checking integrity... done (0 conflicting) Deinstallation has been requested for the following 1 packages (of 0 = packages in the universe): Installed packages to be REMOVED: llvm40-4.0.0 Number of packages to be removed: 1 The operation will free 1 GiB. Proceed with deinstalling packages? [y/N]: n # du -sg /usr/obj/portswork/usr/ports/devel/llvm40/ 5 /usr/obj/portswork/usr/ports/devel/llvm40/ So the alternatives (with everything built each time): (5+1) GiBytes =3D=3D 6 GiBytes. (without WITH_DEBUG=3D) vs. (102+42) GiBytes =3D=3D 146 GiBytes. (WITH_DEBUG=3D but the adjusted = llvm40/Makefiele) vs. (118+49) GiBytes =3D=3D 167 GiBytes. (WITH_DEBUG=3D with Makefile = adjustment) I'll likely end up having /etc/make.conf contain something like the following for most or all of my FreeBSD environments: # # =46rom a local /usr/ports/Mk/bsd.port.mk extension: ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG=3D # .if ${.CURDIR:M*/devel/llvm*} #WITH_DEBUG=3D .else WITH_DEBUG=3D .endif WITH_DEBUG_FILES=3D Along with using: # svnlite diff /usr/ports/Mk/ Index: /usr/ports/Mk/bsd.port.mk =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/ports/Mk/bsd.port.mk (revision 436747) +++ /usr/ports/Mk/bsd.port.mk (working copy) @@ -1646,7 +1646,11 @@ STRIP_CMD=3D ${TRUE} .endif DEBUG_FLAGS?=3D -g +.if defined(ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG) +CFLAGS:=3D ${CFLAGS} ${DEBUG_FLAGS} +.else CFLAGS:=3D ${CFLAGS:N-O*:N-fno-strict*} ${DEBUG_FLAGS} +.endif .if defined(INSTALL_TARGET) INSTALL_TARGET:=3D ${INSTALL_TARGET:S/^install-strip$/install/g} .endif [Although apparently the ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG use has no effect for devel/llvm40 .] =3D=3D=3D Mark Millard markmi at dsl-only.net