Date: Wed, 9 Apr 2014 18:17:00 +0000 (UTC) From: Dimitry Andric <dim@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org Subject: svn commit: r264303 - in stable: 10/bin 10/gnu/usr.bin 10/lib 10/lib/clang 10/sbin 10/share/mk 10/usr.bin 10/usr.bin/clang 10/usr.sbin 8/bin 8/gnu/usr.bin 8/lib 8/sbin 8/share/mk 8/usr.bin 8/usr.sb... Message-ID: <201404091817.s39IH0a3067606@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: dim Date: Wed Apr 9 18:16:58 2014 New Revision: 264303 URL: http://svnweb.freebsd.org/changeset/base/264303 Log: MFC r263778: Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process all the SUBDIR entries in parallel, instead of serially. Apply this option to a selected number of Makefiles, which can greatly speed up the build on multi-core machines, when using make -j. This can be extended to more Makefiles later on, whenever they are verified to work correctly with parallel building. I tested this on a 24-core machine, with make -j48 buildworld (N = 6): before stddev after stddev ======= ====== ======= ====== real time 1741.1 16.5 959.8 2.7 user time 12468.7 16.4 14393.0 16.8 sys time 1825.0 54.8 2110.6 22.8 (user+sys)/real 8.2 17.1 E.g. the build was approximately 45% faster in real time. On machines with less cores, or with lower -j settings, the speedup will not be as impressive. But at least you can now almost max out a machine with buildworld! Submitted by: jilles MFC r263833: Enable parallel building for gnu/usr.bin and usr.bin/clang too. Modified: stable/8/bin/Makefile stable/8/gnu/usr.bin/Makefile (contents, props changed) stable/8/lib/Makefile (contents, props changed) stable/8/sbin/Makefile stable/8/share/mk/bsd.subdir.mk stable/8/usr.bin/Makefile stable/8/usr.sbin/Makefile (contents, props changed) Directory Properties: stable/8/bin/ (props changed) stable/8/gnu/ (props changed) stable/8/gnu/usr.bin/ (props changed) stable/8/lib/ (props changed) stable/8/sbin/ (props changed) stable/8/share/ (props changed) stable/8/share/mk/ (props changed) stable/8/usr.bin/ (props changed) stable/8/usr.sbin/ (props changed) Changes in other areas also in this revision: Modified: stable/10/bin/Makefile stable/10/gnu/usr.bin/Makefile stable/10/lib/Makefile stable/10/lib/clang/Makefile stable/10/sbin/Makefile stable/10/share/mk/bsd.subdir.mk stable/10/usr.bin/Makefile stable/10/usr.bin/clang/Makefile stable/10/usr.sbin/Makefile stable/9/bin/Makefile stable/9/gnu/usr.bin/Makefile stable/9/lib/Makefile (contents, props changed) stable/9/lib/clang/Makefile stable/9/sbin/Makefile stable/9/share/mk/bsd.subdir.mk stable/9/usr.bin/Makefile stable/9/usr.bin/clang/Makefile stable/9/usr.sbin/Makefile (contents, props changed) Directory Properties: stable/10/ (props changed) stable/9/bin/ (props changed) stable/9/lib/ (props changed) stable/9/lib/clang/ (props changed) stable/9/sbin/ (props changed) stable/9/share/ (props changed) stable/9/share/mk/ (props changed) stable/9/usr.bin/ (props changed) stable/9/usr.bin/clang/ (props changed) stable/9/usr.sbin/ (props changed) Modified: stable/8/bin/Makefile ============================================================================== --- stable/8/bin/Makefile Wed Apr 9 14:50:55 2014 (r264302) +++ stable/8/bin/Makefile Wed Apr 9 18:16:58 2014 (r264303) @@ -54,4 +54,6 @@ _rmail= rmail _csh= csh .endif +SUBDIR_PARALLEL= + .include <bsd.subdir.mk> Modified: stable/8/gnu/usr.bin/Makefile ============================================================================== --- stable/8/gnu/usr.bin/Makefile Wed Apr 9 14:50:55 2014 (r264302) +++ stable/8/gnu/usr.bin/Makefile Wed Apr 9 18:16:58 2014 (r264303) @@ -63,4 +63,6 @@ _cc= cc _gdb= gdb .endif +SUBDIR_PARALLEL= + .include <bsd.subdir.mk> Modified: stable/8/lib/Makefile ============================================================================== --- stable/8/lib/Makefile Wed Apr 9 14:50:55 2014 (r264302) +++ stable/8/lib/Makefile Wed Apr 9 18:16:58 2014 (r264303) @@ -203,4 +203,8 @@ _libusbhid= libusbhid _libusb= libusb .endif +.if !make(install) +SUBDIR_PARALLEL= +.endif + .include <bsd.subdir.mk> Modified: stable/8/sbin/Makefile ============================================================================== --- stable/8/sbin/Makefile Wed Apr 9 14:50:55 2014 (r264302) +++ stable/8/sbin/Makefile Wed Apr 9 18:16:58 2014 (r264303) @@ -154,4 +154,6 @@ _mca= mca _sunlabel= sunlabel .endif +SUBDIR_PARALLEL= + .include <bsd.subdir.mk> Modified: stable/8/share/mk/bsd.subdir.mk ============================================================================== --- stable/8/share/mk/bsd.subdir.mk Wed Apr 9 14:50:55 2014 (r264302) +++ stable/8/share/mk/bsd.subdir.mk Wed Apr 9 18:16:58 2014 (r264303) @@ -69,7 +69,26 @@ ${SUBDIR}: .PHONY depend distribute lint maninstall manlint \ obj objlink realinstall regress tags \ ${SUBDIR_TARGETS} +.ifdef SUBDIR_PARALLEL +.for __dir in ${SUBDIR} +${__target}: ${__target}_subdir_${__dir} +${__target}_subdir_${__dir}: .MAKE + @${_+_}set -e; \ + if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ + ${ECHODIR} "===> ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__target:realinstall=install})"; \ + edir=${__dir}.${MACHINE_ARCH}; \ + cd ${.CURDIR}/$${edir}; \ + else \ + ${ECHODIR} "===> ${DIRPRFX}${__dir} (${__target:realinstall=install})"; \ + edir=${__dir}; \ + cd ${.CURDIR}/$${edir}; \ + fi; \ + ${MAKE} ${__target:realinstall=install} \ + DIRPRFX=${DIRPRFX}$$edir/ +.endfor +.else ${__target}: _SUBDIR +.endif .endfor .for __target in files includes Modified: stable/8/usr.bin/Makefile ============================================================================== --- stable/8/usr.bin/Makefile Wed Apr 9 14:50:55 2014 (r264302) +++ stable/8/usr.bin/Makefile Wed Apr 9 18:16:58 2014 (r264303) @@ -393,4 +393,6 @@ _smbutil= smbutil _smbutil= smbutil .endif +SUBDIR_PARALLEL= + .include <bsd.subdir.mk> Modified: stable/8/usr.sbin/Makefile ============================================================================== --- stable/8/usr.sbin/Makefile Wed Apr 9 14:50:55 2014 (r264302) +++ stable/8/usr.sbin/Makefile Wed Apr 9 18:16:58 2014 (r264303) @@ -520,4 +520,6 @@ _eeprom= eeprom _ofwdump= ofwdump .endif +SUBDIR_PARALLEL= + .include <bsd.subdir.mk>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201404091817.s39IH0a3067606>