From owner-freebsd-stable Wed Dec 24 03:20:54 1997 Return-Path: Received: (from root@localhost) by hub.freebsd.org (8.8.7/8.8.7) id DAA01117 for stable-outgoing; Wed, 24 Dec 1997 03:20:54 -0800 (PST) (envelope-from owner-freebsd-stable) Received: from silvia.HIP.Berkeley.EDU (ala-ca34-23.ix.netcom.com [207.93.143.151]) by hub.freebsd.org (8.8.7/8.8.7) with ESMTP id DAA01078; Wed, 24 Dec 1997 03:20:35 -0800 (PST) (envelope-from asami@vader.cs.berkeley.edu) Received: (from asami@localhost) by silvia.HIP.Berkeley.EDU (8.8.8/8.6.9) id DAA27011; Wed, 24 Dec 1997 03:20:33 -0800 (PST) Date: Wed, 24 Dec 1997 03:20:33 -0800 (PST) Message-Id: <199712241120.DAA27011@silvia.HIP.Berkeley.EDU> To: current@freebsd.org, stable@freebsd.org Reply-to: current@freebsd.org Subject: p-make From: asami@cs.berkeley.edu (Satoshi Asami) Sender: owner-freebsd-stable@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Well, here is a new set of patches. My previous one didn't work because it ran out of processes -- the build was spawning N processes at each level -- so, at the 4th level, there were N^4 processes. Some makes got "no more processes" errors, but continued anyway (why?) and the build died much later. So, I made it so that the forking only happens at one level. When bsd.subdir.mk parallelizes a make, it will append -B to the flags so it won't be forked again. First, the numbers from individual directories. The four numbers are "make -j4 all" async/sync for /usr/{src,obj} and with parent dir pmake/leaf node pmake. The machines are both P6-200, /usr/{src,obj} are on 8-disk CCDs of IBM 9GB 7,200RPM drives (the first machine has larger partition though...the total size is 8GB for 2.2-stable and 3GB for 3.0-current). 3.0-current has NOTCL defined. 2.2-stable: === bin 72.73 real 62.09 user 9.62 sys <- async, parent 83.56 real 62.28 user 10.08 sys <- async, leaf 73.00 real 62.24 user 9.77 sys <- sync, parent 84.80 real 62.14 user 10.49 sys <- sync, leaf sbin 95.27 real 80.57 user 13.18 sys 113.60 real 80.20 user 13.05 sys 95.29 real 80.72 user 12.98 sys 115.11 real 80.13 user 13.40 sys lib 843.87 real 599.77 user 223.81 sys 708.49 real 462.48 user 178.23 sys 906.87 real 602.89 user 222.62 sys 759.95 real 466.66 user 177.76 sys libexec 55.12 real 46.02 user 8.26 sys 65.73 real 46.00 user 8.34 sys 55.43 real 45.80 user 8.55 sys 67.20 real 46.25 user 8.26 sys share 111.31 real 88.32 user 19.70 sys 139.89 real 87.36 user 18.77 sys 109.44 real 87.19 user 19.94 sys 144.42 real 87.45 user 19.10 sys usr.bin 446.97 real 379.21 user 62.02 sys 520.46 real 380.81 user 60.70 sys 447.28 real 378.71 user 62.14 sys 526.67 real 380.85 user 62.48 sys usr.sbin 402.24 real 342.53 user 54.49 sys 461.16 real 342.22 user 52.96 sys 401.83 real 342.56 user 53.78 sys 467.48 real 343.44 user 53.25 sys gnu/lib 537.30 real 445.09 user 84.75 sys 562.43 real 447.25 user 87.67 sys 546.56 real 447.99 user 85.05 sys 583.22 real 447.74 user 87.56 sys gnu/usr.bin 1094.20 real 942.52 user 93.68 sys 1089.30 real 938.02 user 93.64 sys 1114.73 real 950.29 user 93.84 sys 1114.97 real 950.11 user 94.73 sys === 3.0-current: === bin 69.65 real 60.11 user 8.17 sys 79.47 real 60.17 user 8.26 sys 69.79 real 60.38 user 7.99 sys 79.92 real 60.06 user 8.52 sys sbin 96.35 real 81.64 user 12.71 sys 115.08 real 81.60 user 11.90 sys 96.39 real 81.63 user 12.64 sys 117.10 real 81.64 user 12.12 sys lib 1259.10 real 918.46 user 251.52 sys 1499.31 real 924.88 user 276.02 sys 1344.95 real 922.70 user 248.34 sys 1625.90 real 927.65 user 281.68 sys libexec 53.42 real 45.15 user 6.94 sys 62.50 real 45.06 user 6.74 sys 53.52 real 45.25 user 6.96 sys 64.59 real 45.20 user 6.72 sys share 107.32 real 85.39 user 15.29 sys 139.84 real 85.76 user 14.84 sys 108.04 real 84.96 user 15.76 sys 144.18 real 85.79 user 14.94 sys usr.bin 463.92 real 395.59 user 57.83 sys 534.99 real 394.94 user 56.71 sys 463.25 real 396.46 user 56.61 sys 540.16 real 396.20 user 56.53 sys usr.sbin 402.00 real 344.13 user 49.20 sys 461.87 real 343.06 user 47.52 sys 401.31 real 344.71 user 48.22 sys 467.61 real 343.75 user 47.95 sys gnu/lib 510.97 real 424.14 user 72.09 sys 543.21 real 423.12 user 76.13 sys 518.31 real 426.08 user 72.46 sys 569.64 real 424.54 user 76.14 sys gnu/usr.bin 1003.39 real 905.61 user 81.43 sys 1055.77 real 907.60 user 81.05 sys 1010.71 real 906.07 user 81.50 sys 1063.74 real 907.70 user 80.91 sys === As you can see, it speeds up places like bin and usr.bin quite a bit. The tradeoff is if you have one or two very large directories (compared to the others), that one will be running a single-process make (because of -B) long after others are done. I actually had to move sh much earlier in the SUBDIR list in bin/Makefile so it doesn't happen. I observed this by looking at the make output; we may need to do similar things in usr.sbin (xntpd is pretty large and is near the end of the list). On the other hand, it doesn't help much in places like lib (it actually hurts in 2.2-stable). It's because there is one directory (libc) that basically dwarfs the rest. My guess on the reason why 3.0-current behaves better is because it also builds libc_r, which is about as big so there's at least a two-way parellism going on even after everything else is done. For lib, I'm actually not sure why it helps at all. For gnu/usr.bin, I assume it's spending most of the time inside cc. Now, for the whole buildworld. The numbers are measured by doing a "rm -rf /usr/obj/usr/src" and doing the actual build with -DNOCLEAN. -j4 async parent -j4 async leaf -j4 sync parent -j4 sync leaf -j2 async parent -j2 async leaf -j2 sync parent -j2 sync leaf 2.2-stable: === 5517.40 real 3785.70 user 927.84 sys 5726.68 real 3766.08 user 960.40 sys 5900.12 real 3787.60 user 945.85 sys 6090.51 real 3768.67 user 970.15 sys 5408.57 real 3816.63 user 860.06 sys 5650.55 real 3749.16 user 851.64 sys 5613.19 real 3776.83 user 843.19 sys 6016.20 real 3756.14 user 852.53 sys === 3.0-current: === 5255.05 real 4080.63 user 832.97 sys 5903.48 real 4073.16 user 859.38 sys 5468.84 real 4101.21 user 829.75 sys 6138.02 real 4076.45 user 863.55 sys 5332.94 real 4074.15 user 811.88 sys 6149.01 real 4059.68 user 846.22 sys 5603.09 real 4083.54 user 799.08 sys 6489.46 real 4068.01 user 857.28 sys === As you can see, it helps quite a bit on -current. It didn't make much difference in -stable. (I disabled forking in lib/Makefile of -stable.) Here are the patches. I'm especially interested in SMP people's numbers. The patches are relative to -current on 11th and last week's -stable. (The -current machine is also a CAM testbox so I only sync the source when there is a CAM snapshot.) 2.2-stable: === Index: Makefile =================================================================== RCS file: /usr/cvs/src/Makefile,v retrieving revision 1.109.2.20 diff -u -r1.109.2.20 Makefile --- Makefile 1997/10/26 22:46:16 1.109.2.20 +++ Makefile 1997/12/22 11:52:52 @@ -640,6 +669,7 @@ .endfor .for __target in clean cleandir obj depend +.if defined(NOPMAKESUBDIR) .for entry in ${SUBDIR} ${entry}.${__target}__D: .PHONY if test -d ${.CURDIR}/${entry}.${MACHINE}; then \ @@ -654,6 +684,9 @@ ${MAKE} ${__target} DIRPRFX=${DIRPRFX}$${edir}/ .endfor par-${__target}: ${SUBDIR:S/$/.${__target}__D/} +.else +par-${__target}: ${__target} +.endif .endfor .include Index: bin/Makefile =================================================================== RCS file: /usr/cvs/src/bin/Makefile,v retrieving revision 1.4.2.1 diff -u -r1.4.2.1 Makefile --- Makefile 1997/03/07 09:39:06 1.4.2.1 +++ Makefile 1997/12/17 05:53:32 @@ -1,8 +1,13 @@ # From: @(#)Makefile 8.1 (Berkeley) 5/31/93 # $Id: Makefile,v 1.4.2.1 1997/03/07 09:39:06 joerg Exp $ -SUBDIR= cat chio chmod cp csh date dd df domainname echo ed expr hostname \ - kill ln ls mkdir mv pax ps pwd rcp rm rmail rmdir sh sleep \ +# XXX put large subdirectories in front to make pmake happy + +SUBDIR= csh sh pax \ + cat chio chmod cp date dd df domainname echo ed expr hostname \ + kill ln ls mkdir mv ps pwd rcp rm rmail rmdir sleep \ stty sync test + +PMAKESUBDIR= t .include Index: games/Makefile =================================================================== RCS file: /usr/cvs/src/games/Makefile,v retrieving revision 1.10.2.1 diff -u -r1.10.2.1 Makefile --- Makefile 1997/09/11 03:03:14 1.10.2.1 +++ Makefile 1997/12/17 05:53:32 @@ -8,4 +8,6 @@ rain random robots rogue sail snake trek wargames worm worms \ wump +PMAKESUBDIR= t + .include Index: gnu/lib/Makefile =================================================================== RCS file: /usr/cvs/src/gnu/lib/Makefile,v retrieving revision 1.13 diff -u -r1.13 Makefile --- Makefile 1996/10/04 08:44:26 1.13 +++ Makefile 1997/12/17 05:53:32 @@ -2,4 +2,6 @@ SUBDIR= libdialog libg++ libgmp libmp libobjc libregex libreadline libstdc++ +PMAKESUBDIR= t + .include Index: gnu/usr.sbin/Makefile =================================================================== RCS file: /usr/cvs/src/gnu/usr.sbin/Makefile,v retrieving revision 1.4 diff -u -r1.4 Makefile --- Makefile 1996/04/28 04:06:06 1.4 +++ Makefile 1997/12/17 05:53:32 @@ -1,6 +1,4 @@ # $Id: Makefile,v 1.4 1996/04/28 04:06:06 wpaul Exp $ -SUBDIR= - .include Index: lib/Makefile =================================================================== RCS file: /usr/cvs/src/lib/Makefile,v retrieving revision 1.47.2.9 diff -u -r1.47.2.9 Makefile --- Makefile 1997/10/08 10:37:20 1.47.2.9 +++ Makefile 1997/12/19 06:37:21 @@ -48,4 +48,6 @@ SUBDIR+= compat +#PMAKESUBDIR= t + .include Index: lib/compat/Makefile =================================================================== RCS file: /usr/cvs/src/lib/compat/Makefile,v retrieving revision 1.1 diff -u -r1.1 Makefile --- Makefile 1996/12/23 05:07:19 1.1 +++ Makefile 1997/12/17 05:53:32 @@ -1,7 +1,5 @@ # $Id: Makefile,v 1.1 1996/12/23 05:07:19 peter Exp $ -SUBDIR= - # Note that compat21 is *not* for the 2.1.x branch! .if defined(COMPAT1X) || defined(RELEASEDIR) SUBDIR+= compat1x Index: lib/libpcap/Makefile =================================================================== RCS file: /usr/cvs/src/lib/libpcap/Makefile,v retrieving revision 1.12.2.1 diff -u -r1.12.2.1 Makefile --- Makefile 1997/10/26 22:48:31 1.12.2.1 +++ Makefile 1997/12/19 21:18:17 @@ -33,6 +33,8 @@ .endfor scanner.o: tokdefs.h +scanner.po: tokdefs.h +scanner.so: tokdefs.h .ORDER: grammar.c tokdefs.h tokdefs.h: grammar.c Index: libexec/Makefile =================================================================== RCS file: /usr/cvs/src/libexec/Makefile,v retrieving revision 1.20.2.3 diff -u -r1.20.2.3 Makefile --- Makefile 1997/07/05 14:23:54 1.20.2.3 +++ Makefile 1997/12/17 05:53:32 @@ -27,4 +27,6 @@ SUBDIR+=rbootd .endif +PMAKESUBDIR= t + .include Index: lkm/Makefile =================================================================== RCS file: /usr/cvs/src/lkm/Makefile,v retrieving revision 1.24.2.1 diff -u -r1.24.2.1 Makefile --- Makefile 1997/11/06 13:03:53 1.24.2.1 +++ Makefile 1997/12/17 05:53:32 @@ -10,4 +10,6 @@ # XXX builds, but not useable with present design # fpu gnufpu union +PMAKESUBDIR= t + .include Index: sbin/Makefile =================================================================== RCS file: /usr/cvs/src/sbin/Makefile,v retrieving revision 1.34.2.1 diff -u -r1.34.2.1 Makefile --- Makefile 1997/03/26 20:05:24 1.34.2.1 +++ Makefile 1997/12/17 05:53:32 @@ -14,4 +14,6 @@ SUBDIR+= ${MACHINE} .endif +PMAKESUBDIR= t + .include Index: share/Makefile =================================================================== RCS file: /usr/cvs/src/share/Makefile,v retrieving revision 1.12.2.2 diff -u -r1.12.2.2 Makefile --- Makefile 1997/07/19 15:31:33 1.12.2.2 +++ Makefile 1997/12/17 05:53:32 @@ -3,4 +3,6 @@ SUBDIR= dict doc examples info man me misc mk skel syscons \ tabset termcap timedef zoneinfo +PMAKESUBDIR= t + .include Index: share/mk/bsd.subdir.mk =================================================================== RCS file: /usr/cvs/src/share/mk/bsd.subdir.mk,v retrieving revision 1.11.2.2 diff -u -r1.11.2.2 bsd.subdir.mk --- bsd.subdir.mk 1997/06/21 15:48:22 1.11.2.2 +++ bsd.subdir.mk 1997/12/17 05:53:33 @@ -29,7 +29,23 @@ .for __target in all checkdpadd clean cleandir depend lint \ maninstall obj objlink .if !target(__target) +.if (defined(PMAKESUBDIR) && !defined(NOPMAKESUBDIR)) +${__target}: ${SUBDIR:S/$/.${__target}__D/} + +.for entry in ${SUBDIR} +${entry}.${__target}__D: .PHONY + @if test -d ${.CURDIR}/${entry}.${MACHINE}; then \ + ${ECHODIR} "===> ${DIRPRFX}${entry}.${MACHINE}"; \ + cd ${.CURDIR}/${entry}.${MACHINE}; \ + else \ + ${ECHODIR} "===> ${DIRPRFX}${entry}"; \ + cd ${.CURDIR}/${entry}; \ + fi; \ + ${MAKE} -B ${__target} +.endfor +.else ${__target}: _SUBDIRUSE +.endif .endif .endfor Index: usr.bin/Makefile =================================================================== RCS file: /usr/cvs/src/usr.bin/Makefile,v retrieving revision 1.73.2.10 diff -u -r1.73.2.10 Makefile --- Makefile 1997/08/21 04:00:56 1.73.2.10 +++ Makefile 1997/12/17 05:53:33 @@ -60,4 +60,6 @@ SUBDIR+=f77 pascal systat vmstat .endif +PMAKESUBDIR= t + .include Index: usr.sbin/Makefile =================================================================== RCS file: /usr/cvs/src/usr.sbin/Makefile,v retrieving revision 1.98.2.6 diff -u -r1.98.2.6 Makefile --- Makefile 1997/09/15 04:56:52 1.98.2.6 +++ Makefile 1997/12/17 05:53:33 @@ -34,4 +34,6 @@ SUBDIR+=config iostat .endif +PMAKESUBDIR= t + .include === 3.0-current: === Index: Makefile =================================================================== RCS file: /usr/cvs/src/Makefile,v retrieving revision 1.155 diff -u -r1.155 Makefile --- Makefile 1997/10/10 13:02:36 1.155 +++ Makefile 1997/12/22 11:03:15 @@ -668,6 +697,7 @@ .endfor .for __target in clean cleandir obj depend +.if defined(NOPMAKESUBDIR) .for entry in ${SUBDIR} ${entry}.${__target}__D: .PHONY if test -d ${.CURDIR}/${entry}.${MACHINE}; then \ @@ -682,6 +712,9 @@ ${MAKE} ${__target} DIRPRFX=${DIRPRFX}$${edir}/ .endfor par-${__target}: ${SUBDIR:S/$/.${__target}__D/} +.else +par-${__target}: ${__target} +.endif .endfor .include Index: bin/Makefile =================================================================== RCS file: /usr/cvs/src/bin/Makefile,v retrieving revision 1.9 diff -u -r1.9 Makefile --- Makefile 1997/12/07 02:26:12 1.9 +++ Makefile 1997/12/18 15:47:16 @@ -1,8 +1,13 @@ # From: @(#)Makefile 8.1 (Berkeley) 5/31/93 # $Id: Makefile,v 1.9 1997/12/07 02:26:12 peter Exp $ -SUBDIR= cat chio chmod cp csh date dd df domainname echo ed expr hostname \ - kill ln ls mkdir mv pax ps pwd rcp rm rmail rmdir sh sleep \ +# XXX put large subdirectories in front to make pmake happy + +SUBDIR= csh sh pax \ + cat chio chmod cp date dd df domainname echo ed expr hostname \ + kill ln ls mkdir mv ps pwd rcp rm rmail rmdir sleep \ stty sync test + +PMAKESUBDIR= t .include Index: games/Makefile =================================================================== RCS file: /usr/cvs/src/games/Makefile,v retrieving revision 1.13 diff -u -r1.13 Makefile --- Makefile 1997/09/11 02:52:59 1.13 +++ Makefile 1997/12/18 15:47:16 @@ -8,4 +8,6 @@ rain random robots rogue sail snake trek wargames worm worms \ wump +PMAKESUBDIR= t + .include Index: gnu/lib/Makefile =================================================================== RCS file: /usr/cvs/src/gnu/lib/Makefile,v retrieving revision 1.15 diff -u -r1.15 Makefile --- Makefile 1997/02/22 15:42:33 1.15 +++ Makefile 1997/12/18 15:47:16 @@ -2,4 +2,6 @@ SUBDIR= libdialog libg++ libgmp libmp libobjc libregex libreadline libstdc++ +PMAKESUBDIR= t + .include Index: gnu/usr.bin/Makefile =================================================================== RCS file: /usr/cvs/src/gnu/usr.bin/Makefile,v retrieving revision 1.36 diff -u -r1.36 Makefile --- Makefile 1997/04/12 07:19:09 1.36 +++ Makefile 1997/12/18 15:47:16 @@ -4,6 +4,8 @@ grep groff gzip ld man patch ptx rcs sdiff send-pr \ sort tar texinfo +PMAKESUBDIR= t + .if !defined(NOPERL) && exists(${.CURDIR}/perl) SUBDIR+=perl .endif Index: gnu/usr.sbin/Makefile =================================================================== RCS file: /usr/cvs/src/gnu/usr.sbin/Makefile,v retrieving revision 1.6 diff -u -r1.6 Makefile --- Makefile 1997/02/22 15:48:30 1.6 +++ Makefile 1997/12/18 15:47:17 @@ -1,6 +1,4 @@ # $Id: Makefile,v 1.6 1997/02/22 15:48:30 peter Exp $ -SUBDIR= - .include Index: lib/Makefile =================================================================== RCS file: /usr/cvs/src/lib/Makefile,v retrieving revision 1.58 diff -u -r1.58 Makefile --- Makefile 1997/12/04 10:48:14 1.58 +++ Makefile 1997/12/18 15:47:17 @@ -45,5 +45,7 @@ .endif SUBDIR+= compat + +PMAKESUBDIR= t .include Index: lib/compat/Makefile =================================================================== RCS file: /usr/cvs/src/lib/compat/Makefile,v retrieving revision 1.3 diff -u -r1.3 Makefile --- Makefile 1997/02/22 14:57:32 1.3 +++ Makefile 1997/12/18 15:47:17 @@ -1,7 +1,5 @@ # $Id: Makefile,v 1.3 1997/02/22 14:57:32 peter Exp $ -SUBDIR= - # Note that compat21 is *not* for the 2.1.x branch! .if defined(COMPAT1X) || defined(RELEASEDIR) SUBDIR+= compat1x Index: libexec/Makefile =================================================================== RCS file: /usr/cvs/src/libexec/Makefile,v retrieving revision 1.26 diff -u -r1.26 Makefile --- Makefile 1997/09/28 09:07:34 1.26 +++ Makefile 1997/12/18 15:47:17 @@ -22,4 +22,6 @@ SUBDIR+=rbootd .endif +PMAKESUBDIR= t + .include Index: lkm/Makefile =================================================================== RCS file: /usr/cvs/src/lkm/Makefile,v retrieving revision 1.28 diff -u -r1.28 Makefile --- Makefile 1997/08/28 14:45:12 1.28 +++ Makefile 1997/12/18 15:47:18 @@ -8,4 +8,6 @@ # XXX builds, but not useable with present design # fpu gnufpu union +PMAKESUBDIR= t + .include Index: sbin/Makefile =================================================================== RCS file: /usr/cvs/src/sbin/Makefile,v retrieving revision 1.41 diff -u -r1.41 Makefile --- Makefile 1997/10/19 11:15:39 1.41 +++ Makefile 1997/12/18 15:47:18 @@ -16,4 +16,6 @@ SUBDIR+= ${MACHINE} .endif +PMAKESUBDIR= t + .include Index: share/Makefile =================================================================== RCS file: /usr/cvs/src/share/Makefile,v retrieving revision 1.14 diff -u -r1.14 Makefile --- Makefile 1997/07/19 15:17:41 1.14 +++ Makefile 1997/12/18 15:47:18 @@ -3,4 +3,6 @@ SUBDIR= dict doc examples info man me misc mk skel syscons \ tabset termcap timedef zoneinfo +PMAKESUBDIR= t + .include Index: share/mk/bsd.subdir.mk =================================================================== RCS file: /usr/cvs/src/share/mk/bsd.subdir.mk,v retrieving revision 1.19 diff -u -r1.19 bsd.subdir.mk --- bsd.subdir.mk 1997/11/15 13:09:52 1.19 +++ bsd.subdir.mk 1997/12/18 15:52:41 @@ -60,7 +60,23 @@ .for __target in all checkdpadd clean cleandir depend lint \ maninstall obj objlink .if !target(${__target}) +.if (defined(PMAKESUBDIR) && !defined(NOPMAKESUBDIR)) +${__target}: ${SUBDIR:S/$/.${__target}__D/} + +.for entry in ${SUBDIR} +${entry}.${__target}__D: .PHONY + @if test -d ${.CURDIR}/${entry}.${MACHINE}; then \ + ${ECHODIR} "===> ${DIRPRFX}${entry}.${MACHINE}"; \ + cd ${.CURDIR}/${entry}.${MACHINE}; \ + else \ + ${ECHODIR} "===> ${DIRPRFX}${entry}"; \ + cd ${.CURDIR}/${entry}; \ + fi; \ + ${MAKE} -B ${__target} +.endfor +.else ${__target}: _SUBDIRUSE +.endif .endif .endfor Index: usr.bin/Makefile =================================================================== RCS file: /usr/cvs/src/usr.bin/Makefile,v retrieving revision 1.91 diff -u -r1.91 Makefile --- Makefile 1997/12/06 05:23:59 1.91 +++ Makefile 1997/12/18 15:47:18 @@ -61,4 +61,6 @@ SUBDIR+=f77 pascal systat vmstat .endif +PMAKESUBDIR= t + .include Index: usr.sbin/Makefile =================================================================== RCS file: /usr/cvs/src/usr.sbin/Makefile,v retrieving revision 1.114 diff -u -r1.114 Makefile --- Makefile 1997/12/07 02:27:48 1.114 +++ Makefile 1997/12/18 15:47:19 @@ -33,5 +33,7 @@ .elif ${MACHINE} == "tahoe" SUBDIR+=config iostat .endif + +PMAKESUBDIR= t .include === (I had to edit the patches by hand to remove other local mods, sorry if I screwed up.) As you can see in bsd.subdir.mk, you can define NOPMAKESUBDIR in the environment to get the current behavior (leaf branching for most, with toplevel branching for clean, obj and depend). Without NOPMAKESUBDIR, you will get my custom branching (try to branch at the thickest part of the tree). Satoshi