From owner-freebsd-current@FreeBSD.ORG Fri Nov 12 08:04:21 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2672316A4CE for ; Fri, 12 Nov 2004 08:04:21 +0000 (GMT) Received: from tigra.ip.net.ua (tigra.ip.net.ua [82.193.96.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 27BB343D49 for ; Fri, 12 Nov 2004 08:04:20 +0000 (GMT) (envelope-from ru@ip.net.ua) Received: from localhost (rocky.ip.net.ua [82.193.96.2]) by tigra.ip.net.ua (8.12.11/8.12.11) with ESMTP id iAC84IR2076929; Fri, 12 Nov 2004 10:04:18 +0200 (EET) (envelope-from ru@ip.net.ua) Received: from tigra.ip.net.ua ([82.193.96.10]) by localhost (rocky.ipnet [82.193.96.2]) (amavisd-new, port 10024) with LMTP id 01322-10; Fri, 12 Nov 2004 10:04:17 +0200 (EET) Received: from heffalump.ip.net.ua (heffalump.ip.net.ua [82.193.96.213]) by tigra.ip.net.ua (8.12.11/8.12.11) with ESMTP id iAC84HKe076926 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 12 Nov 2004 10:04:17 +0200 (EET) (envelope-from ru@ip.net.ua) Received: (from ru@localhost) by heffalump.ip.net.ua (8.13.1/8.13.1) id iAC84Gob049684; Fri, 12 Nov 2004 10:04:16 +0200 (EET) (envelope-from ru) Date: Fri, 12 Nov 2004 10:04:16 +0200 From: Ruslan Ermilov To: Poul-Henning Kamp Message-ID: <20041112080416.GA41844@ip.net.ua> References: <80546.1100202141@critter.freebsd.dk> <85042.1100216213@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xHFwDpU9dbj6ez1V" Content-Disposition: inline In-Reply-To: <85042.1100216213@critter.freebsd.dk> User-Agent: Mutt/1.5.6i X-Virus-Scanned: by amavisd-new at ip.net.ua cc: current@FreeBSD.org Subject: Re: [TEST] make -j patch [take 2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Nov 2004 08:04:21 -0000 --xHFwDpU9dbj6ez1V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Poul-Henning, On Fri, Nov 12, 2004 at 12:36:53AM +0100, Poul-Henning Kamp wrote: > In message <80546.1100202141@critter.freebsd.dk>, Poul-Henning Kamp write= s: >=20 > Here is take two of my "make -j" patch. Further testing found > a couple of buglets. If this survices further testing, it will > be committed in a couple of days. >=20 > With this patch "make -j N" will put the load average as close > to N as the makefiles will allow. >=20 This will make it impossible for "make universe" to build four worlds in parallel, each world with 8 jobs. It's possible now, please see the commit log for src/Makefile,v 1.308 for details. It would be great if -j specified on the command line (rather than derived from MAKEFLAGS environment variable on startup) would cause make(1) to "forget" its group membership regarding parallelism. This can be implemented as simple as unsetting the MAKE_JOBS_FIFO environment variable if -j was specified on the command line. If this is done, I'll be quite happy to see this committed, otherwise not. FWIW. ;) > Included is also a patch I've been using to make the SUBDIR targets > go parallel. Ruslan will have to fix all the mistakes I've made > in that one before it gets committed. >=20 SUBDIR targets should NOT be built in parallel; there are a LOT of places in our world that assume some order (let me know if you need some examples). For this whole reason, the _SUBDIR macro is constructed so that it cannot be parallelized. Your patch to bsd.subdir.mk doesn't change it in any good way. I mean, using .for loops doesn't make it any more parallel. One should think of .for foo in a b do ${foo} .endfor as do a do b That's how make(1) internally parses it, and the latter in NOT parallelizeable. To make something parallel, you should split the job into multiple targets, for example: all: do_a do_b =2Efor t in a b do_${t}: do ${foo} =2Eendfor Then do_a and do_b can be built in parallel, unless told otherwise by the .ORDER directive. This is what the last chunk of Makefile.inc1 does for par-* targets: SUBDIRs will be built in parallel in Makefile.inc1 (sometimes, when it's safe). Your patch replaces the shell loop with an equivalent make loop, causing N shells to be used instead of only one, where N is the SUBDIR list length. This is not good. It also adds some style and functional bugs, like loosing the ability for "make -nn" (double-n) to descend into subdirs. This is quite nice that we have now (today, one can "make -nn buildworld" and see what happens). Please abandon your bsd.subdir.mk patch completely. ;) Cheers, --=20 Ruslan Ermilov ru@FreeBSD.org FreeBSD committer --xHFwDpU9dbj6ez1V Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFBlG6AqRfpzJluFF4RAtOGAKCdpEKN3mHghkslcIvbbZr0/2x3rQCgldqF reJsDxIiO93cSfrnTwl8DiA= =Vd1g -----END PGP SIGNATURE----- --xHFwDpU9dbj6ez1V--