From owner-freebsd-current@FreeBSD.ORG Fri Nov 12 09:35:03 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 28D8F16A4CE; Fri, 12 Nov 2004 09:35:03 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5BA0B43D39; Fri, 12 Nov 2004 09:35:02 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.1/8.13.1) with ESMTP id iAC9Z0Vr094519; Fri, 12 Nov 2004 10:35:01 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Ruslan Ermilov From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 12 Nov 2004 11:09:05 +0200." <20041112090905.GD41844@ip.net.ua> Date: Fri, 12 Nov 2004 10:35:00 +0100 Message-ID: <94518.1100252100@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: current@FreeBSD.org Subject: Re: [TEST] make -j patch [take 2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Nov 2004 09:35:03 -0000 In message <20041112090905.GD41844@ip.net.ua>, Ruslan Ermilov writes: >But you don't give an opportunity to control this on a sub-make >level (that's what I ask for). Why would that be of any use ? If you run "make universe" the task at hand is to get "make universe" to complete. You should not care which partcular submake starts how many jobs when, you should only care that it works as efficient as possible. >level (that's what I ask for). If you implement what I suggest, >"make -j 12 universe" will still do as you saw it last night, >while still allowing "make -j4 universe JFLAG=3D-j3" to build four >worlds in parallel, each with three jobs. Isn't it nice? ;) You're wrong about how this works. Try plotting your load-avg over time and you will see that you have neither enforced the 4 nor the 3 jobs. Or look at the one I just posted: http://phk.freebsd.dk/misc/make.png Then explain to me how you think it is beneficial that "-j 12" gives make(1) permission to start 65 jobs. When I say "make -j 12" I mean "12 jobs, damnit!", not "up to 12^(number of levels of subdirectories)". >I also have a question. What will happen (with your patch) if >make was given -j4, and then I give a sub-make an explicit >argument of -j8? I think it will cause the main "make" to >behave like -j1, supposing there're a lot of jobs in sub-make >and they're completed and started quickly. Yes/no? The main make controls. If you start the main make with -j 4 a global pool of 4 jobs are available to all the submakes spawned as part of that job. The submakes will not respect an explicit "-j N" argument if they find the environment variable. You can of course eliminate that if you want to override this behaviour. >I've spent some time explaining why your patch doesn't add any >parallelism to bsd.subdir.mk, and why parallelism in bsd.subdir.mk >should not be allowed. Did you spend your time trying to >understand it? ;) When theory and measurement disagree, either can have a problem, the main difference in approach is that theory has got little to back up it's claims and must resort to prove experimental error to survive. >Or perhaps you could explain how your patch helps parallelism >or improves something, I fail to see it. All I see is that >it causes N shells to be used instead of one, but this is >unmeasureable in buildworld times. I don't know the reason, but the result was about 5% speedup on parallel buildworld. I'll leave it to you if anything needs done, You're the makefile wizard. I've done the bit I set out to do: prevent boundless creation of jobs. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.