From owner-freebsd-current@FreeBSD.ORG Fri Nov 12 18:57:35 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9C69316A4D0; Fri, 12 Nov 2004 18:57:35 +0000 (GMT) Received: from tigra.ip.net.ua (tigra.ip.net.ua [82.193.96.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9902543D1F; Fri, 12 Nov 2004 18:57:34 +0000 (GMT) (envelope-from ru@ip.net.ua) Received: from localhost (rocky.ip.net.ua [82.193.96.2]) by tigra.ip.net.ua (8.12.11/8.12.11) with ESMTP id iACIvWdL047388; Fri, 12 Nov 2004 20:57:32 +0200 (EET) (envelope-from ru@ip.net.ua) Received: from tigra.ip.net.ua ([82.193.96.10]) by localhost (rocky.ipnet [82.193.96.2]) (amavisd-new, port 10024) with LMTP id 18067-20; Fri, 12 Nov 2004 20:57:31 +0200 (EET) Received: from heffalump.ip.net.ua (heffalump.ip.net.ua [82.193.96.213]) by tigra.ip.net.ua (8.12.11/8.12.11) with ESMTP id iACIvVs9047384 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 12 Nov 2004 20:57:31 +0200 (EET) (envelope-from ru@ip.net.ua) Received: (from ru@localhost) by heffalump.ip.net.ua (8.13.1/8.13.1) id iACIvV4U088544; Fri, 12 Nov 2004 20:57:31 +0200 (EET) (envelope-from ru) Date: Fri, 12 Nov 2004 20:57:21 +0200 From: Ruslan Ermilov To: Poul-Henning Kamp Message-ID: <20041112185721.GA88473@ip.net.ua> References: <20041112130548.GA86162@ip.net.ua> <6750.1100271157@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vkogqOf2sHV7VnPd" Content-Disposition: inline In-Reply-To: <6750.1100271157@critter.freebsd.dk> User-Agent: Mutt/1.5.6i X-Virus-Scanned: by amavisd-new at ip.net.ua cc: Harti Brandt cc: current@freebsd.org Subject: Re: [TEST] make -j patch [take 2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Nov 2004 18:57:35 -0000 --vkogqOf2sHV7VnPd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 12, 2004 at 03:52:37PM +0100, Poul-Henning Kamp wrote: > In message <20041112130548.GA86162@ip.net.ua>, Ruslan Ermilov writes: >=20 > >> At the very least, do not commit your patch until you have managed > >> to come up with at least one instance of real world data where it > >> is a good idea. > >> > >I gave you one example already, and Harti gave you another. > > > >"make universe" builds 6 worlds today. By starting it with > >"make -j12", you build 6 worlds in parallel, and another 6 > >jobs are made available for building these worlds. When one > >world finishes, 7 jobs will be available for building 5 other > >worlds, etc. This is fine as long as you're interested in > >the final result of "make universe". >=20 > You have not actually _tried_ running make universe with > my patch, have you ? >=20 > I thought so :-( >=20 > make -j 12 universe will make 12 jobs available. The main > make will grab six of these to start the buildworlds, and > those six buildworlds compete for the remaining 6 tokens. >=20 > The result is that those six buildworlds progress _in parallel_ > each of them using as many tokens as they can lay their hands on > at any one time. >=20 This is exactly what I said above. I well understand that six buildworlds will compete for six remaining slots, then five buildworld will compete for seven slots, and so on. > If you want them to have more slots, say "-j 18" instead > and they will have 12 slots to compete for. >=20 > This is, as far as I can tell, just the thing you want to happen, > only it doesn't suffer from pointless compartementalization > of resources such as in your proposal. >=20 No, this is not what I want. :-) > The crucial point is that if at one particular instance the > i386.buildworld needs only one token, it will only claim > one token, and the other jobs can get more tokens and start > more things. Once the i386 needs to run more things in > parallel, it will compete for tokens again and run as much > in parallel as it can get tokens for. >=20 I well understand this. > In your scenario, the tokens not used by i386 cannot be > used by the other buildworlds and they are wasted. >=20 Not wasted, they just don't share the same pool of jobs. ;) > >If you're by chance familiar with how you can ask Oracle to > >optimize your queries, you can either optimize for ALL ROWS or > >for FIRST ROWS, and it's up to you to decide on this. The > >"rows" in "make universe" are results of doing buildworld > >and buildkernel, and you have control over this. >=20 > This is not a database. >=20 > This is not even a cartesian problem space. >=20 I'm interested in completing i386 buildworld as fast as possible. In "make -j12 universe", the i386 world will be built in approximately (TOTAL) seconds (let's assume for the moment that we don't build kernels). If I use "make universe JFLAG=3D-j12", i386 world will complete in (TOTAL / 6) seconds. If I use "make -j2 universe JFLAG=3D-j6", i386 and pc98 worlds will complete in (TOTAL / 6 * 2) seconds. Can you see a difference? I'm interested in "first rows". And I'm not saying I'm always interested in them, just want to be able to test my patches as quickly as possible. ;) > >Do you disagree with anything of the above? >=20 > Yes, I disagree with pretty much all of it. It is clear that > you have not actually tried running with the code which is in > the tree. >=20 Why? It's clear that I understand how it works, and you fail to see _my_ point. ;) > >Can you agree to the below patch? It doesn't change anything > >except when JFLAG is set explicitly to request another level > >of parallelism for building individual worlds and kernels? >=20 > The patch is pointless, but harmless once people learn not to set > JFLAG. >=20 > But I will not agree to this patch _until_ you show me _actual_ _real > _life_ data that shows it performs better than just running without > a JFLAG at all. >=20 You disagree that the i386 part of "make universe" will complete faster in my scenario, while still providing an overall limit of 12 jobs? I understand the drawback of my approach -- that the total time for "make universe" will be longer in my case. I now feel like everything that should have been said was said, and this thread should die. ;) Cheers, --=20 Ruslan Ermilov ru@FreeBSD.org FreeBSD committer --vkogqOf2sHV7VnPd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFBlQeRqRfpzJluFF4RAqQIAJ9u4pr0F/B+lXwUC6O4jKWhgakneACfc0aJ eoZ5B4M08eGbh/xC8Ig9j2Y= =8NEY -----END PGP SIGNATURE----- --vkogqOf2sHV7VnPd--