Date: Fri, 12 Nov 2004 20:57:21 +0200 From: Ruslan Ermilov <ru@freebsd.org> To: Poul-Henning Kamp <phk@phk.freebsd.dk> Cc: current@freebsd.org Subject: Re: [TEST] make -j patch [take 2] Message-ID: <20041112185721.GA88473@ip.net.ua> In-Reply-To: <6750.1100271157@critter.freebsd.dk> References: <20041112130548.GA86162@ip.net.ua> <6750.1100271157@critter.freebsd.dk>
next in thread | previous in thread | raw e-mail | index | archive | help
--vkogqOf2sHV7VnPd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 12, 2004 at 03:52:37PM +0100, Poul-Henning Kamp wrote: > In message <20041112130548.GA86162@ip.net.ua>, Ruslan Ermilov writes: >=20 > >> At the very least, do not commit your patch until you have managed > >> to come up with at least one instance of real world data where it > >> is a good idea. > >> > >I gave you one example already, and Harti gave you another. > > > >"make universe" builds 6 worlds today. By starting it with > >"make -j12", you build 6 worlds in parallel, and another 6 > >jobs are made available for building these worlds. When one > >world finishes, 7 jobs will be available for building 5 other > >worlds, etc. This is fine as long as you're interested in > >the final result of "make universe". >=20 > You have not actually _tried_ running make universe with > my patch, have you ? >=20 > I thought so :-( >=20 > make -j 12 universe will make 12 jobs available. The main > make will grab six of these to start the buildworlds, and > those six buildworlds compete for the remaining 6 tokens. >=20 > The result is that those six buildworlds progress _in parallel_ > each of them using as many tokens as they can lay their hands on > at any one time. >=20 This is exactly what I said above. I well understand that six buildworlds will compete for six remaining slots, then five buildworld will compete for seven slots, and so on. > If you want them to have more slots, say "-j 18" instead > and they will have 12 slots to compete for. >=20 > This is, as far as I can tell, just the thing you want to happen, > only it doesn't suffer from pointless compartementalization > of resources such as in your proposal. >=20 No, this is not what I want. :-) > The crucial point is that if at one particular instance the > i386.buildworld needs only one token, it will only claim > one token, and the other jobs can get more tokens and start > more things. Once the i386 needs to run more things in > parallel, it will compete for tokens again and run as much > in parallel as it can get tokens for. >=20 I well understand this. > In your scenario, the tokens not used by i386 cannot be > used by the other buildworlds and they are wasted. >=20 Not wasted, they just don't share the same pool of jobs. ;) > >If you're by chance familiar with how you can ask Oracle to > >optimize your queries, you can either optimize for ALL ROWS or > >for FIRST ROWS, and it's up to you to decide on this. The > >"rows" in "make universe" are results of doing buildworld > >and buildkernel, and you have control over this. >=20 > This is not a database. >=20 > This is not even a cartesian problem space. >=20 I'm interested in completing i386 buildworld as fast as possible. In "make -j12 universe", the i386 world will be built in approximately (TOTAL) seconds (let's assume for the moment that we don't build kernels). If I use "make universe JFLAG=3D-j12", i386 world will complete in (TOTAL / 6) seconds. If I use "make -j2 universe JFLAG=3D-j6", i386 and pc98 worlds will complete in (TOTAL / 6 * 2) seconds. Can you see a difference? I'm interested in "first rows". And I'm not saying I'm always interested in them, just want to be able to test my patches as quickly as possible. ;) > >Do you disagree with anything of the above? >=20 > Yes, I disagree with pretty much all of it. It is clear that > you have not actually tried running with the code which is in > the tree. >=20 Why? It's clear that I understand how it works, and you fail to see _my_ point. ;) > >Can you agree to the below patch? It doesn't change anything > >except when JFLAG is set explicitly to request another level > >of parallelism for building individual worlds and kernels? >=20 > The patch is pointless, but harmless once people learn not to set > JFLAG. >=20 > But I will not agree to this patch _until_ you show me _actual_ _real > _life_ data that shows it performs better than just running without > a JFLAG at all. >=20 You disagree that the i386 part of "make universe" will complete faster in my scenario, while still providing an overall limit of 12 jobs? I understand the drawback of my approach -- that the total time for "make universe" will be longer in my case. I now feel like everything that should have been said was said, and this thread should die. ;) Cheers, --=20 Ruslan Ermilov ru@FreeBSD.org FreeBSD committer --vkogqOf2sHV7VnPd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFBlQeRqRfpzJluFF4RAqQIAJ9u4pr0F/B+lXwUC6O4jKWhgakneACfc0aJ eoZ5B4M08eGbh/xC8Ig9j2Y= =8NEY -----END PGP SIGNATURE----- --vkogqOf2sHV7VnPd--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20041112185721.GA88473>