From owner-freebsd-current@FreeBSD.ORG Mon Feb 11 17:46:29 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 69FD370A; Mon, 11 Feb 2013 17:46:29 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay01.ispgateway.de (smtprelay01.ispgateway.de [80.67.18.43]) by mx1.freebsd.org (Postfix) with ESMTP id E21BA11F; Mon, 11 Feb 2013 17:46:28 +0000 (UTC) Received: from [87.79.249.155] (helo=fabiankeil.de) by smtprelay01.ispgateway.de with esmtpsa (SSLv3:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1U4xSO-0004Hb-7h; Mon, 11 Feb 2013 18:46:20 +0100 Date: Mon, 11 Feb 2013 18:44:19 +0100 From: Fabian Keil To: David Chisnall Subject: Re: 7+ days of dogfood Message-ID: <20130211184419.219ee96d@fabiankeil.de> In-Reply-To: <5BC7246C-4339-48B1-BC7E-9FF6CE1FEF1C@cl.cam.ac.uk> References: <20130210000723.GA73630@troutmask.apl.washington.edu> <20130211114811.09e56b55@fabiankeil.de> <17E009FB-23FA-4E04-8437-DE81033164DE@FreeBSD.org> <20130211145647.79a01f7e@fabiankeil.de> <5BC7246C-4339-48B1-BC7E-9FF6CE1FEF1C@cl.cam.ac.uk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/spZo5htsGvTI3oxP.vESby3"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 Cc: freebsd-current@freebsd.org, Steve Kargl X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Feb 2013 17:46:29 -0000 --Sig_/spZo5htsGvTI3oxP.vESby3 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable David Chisnall wrote: > On 11 Feb 2013, at 13:56, Fabian Keil > wrote: >=20 > > real 350m42.363s > > user 253m5.477s > > sys 50m0.024s >=20 > These numbers look a bit wrong. You've got 300 minutes of CPU time, but > 350 minutes of real time. In an ideal world, on your dual-core system > you'd see 150 minutes of real time. Seeing more than 300 implies that > you're spending a lot of time waiting for I/O. The normal > recommendation is to use -j x where x is 1.5 times the number of cores, > or 1x the number of GBs of RAM, whichever is smaller. With only 2GB of > RAM you might have linking problems with -j3, but it's still worth > trying. With only 2 GB of RAM (parts of which are needed elsewhere) I'm already having linking problems without using -j at all and the I/O I'm waiting for is the disk serving the swap partition. I've used -j2 and occasionally -j4 when building world with gcc in the past, but when using clang I'd risk temporarily having three (or more) processes compete for swap space and bandwidth, so I stopped doing that. I'd expect that another 2 GB of RAM would prevent the swapping and thus reduce the buildworld time quite a bit, but as I intend to replace the system anyway I can't be bothered to investigate what kind of RAM I'd need and where to get it. > One of the more serious problems with our current build system is that > it doesn't scale well to large numbers of cores. On a 32-core system, > with -j64, we're very rarely managing to have even 8 things able to run > in parallel. This should be addressed when the bmake import is fully > integrated and we can use meta mode for better dependency tracking. =20 > > Ninja has a concept of pools, so you can say 'only run one link job at a > time, but you can do two C++ compile jobs or 4 C compile jobs', and it > might be interesting to look at adding something similar to bmake, as > this can improve scalability a lot. Unfortunately I don't have these issues (yet). Fabian --Sig_/spZo5htsGvTI3oxP.vESby3 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlEZLfYACgkQBYqIVf93VJ3zQgCfQ1etO41ofWaizmTYVu7MlmP+ nwwAoKNgk69rL75myNWAvW0JD1uA4dWb =2rnR -----END PGP SIGNATURE----- --Sig_/spZo5htsGvTI3oxP.vESby3--