Date: Tue, 10 Apr 2007 04:52:37 +0200 From: Benjamin Lutz <mail@maxlor.com> To: ports@freebsd.org Subject: parallel builds revisited Message-ID: <200704100452.40574.mail@maxlor.com>
next in thread | raw e-mail | index | archive | help
--nextPart4368234.vHAbUkZaq5 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hello, Some time ago, after buying a Core 2 Duo system, I've become interested=20 in doing something about the inherent single-threadedness of the ports.=20 Even though I have a dualcore machine, ports builds only ever use one=20 core. I started thinking about various approaches to introduce=20 parallelism to ports builds and wrote down my thoughts here: http://marc.info/?l=3Dfreebsd-ports&m=3D116124997126657&w=3D2 Well, since then I've tinkered with various approaches. I concentrated=20 on using make's -j feature. After adding the flag to the gmake=20 invocation in bsd.port.mk, I quickly noticed that some ports can take=20 advantage of the flag and thus build much more quickly (eg, all the KDE=20 ports), others are still single-threaded (X.org), but of course there=20 are also ports that fail to build (Openoffice.org). This means that a=20 per-port switch is required. To make this change as as unintrusive as possible, and to not have to=20 patch a large number of files after every portsnap, I've decided to=20 build a whitelist of ports which work with (or despite? :) ) parallel=20 building. So here's what I use currently: 1) Put "USE_LOCAL_MK=3Dyes" into /etc/make.conf 2) Use this as /usr/ports/Mk/bsd.local.mk: http://www.maxlor.com/temp/bsd.local.mk . (if you want just my changes in that file, you can run=20 "grep PARALLEL bsd.local.mk") 3) Save this to /usr/local/etc/parallel_builds.conf: http://www.maxlor.com/temp/parallel_builds.conf . This is a list of ports as stored in PKGORIGIN, or as pkg_info -o reports them. The code will run make with -jx, where x is the number of cores your=20 machine has, as reported by kern.smp.cpus. So now I would like to invite you test, comment, or simply philosophize=20 on these changes. Cheers Benjamin --nextPart4368234.vHAbUkZaq5 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQBGGvv1zZEjpyKHuQwRAjTfAJwKX+1bvi9FMLdyLvDCNqfGvt7pQACdHz9h AmhjMEgAdLmC5z8dY4OLAtE= =uEM0 -----END PGP SIGNATURE----- --nextPart4368234.vHAbUkZaq5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200704100452.40574.mail>