From owner-freebsd-ports@FreeBSD.ORG Tue Apr 10 03:04:46 2007 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DF80C16A404 for ; Tue, 10 Apr 2007 03:04:46 +0000 (UTC) (envelope-from mail@maxlor.com) Received: from popeye1.ggamaur.net (popeye1.ggamaur.net [213.160.40.50]) by mx1.freebsd.org (Postfix) with ESMTP id 7398613C44C for ; Tue, 10 Apr 2007 03:04:46 +0000 (UTC) (envelope-from mail@maxlor.com) Received: from maxlor.mine.nu (maxlor@c-82-192-240-247.customer.ggaweb.ch [82.192.240.247]) by popeye1.ggamaur.net (8.13.7/8.13.7/Submit) with ESMTP id l3A2qwWE022999 for ; Tue, 10 Apr 2007 04:52:59 +0200 (CEST) (envelope-from mail@maxlor.com) Received: from localhost (unknown [127.0.0.1]) by maxlor.mine.nu (Postfix) with ESMTP id D171E2E1D9 for ; Tue, 10 Apr 2007 04:52:57 +0200 (CEST) X-Virus-Scanned: amavisd-new at atlantis.intranet Received: from maxlor.mine.nu ([127.0.0.1]) by localhost (atlantis.intranet [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IbhLpd802oxp for ; Tue, 10 Apr 2007 04:52:40 +0200 (CEST) Received: from merlin.intranet (merlin.intranet [10.0.0.16]) by maxlor.mine.nu (Postfix) with ESMTP id D345D2E19C for ; Tue, 10 Apr 2007 04:52:40 +0200 (CEST) From: Benjamin Lutz To: ports@freebsd.org Date: Tue, 10 Apr 2007 04:52:37 +0200 User-Agent: KMail/1.9.6 X-Face: $Ov27?7*N,h60fIEfNJdb!m,@#4T/d; 1hw|W0zvsHM(a$Yn6BYQ0^SEEXvi8>D`|V*F"=?utf-8?q?=5F+=0A=09R2?=@Aq>+mNb4`,'[[%z9v0Fa~]AD1}xQO3|>b.z&}l#R-_(P`?@Mz"kS; XC>Eti,i3>%@=?utf-8?q?g=3F=0A=094f?=,\c7|Ghwb&ky$b2PJ^\0b83NkLsFKv|smL/cI4UD%Tu8alAD MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart4368234.vHAbUkZaq5"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200704100452.40574.mail@maxlor.com> X-Scanned-By: MIMEDefang 2.61 on 213.160.40.60 Cc: Subject: parallel builds revisited X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 03:04:47 -0000 --nextPart4368234.vHAbUkZaq5 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hello, Some time ago, after buying a Core 2 Duo system, I've become interested=20 in doing something about the inherent single-threadedness of the ports.=20 Even though I have a dualcore machine, ports builds only ever use one=20 core. I started thinking about various approaches to introduce=20 parallelism to ports builds and wrote down my thoughts here: http://marc.info/?l=3Dfreebsd-ports&m=3D116124997126657&w=3D2 Well, since then I've tinkered with various approaches. I concentrated=20 on using make's -j feature. After adding the flag to the gmake=20 invocation in bsd.port.mk, I quickly noticed that some ports can take=20 advantage of the flag and thus build much more quickly (eg, all the KDE=20 ports), others are still single-threaded (X.org), but of course there=20 are also ports that fail to build (Openoffice.org). This means that a=20 per-port switch is required. To make this change as as unintrusive as possible, and to not have to=20 patch a large number of files after every portsnap, I've decided to=20 build a whitelist of ports which work with (or despite? :) ) parallel=20 building. So here's what I use currently: 1) Put "USE_LOCAL_MK=3Dyes" into /etc/make.conf 2) Use this as /usr/ports/Mk/bsd.local.mk: http://www.maxlor.com/temp/bsd.local.mk . (if you want just my changes in that file, you can run=20 "grep PARALLEL bsd.local.mk") 3) Save this to /usr/local/etc/parallel_builds.conf: http://www.maxlor.com/temp/parallel_builds.conf . This is a list of ports as stored in PKGORIGIN, or as pkg_info -o reports them. The code will run make with -jx, where x is the number of cores your=20 machine has, as reported by kern.smp.cpus. So now I would like to invite you test, comment, or simply philosophize=20 on these changes. Cheers Benjamin --nextPart4368234.vHAbUkZaq5 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQBGGvv1zZEjpyKHuQwRAjTfAJwKX+1bvi9FMLdyLvDCNqfGvt7pQACdHz9h AmhjMEgAdLmC5z8dY4OLAtE= =uEM0 -----END PGP SIGNATURE----- --nextPart4368234.vHAbUkZaq5--