From owner-freebsd-ports  Wed Nov 11 15:53:23 1998
Return-Path: <owner-freebsd-ports@FreeBSD.ORG>
Received: (from majordom@localhost)
          by hub.freebsd.org (8.8.8/8.8.8) id PAA20721
          for freebsd-ports-outgoing; Wed, 11 Nov 1998 15:53:23 -0800 (PST)
          (envelope-from owner-freebsd-ports@FreeBSD.ORG)
Received: from mail.HiWAAY.net (fly.HiWAAY.net [208.147.154.56])
          by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id PAA20713;
          Wed, 11 Nov 1998 15:53:20 -0800 (PST)
          (envelope-from sprice@hiwaay.net)
Received: from localhost (sprice@localhost)
	by mail.HiWAAY.net (8.9.0/8.9.0) with SMTP id RAA23576;
	Wed, 11 Nov 1998 17:53:02 -0600 (CST)
Date: Wed, 11 Nov 1998 17:53:01 -0600 (CST)
From: Steve Price <sprice@hiwaay.net>
To: Satoshi Asami <asami@FreeBSD.ORG>
cc: ports@FreeBSD.ORG, msmith@FreeBSD.ORG
Subject: Re: auto ports build script
In-Reply-To: <199811112340.PAA17073@silvia.hip.berkeley.edu>
Message-ID: <Pine.OSF.4.02.9811111742390.9547-100000@fly.HiWAAY.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-freebsd-ports@FreeBSD.ORG
Precedence: bulk
X-Loop: FreeBSD.org

On Wed, 11 Nov 1998, Satoshi Asami wrote:

# The problem here is that dependencies are not trees.  What we have is
# an directed acyclic graph (DAG).  You can't guarantee ordering on
# DAG's unless you wait for completions of some processes, which a shell 
# "&" won't do while make does.

Yes they are DAG's.  And yes parallelizing within a DAG might
be difficult to achieve, but with 746 of these little beasties
it is trivial to do it among them.  That is, theoretically
all 746 could be run concurrently. :)

#  * 1) build C's package
#  * 2) build B's package
#  * 3) build A's package, pkg_delete A B C
#  * 
# This is nice, but what do you do if there is another port, D, that
# depends on B?  You don't have to wait for 3 to complete to start the
# build of D, you only need 2.

The recurse method takes care of it.  It would do something
like this:

1) build C's package
2) build B's pacakge
3) build D's package, pkg_delete D
4) build A's package, pkg_delete A B C

#  * Another example of a more complex tree could be: A depends
#  * on C, B depends on C, and C is standalone.  Your approach
#  * would do this:
#  * 
#  * 1) build C's package and pkg_delete C
#  * 2) pkg_add C, build A's package, pkg_delete A C
#  * 3) pkg_add C, build B's package, pkg_delete B C
# 
# Note 2 and 3 can be done in parallel.

Yes, they can but only in separate chroot envs.  As I said
earlier, you can achieve parallelism among DAGs just as easy
and with less overhead wrt to CPU and mass storage.

#  * My little script does this:
#  * 
#  * 1) build C's package
#  * 2) build A's package, pkg_delete A
#  * 3) build B's package, pkg_delete B C
#  * 
#  * Unless I'm missing something fundamental here, doing the
#  * latter in both cases seems to not only do the "same thing",
#  * but does it in a more efficient manner.  Please tell me
#  * how come I can't see the forest for the trees. :)
# 
# It's surely more efficient on one machine, but I'm just not sure how
# well it parallelizes. :)

See above. :)

-steve

# Satoshi
# 


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-ports" in the body of the message