From owner-freebsd-ports@FreeBSD.ORG Fri Apr 10 01:40:20 2009 Return-Path: Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72BCD1065679; Fri, 10 Apr 2009 01:40:20 +0000 (UTC) (envelope-from reg@openpave.org) Received: from warsaw.ucdavis.edu (warsaw.ucdavis.edu [128.120.32.41]) by mx1.freebsd.org (Postfix) with ESMTP id 54DC08FC1E; Fri, 10 Apr 2009 01:40:20 +0000 (UTC) (envelope-from reg@openpave.org) Received: from flint.openpave.org ([169.237.230.40]) by warsaw.ucdavis.edu (8.13.7/8.13.1/it-defang-5.4.0) with ESMTP id n3A1CYvc022234 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Apr 2009 18:12:34 -0700 (PDT) Received: from sandy.local (flint.local [192.168.1.5]) by flint.openpave.org (8.14.3/8.14.3) with ESMTP id n3A1CYJo021925; Fri, 10 Apr 2009 01:12:34 GMT (envelope-from reg@sandy.local) Received: (from reg@localhost) by sandy.local (8.14.3/8.14.3/Submit) id n3A1CYKm021924; Thu, 9 Apr 2009 18:12:34 -0700 (PDT) (envelope-from reg) Date: Thu, 9 Apr 2009 18:12:34 -0700 From: Jeremy Lea To: David Forsythe Message-ID: <20090410011234.GA20765@flint.openpave.org> Mail-Followup-To: Jeremy Lea , David Forsythe , freebsd-ports@FreeBSD.ORG, pav@FreeBSD.ORG References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on av4 X-Virus-Status: Clean X-Scanned-By: MIMEDefang 2.57 on 128.120.32.41 Cc: pav@FreeBSD.ORG, freebsd-ports@FreeBSD.ORG Subject: Re: Parallel builds, build locks, pkg_dbdir locked X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2009 01:40:21 -0000 Hi, On Thu, Apr 02, 2009 at 05:57:05PM -0400, David Forsythe wrote: > patch: http://dmz2.khome.utcorp.net/~dforsyth/port.mk-locks.diff Since no one else seems to have commented, a few quick thoughts. I have not actually tried your patch. 1. Is there any reason not to do the locking? Removing NO_LOCKS would make things much simpler. Would it be possible to not lock the whole initial make call and only lock the main targets (grep for USE_SUBMAKE). 2. The locks seem to be created in ${PORTSDIR}/${PKGORIGIN}, which might be read only. The locks should be created in ${PORT_DBDIR}. Or use ${PORTSDIR}/${PKGORIGIN}/Makefile and make lockf always keep the file! 3. I don't see how parallel fetching is useful. If you've got enough bandwidth to make it work, then you've got enough bandwidth to be considered rude to the FTP site at the receiving end. If you haven't then you're not going to benefit from multiple fetches. For checksums, you also don't want to stress your disks with parallel reads. 4. Use: .if defined(MAX_PARALLEL_DEPENDS) && ${MAX_PARALLEL_DEPENDS} != "1" _P_DEPENDS = -j ${MAX_PARALLEL_DEPENDS} .endif Forcing BATCH is bad. It changes how some ports work. Rather, it would be better to force a config-recursive if someone wants this. 5. Can you use "${MAKE} -C $DIR ..." instead of "${SH} -c {cd ${DIR} && $MAKE ...}" as an argument for ${LOCKF} 6. It is not clear to me that all of the merging of lib-depends into the other depends targets buys you anything. I don't think it allows you to run the different types of depends targets in parallel? The logic of parallel depends would be cleaner if you just defined all of the individual targets for each depends type in one loop, the lib depends in second loop, and built the locking targets and things a third loop. Not changing the whitespace or changing $$i to $${dependency} would be good, because they would make the patch easier to read. 7. The locking for PKGDIR is different to the other locking, and probably should always be done. You could wrap the entire target into the ${SH} call. Do you need to wrap the PKG_INFO command in a lockf call? Otherwise, thank you for working on this important feature! I often try to launch multiple port builds in different terminals and hate it when they crash... Regards, -Jeremy -- FreeBSD - Because the best things in life are free... http://www.freebsd.org/