From owner-freebsd-ports@FreeBSD.ORG Wed Jun 10 23:47:31 2009 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25B851065678; Wed, 10 Jun 2009 23:47:31 +0000 (UTC) (envelope-from mel.flynn+fbsd.ports@mailing.thruhere.net) Received: from mailhub.rachie.is-a-geek.net (rachie.is-a-geek.net [66.230.99.27]) by mx1.freebsd.org (Postfix) with ESMTP id A69898FC16; Wed, 10 Jun 2009 23:47:30 +0000 (UTC) (envelope-from mel.flynn+fbsd.ports@mailing.thruhere.net) Received: from smoochies.rachie.is-a-geek.net (mailhub.rachie.is-a-geek.net [192.168.2.11]) by mailhub.rachie.is-a-geek.net (Postfix) with ESMTP id 74A5D7E837; Wed, 10 Jun 2009 15:47:29 -0800 (AKDT) From: Mel Flynn To: freebsd-ports@freebsd.org Date: Wed, 10 Jun 2009 15:47:27 -0800 User-Agent: KMail/1.11.3 (FreeBSD/8.0-CURRENT; KDE/4.2.3; i386; ; ) References: <20090610132647.GC40713@atarininja.org> <200906101244.43479.mel.flynn+fbsd.ports@mailing.thruhere.net> <20090610205236.GD65705@atarininja.org> In-Reply-To: <20090610205236.GD65705@atarininja.org> MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_QYEMKypzS0D4k62" Message-Id: <200906101547.28358.mel.flynn+fbsd.ports@mailing.thruhere.net> X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: ports@freebsd.org, Wesley Shields , python@freebsd.org, miwi@freebsd.org Subject: Re: Python 2.6 update with portmaster X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jun 2009 23:47:31 -0000 --Boundary-00=_QYEMKypzS0D4k62 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline On Wednesday 10 June 2009 12:52:36 Wesley Shields wrote: > On Wed, Jun 10, 2009 at 12:44:42PM -0800, Mel Flynn wrote: > > On Wednesday 10 June 2009 05:26:47 Wesley Shields wrote: > > > It will be quite slow compared to using pkg_which (the normal method) > > > so be patient. > > > > Does pkg_which use the pkgdb? If so, maybe we could have pkg_info grow a > > cache for future encounters, or we can create one: > > My guess is that pkg_which is using pkgdb, which is why it is > significantly faster. > > > % grep -v '^@' /var/db/pkg/*/+CONTENTS >/tmp/pkg_info.cache > > % FOUND=`sed -ne 's,^/var/db/pkg/\(.*\)/+CONTENTS:lib/python2.6/site- > > packages/xcbgen/error.py$,\1,p' /tmp/pkg_info.cache`; echo $FOUND > > xcb-proto-1.4 > > > > So: > > 's,^${PKG_DBDIR}/\(.*\)/+CONTENTS:$$f$$,\1,p' > > I think there is a SoC project in the works to make our pkg utilities > much better. I don't know any of the details but I can only hope that > speed is one of the things being improved. True, however the above can be done now, though it's a bit tricky to squeeze into the find command, one could put this as shell script in files/pkg_which. And patch Makefile as attached. I haven't got a system to run this on, since I switched to 2.6 everywhere a long time ago, but the find command on the shell produced the desired output. > > > [1]: http://people.freebsd.org/~wxs/python26-portmaster.diff > > > > What's this for? > > +JOBS= sysctl -n kern.smp.cpus > > > > Should be != if it's gonna be used, but it's not referenced anywhere else > > that I can see. > > It's used in XARGS_CMD to provide a bit of parallelism in an effort to > be a bit faster. That was pretty blind of me. -- Mel --Boundary-00=_QYEMKypzS0D4k62 Content-Type: text/x-patch; charset="ISO-8859-1"; name="patch-lang-python-Makefile" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="patch-lang-python-Makefile" Index: python/Makefile =================================================================== RCS file: /home/ncvs/ports/lang/python/Makefile,v retrieving revision 1.161 diff -u -r1.161 Makefile --- python/Makefile 31 Jul 2007 16:28:32 -0000 1.161 +++ python/Makefile 10 Jun 2009 23:03:32 -0000 @@ -24,8 +24,19 @@ @${DO_NADA} # Major upgrade support -PORTUPGRADE_CMD?= ${LOCALBASE}/sbin/portupgrade -PKG_WHICH_CMD?= ${LOCALBASE}/sbin/pkg_which +PORTUPGRADE_CMD= ${LOCALBASE}/sbin/portupgrade +PKG_WHICH_CMD= ${LOCALBASE}/sbin/pkg_which +XARGS_CMD= ${XARGS} -0 + +.if defined(USE_PORTMASTER) +PORTUPGRADE_CMD= ${LOCALBASE}/sbin/portmaster +PORTUPGRADE_ARGS?= "" +PKG_WHICH_CMD= ${SETENV} LOCALBASE=${LOCALBASE} PKG_DBDIR=${PKG_DBDIR} \ + ${SH} ${FILESDIR}/pkg_which +JOBS= sysctl -n kern.smp.cpus +XARGS_CMD= ${XARGS} -L 1 -0 -P $$(${JOBS}) +.endif + upgrade-site-packages: @if [ ! -x ${PORTUPGRADE_CMD} ]; then \ ${ECHO_MSG} "Please install ports-mgmt/portupgrade."; \ @@ -36,14 +47,21 @@ if [ -d ${PREFIX}/lib/python$$ver ]; then \ UPD=`${FIND} ${PREFIX}/lib/python$$ver \ -type f -print0 | \ - ${XARGS} -0 ${PKG_WHICH_CMD} | \ + ${XARGS_CMD} ${PKG_WHICH_CMD} | \ ${GREP} -Fv '?' | \ ${EGREP} -v '^python2?[0-5]?-2' | \ ${SORT} -u`; \ if [ "$$UPD" ]; then \ - ${PORTUPGRADE_CMD} -f $$UPD; \ + if [ -n ${PORTUPGRADE_ARGS} ]; then \ + ${PORTUPGRADE_CMD} ${PORTUPGRADE_ARGS} -f $$UPD; \ + else \ + ${PORTUPGRADE_CMD} -f $$UPD; \ + fi; \ fi; \ fi; \ - done \ + done +.if defined(USE_PORTMASTER) + ${PKG_WHICH_CMD} -r +.endif .include --Boundary-00=_QYEMKypzS0D4k62--