Date: Fri, 29 Sep 2006 18:38:45 -0400 From: Kris Kennaway <kris@obsecurity.org> To: ports@freebsd.org Subject: Re: Yet another INDEX builder. Message-ID: <20060929223845.GA38759@xor.obsecurity.org> In-Reply-To: <20060929222519.GB63185@graf.pompo.net> References: <20060929172122.GA80952@lpthe.jussieu.fr> <20060929222519.GB63185@graf.pompo.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--DocE+STaALJfprDB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 30, 2006 at 12:25:19AM +0200, Thierry Thomas wrote: > Le Ven 29 sep 06 ? 19:21:22 +0200, Michel Talon <talon@lpthe.jussieu.fr> > ?crivait?: > > Hello, >=20 > Hi, >=20 > > i have written yet another index builder in python, which can be found = here: > > http://www.lpthe.jussieu.fr/~talon/build_index.py > > I think it works relatively well and fast, and can provide building blo= cks for > > doing a package updater alternative to portupgrade. Of course, like the > > documentation, it is completely free. I would be happy if some people c= ould > > test it on a real biproc machine, or a machine with fast disks, to see = the > > timings (*) one gets. On my machine, a P4 3Ghz, oldish, it takes 23 min= utes to > > complete building the INDEX, but the python script doesn't run more tha= n 1 > > minute. Everything else is IO, i think. So one cannot expect any perfor= mance >=20 > It took longer on my P4 3.6GHz: >=20 > The tree (15689ports) takes 2789.84287214 seconds to explore. > Now computing the recursively extended dependencies. > Took 3.02815794945 seconds. > Now converting to packages and sorting. > Last phase takes 4.7659368515 seconds. > Total time spent: 2800.94901609 seconds. > python build_index.py 3024,42s user 2050,37s system 181% cpu 46:41,64 to= tal >=20 > Remarks: > - this is not a benchmark, and I was using the machine in the same time; > - I have several local ports or local tweaks which have been reported as > "The port xxx/yyy is obsolete". As Michel mentions, INDEX build is almost completely I/O bound, so it almost entirely depends on the speed of your disk and how congested it is from other I/O. Thus it's not really meaningful to compare numbers from one machine to another, but it still may be useful to compare to 'make index' times. You should not expect a significant difference unless something is going wrong though. FYI, I'm not sure if the python version is parallelizable, but you do get a small benefit from using parallelized 'make index' builds (via INDEX_JOBS) on a typical SMP machine. Kris --DocE+STaALJfprDB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFFHaB0Wry0BWjoQKURAjWKAJ9vl/LtfBi88+nHat6l/yra0/WWiwCgwN6E PBfhpqZAPiyvqdgWpsfCQdc= =wdwt -----END PGP SIGNATURE----- --DocE+STaALJfprDB--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060929223845.GA38759>