Date: Fri, 21 Mar 2008 17:21:10 +1100 From: Peter Jeremy <peterjeremy@optushome.com.au> To: Doug Barton <dougb@freebsd.org> Cc: Michel Talon <talon@lpthe.jussieu.fr>, freebsd-ports@freebsd.org Subject: Re: Utility for safe updating of ports in base system Message-ID: <20080321062110.GB85901@server.vk2pj.dyndns.org> In-Reply-To: <47E2BF70.2050109@FreeBSD.org> References: <20080320001048.GA39125@lpthe.jussieu.fr> <alpine.BSF.1.10.0803200047360.54264@ync.qbhto.arg> <20080320094246.GA40807@lpthe.jussieu.fr> <47E2BF70.2050109@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--i9LlY+UWpKt15+FH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 20, 2008 at 12:48:00PM -0700, Doug Barton wrote: >Michel Talon wrote: >> Since the compilation will take most of the >> time, it is not relevant to consider performance questions on the >> portmaster side. > >Having spent a substantial amount of time doing performance tuning on >portmaster, I beg to differ. I also agree with Doug here. Compilation may take most of the time but any time used by the ports upgrading infrastructure will directly add to the total time taken by an upgrade. And, with the switch to "modular X.org", systems are likely to have lots more ports than before so any per-port overhead is much more of issue than before. The "compilation will take most of the time so don't worry about anything else" approach leads to the GNU configure and libtool idiocy - where the port spends several minutes determining the system configuration and then 30 seconds compiling the port. >> Several causes of slowness have already been identified. Parsing >> /var/db/pkg is slow, > >s/is/can be/. There are some things I need to dig out of /var/db/pkg >that are relatively slow, yes. By far the most expensive is >determining the name of an installed port by grep'ing for ORIGIN in >all the +CONTENTS files. =2E.. >Now, would having some kind of database be better? Maybe, maybe not. Note that UFS is a database: If I've understood you correctly, the main problem is that there is no appropriate index to map a port directory to an installed package name. This could be fixed... There are regular discussions about moving more configuration information into some sort of database format. We already use Berkeley DB for some things and importing SQLite has been suggested on at least one occasion. It would be interesting to see if changing /var/db/pkg into a "traditional" database will improve performance without adversely affecting administration. >> this is why the idea of using a database has been >> advocated. Much worse, running make in a port directory, even only to >> get the value of a variable is slow. > >Now on that I agree with you completely. But now you're talking about >a much more fundamental redesign issue that is way outside the scope >of the project we're discussing. One option here might be to restrict port Makefiles to a subset of pmake's functionality and move much of the configuration data currently embedded in /usr/ports/Mk/* into separate files so that 'make describe' can be implemented using an alternative tool that doesn't need to parse most of /usr/ports/Mk/*. >Doing N number of downloads in parallel (where N should be user >configurable) is preferable. ... OTOH, I've >never had a user complain that I'm swamping their machine with >distfile downloads, so it's not been a high priority. I've seen a couple of FTP mirror sites that explicitly state "no more than one client connection at a time is allowed from a single computer" as part of their conditions of use. Swamping a mirror site with parallel downloads may be seen as unfriendly by the site admins. >> Probably the present package management system doesn't keep enough state >> on each installed package so that an upgrade system works reliably. =2E.. >Here you and I are in agreement. <metoo> I doubt we will ever manage to move to a totally binary upgrade approach for all ports - for licensing reasons if nothing else. This means that an end user will probably wind up building some ports. It's also unlikely that the "one-size-fits-all" approach implied by the package system will suit everyone - my favourite gotcha is the lack of mod_php when you use the php and Apache packages. I also occasionally say "make -DWITH_foo -DWITHOUT_bar" and it would be handy to know what options I actually used when I go to rebuild the port (or try to work out why a dependency isn't working as expected). --=20 Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. --i9LlY+UWpKt15+FH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.8 (FreeBSD) iEYEARECAAYFAkfjU9YACgkQ/opHv/APuIfKHgCeJHqNP+Rfn2aLPz8orefX3UIr ffoAoMKLjbRem6p5yqFR1IjSsyvHYua7 =ObVU -----END PGP SIGNATURE----- --i9LlY+UWpKt15+FH--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080321062110.GB85901>