Date: Thu, 20 Mar 2008 10:42:46 +0100 From: Michel Talon <talon@lpthe.jussieu.fr> To: Doug Barton <dougb@FreeBSD.org>, freebsd-ports@FreeBSD.org Subject: Re: Utility for safe updating of ports in base system Message-ID: <20080320094246.GA40807@lpthe.jussieu.fr> In-Reply-To: <alpine.BSF.1.10.0803200047360.54264@ync.qbhto.arg> References: <20080320001048.GA39125@lpthe.jussieu.fr> <alpine.BSF.1.10.0803200047360.54264@ync.qbhto.arg>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Mar 20, 2008 at 01:05:27AM -0700, Doug Barton wrote: > On Thu, 20 Mar 2008, Michel Talon wrote: > > >Doug Barton wrote: > >i would venture to say that such an utility > >should be able to upgrade things based of *binary* packages, and > >consequently that portmaster is not a suitable candidate. > > That ability is not included in the current requirements document, and was > not specifically mentioned the last time we had the discussion on the > list. If the portmgr folks intend that to be a requirement, the current > ideas list entry should be amended. Indeed you are right, but i think this should be a requirement for the reasons discussed below, and is implicit in the fact that the projects refers to a "portupgrade written in C", and portupgrade has such a feature. > > >For example > >pkg_add installs a binary package, if you want to compile and install > >you run "make all install clean" in the ports tree. > > Um, you lost me there. This is simple, all pkg_* tools operate on binary packages at present, it would be strange that a newcomer, pkg_upgrade has no way to operate on binary packages. > > >One of the > >requirements of an upgrade system is predictability, this can only > >be achieved by using binary packages. > > You gain a certain amount of flexibility with packages, at the expense of > being able to customize things. As long as the user understands that, then > it's fine. > I agree that the possibility of compiling from source brings ability to costumize things. However this very ability ruins all hope of having a smooth upgrade system. Due to customization, as soon as you have many ports installed, there is a combinatorial explosion of possibilities and nobody can test that the upgrade process works. OpenBSD people have been converted to this idea, and now push for an upgrade from binary packages exclusively. So i think that the user should have two possibilities: - either he wants to customize, and he is on his own. He will need to compile his software, and in this case portmaster is the perfect tool for managing his upgrades. Since the compilation will take most of the time, it is not relevant to consider performance questions on the portmaster side. - or he wants to use a tried and true upgrade path, he doesn't want to spend time compiling, he wants speed. Basically he wants the Debian or Ubuntu experience. In this case, using prebuilt binary packages is the only reasonable way of achieving the result. I agree that due to licence peculiarities (think e.g. Java) there are a small number of ports which will need compilation, so the experience cannot be perfect. > >Another requirement, in my opinion, > >is speed, and the lack of speed, which is completely hidden when you > >compile your packages will be immediately apparent if you try to use > >packages. Indeed portupgrade has options -P and -PP to work with > >packages which could serve as a prototype for a "pkg_upgrade" written > >in C, except that they work poorly, and in particular run slowly. > > Where do you think the slowness is? Several causes of slowness have already been identified. Parsing /var/db/pkg is slow, this is why the idea of using a database has been advocated. Much worse, running make in a port directory, even only to get the value of a variable is slow. Programs like portupgrade do such things repeatedly without caching the results in memory and incur large time penalties. Similarly for each package he wants to download, portupgrade opens an ftp connection and retreives it independently, etc. Obviously no effort at all has been spent so that things are fast. > >One of its features, that i consider very important for correct > >operation, is that it computes the list of all packages to be deleted > >and all packages to be installed and asks the user if he agrees before > >doing anything. > > Why do you consider this an important feature? (I'm not disagreeing, just > curious about your thought process here.) Because you can see that something is going to break in advance and renounce to the upgrade. This occurred several times for me on Debian. Usually this is because some package has a security problem, and this is solved waiting one or two days. > > >It fetches all necessary packages before installing or > >deleting anything. > > That seems sensible, thanks for mentioning that bit. > > >Hence you can be sure that the upgrade process will > >not end in a mess if something crashes in the middle, like it is the > >case with all present standard FreeBSD upgraders. > > Not sure if this helps the situation you're referring to or not, but > portmaster will by default make a backup package of each port that it > updates, so if something dies in the middle you could back out of it by > hand if you need to. > Yes, so does portupgrade, but it deletes the backup as soon as the installation of the new port succeeds. This means that in the event of a crash in the middle you remain with a half upgraded system, and no way to backup completely to the previous working state. In my experience, such half upgraded systems sometimes work OK, and sometimes half work. > Now all that said, I'd love to see us move to a much more robust package > management system, or even just a better interface to the one we have. The > problem is that I don't have the time to do that as a volunteer project, > and I don't think anyone else does either. :) Probably the present package management system doesn't keep enough state on each installed package so that an upgrade system works reliably. For example, referring to the possibilities to tweak options, i think that keeping a copy of the options used for compilation of the port in the package could help, or at least a flag indicating if the package is basically standard or has been tweaked. Similarly keeping track of wether a package has been installed by order of the end user or as a dependency could help to unclutter after an upgrade. On the other hand, basic libraries, which, when upgraded, require to upgrade a lot of ports should have a better way to require that, rather that asking the end user to read UPGRADING. Finally both the speed questions and the necessity of keeping more state point to the idea of using a database instead of abusing the filesystem for doing a database job. But this is precisely one of the Soc projects. -- Michel TALON
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080320094246.GA40807>