From owner-freebsd-ports@FreeBSD.ORG Sat Jul 12 13:19:28 2008 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2809A1065676 for ; Sat, 12 Jul 2008 13:19:28 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from shiva.jussieu.fr (shiva.jussieu.fr [134.157.0.129]) by mx1.freebsd.org (Postfix) with ESMTP id B252F8FC1A for ; Sat, 12 Jul 2008 13:19:27 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from parthe.lpthe.jussieu.fr (parthe.lpthe.jussieu.fr [134.157.10.1]) by shiva.jussieu.fr (8.14.3/jtpda-5.4) with ESMTP id m6CD4GhL013546 for ; Sat, 12 Jul 2008 15:04:17 +0200 (CEST) X-Ids: 164 Received: from niobe.lpthe.jussieu.fr (niobe.lpthe.jussieu.fr [134.157.10.41]) by parthe.lpthe.jussieu.fr (Postfix) with ESMTP id 782F7237190 for ; Sat, 12 Jul 2008 15:04:15 +0200 (CEST) Received: by niobe.lpthe.jussieu.fr (Postfix, from userid 2005) id 6600A8B; Sat, 12 Jul 2008 15:04:15 +0200 (CEST) Date: Sat, 12 Jul 2008 15:04:15 +0200 From: Michel Talon To: freebsd-ports@freebsd.org Message-ID: <20080712130415.GA97957@lpthe.jussieu.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0 (shiva.jussieu.fr [134.157.0.164]); Sat, 12 Jul 2008 15:04:17 +0200 (CEST) X-Virus-Scanned: ClamAV 0.93/7417/Tue Jun 10 03:14:29 2008 on shiva.jussieu.fr X-Virus-Status: Clean X-Miltered: at jchkmail2.jussieu.fr with ID 48788FB2.004 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 48788FB2.004/134.157.10.1/parthe.lpthe.jussieu.fr/parthe.lpthe.jussieu.fr/ X-j-chkmail-Score: MSGID : 48788FB2.004 on jchkmail2.jussieu.fr : j-chkmail score : . : R=. U=. O=. B=0.013 -> S=0.013 X-j-chkmail-Status: Ham Subject: Upgrading through packages: an experience. X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2008 13:19:28 -0000 Hello, Since KDE recently appeared in the Latest prebuilt packages, and my main desktop ports (running FreeBSD-7.0-RELEASE) had not been upgraded since more than a year i decided to test my pkgupgrade (www.lpthe.jussieu.fr/~talon/pkgupgrade) on a machine with many ports installed (all Gnome and KDE basically, plus many other things) and going through several important changes in the ports system (the new modularized Xorg, the new gettext, etc.). I wanted a worst case test, including using the Latest packages and not a RELEASE. The net result is that it went through without problems, at a speed comparable to that one observes for Debian upgrades, but at the end some small glitches remained, due to using Latest packages. Some details follow: -first the script pkgupgrade crashed. This is due to the fact that the old python version used libpthread, and there are apparent bugs on 7.0, while it worked OK on 6.2. This was solved using libmap.conf to force use of libthr for threading. Since KSE is now deprecated it is not useful to describe this bug more fully. Anyways programs doing IO and threading don't act reliably with libpthread under 7.0. - after that things were smooth. There has been fantastic progress in the utility pkg_add, it runs at least twice faster than last year. This explains the very speedy upgrade i had. More precisely, first the specs of the machine, it is a 4 years old Pentium 4 with IDE disk, so nothing particularly fast. There were 744 installed ports initially. Some of them i put on hold, since they are very heavy to build or install (java, tetex). So finally pkgupgrade removed 616 old packages and installed 877 new packages (the difference is of course because of the new modular Xorg). A few remained to be compiled. At the end i have now 1017 installed ports. - the speed of this upgrade was really remarkable. The initial analysis by pkgupgrade took 2 min, the download of necessary packages (for a total of 1.3 GB) from the french FreeBSD mirror took 5 min (i have a 100Mb/s connection to it, but this also shows the gains of maintaining a unique ftp connection to the server for the whole download), and was simultaneous with backing up the old packages (only shared libraries and config files) which took 28 mn. So after half an hour i had a shell script that i reviewed fully to avoid removing stuff that i had forgotten to put on hold. - then i runned the shell script UpgradeShell, which removed old packages (took 18 mn, pkg_delete is still slow, compared to pkg_add) then installed 877 packages in 48 mn! This is absolutely fantastic, last year i spent here 2 hours in a much smaller installation. Finally it compiled a few packages, notably one of them kdewebdev took 2 hours by itself, that is much more than the complete upgrade procedure. This illustrates the point that compiling from source is a waste of time. I have put the Logs here http://www.lpthe.jussieu.fr/~talon/2008-07-11.tgz so that one can see how it runs in practice. Now the problems. They come from the fact that the Latest packages are not always coherent between themselves or with the libraries in FreeBSD-7.0. So, while most programs runned perfectly well, a few crashed when i tried them. I traced most problems to the fact that the use of fcntl() to lock files did not work in some cases. This was for example the case for tin when it tries to lock the "posted" file. I had to recompile such ports, now they run fine. An example which doesn't run is kdm because it needs to lock /var/run/kdm.lock. This is very annoying since kdebase is huge to recompile. All other KDE stuff runs OK. The lesson of this story is that it is *not* a good idea to use the Latest packages when doing binary upgrades, one should stick to RELEASE. Conclusion: binary upgrades, even in very messy situation are very doable. They take a reasonable time, completely comparable to what one may expect under Linux. Compiling a single KDE port took longer that the whole upgrade procedure except the compilation. -- Michel TALON