From owner-freebsd-ports-announce@FreeBSD.ORG Thu Oct 3 08:49:13 2013 Return-Path: Delivered-To: ports-announce@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4C7B1291 for ; Thu, 3 Oct 2013 08:49:13 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CCE352D55 for ; Thu, 3 Oct 2013 08:49:12 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id p61so2311326wes.30 for ; Thu, 03 Oct 2013 01:49:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=5BEYDjYvoFsSgP+9rbAOf2qKif6oAulLbk+N7cEX1xA=; b=fSf20fV4HDmYGEZlFV79yDUtHndW7AC+gK2F80PDG+qeoO/7++SKKYu+uVWbQxNUyo ytcGUsXg2j//0Q/g5qyTW2dxBJT6N9Vq1DtKvxxftAa+/7ix32gKn2cH2bdcCL3mptUC 6819jxLjJ2k+mayXtmg8SU/MJZBDP4wa7E0JY3EcdLmH9iS3TnJ2Czn1TeKYA47TMhnu mkofdydbd6m2cjT/PYrcG4TVQN7S4MbeQPBYKG+qna0V9KGEVBTZVAKfS0EMGA0hryRv vFwBWi/l3TQAqa667p4aOmbXGIMPAEO3Pq8LMqJ5NlrqxgfOQ2ehCQyMZgo+99tOwlKi ICxg== X-Received: by 10.194.235.138 with SMTP id um10mr6202078wjc.30.1380790151168; Thu, 03 Oct 2013 01:49:11 -0700 (PDT) Received: from ithaqua.etoilebsd.net (ithaqua.etoilebsd.net. [37.59.37.188]) by mx.google.com with ESMTPSA id j5sm11594374wia.4.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 03 Oct 2013 01:49:07 -0700 (PDT) Sender: Baptiste Daroussin Date: Thu, 3 Oct 2013 10:49:05 +0200 From: Baptiste Daroussin To: ports-announce@FreeBSD.org Message-ID: <20131003084905.GC99713@ithaqua.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="i7F3eY7HS/tUJxUd" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [FreeBSD-Ports-Announce] [HEADSUP] Staging, packaging and more X-BeenThere: freebsd-ports-announce@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Important news and instructions about the FreeBSD " ports collection" \[moderated\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Oct 2013 08:49:13 -0000 --i7F3eY7HS/tUJxUd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi all, I want to give to all porters infomations on the rationnale behind all the changes, to explain why some things has happened, the rationale behind what we still need to do to get the ports tree into a modern binary world. 1/ Staging You may has notice that staging has hit the ports tree, staging is something really important, all packages system are using that feature for eons, sometime called DESTDIR sometime called FAKEDIR. Staging is consistent in adding a new step while building packages: install everything into ${STAGEDIR}. Then we can directly create packages out of that directory without having to install into /. What the implementation does is: With pkg_install (legacy package tools): - Create a package from the stage directory - Install that package. With pkgng: - Create a package from that stage directory _OR_ - Install to / (pkgng can consider the stage directory as if it is a package) What it means is, that it is the end of the bad plist, only things in plist are really installed, this is the end of the broken packages that break the system because they fail in the middle of make install as a package will only be installed once we are sure the build process properly pass. It allows right now to build and create a package without the need to be root. It gives us new targets: - make makeplist (to create the pkg-plist) - make check-orphans (to print what is in the stage directory but not in pkg-plist) It reduces code duplication: in the end the installation is being done via a package installation meaning that PKG-MESSAGE is automatically printed, all pre/post installation scripts are executed. It reduces patches: no need anymore to patch upstream Makefiles to avoid installing the DOCS just do not list them in pkg-plist or conditionnaly list them in the pkg-plist. This also allows lots of new features to come: - Allow to create sub-packages - Allow to create debuginfo packages. - Allow to do a lot of sanity check in the staging area to improve our QA To convert your ports refer to: https://wiki.freebsd.org/ports/StageDir Along with stage you have noticed that MAN* and MLINKS has gone, and now all the manpages should be added to the plist. BTW MANCOMPRESSED is also not necessary now. There is 3 reason behind making it go and not being replaced: - The initial goal of those macros was to be able to compress the right manpages and to handle the different hardlinks/symlinks between those pages. Because it was directly installed in / rather than using a stage directory, it was necessary to at a point list them. and to properly track the different links in MLINKS (which wasn't done properly in most ports btw :)) With stage, we have a new compress-man which does it properly on its own without the need to get the list of the manpages, without the need of an explicit declaration of the links. This syntax to handle localized man pages was also terrific :) and if you have a port mixing manpages in "non regular" and "regular" localtion, it was totally messing - Consistency, a port is basically: some metadata, a plist and some operations to perform. for metadata all metadata are in the Makefile, all operations are in the Makefile but plist can be a mix of pkg-plist and Makefile, this is inconsistent, it makes sometime hard to figure out why a file has been added to the plist etc. Also this makes us having tons of targets define to handle those extras entries and results in highly inefficient make(1) usage. - stage is also a first step to go to sub-packages, sub-packages will basically be: 1 port able to produce N packages, to be able to do this we will use multiple plist, having the files properly defined already in plist will help that. Having files defined in macros on Makefile will make it hard to determine which one should go in which plist. Last thing I would like to add about it: I don't see the difference personnaly between listing N lines of manpages into Makefile MAN* macros where btw you have to manually define the categories where to put them and adding those N lines directly into the plist where make makeplist and/or make check-orphans can help you getting the exact lines automatically? 2/ Binary packages For the sake of being able to have clean working binary packages, pkgng has had to use a dirty hack: origin is used has an identifier instead of the package name. Why: How to determine than ImageMagick-nox11 and ImageMagick are the same package? and if I don't know they are the same package how to upgrade them properly? same goes how the package manager can know that mysql-client-5.2.1 is not an upgrade of mysql-client-5.0.1? How can pkgng determine which subversion is the right version to use: 'pkg install subversion' will propose all possible subversion, in that case the higher version is probably the default version, but in perl case, the higher version is probably not the one you want. Some packages are totally different project but have the same package name... How can pkgng differenciate them. So please stop using LATEST_LINK to differenciate the different port but directly make _unique_ package names, pkgng will switch back to package name as soon as possible (that will also solve the ugly and stupid pkg set -o). There is multiple possibility to make sure to properly handle multiple versions of packages: 1/ suffix everything with part of the version (like tcl) 2/ suffix everything but the default with part of the version 3/ have only 2 or 3 different version: project-legacy project project-devel Please stop renaming the packages base on options! options are now registered with the package with both pkgng and pkg_install, please stop adding suffix base on options! Think about the end user, he will try to install a package, it should be as transparent for him as possible. Note that the 2 above are mandatory if you want to provide sub packages and flavours! regards, Bapt --i7F3eY7HS/tUJxUd Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlJNL4EACgkQ8kTtMUmk6EzrLgCePO+IiBELpekGWD9QwE8yHgmA deYAoLFlKB9QCUwwTaBaA6yWbnWTdi/m =0i75 -----END PGP SIGNATURE----- --i7F3eY7HS/tUJxUd--