From owner-freebsd-ports@FreeBSD.ORG Thu Oct 3 08:48:21 2013 Return-Path: Delivered-To: ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CD04819D for ; Thu, 3 Oct 2013 08:48:21 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 59A402D43 for ; Thu, 3 Oct 2013 08:48:21 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id hj3so8552097wib.1 for ; Thu, 03 Oct 2013 01:48:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=IThVi0a5jIvzsCcgzdSxM23xBtznCBi6yCOmIYeMrTs=; b=fx/XtLHCPDvAbUgHGuttmBXyI79zjfLdbceZ6lOviif+slzZ6W9f39xN+xmr8iW/DV h26FgTqTckVtv3K3LOjQheTQ5CQ8/sXmE/XuaTitkeGkV6eyOPMXiSc0nAxCas9ZVlk2 MZyBiw5DXrlPM8eTcMXDzT7KWe6iGJzObJ5eWVjJvb40a7EF247BvXP176/m0llvE8Ll jF8nKJtYbg5vV3+l3PuN6x8+9gcvCPyZp6Hg2gCODkxsnt3wzDSZHCIha4TIAUe8R7H4 Hbq6QqswjjMx6NWOtLCf97QremjI20JGs9zb1WCgLNyEuiAocuhH/5H0nMbz7EnwzbFw C6BA== X-Received: by 10.194.60.73 with SMTP id f9mr384582wjr.65.1380790099565; Thu, 03 Oct 2013 01:48:19 -0700 (PDT) Received: from ithaqua.etoilebsd.net (ithaqua.etoilebsd.net. [37.59.37.188]) by mx.google.com with ESMTPSA id e1sm24548137wij.6.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 03 Oct 2013 01:48:18 -0700 (PDT) Sender: Baptiste Daroussin Date: Thu, 3 Oct 2013 10:48:14 +0200 From: Baptiste Daroussin To: ports@FreeBSD.org Subject: [HEADSUP] Staging, packaging and more Message-ID: <20131003084814.GB99713@ithaqua.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qcHopEYAB45HaUaB" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Oct 2013 08:48:21 -0000 --qcHopEYAB45HaUaB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi all, I want to give to all porters infomations on the rationnale behind all the changes, to explain why some things has happened, the rationale behind what we still need to do to get the ports tree into a modern binary world. 1/ Staging You may has notice that staging has hit the ports tree, staging is something really important, all packages system are using that feature for eons, sometime called DESTDIR sometime called FAKEDIR. Staging is consistent in adding a new step while building packages: install everything into ${STAGEDIR}. Then we can directly create packages out of that directory without having to install into /. What the implementation does is: With pkg_install (legacy package tools): - Create a package from the stage directory - Install that package. With pkgng: - Create a package from that stage directory _OR_ - Install to / (pkgng can consider the stage directory as if it is a package) What it means is, that it is the end of the bad plist, only things in plist are really installed, this is the end of the broken packages that break the system because they fail in the middle of make install as a package will only be installed once we are sure the build process properly pass. It allows right now to build and create a package without the need to be root. It gives us new targets: - make makeplist (to create the pkg-plist) - make check-orphans (to print what is in the stage directory but not in pkg-plist) It reduces code duplication: in the end the installation is being done via a package installation meaning that PKG-MESSAGE is automatically printed, all pre/post installation scripts are executed. It reduces patches: no need anymore to patch upstream Makefiles to avoid installing the DOCS just do not list them in pkg-plist or conditionnaly list them in the pkg-plist. This also allows lots of new features to come: - Allow to create sub-packages - Allow to create debuginfo packages. - Allow to do a lot of sanity check in the staging area to improve our QA To convert your ports refer to: https://wiki.freebsd.org/ports/StageDir Along with stage you have noticed that MAN* and MLINKS has gone, and now all the manpages should be added to the plist. BTW MANCOMPRESSED is also not necessary now. There is 3 reason behind making it go and not being replaced: - The initial goal of those macros was to be able to compress the right manpages and to handle the different hardlinks/symlinks between those pages. Because it was directly installed in / rather than using a stage directory, it was necessary to at a point list them. and to properly track the different links in MLINKS (which wasn't done properly in most ports btw :)) With stage, we have a new compress-man which does it properly on its own without the need to get the list of the manpages, without the need of an explicit declaration of the links. This syntax to handle localized man pages was also terrific :) and if you have a port mixing manpages in "non regular" and "regular" localtion, it was totally messing - Consistency, a port is basically: some metadata, a plist and some operations to perform. for metadata all metadata are in the Makefile, all operations are in the Makefile but plist can be a mix of pkg-plist and Makefile, this is inconsistent, it makes sometime hard to figure out why a file has been added to the plist etc. Also this makes us having tons of targets define to handle those extras entries and results in highly inefficient make(1) usage. - stage is also a first step to go to sub-packages, sub-packages will basically be: 1 port able to produce N packages, to be able to do this we will use multiple plist, having the files properly defined already in plist will help that. Having files defined in macros on Makefile will make it hard to determine which one should go in which plist. Last thing I would like to add about it: I don't see the difference personnaly between listing N lines of manpages into Makefile MAN* macros where btw you have to manually define the categories where to put them and adding those N lines directly into the plist where make makeplist and/or make check-orphans can help you getting the exact lines automatically? 2/ Binary packages For the sake of being able to have clean working binary packages, pkgng has had to use a dirty hack: origin is used has an identifier instead of the package name. Why: How to determine than ImageMagick-nox11 and ImageMagick are the same package? and if I don't know they are the same package how to upgrade them properly? same goes how the package manager can know that mysql-client-5.2.1 is not an upgrade of mysql-client-5.0.1? How can pkgng determine which subversion is the right version to use: 'pkg install subversion' will propose all possible subversion, in that case the higher version is probably the default version, but in perl case, the higher version is probably not the one you want. Some packages are totally different project but have the same package name... How can pkgng differenciate them. So please stop using LATEST_LINK to differenciate the different port but directly make _unique_ package names, pkgng will switch back to package name as soon as possible (that will also solve the ugly and stupid pkg set -o). There is multiple possibility to make sure to properly handle multiple versions of packages: 1/ suffix everything with part of the version (like tcl) 2/ suffix everything but the default with part of the version 3/ have only 2 or 3 different version: project-legacy project project-devel Please stop renaming the packages base on options! options are now registered with the package with both pkgng and pkg_install, please stop adding suffix base on options! Think about the end user, he will try to install a package, it should be as transparent for him as possible. Note that the 2 above are mandatory if you want to provide sub packages and flavours! regards, Bapt --qcHopEYAB45HaUaB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlJNL04ACgkQ8kTtMUmk6EyT+wCgtCuGf4xKb9/w4JKNRdtdEsR4 wUUAnimJFnpww9m8z8B+9L8TAmd3snBR =SKNe -----END PGP SIGNATURE----- --qcHopEYAB45HaUaB--