From owner-freebsd-ports@FreeBSD.ORG Tue Jun 26 12:31:18 2012 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 648101065670; Tue, 26 Jun 2012 12:31:18 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id CADD88FC18; Tue, 26 Jun 2012 12:31:17 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id q5QCVErn037517 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 26 Jun 2012 13:31:14 +0100 (BST) (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: OpenDKIM Filter v2.5.2 smtp.infracaninophile.co.uk q5QCVErn037517 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1340713874; bh=6IENFVmmWLXZkyocGAlKkBrIDkO62AlctA3tnHU4xNE=; h=Date:From:To:CC:Subject:References:In-Reply-To:Content-Type: Message-ID:Mime-Version; b=dMXyJ5ZyuB2f+sUKOaH4duh1svxF8AfVhNDib12Di/PA/3VpjHdf4rffHRQQup64O XFUFCCO0gzpwvgrj8cchwnPUFw91Liia0Pa3N01gMp9DoiBntWPVrOqcQluyZudav2 819X7dmNbYsNIRKpCVN7mzjXrM+Yx/LIJwMOyOTY= Message-ID: <4FE9AB85.3070106@infracaninophile.co.uk> Date: Tue, 26 Jun 2012 13:31:01 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Marcus von Appen References: <4FE8E4A4.9070507@gmail.com> <20120626065732.GH41054@ithaqua.etoilebsd.net> <20120626092645.Horde.HytQbVNNcXdP6WQ1aMtjoMA@webmail.df.eu> <4FE96BA0.6040005@infracaninophile.co.uk> <4FE97008.2060501@netfence.it> <4FE97AE1.9080109@infracaninophile.co.uk> <4FE9817C.7020905@netfence.it> <4FE99200.7050107@infracaninophile.co.uk> <20120626130715.Horde.eb3fPtjz9kRP6ZfjA7sSFoA@webmail.df.eu> In-Reply-To: <20120626130715.Horde.eb3fPtjz9kRP6ZfjA7sSFoA@webmail.df.eu> X-Enigmail-Version: 1.4.2 OpenPGP: id=60AE908C Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigD4273BF32AE06AFF36E2989E" X-Virus-Scanned: clamav-milter 0.97.5 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_ADSP_ALL,DKIM_SIGNED,T_DKIM_INVALID autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Cc: freebsd-ports@freebsd.org Subject: Re: Port system "problems" X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jun 2012 12:31:18 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigD4273BF32AE06AFF36E2989E Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 26/06/2012 12:07, Marcus von Appen wrote: >> Sub-ports should be much more efficient, as there's a lot of the work >> involved in installing which is a one-time thing when installing port >> plus some collection of sub-ports. > How does that look like in detail? Are there any concepts on about > how it is supposed to work, what the implications are, and so forth? Please understand that a lot of this stuff exists only in various people's heads and is subject to arbitrary change as ideas evolve. Other parts are more-or-less documented; some is already under development and may or may not require various degrees of revision in the light of experience. Anyhow: the various stages of building a port at the moment are well know= n: 1 options dialogue 2 fetch & verify distfiles (*) 3 extract (*) 4 patch (*) 5 configure (*) 6 compile 7 install (*) 8 post-install scripts 9 register in pkg DB Not all ports require all those stages. The stages marked (*) can involve resolving various dependencies by installing other ports -- although {FETCH,EXTRACT,PATCH}_DEPENDS are relatively rare, and it's usually {BUILD,LIB}_DEPENDS before configuring and RUN_DEPENDS before installation that take up most of the time. You have to do each of these steps (where applicable) for each port you install. There will be some time savings for installing several closely related ports. For instance: where you have -client and -server slave ports, you'll generally not have to download any extra distfiles (although you would still compare all the checksums.) One of those ports would be usually a BUILD_DEPENDS of the other, so a lot of the dependency chasing should already be done. What's different in the new scheme? 1 options dialogue 2 fetch & verify distfiles 3 extract 4 patch 5 configure 6 compile 7 install to staging directory tree *** 8 create packages, sub-packages *** 9 install packages and sub-packages as selected *** where running post-install scripts is incorporated into stage 9. (This also means post-install scripts only have to be implemented once, unlike currently where you have to implement both a post-install Makefile target and a post-install script for use with pkgs.) Stages 1-6 are essentially the same as at the moment: they may differ in detail due to amalgamating master- and slave-port options etc. but they only need to happen *once* for the whole collection of port and sub-ports. The same sort of dependency chasing will happen as currently.= Installing into a staging directory tree involves installing all the executables, data, documentation and other files from the port irrespective of what sub-port they might belong to. This bit is an addition compared to the current ports. Again though, it's a one-time-for-all step. Creating packages in the new scheme (ie. assuming pkgng) is a matter of tarring up selected files from the staging directory along with two blobs of meta data -- the YAML file containing the runtime dependency data, package plist, various scripts to be run on installation/deinstallation and the checksums of all the files, plus various other information; and an MTREE file for creating the directory hierarchy to install into. It's comparable to the current 'fake-pkg' target in bsd.ports.mk. This step is repeated for each of the different sub-packages, packing up different selections of the staging directory contents each time. You may well be creating more {sub-,}packages here than you ultimately intend to install. Finally, installing the selected package and sub-packages -- this will be identical to installing pre-compiled packages downloaded from one of the standard package repositories, and involves unpacking and verifying the tarball contents, making any necessary directory structure, moving the files into place, running any scripts as specified in the metadata and updating the package DB -- now an actual relational database stored in a single file: ${PKG_DBDIR}/local.sqlite Whether the extra/different work done in stages 7, 8 and 9 will negate the savings from only doing stages 1-6 once remains to be seen. My prediction is that mostly you'ld come out ahead, but whether you do, and by how much will vary significantly between individual ports. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matthew@infracaninophile.co.uk Kent, CT11 9PW --------------enigD4273BF32AE06AFF36E2989E Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/pq5EACgkQ8Mjk52CukIwHTACfTlJU24s7E+MSZ879s9PR+ZcQ zpYAn0qypK0FKCTMzv1UQRwOYX17iWJ3 =teHY -----END PGP SIGNATURE----- --------------enigD4273BF32AE06AFF36E2989E--