From owner-freebsd-questions@FreeBSD.ORG Sat Feb 14 20:00:49 2015 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BDC3D815 for ; Sat, 14 Feb 2015 20:00:49 +0000 (UTC) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "ca.infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 580E0E3C for ; Sat, 14 Feb 2015 20:00:49 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.2.117.100]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.15.1/8.15.1) with ESMTPSA id t1EK0hju007131 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sat, 14 Feb 2015 20:00:44 GMT (envelope-from m.seaman@infracaninophile.co.uk) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=infracaninophile.co.uk DKIM-Filter: OpenDKIM Filter v2.9.2 smtp.infracaninophile.co.uk t1EK0hju007131 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1423944044; bh=qv5PfykOllAgtAvD46wZ9c2PxGSZmdkWkRjSwtnCwRU=; h=Date:From:To:Subject:References:In-Reply-To; z=Date:=20Sat,=2014=20Feb=202015=2020:00:34=20+0000|From:=20Matthew =20Seaman=20|To:=20freebsd-questi ons@freebsd.org|Subject:=20Re:=20Ports/Packages=20and=20release=20 engineering|References:=20<54DF89BE.6010005@complete.org>|In-Reply -To:=20<54DF89BE.6010005@complete.org>; b=KCsNlIQax7SgvZzrfGXCHIEECEiIeTHtS5UrSLdft8FZcCIIaKMUau5I3U0VvBqH6 S/vumVTRa1NXfIHSP8RbJIWdTg75t+Fs+d8wTFdBWYYcAMXmgX/Z1z1NtFtsm1Drot lkRjIZNfRggIYV+ZwhvJ3fK2ZSPSelAGRuVIbMCY= Message-ID: <54DFA962.2010509@infracaninophile.co.uk> Date: Sat, 14 Feb 2015 20:00:34 +0000 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-questions@freebsd.org Subject: Re: Ports/Packages and release engineering References: <54DF89BE.6010005@complete.org> In-Reply-To: <54DF89BE.6010005@complete.org> OpenPGP: id=E1ECF9BB Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="fDlnNM4IELDQL1VUsoEwKAeU5VA7ikuvt" X-Virus-Scanned: clamav-milter 0.98.6 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Feb 2015 20:00:49 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --fDlnNM4IELDQL1VUsoEwKAeU5VA7ikuvt Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 14/02/2015 17:45, John Goerzen wrote: > So, it looks to me that ports is essentially always a -CURRENT tree. I= > don't see any reference in the handbook to selecting different branches= > of ports, or different branches that might happen to match different > RELEASE versions. How, then, is quality maintained within the ports?=20 > Is it just luck of the draw on when a local update happens that it gets= > a version that didn't have a silly bug that got corrected a day later, > or how do ports get tested before being released to everyone? Correct. There can be mistakes committed to the ports. In fact it happens reasonably frequently. However, we do have some quite stringent requirements about integration testing before and after commits are made: certainly anyone submitting patches is nowadays asked to provide 'poudriere testport' output showing the package builds successfully and passes the built-in tests the ports tree uses. Committers are also meant to run these sorts of tests before they commit anything, but they are generally trusted to behave sensibly and don't have to provide proof of doing so. There's also an automated testing process which will generate warning messages to port maintainers / committers if any port fails to build properly. So while mistakes do happen, they tend to only affect individual ports and then for only fairly short periods of time. Generally you're pretty unlikely to run into problems on that score. > One thing I know from Debian is that there we often have "transitions" > from Debian's unstable (roughly -CURRENT) to testing (roughly -STABLE) > when certain groups of packages have to be transitioned in sync in orde= r > to avoid breakage. This might be things relating to KDE (such as > Digikam), or another example is Haskell, where new versions of the > compiler ghc introduce a different ABI requiring all the Haskell module= > packages to be rebuilt. There are plenty of examples, though. How are= > these handled in FreeBSD? Yes. When really important sub-systems get updated, or changes are made that affect large numbers of ports, there will be an exp-run, which is an experimental build of the entire ports tree to ensure everything still works. > When packages are updated, what happens to their config files and rc.d > scripts? Are files in /usr/local/etc, /etc, {/usr/local,}/etc/rc.d > never modified automatically, or is the administrator prompted on what > to do (as in Debian)? If they are never modified automatically, how > does a person know what changes to make after an update? Ports don't put any config files into /etc -- only /usr/local/etc. Similarly ports only put startup scripts into /usr/local/etc/rc.d -- this division of ports and base is one of the FreeBSD fundamentals, which I'm sure you'll be recalling from your previous experience now. Configuration files are *not* owned by ports per-se, so they aren't necessarily removed or modified when a port is deleted or updated. Instead the port owns various sample configuration files, which generally have the same name as the actual config with '.sample' appended (other naming variations exist, but '.sample' is by far the commonest.) These the port will replace at will. When a port is installed the first time then the sample config file is copied to the real config file name. When the port is removed or upgraded, then the config file is removed if and only if it is identical to the sample file.= There isn't a capacity for doing a three-way merge in the ports at the moment, so if the upstream has made incompatible changes to the config file format the administrator will have to merge those manually. This is fortunately a fairly rare event. 3-way merge capability has been added to pkg(8) but that capability hasn't been applied to the ports yet.= > This brings me to the question of how the ports/packages relate to the > base system. A couple of things surprised me in researching this > question: the first being the comment in the Handbook to reinstall the > entire ports/packages system after a base system upgrade to a new major= > version, due to ABI incompatibilities. I would have thought backward > ABI compatibility would be a pretty important design goal in FreeBSD.=20 > The other thing was that ports are only tested against -CURRENT and > -STABLE, therefore implying that even the current -RELEASE may not have= > working ports (let alone even an LTS -RELEASE). That leaves me > concerned about the long-term viability of managing FreeBSD systems, bu= t > also wondering if I'm missing out on some important bit of information = here. Reinstalling all your ports on a major version upgrade is still a requirement, yes. But it is something that may not remain so for a great deal of time longer, given the advent of symbol versioning in the base system shared libraries. While you can still *run* a port installed for, say, 9.3-RELEASE once you've upgraded to 10.1-RELEASE, until everything in the base system has appropriate symbol versioning applied you'll very quickly get into a pickle if you try and update some ports selectively: effectively you would end up with one binary trying to dynamically link against two or more different versions of the same shared library: something that always ends in tears. So, yes, for now: reinstall all your ports when you do a major version upgrade. Note that freebsd-update(8) advises reinstalling all ports even if you're only doing a minor version upgrade (eg. 10.0 to 10.1) This is *not* necessary. > Speaking of which, when the base system is upgraded, how are > administrator customizations in /etc and related configuration location= s > preserved? (Same question as with the ports updates above.) It depends how you go about updating the base system, but all the widely used methods run you through some sort of 3-way merge process to handle stuff under /etc and the like. It's built into freebsd-update(8), and if you're upgrading by building from source, then you should be running mergemaster(8) as a standard part of that process. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey JID: matthew@infracaninophile.co.uk --fDlnNM4IELDQL1VUsoEwKAeU5VA7ikuvt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.20 (Darwin) iQJ8BAEBCgBmBQJU36lrXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkATEGwQAIBdRuVKN1rnKGJ+6pIC7wiQ bW2RlbIAOrwM9CDXHri24DLMMxmgGnFU2QGAduKY1QzwBsmXQ+9yIANSI3NG5X1h KzZYKC2IzZiad6dKBiH0i2dKj2w5bCsTzXO5lS0/ImcuPAzHpYWhh4fODmnDqlUB eocRtU2qcWYDPsiYjeiUdrFKhO4ms4yvePG+aOh/c7x5AdHM9WErTL/1upNShgoY 5Rb42uOJoKZwGVQ5hbCEnf1ZoaNDkR1rHV5H+A1lAwb4p42a9V9HJIe3aGEFe6ss jkmu3vSa3KWfL7XnhSyi5DS3/5wu4oXEfs7a6OIhAsxkpqTgBY88KAmK61+8cCl9 0T5t1T65WXjc8xgCH8fSX/4Jw5leAytHqxuxzJqfRP9IZbHArOG+jLV/2TVT43+N ZIbla9Nl66KSZYimJ5O33dMv0HCYbN9ll4XOWHvWpHDggmdGd6yhuYaeU8Kzso2R Pies6e9oySSJosjqhvFSiyH1YQl/gEBbfCAMSbEfhIQboS3FQsC6CG2NOHaMlL7a 1z6ClqljuaxXxuZVK63P+t60jEbwiMx+4FOZ65bhoxX/QKTrJu0oBn39E03YRgA+ GYNnU18cc27wjhFkV4TzftmjO9ZPMIArEee2YRR+ik7z5LKRAoHFWiWX4xYz5O95 P/Zxm/yK4nV5NsRHaaAf =fG9v -----END PGP SIGNATURE----- --fDlnNM4IELDQL1VUsoEwKAeU5VA7ikuvt--