From owner-freebsd-ports@freebsd.org Sun Nov 11 14:27:15 2018 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D12FF11069BF for ; Sun, 11 Nov 2018 14:27:15 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [81.2.117.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6397832A9 for ; Sun, 11 Nov 2018 14:27:14 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from liminal.local (unknown [IPv6:2001:8b0:151:1:c07a:d7a5:dca0:255d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id 8C7D66095 for ; Sun, 11 Nov 2018 14:27:07 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none (p=none dis=none) header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/8C7D66095; dkim=none; dkim-atps=neutral Subject: Re: Few how-it-works questions To: freebsd-ports@freebsd.org References: <73366a94-238a-ed9a-5dee-d5955525e851@gjunka.com> From: Matthew Seaman Openpgp: preference=signencrypt Autocrypt: addr=matthew@FreeBSD.org; prefer-encrypt=mutual; keydata= xsFNBFJIL80BEADi7/VbnnErDU6pjEhI/SzEZ/HbDRkJ5g7HroAtqIRm6nj8ZwOAgZ/2ZnWn 5F+fXTuLsG0FLNtkd17FoVcuCi5e/GPliXI5cmamV7E1Yz4T8UsJ7RQolimyxVexccKd16Tc AA7B9bFlJSKkBUSD0buj7VjT07xWhRzu6Vgi5r0UjLALYJz977uZA0F1aOGOXREDEAOhdcNc kSNjynqAwDA6dCT1Elpi4key1fYjv4jyDF+GU/YXul2Y/rguA8FCkHd9vyym5eAsLQ5mG00V V9fkEHIpH5KorNVnl/ufHXnkZqmHAZVpFDcrshb7aZ/pL45PXyWgLj+e6etelgj3a2bZi0JF cVdXCnBZVP2oIyYblM11ugTbfCwodORU8a5KfPeztMdAtDr4e+32NTrPdPi5rLT+GUsYz+PL 3A3m3u8bdsFp40DlIrBtSByVjqERxcfhphrEB4J8BXHUG7OAtXkZMlW/PGKDwXJq0O6Z5Tcg YHAoEiSWbXiexHgXNJyP+sqnIlhLWhSJGeJ+C83wqI6oYlZUCW00NkPxcIHnQPV/z+5wQVci TMyaWC2YCIHz4Ljs+TnwWMz0E8PNFDfHVbQ0W4PRGV7gRAqxfL+yKufauIEGbEq8rNDbSwL3 bcUCxR4ZDlaUEUwT4J8naf7rjdgiEYHs2Ig3jeK1+ER4FPG1sQARAQABzTBNYXR0aGV3IFNl YW1hbiA8bS5zZWFtYW5AaW5mcmFjYW5pbm9waGlsZS5jby51az7CwZcEEwEKAEECGwMFCwkI BwMFFQoJCAsFFgIDAQACHgECF4ACGQEWIQRyz6whebywJLW1RZADb2ye5/OevwUCWttU4QUJ DFmAlAAKCRADb2ye5/Oevwb5EACipbOazgwl5IbqkQI4gELpCh5dqDASS9DQqAD35n/cI91P 0lrYcdyCQbOXadQi5bswnP4AcJqX83mITXbcApDdxVxHujw7VODI069eV3/I9Qz72mHYYAAj w0CHNx4bKED2YCSVS6+jV5hq2sywNEUxL+4I218Oc+IsLts62m4tQ8UxX9fQ2H1kQOvdrYpj x7je5qJX/yujLc+9WWZ8ZBSdP/HVJUEdRgQotwAlgfMp3mRQEE73MAJisG/olj/dSxd+oHIP NbJt1yxMqhZekuEGqZpm3tWvqYgpGcEXdhphJSxeK6oLpTLghuAb7/WdOBrpfL7c2OQYBgOw DK+7Io9NBt/d/rCxL39jmUONW8ohrhnNQ2SALnyYTvZgruxA4tXxOOyM9up0/8mB5E8YC9ML 5YuxRPNTXYeWCexa0zktnkCgT7PhS33evf5gsA0B9Snv7TFCFN9adPAdHlsppZIWfTHDG8e2 Jik8PmvsUG34XNif5k6Ui3++2ZA8ZoKvOyLeomuno1hN8yk1APw8SbX1SPNz9UVbl8W/YgGj 3GhYOuQt4HcMiLyTby6R4lC4nsBaHS1MX+57f6Zxzf2wNjSKxiJK9qS7azbu/GxpafNhbz1Z +iUDIaJkRWA1Gs8C7SMcfVsI5zDtvqHGYtTCgooVMYJ6vRyB68M4bljUYMxRTs7BTQRSUUK4 ARAA1FhWoOejtwmsnGshoIbda2FmM+z/f97OzpagLhACHfP5Es/I18wG/0G+rdNuO2tjA9IM Z44GUMtjokDrDk63N9S+rVKy1QEy+UN6CiIfYTpTTAPnEY7IGN1JjGksPhn7aeuBCQwUMAV1 k+wklBCcOD6s8DD4kx0ZJqkH83XzWoBSVamdHvnM56C8yPVr5HHMC1tZInAWBMrF+cjl1EPf z3CqkVnG8Sxc5ydeibMS9Q3lHLeVkVlMRAmNqzNLfgJDUWtzac7JIjFEsxYYhpiaPcsstUUu Ha4zIRJ/yHDNbDttWRf1lrlFZLpeuap4BZ2hQw0UOZVNwGoFoS4ZqaZiv8mm0lX6s9/AdQD6 AVrpXWKa7JU2wDiay9sRbYh+5vVWGz9mhncK/Vfwtu5IjVp5v5WMz/WfnUxZMcNlfgTo4i1s www+qRBO2A4Yj8qKKWnTsl7aCX92itTiPgwbt6YgQPwgww72r67jPt5o8VMXDqPMPKzGicw1 AyxtMjsoSlnn91FuZctwil3vPpvzGXtBmrzQSbdDmy0KT5p5/W9pD/8UtLLLM6PLs5X0jIho vQHnQKEUO7xV3yNDAW9DPICeh7f/o9W+QJfQAXngNz0brvmgScAUXRaeAFeQbAmtEG92qlSV D7gb7WOemllgfbEn0Nanrv5aEcZCWx4WjybMLHEAEQEAAcLBfAQYAQoAJgIbDBYhBHLPrCF5 vLAktbVFkANvbJ7n856/BQJa21VJBQkMUG4RAAoJEANvbJ7n856/mAcP/0ybQAvXfxWEEByk IP0DhJHAC/EMeBwNkiAp4Sqr+uIz3GCFGKHDjvEGsofiFQ2ujBpG7FncHlBbnsTLFvte3ahE 30I1AKcd9k1MBeOFoCBHwES1ts0XUXF37E+ANrECQrzSayZx95csIiYvlfOPEOLAt7EiURKX CXdO6HNo8UimcmGdQwT3ytTMosHAbdrhQk13chTIWptmmCwz9iWLxT9PLY01ACCoXuAdGz07 ZXQn+bB+avMa6Wh5yh39J+6jJiuzbRlv/Uelogq7ojbC5zveX5rNbcyinwOEFyGAhFpfF7ES sKedR2Q40LvysT7I5ugS+Hk4Z2nvbd2bOSdC4j8aBWzfqVu2p37d2AnnswfPoLrOyNUZ+ciT EcmEUVR7WWUwQ0H6A6h4C2NeBmLRRjk9CEfzrgM2DNQqDL1RMYKlVosQ8BeUR9ThztUwDakx nK0ZtZb2rAliKYaaEFbZDePz1xmvjYc7EZq/3OTlGMUDa6BPHHbCvJjiAUc/Q9iaRe3dp69V /rwOM5NiS+tWgp3OtgX0mDWVoQnDjyWVIRU/QagJHsNJJCc0N48BxgIX3H6M0x6BbA9PKgFt DlK4hLR/hDl5fnWG45TVIxT4ybuPXGW7af9U6bGDgXTBNUCzNUz2p2F2u7W/iK0WTfjovYvV Vcptegyu6ttZN49KkQtL Message-ID: Date: Sun, 11 Nov 2018 14:27:04 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <73366a94-238a-ed9a-5dee-d5955525e851@gjunka.com> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="fxnYY8hyzuSWR0sdfqIbMjEuCGpFXmi1z" X-Rspamd-Queue-Id: D6397832A9 X-Spamd-Result: default: False [-105.20 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ALLOW_DOMAIN_WHITELIST(-100.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_ATTACHMENT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ports@freebsd.org]; TO_DN_NONE(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; DMARC_NA(0.00)[FreeBSD.org]; MX_GOOD(-0.01)[cached: mx1.FreeBSD.org]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; IP_SCORE(-0.02)[country: GB(-0.10)]; ASN(0.00)[asn:20712, ipnet:81.2.64.0/18, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Nov 2018 14:27:16 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --fxnYY8hyzuSWR0sdfqIbMjEuCGpFXmi1z Content-Type: multipart/mixed; boundary="NLQWEpMMScxieNvHpVNtUREIyeMdcGnPC"; protected-headers="v1" From: Matthew Seaman To: freebsd-ports@freebsd.org Message-ID: Subject: Re: Few how-it-works questions References: <73366a94-238a-ed9a-5dee-d5955525e851@gjunka.com> In-Reply-To: <73366a94-238a-ed9a-5dee-d5955525e851@gjunka.com> --NLQWEpMMScxieNvHpVNtUREIyeMdcGnPC Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: quoted-printable On 11/11/2018 12:34, Grzegorz Junka wrote: > Hi All, >=20 > I would like to understand a bit better how the ports infrastructure wo= rks. >=20 > 1. Recommended way of upgrading ports is "poudriere ports -p local -u",= > right? But this always gets me the latest version, in which some ports > may not compile, depending on my luck. I know I can use SVN to checkout= > a specific version of ports instead, but is it possible to find out in > which SVN version which ports are compiling and which are not? In other= > words, can I open the history of builds of FreeBSD ports on the build > servers and check which ports are building in a specific SVN version, > then checkout that version to build on my server? Firstly ports rarely stay uncompilable for very long. There are exceptions mostly due to situations like the problems with openssl-1.1.1 support at the moment. So there are some additional strategies to apply:= 1) Wait. If you can just put off updating for a day or two, problems are quite likely to get fixed. Of course, you can help this process along by opening bug reports and/or submitting fixes or even just posting to the various lists about problems you're experiencing. 2) Change versions: since you're building your own packages, you don't need to stick with the default versions of things like python, postgres, openssl etc. set in the ports. Sometimes just substituting in a different version of a major dependency will solve your compilation problems. At the moment this is particularly fruitful with packages that need OpenSSL or an equivalent, given compatibility problems with libressl and recent openssl. 3) Change options: do you really need support for all of the possible whizz-bang features available in the downstream port? In many cases you can prune the dependency tree of a package pretty effectively by tweaking package options. Limit what's built to just what you actually need to use and you've cut out many places where failures could occur. By doing these things, you can generally get yourself a set of packages that you can build fairly reliably from the head of the ports tree. Now, you can play with checking out an older version of the ports and you can look at the results from the package build cluster eg. here: https://pkg-status.freebsd.org/?all=3D1 where there are historical logs, and you can compare against the commit history for your ports of interest to find when a problem was introduced and checkout the corresponding version of the ports tree. However... see below. > - also, is the ports tree mirrored in Git/GitHub? Yes. Here: https://github.com/freebsd/freebsd-ports.git > 2. Every time poudriere builds a new set of packages, it deletes those > which have changed in ports or for which the options have changed, then= > it tries to build them again. But because the ports change, some may no= > longer build. In that case I am left with no packages to install on a > system unless I am lucky enough to be able to cleanly build all ports > again. What is the preferred strategy to maintain old packages until a > new set is build correctly? Copy the whole folder aside? Is it possible= > to tell poudriere to create a new folder with packages for each new > build (eventually with the option to use already built packages when > they have not changed)? I suspect poudriere doesn't maintain such state= > internally, it simply deletes what's no longer relevant and then builds= > what's missing? Yes, poudriere can keep a history of the state of your package repository over time. Check the values of these settings in your poudriere.conf: ATOMIC_PACKAGE_REPOSITORY=3Dyes KEEP_OLD_PACKAGES=3Dyes KEEP_OLD_PACKAGES_COUNT=3D15 Poudriere will create a new directory tree for each package bulk you run, and link the unchanged packages into each tree for an incremental update so it's as space efficient as it can be. Poudriere will create a symbolic link in eg. /usr/local/poudriere/data/packages/ like: % ls -lad .latest lrwxr-xr-x 1 root wheel 16 Nov 10 12:07 .latest@ -> .real_1541851673 Changing that link to point at an earlier version of the repository will achieve the effect you desire, although be aware that poudriere will tend to update it to point to the lasted build results each time it is ru= n. > 3. When a package doesn't build because of a patch error: >=20 > =3D=3D=3D>=C2=A0 Patching for ImageMagick6-6.9.10.14,1 > =3D=3D=3D>=C2=A0 Applying FreeBSD patches for ImageMagick6-6.9.10.14,1 > 1 out of 1 hunks failed--saving rejects to config/policy.xml.rej > =3D> FreeBSD patch patch-config_policy.xml failed to apply cleanly. > *** Error code 1 >=20 > Is this a problem with the ports tree, the upstream package, or some > other package? In other words, can that problem be fixed without having= > an updated version of the port (e.g. assuming the upstream package or > the other package gets updated and rebuilding the same package without = a > new version of the ports tree may fix this). Are patches always kept in= > the ports tree or sometimes they may be kept separately (e.g. in anothe= r > repo versioned independently)? Patches that fail to apply cleanly is an obvious problem that port maintainers will deal with routinely at a very early stage whenever updating a port. Even if someone managed to accidentally commit a broken patch, it would generally be fixed in pretty short order. If you're seeing this frequently in your checked out ports tree then it's probably due to 3rd party patches you've added to the tree or old patches that have been deleted upstream but that somehow still exist in your checkout. 'svn status' or 'git status' will tell you about locally added files or various forms of tree conflict that prevent removing a no-longer-required patch. If you need to maintain your own set of private patches to the ports tree, that is certainly possible. I'd recommend uing git to clone the freebsd-ports repo and creating your own branch as that is probably the lowest effort way of being able to merge in upstream changes. Cheers, Matthew --NLQWEpMMScxieNvHpVNtUREIyeMdcGnPC-- --fxnYY8hyzuSWR0sdfqIbMjEuCGpFXmi1z Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEGfFU7L8RLlBUTj8wAFE/EOCp5OcFAlvoPDlfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDE5 RjE1NEVDQkYxMTJFNTA1NDRFM0YzMDAwNTEzRjEwRTBBOUU0RTcACgkQAFE/EOCp 5Odg9w//cR31Z+RquUFk0NfFqIwvDhUL5sHGgPXL4Y3G/ns/8H7rnlmUsIbW7F/g eYKo7tWEShfdwwC1GydIzmbeqOS2EGWN64V/NyOv0Zd0a4QZrRXzJUXraK0dODH7 7knNkWPEZ8VtHZMiVzzC8Ae7rsriXU2HMnWh4REfawdP2ocT/jaK8tZGMSRxY04G F+hMwUAWvOoarIq1hwimmp07WXQ7g/LfXCn4Y1NGv/ICHcCZTD0QahrN7i3KL4Ma hlDQ09E5LAxQY9iq1AkAcj8ZWMN2v3xF8vKejUrLZQsNwWdR5jJ0tRcK8+0dhTdV pMrZskOx8S8k1U3gABlhbDWwVVJSDfJddDFkoYvFMCA7q0eBJ4UtDJbCoU9d4efY LwhDw/xVvEVJwufJ3UXEfIX0UwK5x8U0Szt8hyavg7Vf9G6msRy0tcRhCQ/mSqyw WDzj0eMXmAnckKIbA0zppm6F0kyuawJ/fPaWRylTlvivg984iQgSgxeeuD4wL6ey 62/eNWqTbLiXH77MyVDQkQDG0P89kseuIesWktUVbrxAI0xNgu2AdxBxNuRXzthO 4PrYMTAgpqudg5TRrIIBJv99jVHUtOToogA46Z1x2AbssQTkkmflh6Zprj/WmjH+ +MSROXu7/Qw3S41bywDiyz2MWONbu4PmDJ/hBjeV3q9Rj8yTzvs= =RIW9 -----END PGP SIGNATURE----- --fxnYY8hyzuSWR0sdfqIbMjEuCGpFXmi1z--