From nobody Sun Oct 9 19:18:22 2022 X-Original-To: ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MlsK54qS2z4fLcw for ; Sun, 9 Oct 2022 19:18:33 +0000 (UTC) (envelope-from freebsd@gushi.org) Received: from prime.gushi.org (prime.gushi.org [IPv6:2620:137:6000:10::142]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "prime.gushi.org", Issuer "RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MlsK43HRgz3JDM for ; Sun, 9 Oct 2022 19:18:32 +0000 (UTC) (envelope-from freebsd@gushi.org) Received: from smtpclient.apple ([IPv6:2601:602:87f:b05d:8566:a4e5:3c91:4432]) (authenticated bits=0) by prime.gushi.org (8.16.1/8.16.1) with ESMTPSA id 299JIRv4019390 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sun, 9 Oct 2022 12:18:28 -0700 (PDT) (envelope-from freebsd@gushi.org) DKIM-Filter: OpenDKIM Filter v2.10.3 prime.gushi.org 299JIRv4019390 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gushi.org; s=prime2014; t=1665343108; bh=XQ/iDHg0BYtmPuICkGVocjX5C+eaigW+Gio1GTK/+UE=; h=From:Subject:Date:To; z=From:=20Dan=20Mahoney=20|Subject:=20Default=20 versions=20and=20pkg=20(slight=20rant)|Date:=20Sun,=209=20Oct=2020 22=2012:18:22=20-0700|To:=20ports@freebsd.org; b=IjhesczHnmrFn/vd1TkpcdLi8yaVydy+oaixojJ41jJ1869Z+Zj855gXoTdT8T02o qmHZGSv/Y88kvmVR3ETgY1BXut+7acS0aOY4qnOKWE5FFiO/GYgR4WvdZZLFTKsOna dF+LpplsyG4j40MEqESEcTaZeBbVFeHqZCEzJ1zLnU4Ql25Fj+Fl74Yxw4ZsvVeQ1j b01Pj5IgRkIG5s3Nc7vXBSdzKIZfl7DeM6G6M2WlZakJcnycTZO2HKgcmLI+OY11YA UVAsPzgSBbAzJ7NqAhclOFVIv5owO+e1KQ6HqPW4sCzBvFYR1+9/w3vX3jGwZKLU+y mp+xMiIdnaI2A== X-Authentication-Warning: prime.gushi.org: Host [IPv6:2601:602:87f:b05d:8566:a4e5:3c91:4432] claimed to be smtpclient.apple From: Dan Mahoney Content-Type: multipart/alternative; boundary="Apple-Mail=_C561DA2D-218E-4114-A43E-D15CD83A05D4" List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Default versions and pkg (slight rant) Message-Id: Date: Sun, 9 Oct 2022 12:18:22 -0700 To: ports@freebsd.org X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4MlsK43HRgz3JDM X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gushi.org header.s=prime2014 header.b=IjhesczH; dmarc=pass (policy=none) header.from=gushi.org; spf=pass (mx1.freebsd.org: domain of freebsd@gushi.org designates 2620:137:6000:10::142 as permitted sender) smtp.mailfrom=freebsd@gushi.org X-Spamd-Result: default: False [-5.20 / 15.00]; DWL_DNSWL_MED(-2.00)[gushi.org:dkim]; URI_COUNT_ODD(1.00)[5]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MV_CASE(0.50)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DMARC_POLICY_ALLOW(-0.50)[gushi.org,none]; R_DKIM_ALLOW(-0.20)[gushi.org:s=prime2014]; RCVD_IN_DNSWL_MED(-0.20)[2620:137:6000:10::142:from]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[ports@freebsd.org]; MLMMJ_DEST(0.00)[ports@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:393507, ipnet:2620:137:6000::/44, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TO_DN_NONE(0.00)[]; HAS_XAW(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[gushi.org:+]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_C561DA2D-218E-4114-A43E-D15CD83A05D4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hey there all, I recently asked about default versions in pkg, and if it was possible = to tell a pkg (without building it yourself) "no, don't conflict with = that", for example to point bind-tools916 at python3.9, because it's = just a shebang line. That thread went a lot of places, but only served = to expose an underlying problem. In asking those questions, I was trying to learm more about the DEFAULTS = changes, and how we can mitigate them, because every time one changes, = there's pain. This stems from a healthy culture of having pkg bite me during what = should be a routine thing. Here's a few examples where we've been burned: * As recently mentioned, we had two versions of python installed on a = system, and removing python3.8 attempted to remove bind-tools, which in = turn would have wanted to deinstall bind9. A more recent bind-tools = didn't require python at all -- but that upgrade path wasn't suggested = as part of it. Pkg remove only attempts to remove things, not fix a = running system. Pkg doesn't have any option to keep leaf packages = installed at all cost. Nor does it fire the solver when doing a pkg = remove. * We run RT, against PostgreSQL. We installed p5-DBD-Pg (which is not a = direct dependency), During a routine pkg upgrade, because of a default = version change, p5-DBD-Pg forced an upgrade from postgresql12-client to = postgresql13-client. It did not, however, cause an upgrade to = postgresql13-server, it only caused postgresql12-server to be removed = because it conflicted with postgresql13-client. The ultimate fix here = was a full dump and restore to get from pg12 to pg13, which took hours. * On a recent upgrade where ruby30 became the default, we were left with = a puppet6 install that had no ruby interpreter it could run with. The = puppet6 port is marked as broken (because it doesn't work with ruby30), = but pkg didn't deinstall it, threw no warnings about it, because its = listed as having ruby30 as a dependency, as opposed to a lower ruby. = The solution was "Well, I guess we're upgrading to puppetserver 7 = tonight, hope there's no fallout from that". Luckily, our manifests are = really solid, but in the older days of puppet, there were a lot of = breaking changes. In all three of the cases, the old no-longer-default dependencies were = still in the tree to be pkg installed. In all three cases, pulling down = a ports tree and editing make.conf to point at the old version might = have been a reasonable solution, that wouldn't have carried forward with = someone who just wants to be able to pkg upgrade their stuff, and it = would have required building some major packages from scratch (in all = three of the above, it would have resulted in building a language -- = python, perl, and ruby, respectively). Here's what I'm asking: * Most of this could be solved by reading /usr/ports/UPDATING. However, = as someone who wants to trust the maintainers, and just use the = packages, is there a way to see "Updating" without pulling down a ports = tree? You've baked a "pkg" command in to read it, but that's half the = work. (Perhaps, maybe, just make /usr/local/share/doc/pkg/UPDATING, = itself a pkg that can be updated)? Maybe, on-upgrade, print those = things out as part of the process -- i.e. if the date on the UPDATING = entry is sometime between now and when my pkg was installed...just print = a "Hey, some warnings exist, you might want to read this, display = [Y/n]". * In the special case of puppetserver6, if the thing is broken because = it doesn't work with ruby, why not list the old ruby as a dependency? = I'm fairly certain in that case that multiple rubies could have = coexisted. (FWIW, it looks like puppet6 will probably never work with = 3.0 per https://tickets.puppetlabs.com/browse/PUP-10957 = ). Nobody made it a = flavored port that could also work against ruby2.7. The "upgrade" list = didn't show that it would be upgraded/touched/deleted, so one was left = to assume that it would "just work". There should have been some way = for the repo to have listed the broken version and warned me about it. = "Hi, this configuration WILL NOT WORK". * Finally, some option to, when removing a dependency package, make the = solver smarter in some way? I know that's vague, but all the info to = say "your currently installed version will be deleted if you remove = this, but the following version would cause that to not happen" is = present in the repo, it's just not clearly presented to the user. -Dan= --Apple-Mail=_C561DA2D-218E-4114-A43E-D15CD83A05D4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hey = there all,

I recently asked about default versions in = pkg, and if it was possible to tell a pkg (without building it yourself) = "no, don't conflict with that", for example to point bind-tools916 at = python3.9, because it's just a shebang line.  That thread went a = lot of places, but only served to expose an underlying = problem.

In asking those questions, I was trying to learm more = about the DEFAULTS changes, and how we can mitigate them, because every = time one changes, there's pain.

This stems from a healthy = culture of having pkg bite me during what should be a routine = thing.

Here's a few examples where we've been = burned:

* As recently mentioned, we had two versions of = python installed on a system, and removing python3.8 attempted to remove = bind-tools, which in turn would have wanted to deinstall bind9.  A = more recent bind-tools didn't require python at all -- but that upgrade = path wasn't suggested as part of it.  Pkg remove only attempts to = remove things, not fix a running system.  Pkg doesn't have any = option to keep leaf packages installed at all cost.  Nor does it = fire the solver when doing a pkg remove.

* We run RT, against = PostgreSQL.  We installed p5-DBD-Pg (which is not a direct = dependency), During a routine pkg upgrade, because of a default version = change, p5-DBD-Pg forced an upgrade from postgresql12-client to = postgresql13-client.  It did not, however, cause an upgrade to = postgresql13-server, it only caused postgresql12-server to be removed = because it conflicted with postgresql13-client.  The ultimate fix = here was a full dump and restore to get from pg12 to pg13, which took = hours.

* On a recent upgrade where ruby30 became the = default, we were left with a puppet6 install that had no ruby = interpreter it could run with.  The puppet6 port is marked as = broken (because it doesn't work with ruby30), but pkg didn't deinstall = it, threw no warnings about it, because its listed as having ruby30 as a = dependency, as opposed to a lower ruby.  The solution was "Well, I = guess we're upgrading to puppetserver 7 tonight, hope there's no fallout = from that".  Luckily, our manifests are really solid, but in the = older days of puppet, there were a lot of breaking changes.

In = all three of the cases, the old no-longer-default dependencies were = still in the tree to be pkg installed.  In all three cases, pulling = down a ports tree and editing make.conf to point at the old version = might have been a reasonable solution, that wouldn't have carried = forward with someone who just wants to be able to pkg upgrade their = stuff, and it would have required building some major packages from = scratch (in all three of the above, it would have resulted in building a = language -- python, perl, and ruby, respectively).

Here's what I'm asking:

* Most of this could be = solved by reading /usr/ports/UPDATING.  However, as someone who = wants to trust the maintainers, and just use the packages, is there a = way to see "Updating" without pulling down a ports tree?  You've = baked a "pkg" command in to read it, but that's half the work. =  (Perhaps, maybe, just make /usr/local/share/doc/pkg/UPDATING, = itself a pkg that can be updated)?  Maybe, on-upgrade, print those = things out as part of the process -- i.e. if the date on the UPDATING = entry is sometime between now and when my pkg was installed...just print = a "Hey, some warnings exist, you might want to read this, display = [Y/n]".

* In the special case of puppetserver6, if the thing = is broken because it doesn't work with ruby, why not list the old ruby = as a dependency?  I'm fairly certain in that case that multiple = rubies could have coexisted.  (FWIW, it looks like puppet6 will = probably never work with 3.0 per https://tickets.puppetlabs.com/browse/PUP-10957). =  Nobody made it a flavored port that could also work against = ruby2.7.  The "upgrade" list didn't show that it would be = upgraded/touched/deleted, so one was left to assume that it would "just = work".  There should have been some way for the repo to have listed = the broken version and warned me about it.  "Hi, this configuration = WILL NOT WORK".

* Finally, some option to, when = removing a dependency package, make the solver smarter in some way? =  I know that's vague, but all the info to say "your currently = installed version will be deleted if you remove this, but the following = version would cause that to not happen" is present in the repo, it's = just not clearly presented to the user.

-Dan= --Apple-Mail=_C561DA2D-218E-4114-A43E-D15CD83A05D4--