From nobody Thu Sep 29 21:21:09 2022 X-Original-To: ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MdmWF4dVZz4YCqH for ; Thu, 29 Sep 2022 21:21:13 +0000 (UTC) (envelope-from se@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MdmWF46J4z3KcF; Thu, 29 Sep 2022 21:21:13 +0000 (UTC) (envelope-from se@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1664486473; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DXPJjsfcsulHtRK7fAteLfPTe5sYHiqD62O47nkqWbg=; b=aj0htHN/L2ItYmqhGKTppt/jCKrNFtfb6kUcTVNuHfxNKOnF3MBCS8KpAi4RhpJQjHY0VS 1/nGzK702KvCLQwCUy2Qwdu1SwD3ql/w3UkJRCwSMaNEMv0m1SZ1iGErOLlJ/mmNzMGhuU CUidIKL6iZlkVbM049mgsd5hNx3G/oqU8gxWoU1dhfww6t5WM9YV0FhOTqXrVPqmOacfXh vLD4oDHJpGjNdW7evu/Jw2z84ORZ1kaFt6cYpx7gDG8MogheOAThxGcOqYbMgEZT1Qk5qj Fz9dwqULf2z4o8IHnn2j7E7qbDKtGuqmZmvwxhwwrsEfWp5QgbsmT347Okygjw== Received: from [IPV6:2003:cd:5f1f:1b00:d0ff:a65f:91a2:c562] (p200300cd5f1f1b00d0ffa65f91a2c562.dip0.t-ipconnect.de [IPv6:2003:cd:5f1f:1b00:d0ff:a65f:91a2:c562]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4MdmWD6Mx7zyT0; Thu, 29 Sep 2022 21:21:12 +0000 (UTC) (envelope-from se@FreeBSD.org) Message-ID: <2c373ee2-785b-d31f-23d3-bc0856c7bf5e@FreeBSD.org> Date: Thu, 29 Sep 2022 23:21:09 +0200 List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.3.1 Subject: Re: Python version dependencies in pkg To: Dan Mahoney Cc: ports@freebsd.org, Alexander Leidinger References: <8B490359-27A3-410C-AE98-C1362D4FA9F0@gushi.org> <89D2B2D7-FA3D-46F5-A93B-4846950A157C@gushi.org> <65597c37-813b-d0ed-ea9f-8591b68cf097@gwdg.de> <20220929083939.Horde.Viq2uLBgUctcrsgmaJxN_1u@webmail.leidinger.net> From: Stefan Esser Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1664486473; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DXPJjsfcsulHtRK7fAteLfPTe5sYHiqD62O47nkqWbg=; b=tZkQXUPbKP36nnj+LyYvWbbCUlAB5UFIATO5gp6S4zuCgo17ePk49dlqmPHfUnoCswwTXx i3FJ6fLmynTmiRjWwTkAvxhAsLCZz4oNysJkOd/CutryZPmNMNNYLOCXphACl20r2VAfis 1UFcVBroZZE9HLPciWQ348/g6R+QftH30LnDqbmwSQiWMRqm00ALbJrEBsFQCNFMc/xgaI bTXCSTqqosUyPHnrI5QQLIsss+zntylkwmk+iN19lRlnmvSnj5AtJTffrOe9JJU06Ux14G JJsK/x7hrhfUkknDDZs/5kExwhAlp7G7Sl00KmpQ2mCvVJ4TZCyTCjkgLdFF4w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1664486473; a=rsa-sha256; cv=none; b=SxYdfR5X8V7+1qTsXyf/Cyb+djaFDTWQ9YNuO+9DbLygwPxHS2Nq9MvVWYIxephXYvBdZu sxdyVTWNxElfMfEozKk6sApfmC0UWAZtgk23y45HioEHoxHWg5bl8QutvD3ZhwYhedtAHv V63Ro3u2XvRMKVN7x7VLMY9n66lmNG5D8U0W8QQpuU3gprD08WVA0ZcVBKHpzc1ndtriYL D60DGch7nhQ+Ol5qjZbr4lkRYa2auPzg2IhoUBpKeyEbQya0a5JprEwwO2qurrHBXr7t2F EZ53H3iM6dYjO9fzXgYX/xtZEnG1xs9+RuI/8tDeBoKVPsFRdjjOZeRS2cDxAA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N Am 29.09.22 um 21:52 schrieb Dan Mahoney:>> On Sep 28, 2022, at 23:39, Alexander Leidinger wrote: >> >> Quoting Rainer Hurling (from Thu, 29 Sep 2022 06:43:49 +0200): >> >>> Hi Dan, >>> >>> Am 28.09.22 um 23:01 schrieb Dan Mahoney: >>>> >>>> >>>>> On Sep 26, 2022, at 20:22, Kevin Oberman > wrote: >>>>> >>>>> On Mon, Sep 26, 2022, 18:13 Dan Mahoney > wrote: >>>> >>>> Okay, so in either the package file, or the repo file, is there a bit that says "whatever is the default" or that explicitly says "if there's not a python, install 3.8"? >>> >>> There is PYTHON_DEFAULT (set to 3.9 ATM) in ports/Mk/bsd.default-versions.mk. You can override this for example by an entry in /etc/make.conf: >>> >>> DEFAULT_VERSIONS+= python=3.10 python3=3.10 > > That's ports. I'm asking about packages. Packages are built based on consistent defaults, e.g. currently for Python-3.9. If you need a different Python version, you can easily build your own packages for a different set of defaults. The package system does not and cannot provide the version independence you are asking for, since dependent packages could be built with different options for changed defaults. > I'm asking on ports@ because there's not any kind of pkgs@ mailing list. I feel like I'm not being clear as to why. Yes, and its a legitimate question - but the simple answer is that the official packages constitute a set with complex interdependencies between its members. >> To extend a little bit on that: >> "pkg info | grep py3" will give you a list of ports which are depending on a specific python version. If it lists py37-xxx it means the xxx port depends upon python 3.7. If you then delete python 3.7 pkg will also ask you if it is ok to delete all the py37-* ports. > > No, it will give me a list of *packages*, except it won't. > > Case in point: bind-tools (which in turn is a dependency of bind9) > > # pkg info | grep bind-tools > bind-tools-9.16.27_1 Command line tools from BIND: delv, dig, host, nslookup... > # pkg all-depends bind-tools > libxml2-2.9.13_1 > python38-3.8.13 > libidn2-2.3.2 > py38-ply-3.11 > libuv-1.42.0 > libedit-3.1.20210910,1 > json-c-0.15_1 > # This command will show packages together with the Python version they depend on: pkg query "%dn %n-%v" | grep -E "^python3[0-9]+ " > The latest version of bind-tools no longer requires python at all (bind 9.18 has rewritten all the utilities in C once again), so I can't just experiment with this. Yes, and the following command shows this information, too: pkg rquery "%dn %n-%v" bind-tools | grep -E "^python3[0-9]+ " > But trying to do a pkg delete python3.8 attempted to delete bind-tools, and I was looking for the magic incantation that says "without reinstalling bind-tools, how can I re-point it at the python39 variants of that package". On a box without a ports tree. Only using pkg. As you say, bind-tools does no longer depend on Python and therefore is not a good example. But you can use the above command to query the python dependency for any pre-built package. And no, you cannot "re-point" a package. "Re-pointing" is done by building the package with different default versions, and you can do it yourself, if the official version do not match your requirements. > If the answer is "you really can't unless you run poudriere and tell it to build a different package with a different dependent version of python", then so be it. But I'd like to be able to install py39-ply, and python39, and have bind-tools not try to deinstall, and instead just re-point itself. Now, I know this is not the same as a library that can build linked against two ssl's where there are real compile-time/link-time differences. This is because one particular file in bind-tools is a python script. The port/package system cannot distinguish cases of multiple packages that could be used as substitutes for each other. There is no simple algorithmic way to identify packages that could be exchanged with each other. It might be possible to manually annotate packages in such a way (e.g. if any version of Python-3.x would do with the matching set of Python modules), but this would require a lot of effort for little value. > If the answer is "some dumb ports will always depend on whatever the default version was on the build farm, so you MUST have those versions installed" that's an answer. Yes, that's *the* answer ... > I know ports support "flavors" but there's no py38-bind-tools, which led me to believe there was a looser coupling. Only a small subset of supported flavors is built by the built servers. But bind-tools is no longer a good example ... > My secondary question: when a pkg is nailed to the "Default" version, does that result in the pkg file itself actually saying 3.8, or just a reference that says "the default was used, but any python will do, but you have to manually edit the pkgdb to update the dependencies". There are search paths in the Python interpreters, some of which include the Python version. In many cases you could move a python module into the search path of a different interpreter version, but there is no guarantee, that this would lead to a working combination of interpreter and module. You could manipulate the pkgdb to reflect such a change in the installed files, but that leads into madness ... > If I had manually installed py39-ply and python3.9, would bind-tools have still required python3.8 and py38-ply? If you install the bind-tools package (and assuming it still depended on Python), then the exact version recorded at the time the bind-tools package was built will be required. >> So for a given system with already installed ports it is not simply installing python 3.9 and deleting 3.7. It is also updating all the ports which depend upon 3.7. Yes. > Again, that's ports. My question the whole time has been "how does this knob, specifed in the ports tree, ultimately translate into the pkg tree", where there is no ports tree present and make.conf is not consulted. > > I am asking about either fields in the pkg file itself, or in the pkg database. You can display all the information in the pkgdb with "pkg query" for installed and "pkg rquery" for available packages. - STefan