From nobody Wed Jan 11 07:48:59 2023 X-Original-To: freebsd-python@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NsKYv3Tgbz2pNx0 for ; Wed, 11 Jan 2023 07:48:51 +0000 (UTC) (envelope-from mizhka@gmail.com) Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com [IPv6:2a00:1450:4864:20::12c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NsKYv0wBsz4HTg; Wed, 11 Jan 2023 07:48:51 +0000 (UTC) (envelope-from mizhka@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x12c.google.com with SMTP id j17so22260346lfr.3; Tue, 10 Jan 2023 23:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=z/DvnhCu8lq67ZYw+DYRM1JRMxL2aALLfrpcLE8wbzE=; b=O67gdCUhl0HmyewxwzffQWFhlzVzJrtBJQ8odZ6v24P9dFumjd3P7wjpnwVWGXsrBc EhlP/dNJ/qAcUHbX3KzuDI6klqkTUOpOWLgbQ5HUOmWv5vCfs4oMTdtfvmywQzPejZQl pUIqTFY7M9YxwNSo64bEA1FCovh/ruOq2QyH6WDn8EgNnhQByichJ/YZrlVACyys+7Jb hk9xsGCVxegF88/KcDr7XzYvwPb4jgFhCXR2WJyv3WORDFYhPXO6/RxwrjafxkmOUXFf cgeCagZQyIrDND+4oprLk+R1e4COvbxQH1R+xXQiiZWOViX2vLOh7wr2cHZyf+/SvXCB Ic0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=z/DvnhCu8lq67ZYw+DYRM1JRMxL2aALLfrpcLE8wbzE=; b=F7rdY7V2hxaeSsW8NoG7lgWnks7dZcgiITV3VmSP6I90f9MIuWI2sHTDzLbF/JxQNE Ys/AHSA8/5Y4xQrWHoSbu6oDSk4rzbcERDT16qpjqmnHJXhsa/LmM+5HeMC7I+Hgb+38 /Li1FIz9O/+FVbYCue46C2CwCFAm70SWysyiG0EJ5fRhyyFNt1++AVj+aiFobeitJw/0 Af+RJ+8pwxkBVswkIZIdcNd7LTTXialKHW9JooesMjMohnxhICsz63Jn9g3+Wuofr6w6 NhrKBgT3NUG1PW7u1sMT9HAHmP217ru5tiF6G1BaJvjcT/5mJk3NzuQ7a50R5icFkB1p pvMw== X-Gm-Message-State: AFqh2kqPqpCR55LKd39YnmG15lkPN57jwlvr1pCESdVzf6F4BLv4UmGy ZTGTZghp56zpAsGzujR7u0HPYMXM7AHETmF4GQhtsFCY X-Google-Smtp-Source: AMrXdXshJoPuGR/bdwXiaExPuWpN+IsxeZ7+ECtWXOJuyZAZpJo50BSvknaYg+zmpMzvIwgP57ojBYNP9myXRtQDUwo= X-Received: by 2002:a05:6512:1516:b0:4ca:f58f:195e with SMTP id bq22-20020a056512151600b004caf58f195emr6690200lfb.319.1673423328694; Tue, 10 Jan 2023 23:48:48 -0800 (PST) List-Id: FreeBSD-specific Python issues List-Archive: https://lists.freebsd.org/archives/freebsd-python List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-python@freebsd.org X-BeenThere: freebsd-python@freebsd.org MIME-Version: 1.0 References: <98589f8e-25e9-0738-2690-d197651ded72@freebsd.org> In-Reply-To: <98589f8e-25e9-0738-2690-d197651ded72@freebsd.org> From: Michael Zhilin Date: Wed, 11 Jan 2023 10:48:59 +0300 Message-ID: Subject: Re: PYTHON_EXT_SUFFIX value for shared libraries, pyc files and others To: Charlie Li Cc: freebsd-python@freebsd.org Content-Type: multipart/alternative; boundary="00000000000010d4a405f1f83b9e" X-Rspamd-Queue-Id: 4NsKYv0wBsz4HTg X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000010d4a405f1f83b9e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Jan 11, 2023 at 9:45 AM Charlie Li wrote: > Michael Zhilin wrote: > > Hi, > > > > I have a problem with PYTHON_EXT_SUFFIX when I try to build ports > > (including python) with the option WITH_DEBUG. > > > > I want to build subset of ports with WITH_DEBUG by poudriere, but > > several python package builds have failed on phase "package" due to > > missing artifacts. The root cause of failure is that ports uses same > > macro PYTHON_EXT_SUFFIX for shared libraries and pyc files, but actual > > file names have different suffixes: > > - pyc files have the suffix ".cpython-39" as expected. > > - so files have the suffix ".cpython-39d". > > > > The "d" is ${PYTHON_ABIVER} and actual ABI flags of Python build. > > According to Python specifications, all pyc files must have a suffix > > without ABI flags. Shared libraries have suffixes with ABI flags, but I > > didn't find any spec about it. > > > PEP-3147 for byte compilation files (.pyc) and PEP-3149 for extension > module files (.so). > > The first idea came to me is to add an extra suffix PYTHON_EXTSO_SUFFIX > > with value ".cpython-${PYTHON_SUFFIX}${PYTHON_ABIVER}". It's easy to ad= d > > it and replace all occurrences in pkg-plist and Makefile, but it may be > > hard to maintain it in future. > > > Looks like some unfinished business from PR 252057. In theory, should be > able to append ${PYTHON_ABIVER} directly to ${PYTHON_EXT_SUFFIX}, since > ${PYTHON_CMD}-config --abiflags returns the correct flags even if empty. > Because the ABI changes when you build Python itself under > --with-pydebug (WITH_DEBUG or DEBUG option), you cannot build only a > subset of Python ports under WITH_DEBUG; it's an all-or-nothing situation= . > > Byte compilation files are not compiled or installed when the port is > built under PEP-517. Compiling and installing byte compilation files > with distutils/setuptools at build/stage-time will be deprecated. A new > install-time hook is under development to replace this functionality, > which has the added benefit of less pkg-plist churn and framework > maintenance. > > -- > Charlie Li > =E2=80=A6nope, still don't have an exit line. > > Thank you! It looks very good to me. I forgot to mention CMake files that contains suffix with ABI flags as well, but approach mentioned by you works for them as well. Looking for deprecation of byte compilation. Please let me know if I can help to move it forward in any way. BR, Michael. --00000000000010d4a405f1f83b9e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Wed, Jan 11, 2023 at 9:45 AM Charlie Li <vishwin@freebsd.org> wrote:
Michael Zhilin wrote:
> Hi,
>
> I have a problem with PYTHON_EXT_SUFFIX when I try to build ports
> (including python) with the option WITH_DEBUG.
>
> I want to build subset of ports with WITH_DEBUG by poudriere, but
> several python package builds have failed on phase "package"= due to
> missing artifacts. The root cause of failure is that ports uses same <= br> > macro PYTHON_EXT_SUFFIX for shared libraries and pyc files, but actual=
> file names have different suffixes:
>=C2=A0 =C2=A0- pyc files have the suffix ".cpython-39" as exp= ected.
>=C2=A0 =C2=A0- so files have the suffix ".cpython-39d".
>
> The "d" is ${PYTHON_ABIVER} and actual ABI flags of Python b= uild.
> According to Python specifications, all pyc files must have a suffix <= br> > without ABI flags. Shared libraries have suffixes with ABI flags, but = I
> didn't find any spec about it.
>
PEP-3147 for byte compilation files (.pyc) and PEP-3149 for extension
module files (.so).
> The first idea came to me is to add an extra suffix PYTHON_EXTSO_SUFFI= X
> with value ".cpython-${PYTHON_SUFFIX}${PYTHON_ABIVER}". It&#= 39;s easy to add
> it and replace all occurrences in pkg-plist and Makefile, but it may b= e
> hard to maintain it in future.
>
Looks like some unfinished business from PR 252057. In theory, should be able to append ${PYTHON_ABIVER} directly to ${PYTHON_EXT_SUFFIX}, since ${PYTHON_CMD}-config --abiflags returns the correct flags even if empty. Because the ABI changes when you build Python itself under
--with-pydebug (WITH_DEBUG or DEBUG option), you cannot build only a
subset of Python ports under WITH_DEBUG; it's an all-or-nothing situati= on.

Byte compilation files are not compiled or installed when the port is
built under PEP-517. Compiling and installing byte compilation files
with distutils/setuptools at build/stage-time will be deprecated. A new install-time hook is under development to replace this functionality,
which has the added benefit of less pkg-plist churn and framework
maintenance.

--
Charlie Li
=E2=80=A6nope, still don't have an exit line.


Thank you!

It looks very good to me.
I forgot to mention CMake files= that contains suffix with ABI flags as well, but approach mentioned by you= works for them as well.
Looking for deprecation of byte com= pilation. Please let me know if I can help to move it forward in any way. <= br>

BR, Michael.
--00000000000010d4a405f1f83b9e--