Date: Wed, 11 Jan 2023 10:48:59 +0300 From: Michael Zhilin <mizhka@gmail.com> To: Charlie Li <vishwin@freebsd.org> Cc: freebsd-python@freebsd.org Subject: Re: PYTHON_EXT_SUFFIX value for shared libraries, pyc files and others Message-ID: <CAF19XBL9yhq00VK9r6xQ38ZXH1x7MJiSbeT72AvRxGx61Hk2bg@mail.gmail.com> In-Reply-To: <98589f8e-25e9-0738-2690-d197651ded72@freebsd.org> References: <CAF19XBKVL=1CdJHsYC1-WZwX7Gvop7Zjv3aADHCx5eoW9rAz_w@mail.gmail.com> <98589f8e-25e9-0738-2690-d197651ded72@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000010d4a405f1f83b9e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On Wed, Jan 11, 2023 at 9:45 AM Charlie Li <vishwin@freebsd.org> wrote:
> Michael Zhilin wrote:
> > Hi,
> >
> > I have a problem with PYTHON_EXT_SUFFIX when I try to build ports
> > (including python) with the option WITH_DEBUG.
> >
> > I want to build subset of ports with WITH_DEBUG by poudriere, but
> > several python package builds have failed on phase "package" due to
> > missing artifacts. The root cause of failure is that ports uses same
> > macro PYTHON_EXT_SUFFIX for shared libraries and pyc files, but actual
> > file names have different suffixes:
> > - pyc files have the suffix ".cpython-39" as expected.
> > - so files have the suffix ".cpython-39d".
> >
> > The "d" is ${PYTHON_ABIVER} and actual ABI flags of Python build.
> > According to Python specifications, all pyc files must have a suffix
> > without ABI flags. Shared libraries have suffixes with ABI flags, but I
> > didn't find any spec about it.
> >
> PEP-3147 for byte compilation files (.pyc) and PEP-3149 for extension
> module files (.so).
> > The first idea came to me is to add an extra suffix PYTHON_EXTSO_SUFFIX
> > with value ".cpython-${PYTHON_SUFFIX}${PYTHON_ABIVER}". It's easy to ad=
d
> > it and replace all occurrences in pkg-plist and Makefile, but it may be
> > hard to maintain it in future.
> >
> Looks like some unfinished business from PR 252057. In theory, should be
> able to append ${PYTHON_ABIVER} directly to ${PYTHON_EXT_SUFFIX}, since
> ${PYTHON_CMD}-config --abiflags returns the correct flags even if empty.
> Because the ABI changes when you build Python itself under
> --with-pydebug (WITH_DEBUG or DEBUG option), you cannot build only a
> subset of Python ports under WITH_DEBUG; it's an all-or-nothing situation=
.
>
> Byte compilation files are not compiled or installed when the port is
> built under PEP-517. Compiling and installing byte compilation files
> with distutils/setuptools at build/stage-time will be deprecated. A new
> install-time hook is under development to replace this functionality,
> which has the added benefit of less pkg-plist churn and framework
> maintenance.
>
> --
> Charlie Li
> =E2=80=A6nope, still don't have an exit line.
>
>
Thank you!
It looks very good to me.
I forgot to mention CMake files that contains suffix with ABI flags as
well, but approach mentioned by you works for them as well.
Looking for deprecation of byte compilation. Please let me know if I can
help to move it forward in any way.
BR, Michael.
--00000000000010d4a405f1f83b9e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail=
_attr">On Wed, Jan 11, 2023 at 9:45 AM Charlie Li <<a href=3D"mailto:vis=
hwin@freebsd.org">vishwin@freebsd.org</a>> wrote:<br></div><blockquote c=
lass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px soli=
d rgb(204,204,204);padding-left:1ex">Michael Zhilin wrote:<br>
> Hi,<br>
> <br>
> I have a problem with PYTHON_EXT_SUFFIX when I try to build ports <br>
> (including python) with the option WITH_DEBUG.<br>
> <br>
> I want to build subset of ports with WITH_DEBUG by poudriere, but <br>
> several python package builds have failed on phase "package"=
due to <br>
> missing artifacts. The root cause of failure is that ports uses same <=
br>
> macro PYTHON_EXT_SUFFIX for shared libraries and pyc files, but actual=
<br>
> file names have different suffixes:<br>
>=C2=A0 =C2=A0- pyc files have the suffix ".cpython-39" as exp=
ected.<br>
>=C2=A0 =C2=A0- so files have the suffix ".cpython-39d".<br>
> <br>
> The "d" is ${PYTHON_ABIVER} and actual ABI flags of Python b=
uild. <br>
> According to Python specifications, all pyc files must have a suffix <=
br>
> without ABI flags. Shared libraries have suffixes with ABI flags, but =
I <br>
> didn't find any spec about it.<br>
> <br>
PEP-3147 for byte compilation files (.pyc) and PEP-3149 for extension <br>
module files (.so).<br>
> The first idea came to me is to add an extra suffix PYTHON_EXTSO_SUFFI=
X <br>
> with value ".cpython-${PYTHON_SUFFIX}${PYTHON_ABIVER}". It&#=
39;s easy to add <br>
> it and replace all occurrences in pkg-plist and Makefile, but it may b=
e <br>
> hard to maintain it in future.<br>
> <br>
Looks like some unfinished business from PR 252057. In theory, should be <b=
r>
able to append ${PYTHON_ABIVER} directly to ${PYTHON_EXT_SUFFIX}, since <br=
>
${PYTHON_CMD}-config --abiflags returns the correct flags even if empty. <b=
r>
Because the ABI changes when you build Python itself under <br>
--with-pydebug (WITH_DEBUG or DEBUG option), you cannot build only a <br>
subset of Python ports under WITH_DEBUG; it's an all-or-nothing situati=
on.<br>
<br>
Byte compilation files are not compiled or installed when the port is <br>
built under PEP-517. Compiling and installing byte compilation files <br>
with distutils/setuptools at build/stage-time will be deprecated. A new <br=
>
install-time hook is under development to replace this functionality, <br>
which has the added benefit of less pkg-plist churn and framework <br>
maintenance.<br>
<br>
-- <br>
Charlie Li<br>
=E2=80=A6nope, still don't have an exit line.<br>
<br></blockquote><div><br></div><div>Thank you! <br></div><div><br></div><d=
iv>It looks very good to me. <br></div><div>I forgot to mention CMake files=
that contains suffix with ABI flags as well, but approach mentioned by you=
works for them as well. <br></div><div>Looking for deprecation of byte com=
pilation. Please let me know if I can help to move it forward in any way. <=
br></div><div><br></div><div>BR, Michael. <br></div></div></div>
--00000000000010d4a405f1f83b9e--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAF19XBL9yhq00VK9r6xQ38ZXH1x7MJiSbeT72AvRxGx61Hk2bg>