Date: Sun, 26 Jan 2014 15:22:45 +0100 From: Marcus von Appen <mva@FreeBSD.org> To: freebsd-python@freebsd.org Subject: Re: PYDISTUTILS_AUTOPLIST question Message-ID: <20140126142245.GB1394@medusa.sysfault.org> In-Reply-To: <52E5007C.8060805@yandex.ru> References: <52E4BCF4.3070507@yandex.ru> <20140126113022.GA1394@medusa.sysfault.org> <52E5007C.8060805@yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
--QKdGvSO+nmPlgiQ/
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On, Sun Jan 26, 2014, Ruslan Makhmatkhanov wrote:
> Marcus von Appen wrote on 26.01.2014 15:30:
> > On, Sun Jan 26, 2014, Ruslan Makhmatkhanov wrote:
> >
> >> Hi,
> >>
> >> while PYDISTUTILS_AUTOPLIST works smoothly for majority of ports, there
> >> is the problem with some of them I come across yesterday. For example,
> >> there is the port devel/py-Products.PasswordResetTool. If I set there
> >> PYDISTUTILS_AUTOPLIST, and try to build, I got packaging error, because
> >> some of the files, that port provides, cannot be compiled (by design).
> >> While they are python code files, they are not supposed to be executed
> >> with python interpreter directly (they are kind of templates).
> >>
> >> So, is there any possibility to:
> >> a) Do not add compiled/optimized filenames to packing list, if the
> >> actual file fails to compile?
> >> b) Make a hook, that allows to list file, that should not have
> >> corresponding *.pyc/*.pyo entries in the packing list?
> >
> > If the port uses distutils, it should not matter, if you do not mess around in
> > the post-install targets yourself, since the file list will be taken from what
> > distutils records - we do not implement magic ourselves for distutils-aware
> > ports (well, a bit :-).
> > If this does not work properly for you, upstream might do something that
> > breaks the intended behaviour of distutils.
> >
> > So, before fixing the symptom, I actually would like to find out, what exactly
> > they do on installation. Are all files properly recorded in ${_PYTHONPKGLIST}?
> > What errors do you get, what does the install target complain about, etc.etc.?
> >
> > Cheers
> > Marcus
>
>
> For example at install stage it fails with pwreset_constructURL.py file:
>
> byte-compiling
> /usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py
> to pwreset_constructURL.pyc
> File
> "/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py",
> line 14
> return "%s/passwordreset/%s" % (host, randomstring)
> SyntaxError: 'return' outside function
>
> writing byte-compilation script '/tmp/tmpqmG0z8.py'
> /usr/local/bin/python2.7 -O /tmp/tmpqmG0z8.py
> File
> "/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py",
> line 14
> return "%s/passwordreset/%s" % (host, randomstring)
> SyntaxError: 'return' outside function
>
> removing /tmp/tmpqmG0z8.py
>
> And here is what on package stage:
> ===> Building package for py27-Products.PasswordResetTool-2.0.15_1
> pkg-static:
> lstat(/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.pyc):
> No such file or directory
> pkg-static:
> lstat(/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.pyo):
> No such file or directory
> *** Error code 1
>
> Stop.
> make: stopped in
> /usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool
>
>
> As we can see *.pyc/*.pyo pkg-plist entries were generated for
> pwreset_constructURL.py despite the fact that byte-compiling had failed.
Thanks - I just checked, what exactly happens within the record magic for
those cases. distutils (and thus setuptools, since it does not do anything
special in that area) just picks up the .py files and adds entries for
matching .pyc and .pyo files without actually checking, if those files
could be compiled successfully.
The code snippet (distutils/install_lib.py, _bytecode_filenames()) actually
needs to verify, if .pyc/.pyo files exist, since the bytecode compiler
function (distutils/util.py, byte_compile() in indirect mode)
does not break on errors, but silently proceeds.
Rather than adding a hack to bsd.python.mk to filter out such entries, I'd
like to see that issue fixed upstream: http://bugs.python.org/issue20397.
My current (quick'n'dirty) idea is to add a simple file existence verification
check into our currently supported versions.
Cheers
Marcus
--QKdGvSO+nmPlgiQ/
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (FreeBSD)
iEYEARECAAYFAlLlGjUACgkQi68/ErJnpkeRkgCfZyLuwiVz+veGdRAkdemlconB
y+UAniK72S8PTQaP8mrR8SDja2SNWyqX
=EyGN
-----END PGP SIGNATURE-----
--QKdGvSO+nmPlgiQ/--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140126142245.GB1394>