From owner-svn-ports-head@freebsd.org Fri Jan 8 16:45:11 2016 Return-Path: Delivered-To: svn-ports-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC387A68A2A; Fri, 8 Jan 2016 16:45:11 +0000 (UTC) (envelope-from koobs@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 82B1D157D; Fri, 8 Jan 2016 16:45:11 +0000 (UTC) (envelope-from koobs@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u08GjAK6005988; Fri, 8 Jan 2016 16:45:10 GMT (envelope-from koobs@FreeBSD.org) Received: (from koobs@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u08Gj9EW005975; Fri, 8 Jan 2016 16:45:09 GMT (envelope-from koobs@FreeBSD.org) Message-Id: <201601081645.u08Gj9EW005975@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: koobs set sender to koobs@FreeBSD.org using -f From: Kubilay Kocak Date: Fri, 8 Jan 2016 16:45:09 +0000 (UTC) To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r405569 - in head/lang: python27 python27/files python32 python32/files python33 python33/files python34 python34/files python35 python35/files X-SVN-Group: ports-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-ports-head@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the ports tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 16:45:11 -0000 Author: koobs Date: Fri Jan 8 16:45:09 2016 New Revision: 405569 URL: https://svnweb.freebsd.org/changeset/ports/405569 Log: lang/python{27,3*}: Backport patch in upstream issue20397 In certain situations, file references (.py[co]) for Python files that fail to compile with compileall() are still added to distutils --record output. This output is used for pkg-plist generation and must only contain references to files that will be installed. One example of a failure condition is when a Python 2/3 compatible package containing a file containing Python 3.x only code is built with Python 2.x, such as Gunicorn's _gaiohttp.py [1] This change backports patches submitted against upstream issue 20397 [2] that has not yet been committed. - For Python 2.7 and 3.5, backport both install_lib and test - For Python 3.2, 3.3 and 3.4, only backport install_lib [1] https://svnweb.freebsd.org/changeset/ports/404558 [2] https://bugs.python.org/issue20397 Thank you to Brendan Molloy for producing and submitting the patches against upstream sources. Reviewed by: sbz (python) MFH: 2016Q1 Differential Revision: D4832 Added: head/lang/python27/files/patch-Lib_distutils_command_install__lib.py (contents, props changed) head/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py (contents, props changed) head/lang/python32/files/patch-Lib_distutils_command_install__lib.py (contents, props changed) head/lang/python33/files/patch-Lib_distutils_command_install__lib.py (contents, props changed) head/lang/python34/files/patch-Lib_distutils_command_install__lib.py (contents, props changed) head/lang/python35/files/patch-Lib_distutils_command_install__lib.py (contents, props changed) head/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py (contents, props changed) Modified: head/lang/python27/Makefile head/lang/python32/Makefile head/lang/python33/Makefile head/lang/python34/Makefile head/lang/python35/Makefile Modified: head/lang/python27/Makefile ============================================================================== --- head/lang/python27/Makefile Fri Jan 8 16:44:10 2016 (r405568) +++ head/lang/python27/Makefile Fri Jan 8 16:45:09 2016 (r405569) @@ -2,6 +2,7 @@ PORTNAME= python27 PORTVERSION= ${PYTHON_PORTVERSION} +PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} DISTNAME= Python-${PORTVERSION} Added: head/lang/python27/files/patch-Lib_distutils_command_install__lib.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lang/python27/files/patch-Lib_distutils_command_install__lib.py Fri Jan 8 16:45:09 2016 (r405569) @@ -0,0 +1,34 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. +--- + Lib/distutils/command/install_lib.py | 17 +++++++++++++---- + Lib/distutils/tests/test_install_lib.py | 8 ++++++-- + 2 files changed, 19 insertions(+), 6 deletions(-) + +--- Lib/distutils/command/install_lib.py.orig 2015-12-05 19:46:56 UTC ++++ Lib/distutils/command/install_lib.py +@@ -168,10 +168,14 @@ class install_lib(Command): + ext = os.path.splitext(os.path.normcase(py_file))[1] + if ext != PYTHON_SOURCE_EXTENSION: + continue +- if self.compile: +- bytecode_files.append(py_file + "c") +- if self.optimize > 0: +- bytecode_files.append(py_file + "o") ++ ++ pyc_file = py_file + "c" ++ if self.compile and os.path.isfile(pyc_file): ++ bytecode_files.append(pyc_file) ++ ++ pyo_file = py_file + "o" ++ if self.optimize > 0 and os.path.isfile(pyo_file): ++ bytecode_files.append(pyo_file) + + return bytecode_files + Added: head/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py Fri Jan 8 16:45:09 2016 (r405569) @@ -0,0 +1,30 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. +--- + Lib/distutils/command/install_lib.py | 17 +++++++++++++---- + Lib/distutils/tests/test_install_lib.py | 8 ++++++-- + 2 files changed, 19 insertions(+), 6 deletions(-) + +--- Lib/distutils/tests/test_install_lib.py.orig 2015-12-05 19:46:57 UTC ++++ Lib/distutils/tests/test_install_lib.py +@@ -64,8 +64,12 @@ class InstallLibTestCase(support.Tempdir + cmd.distribution.packages = [pkg_dir] + cmd.distribution.script_name = 'setup.py' + +- # get_output should return 4 elements +- self.assertGreaterEqual(len(cmd.get_outputs()), 2) ++ # Create rubbish, uncompilable file ++ f = os.path.join(pkg_dir, 'rubbish.py') ++ self.write_file(f, 'rubbish()') ++ ++ # get_output should return 3 elements ++ self.assertEqual(len(cmd.get_outputs()), 3) + + def test_get_inputs(self): + pkg_dir, dist = self.create_dist() Modified: head/lang/python32/Makefile ============================================================================== --- head/lang/python32/Makefile Fri Jan 8 16:44:10 2016 (r405568) +++ head/lang/python32/Makefile Fri Jan 8 16:45:09 2016 (r405569) @@ -2,6 +2,7 @@ PORTNAME= python32 PORTVERSION= ${PYTHON_PORTVERSION} +PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} DISTNAME= Python-${PORTVERSION} Added: head/lang/python32/files/patch-Lib_distutils_command_install__lib.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lang/python32/files/patch-Lib_distutils_command_install__lib.py Fri Jan 8 16:45:09 2016 (r405569) @@ -0,0 +1,34 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. + +--- Lib/distutils/command/install_lib.py.orig 2014-10-12 06:52:02 UTC ++++ Lib/distutils/command/install_lib.py +@@ -165,11 +165,18 @@ class install_lib(Command): + if ext != PYTHON_SOURCE_EXTENSION: + continue + if self.compile: +- bytecode_files.append(imp.cache_from_source( +- py_file, debug_override=True)) ++ candidate = imp.cache_from_source( ++ py_file, debug_override=True) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + if self.optimize > 0: +- bytecode_files.append(imp.cache_from_source( +- py_file, debug_override=False)) ++ candidate = imp.cache_from_source( ++ py_file, debug_override=False) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) + + return bytecode_files + Modified: head/lang/python33/Makefile ============================================================================== --- head/lang/python33/Makefile Fri Jan 8 16:44:10 2016 (r405568) +++ head/lang/python33/Makefile Fri Jan 8 16:45:09 2016 (r405569) @@ -2,7 +2,7 @@ PORTNAME= python33 PORTVERSION= ${PYTHON_PORTVERSION} -PORTREVISION= 2 +PORTREVISION= 3 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} DISTNAME= Python-${PORTVERSION} Added: head/lang/python33/files/patch-Lib_distutils_command_install__lib.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lang/python33/files/patch-Lib_distutils_command_install__lib.py Fri Jan 8 16:45:09 2016 (r405569) @@ -0,0 +1,34 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. + +--- Lib/distutils/command/install_lib.py.orig 2014-10-12 06:52:02 UTC ++++ Lib/distutils/command/install_lib.py +@@ -165,11 +165,18 @@ class install_lib(Command): + if ext != PYTHON_SOURCE_EXTENSION: + continue + if self.compile: +- bytecode_files.append(imp.cache_from_source( +- py_file, debug_override=True)) ++ candidate = imp.cache_from_source( ++ py_file, debug_override=True) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + if self.optimize > 0: +- bytecode_files.append(imp.cache_from_source( +- py_file, debug_override=False)) ++ candidate = imp.cache_from_source( ++ py_file, debug_override=False) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) + + return bytecode_files + Modified: head/lang/python34/Makefile ============================================================================== --- head/lang/python34/Makefile Fri Jan 8 16:44:10 2016 (r405568) +++ head/lang/python34/Makefile Fri Jan 8 16:45:09 2016 (r405569) @@ -3,7 +3,7 @@ PORTNAME= python34 PORTVERSION= ${PYTHON_PORTVERSION} -PORTREVISION= 0 +PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} DISTNAME= Python-${PORTVERSION} Added: head/lang/python34/files/patch-Lib_distutils_command_install__lib.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lang/python34/files/patch-Lib_distutils_command_install__lib.py Fri Jan 8 16:45:09 2016 (r405569) @@ -0,0 +1,35 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. + +--- Lib/distutils/command/install_lib.py.orig 2016-01-08 12:38:49 UTC ++++ Lib/distutils/command/install_lib.py +@@ -165,11 +165,19 @@ class install_lib(Command): + if ext != PYTHON_SOURCE_EXTENSION: + continue + if self.compile: +- bytecode_files.append(importlib.util.cache_from_source( +- py_file, debug_override=True)) ++ candidate = importlib.util.cache_from_source( ++ py_file, debug_override=True) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + if self.optimize > 0: +- bytecode_files.append(importlib.util.cache_from_source( +- py_file, debug_override=False)) ++ candidate = importlib.util.cache_from_source( ++ py_file, debug_override=False) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + + return bytecode_files + Modified: head/lang/python35/Makefile ============================================================================== --- head/lang/python35/Makefile Fri Jan 8 16:44:10 2016 (r405568) +++ head/lang/python35/Makefile Fri Jan 8 16:45:09 2016 (r405569) @@ -3,6 +3,7 @@ PORTNAME= python DISTVERSION= ${PYTHON_PORTVERSION} +PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PYTHON_PORTVERSION} PKGNAMESUFFIX= ${PYTHON_SUFFIX} Added: head/lang/python35/files/patch-Lib_distutils_command_install__lib.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lang/python35/files/patch-Lib_distutils_command_install__lib.py Fri Jan 8 16:45:09 2016 (r405569) @@ -0,0 +1,41 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. +--- + Lib/distutils/command/install_lib.py | 17 +++++++++++++---- + Lib/distutils/tests/test_install_lib.py | 8 ++++++-- + 2 files changed, 19 insertions(+), 6 deletions(-) + +--- Lib/distutils/command/install_lib.py.orig 2015-12-07 01:39:07 UTC ++++ Lib/distutils/command/install_lib.py +@@ -164,12 +164,21 @@ class install_lib(Command): + ext = os.path.splitext(os.path.normcase(py_file))[1] + if ext != PYTHON_SOURCE_EXTENSION: + continue ++ + if self.compile: +- bytecode_files.append(importlib.util.cache_from_source( +- py_file, optimization='')) ++ candidate = importlib.util.cache_from_source( ++ py_file, optimization='') ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + if self.optimize > 0: +- bytecode_files.append(importlib.util.cache_from_source( +- py_file, optimization=self.optimize)) ++ candidate = importlib.util.cache_from_source( ++ py_file, optimization=self.optimize) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + + return bytecode_files + Added: head/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py Fri Jan 8 16:45:09 2016 (r405569) @@ -0,0 +1,33 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. +--- + Lib/distutils/command/install_lib.py | 17 +++++++++++++---- + Lib/distutils/tests/test_install_lib.py | 8 ++++++-- + 2 files changed, 19 insertions(+), 6 deletions(-) + +--- Lib/distutils/tests/test_install_lib.py.orig 2015-12-07 01:39:07 UTC ++++ Lib/distutils/tests/test_install_lib.py +@@ -64,11 +64,15 @@ class InstallLibTestCase(support.Tempdir + cmd.distribution.ext_modules = [Extension('foo', ['xxx'])] + cmd.distribution.packages = ['spam'] + cmd.distribution.script_name = 'setup.py' ++ ++ # Create rubbish, uncompilable file ++ f = os.path.join(project_dir, 'spam', 'rubbish.py') ++ self.write_file(f, 'rubbish()') + + # get_outputs should return 4 elements: spam/__init__.py and .pyc, +- # foo.import-tag-abiflags.so / foo.pyd ++ # foo.import-tag-abiflags.so / foo.pyd and rubbish.py (no .pyc) + outputs = cmd.get_outputs() +- self.assertEqual(len(outputs), 4, outputs) ++ self.assertEqual(len(outputs), 5, outputs) + + def test_get_inputs(self): + project_dir, dist = self.create_dist()