Date: Sat, 9 Jan 2016 07:04:06 +0000 (UTC) From: Kubilay Kocak <koobs@FreeBSD.org> To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-branches@freebsd.org Subject: svn commit: r405609 - in branches/2016Q1/lang: python27 python27/files python32 python32/files python33 python33/files python34 python34/files python35 python35/files Message-ID: <201601090704.u09746aN066739@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: koobs Date: Sat Jan 9 07:04:05 2016 New Revision: 405609 URL: https://svnweb.freebsd.org/changeset/ports/405609 Log: MFH: r405569: lang/python{27,3*}: Backport patch in upstream issue20397 In certain situations, file references (.py[co]) for Python files that fail to compile with compileall() are still added to distutils --record output. This output is used for pkg-plist generation and must only contain references to files that will be installed. One example of a failure condition is when a Python 2/3 compatible package containing a file containing Python 3.x only code is built with Python 2.x, such as Gunicorn's _gaiohttp.py [1] This change backports patches submitted against upstream issue 20397 [2] that has not yet been committed. - For Python 2.7 and 3.5, backport both install_lib and test - For Python 3.2, 3.3 and 3.4, only backport install_lib [1] https://svnweb.freebsd.org/changeset/ports/404558 [2] https://bugs.python.org/issue20397 Thank you to Brendan Molloy for producing and submitting the patches against upstream sources. Reviewed by: sbz (python) Differential Revision: D4832 Approved by: ports-secteam (miwi) Added: branches/2016Q1/lang/python27/files/patch-Lib_distutils_command_install__lib.py - copied unchanged from r405569, head/lang/python27/files/patch-Lib_distutils_command_install__lib.py branches/2016Q1/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py - copied unchanged from r405569, head/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py branches/2016Q1/lang/python32/files/patch-Lib_distutils_command_install__lib.py - copied unchanged from r405569, head/lang/python32/files/patch-Lib_distutils_command_install__lib.py branches/2016Q1/lang/python33/files/patch-Lib_distutils_command_install__lib.py - copied unchanged from r405569, head/lang/python33/files/patch-Lib_distutils_command_install__lib.py branches/2016Q1/lang/python34/files/patch-Lib_distutils_command_install__lib.py - copied unchanged from r405569, head/lang/python34/files/patch-Lib_distutils_command_install__lib.py branches/2016Q1/lang/python35/files/patch-Lib_distutils_command_install__lib.py - copied unchanged from r405569, head/lang/python35/files/patch-Lib_distutils_command_install__lib.py branches/2016Q1/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py - copied unchanged from r405569, head/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py Modified: branches/2016Q1/lang/python27/Makefile branches/2016Q1/lang/python32/Makefile branches/2016Q1/lang/python33/Makefile branches/2016Q1/lang/python34/Makefile branches/2016Q1/lang/python35/Makefile Directory Properties: branches/2016Q1/ (props changed) Modified: branches/2016Q1/lang/python27/Makefile ============================================================================== --- branches/2016Q1/lang/python27/Makefile Sat Jan 9 05:05:44 2016 (r405608) +++ branches/2016Q1/lang/python27/Makefile Sat Jan 9 07:04:05 2016 (r405609) @@ -2,6 +2,7 @@ PORTNAME= python27 PORTVERSION= ${PYTHON_PORTVERSION} +PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} DISTNAME= Python-${PORTVERSION} Copied: branches/2016Q1/lang/python27/files/patch-Lib_distutils_command_install__lib.py (from r405569, head/lang/python27/files/patch-Lib_distutils_command_install__lib.py) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ branches/2016Q1/lang/python27/files/patch-Lib_distutils_command_install__lib.py Sat Jan 9 07:04:05 2016 (r405609, copy of r405569, head/lang/python27/files/patch-Lib_distutils_command_install__lib.py) @@ -0,0 +1,34 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy <brendan@bbqsrc.net> +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. +--- + Lib/distutils/command/install_lib.py | 17 +++++++++++++---- + Lib/distutils/tests/test_install_lib.py | 8 ++++++-- + 2 files changed, 19 insertions(+), 6 deletions(-) + +--- Lib/distutils/command/install_lib.py.orig 2015-12-05 19:46:56 UTC ++++ Lib/distutils/command/install_lib.py +@@ -168,10 +168,14 @@ class install_lib(Command): + ext = os.path.splitext(os.path.normcase(py_file))[1] + if ext != PYTHON_SOURCE_EXTENSION: + continue +- if self.compile: +- bytecode_files.append(py_file + "c") +- if self.optimize > 0: +- bytecode_files.append(py_file + "o") ++ ++ pyc_file = py_file + "c" ++ if self.compile and os.path.isfile(pyc_file): ++ bytecode_files.append(pyc_file) ++ ++ pyo_file = py_file + "o" ++ if self.optimize > 0 and os.path.isfile(pyo_file): ++ bytecode_files.append(pyo_file) + + return bytecode_files + Copied: branches/2016Q1/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py (from r405569, head/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ branches/2016Q1/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py Sat Jan 9 07:04:05 2016 (r405609, copy of r405569, head/lang/python27/files/patch-Lib_distutils_tests_test__install__lib.py) @@ -0,0 +1,30 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy <brendan@bbqsrc.net> +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. +--- + Lib/distutils/command/install_lib.py | 17 +++++++++++++---- + Lib/distutils/tests/test_install_lib.py | 8 ++++++-- + 2 files changed, 19 insertions(+), 6 deletions(-) + +--- Lib/distutils/tests/test_install_lib.py.orig 2015-12-05 19:46:57 UTC ++++ Lib/distutils/tests/test_install_lib.py +@@ -64,8 +64,12 @@ class InstallLibTestCase(support.Tempdir + cmd.distribution.packages = [pkg_dir] + cmd.distribution.script_name = 'setup.py' + +- # get_output should return 4 elements +- self.assertGreaterEqual(len(cmd.get_outputs()), 2) ++ # Create rubbish, uncompilable file ++ f = os.path.join(pkg_dir, 'rubbish.py') ++ self.write_file(f, 'rubbish()') ++ ++ # get_output should return 3 elements ++ self.assertEqual(len(cmd.get_outputs()), 3) + + def test_get_inputs(self): + pkg_dir, dist = self.create_dist() Modified: branches/2016Q1/lang/python32/Makefile ============================================================================== --- branches/2016Q1/lang/python32/Makefile Sat Jan 9 05:05:44 2016 (r405608) +++ branches/2016Q1/lang/python32/Makefile Sat Jan 9 07:04:05 2016 (r405609) @@ -2,6 +2,7 @@ PORTNAME= python32 PORTVERSION= ${PYTHON_PORTVERSION} +PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} DISTNAME= Python-${PORTVERSION} Copied: branches/2016Q1/lang/python32/files/patch-Lib_distutils_command_install__lib.py (from r405569, head/lang/python32/files/patch-Lib_distutils_command_install__lib.py) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ branches/2016Q1/lang/python32/files/patch-Lib_distutils_command_install__lib.py Sat Jan 9 07:04:05 2016 (r405609, copy of r405569, head/lang/python32/files/patch-Lib_distutils_command_install__lib.py) @@ -0,0 +1,34 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy <brendan@bbqsrc.net> +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. + +--- Lib/distutils/command/install_lib.py.orig 2014-10-12 06:52:02 UTC ++++ Lib/distutils/command/install_lib.py +@@ -165,11 +165,18 @@ class install_lib(Command): + if ext != PYTHON_SOURCE_EXTENSION: + continue + if self.compile: +- bytecode_files.append(imp.cache_from_source( +- py_file, debug_override=True)) ++ candidate = imp.cache_from_source( ++ py_file, debug_override=True) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + if self.optimize > 0: +- bytecode_files.append(imp.cache_from_source( +- py_file, debug_override=False)) ++ candidate = imp.cache_from_source( ++ py_file, debug_override=False) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) + + return bytecode_files + Modified: branches/2016Q1/lang/python33/Makefile ============================================================================== --- branches/2016Q1/lang/python33/Makefile Sat Jan 9 05:05:44 2016 (r405608) +++ branches/2016Q1/lang/python33/Makefile Sat Jan 9 07:04:05 2016 (r405609) @@ -2,7 +2,7 @@ PORTNAME= python33 PORTVERSION= ${PYTHON_PORTVERSION} -PORTREVISION= 2 +PORTREVISION= 3 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} DISTNAME= Python-${PORTVERSION} Copied: branches/2016Q1/lang/python33/files/patch-Lib_distutils_command_install__lib.py (from r405569, head/lang/python33/files/patch-Lib_distutils_command_install__lib.py) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ branches/2016Q1/lang/python33/files/patch-Lib_distutils_command_install__lib.py Sat Jan 9 07:04:05 2016 (r405609, copy of r405569, head/lang/python33/files/patch-Lib_distutils_command_install__lib.py) @@ -0,0 +1,34 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy <brendan@bbqsrc.net> +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. + +--- Lib/distutils/command/install_lib.py.orig 2014-10-12 06:52:02 UTC ++++ Lib/distutils/command/install_lib.py +@@ -165,11 +165,18 @@ class install_lib(Command): + if ext != PYTHON_SOURCE_EXTENSION: + continue + if self.compile: +- bytecode_files.append(imp.cache_from_source( +- py_file, debug_override=True)) ++ candidate = imp.cache_from_source( ++ py_file, debug_override=True) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + if self.optimize > 0: +- bytecode_files.append(imp.cache_from_source( +- py_file, debug_override=False)) ++ candidate = imp.cache_from_source( ++ py_file, debug_override=False) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) + + return bytecode_files + Modified: branches/2016Q1/lang/python34/Makefile ============================================================================== --- branches/2016Q1/lang/python34/Makefile Sat Jan 9 05:05:44 2016 (r405608) +++ branches/2016Q1/lang/python34/Makefile Sat Jan 9 07:04:05 2016 (r405609) @@ -3,7 +3,7 @@ PORTNAME= python34 PORTVERSION= ${PYTHON_PORTVERSION} -PORTREVISION= 0 +PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} DISTNAME= Python-${PORTVERSION} Copied: branches/2016Q1/lang/python34/files/patch-Lib_distutils_command_install__lib.py (from r405569, head/lang/python34/files/patch-Lib_distutils_command_install__lib.py) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ branches/2016Q1/lang/python34/files/patch-Lib_distutils_command_install__lib.py Sat Jan 9 07:04:05 2016 (r405609, copy of r405569, head/lang/python34/files/patch-Lib_distutils_command_install__lib.py) @@ -0,0 +1,35 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy <brendan@bbqsrc.net> +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. + +--- Lib/distutils/command/install_lib.py.orig 2016-01-08 12:38:49 UTC ++++ Lib/distutils/command/install_lib.py +@@ -165,11 +165,19 @@ class install_lib(Command): + if ext != PYTHON_SOURCE_EXTENSION: + continue + if self.compile: +- bytecode_files.append(importlib.util.cache_from_source( +- py_file, debug_override=True)) ++ candidate = importlib.util.cache_from_source( ++ py_file, debug_override=True) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + if self.optimize > 0: +- bytecode_files.append(importlib.util.cache_from_source( +- py_file, debug_override=False)) ++ candidate = importlib.util.cache_from_source( ++ py_file, debug_override=False) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + + return bytecode_files + Modified: branches/2016Q1/lang/python35/Makefile ============================================================================== --- branches/2016Q1/lang/python35/Makefile Sat Jan 9 05:05:44 2016 (r405608) +++ branches/2016Q1/lang/python35/Makefile Sat Jan 9 07:04:05 2016 (r405609) @@ -3,6 +3,7 @@ PORTNAME= python DISTVERSION= ${PYTHON_PORTVERSION} +PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PYTHON_PORTVERSION} PKGNAMESUFFIX= ${PYTHON_SUFFIX} Copied: branches/2016Q1/lang/python35/files/patch-Lib_distutils_command_install__lib.py (from r405569, head/lang/python35/files/patch-Lib_distutils_command_install__lib.py) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ branches/2016Q1/lang/python35/files/patch-Lib_distutils_command_install__lib.py Sat Jan 9 07:04:05 2016 (r405609, copy of r405569, head/lang/python35/files/patch-Lib_distutils_command_install__lib.py) @@ -0,0 +1,41 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy <brendan@bbqsrc.net> +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. +--- + Lib/distutils/command/install_lib.py | 17 +++++++++++++---- + Lib/distutils/tests/test_install_lib.py | 8 ++++++-- + 2 files changed, 19 insertions(+), 6 deletions(-) + +--- Lib/distutils/command/install_lib.py.orig 2015-12-07 01:39:07 UTC ++++ Lib/distutils/command/install_lib.py +@@ -164,12 +164,21 @@ class install_lib(Command): + ext = os.path.splitext(os.path.normcase(py_file))[1] + if ext != PYTHON_SOURCE_EXTENSION: + continue ++ + if self.compile: +- bytecode_files.append(importlib.util.cache_from_source( +- py_file, optimization='')) ++ candidate = importlib.util.cache_from_source( ++ py_file, optimization='') ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + if self.optimize > 0: +- bytecode_files.append(importlib.util.cache_from_source( +- py_file, optimization=self.optimize)) ++ candidate = importlib.util.cache_from_source( ++ py_file, optimization=self.optimize) ++ ++ if os.path.isfile(candidate): ++ bytecode_files.append(candidate) ++ + + return bytecode_files + Copied: branches/2016Q1/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py (from r405569, head/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ branches/2016Q1/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py Sat Jan 9 07:04:05 2016 (r405609, copy of r405569, head/lang/python35/files/patch-Lib_distutils_tests_test__install__lib.py) @@ -0,0 +1,33 @@ +From 9934ce31b8447667f71c211e559a8de71e8263db Mon Sep 17 00:00:00 2001 +From: Brendan Molloy <brendan@bbqsrc.net> +Date: Mon, 4 Jan 2016 23:14:06 +1100 +Subject: [PATCH] Check bytecode file actually exists and tests + +Should solve issue 20397, where using the --record argument results +in files that failed to generate bytecode files are added to the +record file nonetheless. +--- + Lib/distutils/command/install_lib.py | 17 +++++++++++++---- + Lib/distutils/tests/test_install_lib.py | 8 ++++++-- + 2 files changed, 19 insertions(+), 6 deletions(-) + +--- Lib/distutils/tests/test_install_lib.py.orig 2015-12-07 01:39:07 UTC ++++ Lib/distutils/tests/test_install_lib.py +@@ -64,11 +64,15 @@ class InstallLibTestCase(support.Tempdir + cmd.distribution.ext_modules = [Extension('foo', ['xxx'])] + cmd.distribution.packages = ['spam'] + cmd.distribution.script_name = 'setup.py' ++ ++ # Create rubbish, uncompilable file ++ f = os.path.join(project_dir, 'spam', 'rubbish.py') ++ self.write_file(f, 'rubbish()') + + # get_outputs should return 4 elements: spam/__init__.py and .pyc, +- # foo.import-tag-abiflags.so / foo.pyd ++ # foo.import-tag-abiflags.so / foo.pyd and rubbish.py (no .pyc) + outputs = cmd.get_outputs() +- self.assertEqual(len(outputs), 4, outputs) ++ self.assertEqual(len(outputs), 5, outputs) + + def test_get_inputs(self): + project_dir, dist = self.create_dist()
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201601090704.u09746aN066739>