From owner-freebsd-python@FreeBSD.ORG Thu May 2 19:40:02 2013 Return-Path: Delivered-To: freebsd-python@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1D69AF92 for ; Thu, 2 May 2013 19:40:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 02C1C1268 for ; Thu, 2 May 2013 19:40:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r42Je1pq090052 for ; Thu, 2 May 2013 19:40:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r42Je1fx090051; Thu, 2 May 2013 19:40:01 GMT (envelope-from gnats) Resent-Date: Thu, 2 May 2013 19:40:01 GMT Resent-Message-Id: <201305021940.r42Je1fx090051@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-python@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Jan Beich Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 771BFD8B for ; Thu, 2 May 2013 19:32:37 +0000 (UTC) (envelope-from jbeich@tormail.org) Received: from outgoing.tormail.org (outgoing.tormail.org [82.221.96.22]) by mx1.freebsd.org (Postfix) with ESMTP id 19BD71238 for ; Thu, 2 May 2013 19:32:36 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=internal.tormail.org) by outgoing.tormail.org with esmtp (Exim 4.72) (envelope-from ) id 1UXzEx-0003nr-Lv for FreeBSD-gnats-submit@freebsd.org; Thu, 02 May 2013 23:32:28 +0400 Received: from jbeich by internal.tormail.org with local (Exim 4.63) (envelope-from ) id 1UXzBk-0002ne-0w for FreeBSD-gnats-submit@freebsd.org; Thu, 02 May 2013 19:29:10 +0000 Message-Id: <1UXzBk-0002ne-0w@internal.tormail.org> Date: Wed, 01 May 2013 21:09:06 -0900 From: Jan Beich To: FreeBSD-gnats-submit@freebsd.org Subject: ports/178301: [patch] lang/python2[67]: SEM option is inconsistent X-BeenThere: freebsd-python@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: FreeBSD-specific Python issues List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2013 19:40:02 -0000 >Number: 178301 >Category: ports >Synopsis: [patch] lang/python2[67]: SEM option is inconsistent >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-python >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Thu May 02 19:40:01 UTC 2013 >Closed-Date: >Last-Modified: >Originator: Jan Beich >Release: FreeBSD 10.0-CURRENT amd64 >Organization: >Environment: SEM unset (default, testing) sem.ko unloaded (default, testing) http://svnweb.freebsd.org/changeset/base/201546 http://svnweb.freebsd.org/changeset/base/212852 >Description: python27 is always built with multiprocessing support on >= 9.0 or on 7.1..8.4 with sem.ko preloaded because autoconf finds working sem_open(). And python2[67] always USE_SEMAPHORES for locking which seems to work fine with libthr's sem_init(pshared=0). >How-To-Repeat: FreeBSD 9.0 or later: $ python3.3 -c 'import multiprocessing.synchronize' $ python2.7 -c 'import multiprocessing.synchronize' $ python2.6 -c 'import multiprocessing.synchronize' Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 33, in " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. FreeBSD 7.2..8.4: $ python3.3 -c 'import multiprocessing.synchronize' Traceback (most recent call last): File "/usr/local/lib/python3.3/multiprocessing/synchronize.py", line 27, in from _multiprocessing import SemLock ImportError: cannot import name SemLock During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.3/multiprocessing/synchronize.py", line 32, in " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. $ python2.7 -c 'import multiprocessing.synchronize' Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/multiprocessing/synchronize.py", line 59, in " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. $ python2.6 -c 'import multiprocessing.synchronize' Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 33, in " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. >Fix: Just an incomplete attempt to make SEM more predictable: - remove checks for EOL releases - limit sem.ko warning and extra patches to 8.x users - warn 9.x users semaphores support cannot be disabled like in python3x - rely on autoconf to find sem_open et al. like netbsd or python3x - enable sem_open for python26 on >= 9.0 by default like python27 - don't use semaphores for locking if SEM is unset on 8.x --- python_sem.diff begins here --- Index: lang/python26/Makefile =================================================================== --- lang/python26/Makefile (revision 317131) +++ lang/python26/Makefile (working copy) @@ -57,16 +57,13 @@ OPTIONS_SINGLE= UCS OPTIONS_SINGLE_UCS= UCS2 UCS4 NLS_DESC= Enable Gettext support for the locale module +SEM_DESC= POSIX semaphores support (cannot be disabled on >= 9.0) .include -.if ${PORT_OPTIONS:MSEM} -.if ${OSVERSION} >= 701106 +.if ${PORT_OPTIONS:MSEM} && ${OSVERSION} < 900007 SEM_MSG= "" .else -IGNORE= POSIX semaphore support only works in FreeBSD 7-STABLE and later -.endif # ${OSVERSION} >= 701106 -.else SEM_MSG= "@comment " .endif @@ -190,11 +187,10 @@ pre-patch: .endif post-patch: -.if ${PORT_OPTIONS:MSEM} -.if ${OSVERSION} >= 701106 +.if ${PORT_OPTIONS:MSEM} && ${OSVERSION} < 900007 @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < ${PATCHDIR}/extra-patch-setup.py + @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < ${PATCHDIR}/extra-patch-Python_thread__pthread.h .endif -.endif .if defined(WITHOUT_NIS) ${REINPLACE_CMD} -e \ 's/disabled_module_list =[^]]*/&, "nis"/' \ Index: lang/python26/files/extra-patch-Python_thread__pthread.h =================================================================== --- lang/python26/files/extra-patch-Python_thread__pthread.h (revision 0) +++ lang/python26/files/extra-patch-Python_thread__pthread.h (working copy) @@ -0,0 +1,11 @@ +--- Python/thread_pthread.h.orig 2010-05-09 22:46:46.000000000 +0800 ++++ Python/thread_pthread.h 2010-08-15 14:27:51.886823397 +0800 +@@ -44,7 +44,7 @@ + #ifdef _POSIX_SEMAPHORES + /* On FreeBSD 4.x, _POSIX_SEMAPHORES is defined empty, so + we need to add 0 to make it work there as well. */ +-#if (_POSIX_SEMAPHORES+0) == -1 ++#if !defined(__FreeBSD__) && (_POSIX_SEMAPHORES+0) == -1 + #define HAVE_BROKEN_POSIX_SEMAPHORES + #else + #include Index: lang/python26/files/extra-patch-setup.py =================================================================== --- lang/python26/files/extra-patch-setup.py (revision 317131) +++ lang/python26/files/extra-patch-setup.py (working copy) @@ -1,26 +1,25 @@ --- setup.py.1 2009-03-12 04:07:36.000000000 +0000 +++ setup.py 2009-03-12 04:08:36.000000000 +0000 -@@ -1265,13 +1265,21 @@ +@@ -1385,20 +1385,12 @@ ) libraries = [] -- elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): -+ elif platform in ('freebsd4', 'freebsd5', 'freebsd6'): +- elif platform in ('freebsd9', 'freebsd10'): +- macros = dict( # at least FreeBSD 9 +- HAVE_SEM_OPEN=1, +- HAVE_SEM_TIMEDWAIT=1, +- HAVE_FD_TRANSFER=1, +- ) +- libraries = [] +- + elif platform.startswith('freebsd'): # FreeBSD's P1003.1b semaphore support is very experimental # and has many known problems. (as of June 2008) -- macros = dict( # FreeBSD -+ macros = dict( # FreeBSD 4-6 - HAVE_SEM_OPEN=0, - HAVE_SEM_TIMEDWAIT=0, - HAVE_FD_TRANSFER=1, -+ ) -+ libraries = [] -+ -+ elif platform in ('freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): -+ macros = dict( # FreeBSD 7+ + macros = dict( # FreeBSD +- HAVE_SEM_OPEN=0, +- HAVE_SEM_TIMEDWAIT=0, + HAVE_SEM_OPEN=1, + HAVE_SEM_TIMEDWAIT=1, -+ HAVE_FD_TRANSFER=1, + HAVE_FD_TRANSFER=1, ) libraries = [] - Index: lang/python26/files/patch-Python_thread__pthread.h =================================================================== --- lang/python26/files/patch-Python_thread__pthread.h (revision 317131) +++ lang/python26/files/patch-Python_thread__pthread.h (working copy) @@ -1,33 +1,5 @@ --- Python/thread_pthread.h.orig 2010-05-09 22:46:46.000000000 +0800 +++ Python/thread_pthread.h 2010-08-15 14:27:51.886823397 +0800 -@@ -26,13 +26,18 @@ - #endif - #endif - -+#ifdef __FreeBSD__ -+#include -+#endif -+ - /* The POSIX spec says that implementations supporting the sem_* - family of functions must indicate this by defining - _POSIX_SEMAPHORES. */ - #ifdef _POSIX_SEMAPHORES - /* On FreeBSD 4.x, _POSIX_SEMAPHORES is defined empty, so - we need to add 0 to make it work there as well. */ --#if (_POSIX_SEMAPHORES+0) == -1 -+#if defined(__FreeBSD__) && __FreeBSD_version < 701104 && \ -+ (_POSIX_SEMAPHORES+0) == -1 - #define HAVE_BROKEN_POSIX_SEMAPHORES - #else - #include -@@ -44,7 +49,6 @@ - in default setting. So the process scope is preferred to get - enough number of threads to work. */ - #ifdef __FreeBSD__ --#include - #if __FreeBSD_version >= 500000 && __FreeBSD_version < 504101 - #undef PTHREAD_SYSTEM_SCHED_SUPPORTED - #endif @@ -149,6 +153,7 @@ { pthread_t th; Index: lang/python26/files/patch-setup.py =================================================================== --- lang/python26/files/patch-setup.py (revision 317131) +++ lang/python26/files/patch-setup.py (working copy) @@ -68,12 +68,20 @@ libraries = curses_libs) ) else: missing.append('_curses') -@@ -1381,7 +1385,7 @@ +@@ -1381,7 +1385,15 @@ ) libraries = [] - elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'): -+ elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): ++ elif platform in ('freebsd9', 'freebsd10'): ++ macros = dict( # at least FreeBSD 9 ++ HAVE_SEM_OPEN=1, ++ HAVE_SEM_TIMEDWAIT=1, ++ HAVE_FD_TRANSFER=1, ++ ) ++ libraries = [] ++ ++ elif platform.startswith('freebsd'): # FreeBSD's P1003.1b semaphore support is very experimental # and has many known problems. (as of June 2008) macros = dict( # FreeBSD Index: lang/python27/Makefile =================================================================== --- lang/python27/Makefile (revision 317131) +++ lang/python27/Makefile (working copy) @@ -57,16 +57,13 @@ OPTIONS_SINGLE= UCS OPTIONS_SINGLE_UCS= UCS2 UCS4 NLS_DESC= Enable Gettext support for the locale module +SEM_DESC= POSIX semaphores support (cannot be disabled on >= 9.0) .include -.if ${PORT_OPTIONS:MSEM} -.if ${OSVERSION} >= 701106 +.if ${PORT_OPTIONS:MSEM} && ${OSVERSION} < 900007 SEM_MSG= "" .else -IGNORE= POSIX semaphore support only works in FreeBSD 7-STABLE and later -.endif # ${OSVERSION} >= 701106 -.else SEM_MSG= "@comment " .endif @@ -205,11 +202,10 @@ pre-patch: .endif post-patch: -.if ${PORT_OPTIONS:MSEM} -.if ${OSVERSION} >= 701106 +.if ${PORT_OPTIONS:MSEM} && ${OSVERSION} < 900007 @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < ${PATCHDIR}/extra-patch-setup.py + @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < ${PATCHDIR}/extra-patch-Python_thread__pthread.h .endif -.endif .if defined(WITHOUT_NIS) ${REINPLACE_CMD} -e \ 's/disabled_module_list =[^]]*/&, "nis"/' \ Index: lang/python27/files/extra-patch-Python_thread__pthread.h =================================================================== --- lang/python27/files/extra-patch-Python_thread__pthread.h (revision 0) +++ lang/python27/files/extra-patch-Python_thread__pthread.h (working copy) @@ -0,0 +1,11 @@ +--- Python/thread_pthread.h.orig 2010-05-09 22:46:46.000000000 +0800 ++++ Python/thread_pthread.h 2010-08-15 14:27:51.886823397 +0800 +@@ -44,7 +44,7 @@ + #ifdef _POSIX_SEMAPHORES + /* On FreeBSD 4.x, _POSIX_SEMAPHORES is defined empty, so + we need to add 0 to make it work there as well. */ +-#if (_POSIX_SEMAPHORES+0) == -1 ++#if !defined(__FreeBSD__) && (_POSIX_SEMAPHORES+0) == -1 + #define HAVE_BROKEN_POSIX_SEMAPHORES + #else + #include Index: lang/python27/files/extra-patch-setup.py =================================================================== --- lang/python27/files/extra-patch-setup.py (revision 317131) +++ lang/python27/files/extra-patch-setup.py (working copy) @@ -1,23 +1,15 @@ --- setup.py.bak 2010-08-15 14:57:00.347134100 +0800 +++ setup.py 2010-08-15 15:00:06.019643300 +0800 -@@ -1402,10 +1402,22 @@ +@@ -1432,10 +1436,14 @@ class PyBuildExt(build_ext): macros = dict() libraries = [] -- elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): -+ elif platform in ('freebsd4', 'freebsd5', 'freebsd6'): +- elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'): ++ elif platform in ('freebsd7', 'freebsd8'): # FreeBSD's P1003.1b semaphore support is very experimental # and has many known problems. (as of June 2008) - macros = dict() -+ macros = dict( # FreeBSD 4-6 -+ HAVE_SEM_OPEN=0, -+ HAVE_SEM_TIMEDWAIT=0, -+ HAVE_FD_TRANSFER=1, -+ ) -+ libraries = [] -+ -+ elif platform in ('freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): -+ macros = dict( # FreeBSD 7+ ++ macros = dict( # at least FreeBSD 7 + HAVE_SEM_OPEN=1, + HAVE_SEM_TIMEDWAIT=1, + HAVE_FD_TRANSFER=1, Index: lang/python27/files/patch-Python_thread__pthread.h =================================================================== --- lang/python27/files/patch-Python_thread__pthread.h (revision 317131) +++ lang/python27/files/patch-Python_thread__pthread.h (working copy) @@ -1,33 +1,5 @@ --- Python/thread_pthread.h.orig 2010-05-09 22:46:46.000000000 +0800 +++ Python/thread_pthread.h 2010-08-15 14:27:51.886823397 +0800 -@@ -26,13 +26,18 @@ - #endif - #endif - -+#ifdef __FreeBSD__ -+#include -+#endif -+ - /* The POSIX spec says that implementations supporting the sem_* - family of functions must indicate this by defining - _POSIX_SEMAPHORES. */ - #ifdef _POSIX_SEMAPHORES - /* On FreeBSD 4.x, _POSIX_SEMAPHORES is defined empty, so - we need to add 0 to make it work there as well. */ --#if (_POSIX_SEMAPHORES+0) == -1 -+#if defined(__FreeBSD__) && __FreeBSD_version < 701104 && \ -+ (_POSIX_SEMAPHORES+0) == -1 - #define HAVE_BROKEN_POSIX_SEMAPHORES - #else - #include -@@ -44,7 +49,6 @@ - in default setting. So the process scope is preferred to get - enough number of threads to work. */ - #ifdef __FreeBSD__ --#include - #if __FreeBSD_version >= 500000 && __FreeBSD_version < 504101 - #undef PTHREAD_SYSTEM_SCHED_SUPPORTED - #endif @@ -149,6 +153,7 @@ { pthread_t th; Index: lang/python27/files/patch-setup.py =================================================================== --- lang/python27/files/patch-setup.py (revision 317131) +++ lang/python27/files/patch-setup.py (working copy) @@ -68,15 +68,17 @@ libraries = curses_libs) ) else: missing.append('_curses') -@@ -1432,7 +1436,7 @@ +@@ -1438,6 +1442,10 @@ macros = dict() libraries = [] -- elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'): -+ elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): - # FreeBSD's P1003.1b semaphore support is very experimental - # and has many known problems. (as of June 2008) ++ elif platform.startswith('freebsd'): ++ macros = dict() ++ libraries = [] ++ + elif platform.startswith('openbsd'): macros = dict() + libraries = [] @@ -1484,7 +1488,7 @@ missing.append('linuxaudiodev') --- python_sem.diff ends here --- >Release-Note: >Audit-Trail: >Unformatted: