From owner-svn-src-stable@FreeBSD.ORG Sun Feb 22 12:58:17 2009 Return-Path: Delivered-To: svn-src-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97356106566B; Sun, 22 Feb 2009 12:58:17 +0000 (UTC) (envelope-from bms@incunabulum.net) Received: from out1.smtp.messagingengine.com (out1.smtp.messagingengine.com [66.111.4.25]) by mx1.freebsd.org (Postfix) with ESMTP id 51BE58FC12; Sun, 22 Feb 2009 12:58:17 +0000 (UTC) (envelope-from bms@incunabulum.net) Received: from compute2.internal (compute2.internal [10.202.2.42]) by out1.messagingengine.com (Postfix) with ESMTP id 755422A1B35; Sun, 22 Feb 2009 07:58:16 -0500 (EST) Received: from heartbeat2.messagingengine.com ([10.202.2.161]) by compute2.internal (MEProxy); Sun, 22 Feb 2009 07:58:16 -0500 X-Sasl-enc: nD4D84ukm2Gth7AmYWsQe5tozmnpjJCEM254SKshovL0 1235307496 Received: from [192.168.123.18] (82-35-112-254.cable.ubr07.dals.blueyonder.co.uk [82.35.112.254]) by mail.messagingengine.com (Postfix) with ESMTPSA id 1E7AB230D1; Sun, 22 Feb 2009 07:58:14 -0500 (EST) Message-ID: <49A14BE5.7020400@incunabulum.net> Date: Sun, 22 Feb 2009 12:58:13 +0000 From: Bruce Simpson User-Agent: Thunderbird 2.0.0.19 (Windows/20081209) MIME-Version: 1.0 To: John Baldwin References: <200902171957.n1HJvqk8026145@svn.freebsd.org> <200902181005.28625.jhb@freebsd.org> <499C269D.7030406@incunabulum.net> <200902181054.35709.jhb@freebsd.org> In-Reply-To: <200902181054.35709.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: svn-src-stable@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, svn-src-stable-7@freebsd.org Subject: Re: svn commit: r188727 - in stable/7: sys sys/contrib/pf sys/dev/ath/ath_hal sys/dev/cxgb sys/kern sys/modules/sem sys/sys tools/regression/posixsem usr.bin/procstat X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Feb 2009 12:58:18 -0000 John Baldwin wrote: > Hmm, by symbols I meant having things built with debug symbols (i.e. "-g"). > OK... I will try to build a Python interpreter w/debug symbols next time around. I am just giving this a dry run for now -- I am collaborating on a project where we've ended up implementing stuff on Linux using Python multiprocessing as it seemed the best fit for the team's capabilities. So I'm posting this here for lists to pick up, hopefully someone looking to use multiprocessing seriously on FreeBSD will come across this. It would be nice to port to FreeBSD at a later date, unfortunately, we're a bit further down this road now... > Also, do you have 'sem.ko' loaded? The only reason I can think of why you > would get a core dump in ksem_open() itself would be if you got a SIGSYS > because the module wasn't loaded. > That seems to have been the issue! Thanks. If I cd src/modules/sem && make && make install, and kldload the module before running Python, then I do not see the core dump. I didn't realise a separate kernel module was needed for POSIX semaphores, the option seems to be P1003_1B_SEMAPHORES according to sem(4). It is quite possible the Python guys totally missed this... It seems that whilst ipcs(1) is the appropriate tool to examine SYSV IPC facilities, there is no such tool for POSIX semaphores on FreeBSD, or am I wrong? When the Python process is running and using sem(4), I see a segment of shared memory in ipcs(1) output. Perhaps the libc wrapper should be taught to return ENOSYS if the system call(s) it requires are not present, rather than SIGSYS? However, the mp_benchmarks.py script just hangs when testing multiprocessing.Queue. A ktrace reveals that a child process is simply waiting for data on a pipe which doesn't arrive. Killing Python at this point and restarting results in future attempts to open a semaphore by the module failing with ENFILE after the 4th ksem_open() call: %%% Traceback (most recent call last): File "Doc/includes/mp_benchmarks.py", line 238, in test() File "Doc/includes/mp_benchmarks.py", line 192, in test multiprocessing.Condition()) File "/usr/local/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-freebsd-7.1-STABLE-amd64.egg/multiprocessing/__init__.py", line 190, in Condition return Condition(lock) File "/usr/local/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-freebsd-7.1-STABLE-amd64.egg/multiprocessing/synchronize.py", line 168, in __init__ self._lock = lock or RLock() File "/usr/local/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-freebsd-7.1-STABLE-amd64.egg/multiprocessing/synchronize.py", line 142, in __init__ SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1) File "/usr/local/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-freebsd-7.1-STABLE-amd64.egg/multiprocessing/synchronize.py", line 49, in __init__ sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue) OSError: [Errno 23] Too many open files in system Exit 1 anglepoise:~/p/multiprocessing-2.6.1.1 % %%%