From owner-freebsd-python@FreeBSD.ORG  Mon May  9 22:31:58 2005
Return-Path: <owner-freebsd-python@FreeBSD.ORG>
Delivered-To: freebsd-python@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 78BA616A4EA
	for <freebsd-python@freebsd.org>;
	Mon,  9 May 2005 22:31:58 +0000 (GMT)
Received: from pimout4-ext.prodigy.net (pimout4-ext.prodigy.net
	[207.115.63.98])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 000D443D81
	for <freebsd-python@freebsd.org>;
	Mon,  9 May 2005 22:31:57 +0000 (GMT)
	(envelope-from ewan@mathcode.net)
Received: from mail.mathcode.net (adsl-68-23-218-132.dsl.wotnoh.ameritech.net
	[68.23.218.132])j49MVvqi013644
	for <freebsd-python@freebsd.org>; Mon, 9 May 2005 18:31:57 -0400
Received: from bsd.mathcode.net (localhost [127.0.0.1])
	by mail.mathcode.net (8.12.11/8.12.11) with ESMTP id j49MWGwW011178
	for <freebsd-python@freebsd.org>; Mon, 9 May 2005 18:32:16 -0400 (EDT)
	(envelope-from ewan@bsd.mathcode.net)
Received: (from ewan@localhost)
	by bsd.mathcode.net (8.12.11/8.12.11/Submit) id j49MWGfS011177
	for freebsd-python@freebsd.org; Mon, 9 May 2005 18:32:16 -0400 (EDT)
	(envelope-from ewan)
Date: Mon, 9 May 2005 18:32:16 -0400
From: Ewan Todd <ewan@mathcode.net>
To: freebsd-python@freebsd.org
Message-ID: <20050509223216.GM281@mathcode.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.1i
X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.64
X-Spam-Checker-Version: SpamAssassin 2.64 (2004-01-11) on bsd.mathcode.net
Subject: fbsd 5.4 python sigprocmask() issue
X-BeenThere: freebsd-python@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>,
	<mailto:freebsd-python-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python>
List-Post: <mailto:freebsd-python@freebsd.org>
List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>,
	<mailto:freebsd-python-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 May 2005 22:31:58 -0000


Hi,

I ran a python diagnostic on my new 5.3 install this morning comparing
it to a 4.10 install.  Basically, the program was taking 130 seconds
to perform a task that should have taken 70 seconds.  The extra time
is being consumed in system overheads.  A whole bunch of folks
confirmed the problem on a variety of machines with several kernels.
Ultimately, after eliminating processor problems, systemwide kernel
problems etc., python was found to be the culprit.  The discussion
starts at

  http://lists.freebsd.org/pipermail/freebsd-stable/2005-May/014575.html

Suleiman Souhlal ran ktrace(1) on it and discovered that python was
continually calling sigprocmask().

  http://lists.freebsd.org/pipermail/freebsd-stable/2005-May/014591.html

I then ran the same diagnostic in PERL, ruby, and C, and confirmed
that it was a python problem.

  http://lists.freebsd.org/pipermail/freebsd-stable/2005-May/014596.html

There were also a few attempts to speculate what the causes might be,
and suggestions for workarounds.  

I haven't yet had the chance to investigate it further, but I did
say that I would get word to the python porters, so that's what this
message is about.

-e

From owner-freebsd-python@FreeBSD.ORG  Mon May  9 23:01:37 2005
Return-Path: <owner-freebsd-python@FreeBSD.ORG>
Delivered-To: freebsd-python@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id DA53816A4EB
	for <freebsd-python@freebsd.org>;
	Mon,  9 May 2005 23:01:37 +0000 (GMT)
Received: from ylpvm15.prodigy.net (ylpvm15-ext.prodigy.net [207.115.57.46])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 7D7C743D6E
	for <freebsd-python@freebsd.org>;
	Mon,  9 May 2005 23:01:37 +0000 (GMT)
	(envelope-from ewan@mathcode.net)
Received: from pimout2-ext.prodigy.net (pimout2-ext.prodigy.net
	[207.115.63.101])j49MvmDE003498
	for <freebsd-python@freebsd.org>; Mon, 9 May 2005 18:57:48 -0400
X-ORBL: [68.23.218.132]
Received: from mail.mathcode.net (adsl-68-23-218-132.dsl.wotnoh.ameritech.net
	[68.23.218.132])j49N1Vmd309752
	for <freebsd-python@freebsd.org>; Mon, 9 May 2005 19:01:35 -0400
Received: from bsd.mathcode.net (localhost [127.0.0.1])
	by mail.mathcode.net (8.12.11/8.12.11) with ESMTP id j49N1oZM011348
	for <freebsd-python@freebsd.org>; Mon, 9 May 2005 19:01:50 -0400 (EDT)
	(envelope-from ewan@bsd.mathcode.net)
Received: (from ewan@localhost)
	by bsd.mathcode.net (8.12.11/8.12.11/Submit) id j49N1o6e011347
	for freebsd-python@freebsd.org; Mon, 9 May 2005 19:01:50 -0400 (EDT)
	(envelope-from ewan)
Date: Mon, 9 May 2005 19:01:50 -0400
From: Ewan Todd <ewan@mathcode.net>
To: freebsd-python@freebsd.org
Message-ID: <20050509230150.GN281@mathcode.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.1i
X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.64
X-Spam-Checker-Version: SpamAssassin 2.64 (2004-01-11) on bsd.mathcode.net
Subject: fbsd 5.4 python sigprocmask() issue
X-BeenThere: freebsd-python@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>,
	<mailto:freebsd-python-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python>
List-Post: <mailto:freebsd-python@freebsd.org>
List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>,
	<mailto:freebsd-python-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 May 2005 23:01:38 -0000

----- Forwarded message from Suleiman Souhlal <ssouhlal@freebsd.org> -----

Hello,

On May 9, 2005, at 3:54 PM, Daniel Eischen wrote:

>On Tue, 10 May 2005, Peter Jeremy wrote:
>
>
>>On Mon, 2005-May-09 11:00:18 -0400, Ewan Todd wrote:
>>
>>>I have what I think is a serious performance issue with fbsd 5.3
>>>release.  I've read about threading issues, and it seems to me that
>>>that is what I'm looking at, but I'm not confident enough to rule  
>>>out
>>>that it might be a hardware issue, a kernel configuration issue, or
>>>something to do with the python port.
>>>
>>
>>There does appear to be a problem in FreeBSD.  Python is built with
>>threading enabled by default, the threading libraries play with the
>>signal mask and there have been extensive changes there.  My
>>
>
>The threading libraries don't play with the signal mask.  In fact,
>libpthread has userland versions of sigprocmask() et. al. and won't
>even make the syscall() unless the threads are system scope.  There
>is a special thread in libpthread that handles signals which does
>use the system sigprocmask(), but unless the application is
>making heavy use of signals in general, it shouldn't matter.

I think I've found the problem: Python uses setjmp/longjmp to protect
against SIGFPU every time it does floating point operations. The
python script does not actually use threads, and libpthread assumes
non-threaded processes are system scope. So, it would end up using
the sigprocmask syscall, even though it doesn't really need to.  The
diff at
http://people.freebsd.org/~ssouhlal/testing/thr_sigmask-20050509.diff
fixes this, by making sure the process is  threaded, before using the
syscall.

--
Suleiman Souhlal     | ssouhlal@vt.edu
The FreeBSD Project  | ssouhlal@FreeBSD.org

_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"

----- End forwarded message -----