From owner-freebsd-python@FreeBSD.ORG Mon May 9 22:31:58 2005 Return-Path: <owner-freebsd-python@FreeBSD.ORG> Delivered-To: freebsd-python@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 78BA616A4EA for <freebsd-python@freebsd.org>; Mon, 9 May 2005 22:31:58 +0000 (GMT) Received: from pimout4-ext.prodigy.net (pimout4-ext.prodigy.net [207.115.63.98]) by mx1.FreeBSD.org (Postfix) with ESMTP id 000D443D81 for <freebsd-python@freebsd.org>; Mon, 9 May 2005 22:31:57 +0000 (GMT) (envelope-from ewan@mathcode.net) Received: from mail.mathcode.net (adsl-68-23-218-132.dsl.wotnoh.ameritech.net [68.23.218.132])j49MVvqi013644 for <freebsd-python@freebsd.org>; Mon, 9 May 2005 18:31:57 -0400 Received: from bsd.mathcode.net (localhost [127.0.0.1]) by mail.mathcode.net (8.12.11/8.12.11) with ESMTP id j49MWGwW011178 for <freebsd-python@freebsd.org>; Mon, 9 May 2005 18:32:16 -0400 (EDT) (envelope-from ewan@bsd.mathcode.net) Received: (from ewan@localhost) by bsd.mathcode.net (8.12.11/8.12.11/Submit) id j49MWGfS011177 for freebsd-python@freebsd.org; Mon, 9 May 2005 18:32:16 -0400 (EDT) (envelope-from ewan) Date: Mon, 9 May 2005 18:32:16 -0400 From: Ewan Todd <ewan@mathcode.net> To: freebsd-python@freebsd.org Message-ID: <20050509223216.GM281@mathcode.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.64 X-Spam-Checker-Version: SpamAssassin 2.64 (2004-01-11) on bsd.mathcode.net Subject: fbsd 5.4 python sigprocmask() issue X-BeenThere: freebsd-python@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>, <mailto:freebsd-python-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python> List-Post: <mailto:freebsd-python@freebsd.org> List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>, <mailto:freebsd-python-request@freebsd.org?subject=subscribe> X-List-Received-Date: Mon, 09 May 2005 22:31:58 -0000 Hi, I ran a python diagnostic on my new 5.3 install this morning comparing it to a 4.10 install. Basically, the program was taking 130 seconds to perform a task that should have taken 70 seconds. The extra time is being consumed in system overheads. A whole bunch of folks confirmed the problem on a variety of machines with several kernels. Ultimately, after eliminating processor problems, systemwide kernel problems etc., python was found to be the culprit. The discussion starts at http://lists.freebsd.org/pipermail/freebsd-stable/2005-May/014575.html Suleiman Souhlal ran ktrace(1) on it and discovered that python was continually calling sigprocmask(). http://lists.freebsd.org/pipermail/freebsd-stable/2005-May/014591.html I then ran the same diagnostic in PERL, ruby, and C, and confirmed that it was a python problem. http://lists.freebsd.org/pipermail/freebsd-stable/2005-May/014596.html There were also a few attempts to speculate what the causes might be, and suggestions for workarounds. I haven't yet had the chance to investigate it further, but I did say that I would get word to the python porters, so that's what this message is about. -e From owner-freebsd-python@FreeBSD.ORG Mon May 9 23:01:37 2005 Return-Path: <owner-freebsd-python@FreeBSD.ORG> Delivered-To: freebsd-python@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DA53816A4EB for <freebsd-python@freebsd.org>; Mon, 9 May 2005 23:01:37 +0000 (GMT) Received: from ylpvm15.prodigy.net (ylpvm15-ext.prodigy.net [207.115.57.46]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7D7C743D6E for <freebsd-python@freebsd.org>; Mon, 9 May 2005 23:01:37 +0000 (GMT) (envelope-from ewan@mathcode.net) Received: from pimout2-ext.prodigy.net (pimout2-ext.prodigy.net [207.115.63.101])j49MvmDE003498 for <freebsd-python@freebsd.org>; Mon, 9 May 2005 18:57:48 -0400 X-ORBL: [68.23.218.132] Received: from mail.mathcode.net (adsl-68-23-218-132.dsl.wotnoh.ameritech.net [68.23.218.132])j49N1Vmd309752 for <freebsd-python@freebsd.org>; Mon, 9 May 2005 19:01:35 -0400 Received: from bsd.mathcode.net (localhost [127.0.0.1]) by mail.mathcode.net (8.12.11/8.12.11) with ESMTP id j49N1oZM011348 for <freebsd-python@freebsd.org>; Mon, 9 May 2005 19:01:50 -0400 (EDT) (envelope-from ewan@bsd.mathcode.net) Received: (from ewan@localhost) by bsd.mathcode.net (8.12.11/8.12.11/Submit) id j49N1o6e011347 for freebsd-python@freebsd.org; Mon, 9 May 2005 19:01:50 -0400 (EDT) (envelope-from ewan) Date: Mon, 9 May 2005 19:01:50 -0400 From: Ewan Todd <ewan@mathcode.net> To: freebsd-python@freebsd.org Message-ID: <20050509230150.GN281@mathcode.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.64 X-Spam-Checker-Version: SpamAssassin 2.64 (2004-01-11) on bsd.mathcode.net Subject: fbsd 5.4 python sigprocmask() issue X-BeenThere: freebsd-python@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>, <mailto:freebsd-python-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python> List-Post: <mailto:freebsd-python@freebsd.org> List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-python>, <mailto:freebsd-python-request@freebsd.org?subject=subscribe> X-List-Received-Date: Mon, 09 May 2005 23:01:38 -0000 ----- Forwarded message from Suleiman Souhlal <ssouhlal@freebsd.org> ----- Hello, On May 9, 2005, at 3:54 PM, Daniel Eischen wrote: >On Tue, 10 May 2005, Peter Jeremy wrote: > > >>On Mon, 2005-May-09 11:00:18 -0400, Ewan Todd wrote: >> >>>I have what I think is a serious performance issue with fbsd 5.3 >>>release. I've read about threading issues, and it seems to me that >>>that is what I'm looking at, but I'm not confident enough to rule >>>out >>>that it might be a hardware issue, a kernel configuration issue, or >>>something to do with the python port. >>> >> >>There does appear to be a problem in FreeBSD. Python is built with >>threading enabled by default, the threading libraries play with the >>signal mask and there have been extensive changes there. My >> > >The threading libraries don't play with the signal mask. In fact, >libpthread has userland versions of sigprocmask() et. al. and won't >even make the syscall() unless the threads are system scope. There >is a special thread in libpthread that handles signals which does >use the system sigprocmask(), but unless the application is >making heavy use of signals in general, it shouldn't matter. I think I've found the problem: Python uses setjmp/longjmp to protect against SIGFPU every time it does floating point operations. The python script does not actually use threads, and libpthread assumes non-threaded processes are system scope. So, it would end up using the sigprocmask syscall, even though it doesn't really need to. The diff at http://people.freebsd.org/~ssouhlal/testing/thr_sigmask-20050509.diff fixes this, by making sure the process is threaded, before using the syscall. -- Suleiman Souhlal | ssouhlal@vt.edu The FreeBSD Project | ssouhlal@FreeBSD.org _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" ----- End forwarded message -----