From owner-freebsd-questions@FreeBSD.ORG Tue Dec 16 15:32:05 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 45A621065677 for ; Tue, 16 Dec 2008 15:32:05 +0000 (UTC) (envelope-from OttK@zzz.ee) Received: from zzz.ee (kalah.zzz.ee [194.204.30.253]) by mx1.freebsd.org (Postfix) with ESMTP id D54B48FC17 for ; Tue, 16 Dec 2008 15:32:04 +0000 (UTC) (envelope-from OttK@zzz.ee) Received: from zzz.ee (localhost.zzz.ee [127.0.0.1]) by zzz.ee (Postfix) with ESMTP id EABB9FD28D for ; Tue, 16 Dec 2008 17:32:03 +0200 (EET) Received: by zzz.ee (Postfix, from userid 3019) id E938FFD28C; Tue, 16 Dec 2008 17:32:03 +0200 (EET) X-Spam-Checker-Version: SpamAssassin on spamassassin.zzz.ee X-Spam-Level: X-Spam-Guessed-Language: X-Spam-Status: No, score=-2.1 required=5.0 tests=ALL_TRUSTED,BAYES_50, HTML_IMAGE_ONLY_28,HTML_MESSAGE X-Spam-Checker-URL: http://info.zzz.ee Received: from [192.168.5.3] (adsl215.uninet.ee [194.204.62.215]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by zzz.ee (Postfix) with ESMTPS id 852BCFD248 for ; Tue, 16 Dec 2008 17:32:00 +0200 (EET) Message-ID: <4947C9EF.1060606@zzz.ee> Date: Tue, 16 Dec 2008 17:31:59 +0200 From: =?ISO-8859-1?Q?Ott_K=F6stner?= Organization: TIGMA AS User-Agent: Thunderbird 2.0.0.18 (X11/20081125) MIME-Version: 1.0 To: freebsd-questions@freebsd.org References: <20081216142942.GA14998@lpthe.jussieu.fr> In-Reply-To: <20081216142942.GA14998@lpthe.jussieu.fr> Content-Type: multipart/mixed; boundary="------------040609050905080404020707" X-Virus-Scanned: ClamAV using ClamSMTP @zzz.ee X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Python with many threads X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Dec 2008 15:32:05 -0000 This is a multi-part message in MIME format. --------------040609050905080404020707 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Michel Talon wrote: > Nothing limits the number of concurrent threads. Personnally i have > checked i can run Grub Next Generation Python Client with 600 threads > without any problem. > niobe% uname -a > FreeBSD niobe.lpthe.jussieu.fr 7.0-STABLE FreeBSD 7.0-STABLE #0: Tue Jul > 22 10:31:01 CEST 2008 > niobe% python > Python 2.5.2 (r252:60911, Jul 5 2008, 13:44:44) > Still, something locks up in the system, when I increase the number Python threads with http://grub.org/?q=en/node/204 With more than 50 threads I start to get random messages like: Exception in thread 40: Traceback (most recent call last): File "/usr/local/lib/python2.5/threading.py", line 486, in __bootstrap_inner self.run() The frequency of these messages increases with 70 threads. And with 200 threads the system locks up totally -- almost no access from web. [I can still access the system and kill the process :) ] Clearly, there is some limit in the system. It is not memory, cpu or bandwidth. If the number of threads is also not limited, it must be something else. What might it be? Network traffic is moderate: IP traffic > Recall that python threads are just native threads, but these threads > are protected from trashing the python state by a single lock, the GIL, > which is released when you do IO. This allows to effectively thread IO, > but not python computation. The FreeBSD thread library has no particular > limitations, you can run hundreds or thousands of threads without much > problem, for example under Java. > > > Regards, O.K. [sorry for HTML message in the list] -- Testi oma Interneti kiirust / Test Your Internet speed: http://speedtest.zzz.ee/ --------------040609050905080404020707--