From owner-freebsd-ports@FreeBSD.ORG Sat Mar 27 02:58:49 2010 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 038121065670; Sat, 27 Mar 2010 02:58:49 +0000 (UTC) (envelope-from andymac@bullseye.apana.org.au) Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by mx1.freebsd.org (Postfix) with ESMTP id 4592E8FC13; Sat, 27 Mar 2010 02:58:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEAAsOrUt5LdR8/2dsb2JhbACbLHS+NoR+BA Received: from ppp121-45-212-124.lns20.cbr1.internode.on.net (HELO bullseye.apana.org.au) ([121.45.212.124]) by ipmail06.adl2.internode.on.net with ESMTP; 27 Mar 2010 13:13:10 +1030 Received: from [192.168.63.10] (tenring.andymac.org [192.168.63.10]) by bullseye.apana.org.au (8.14.2/8.14.2) with ESMTP id o2R2qSvv085119; Sat, 27 Mar 2010 13:52:30 +1100 (EST) (envelope-from andymac@bullseye.andymac.org) Message-ID: <4BAD6D88.1020902@bullseye.andymac.org> Date: Sat, 27 Mar 2010 13:29:28 +1100 From: Andrew MacIntyre User-Agent: Thunderbird 2.0.0.23 (OS/2/20090822) MIME-Version: 1.0 To: Doug Barton References: <4702BA39-7C18-45C3-9920-9E460502B58F@freebsd.org> <4BAA9C32.6040606@delphij.net> <2D129848-8A41-4BB5-A58C-A9A35D5FBD9A@mac.com> <6201873e1003242207m49351c55id94341d872fd8e17@mail.gmail.com> <4BAD4078.6000300@FreeBSD.org> In-Reply-To: <4BAD4078.6000300@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Steven Kreuzer , freebsd-ports@FreeBSD.org, freebsd-python@FreeBSD.org Subject: Re: python and HUGE_STACK_SIZE X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 02:58:49 -0000 Doug Barton wrote: > On 03/26/10 14:03, Steven Kreuzer wrote: >> So, it seems like most of the time python scripts will work with HUGE_STACK_SIZE >> turned off, but every once and a while some scripts will fail in non obvious ways >> that could leave a person scratching their head for weeks trying to get to the bottom of it >> >> To me, it seems like the best behavior would be to default to compiling with that set. I'll create >> a patch over the weekend and open a PR > >>From the discussion (not speaking from experience or python knowledge) > it seems like an OPTION is the way to go, with the open question being > defaults to on or defaults to off. > > What is the impact of HUGE_STACK_SIZE when it's compiled in, and how > will it affect those running python stuff who don't actually need it? > > If it turns out that only a few ports need it and the impact is > undesirable those ports that do need it could be adapted to test for it > somehow and suggest that the user re-install python with the option. HUGE_STACK_SIZE actually refers to the stack size used for each thread other than the primary thread. As I understand it, these stacks are fully committed while the thread exists, which is different to the handling of the primary thread's stack (where only the pages used are committed). On 32 bit systems, careless use of large thread stacks can cause non-trivial address space wastage. People don't see the same issues on Linux as it has a much larger default thread stack size and I'm lead to believe that only the pages in use in the thread stacks are committed. Since Python 2.5, there has been a way to change the thread stack size at runtime: threading.stack_size(). Ports patches using these calls for packages likely to be affected (such as Zope) could make this easier for a lot of people, if support for them can't be encouraged upstream. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia