Date: Sat, 27 Mar 2010 13:29:28 +1100 From: Andrew MacIntyre <andymac@bullseye.apana.org.au> To: Doug Barton <dougb@FreeBSD.org> Cc: Steven Kreuzer <skreuzer@exit2shell.com>, freebsd-ports@FreeBSD.org, freebsd-python@FreeBSD.org Subject: Re: python and HUGE_STACK_SIZE Message-ID: <4BAD6D88.1020902@bullseye.andymac.org> In-Reply-To: <4BAD4078.6000300@FreeBSD.org> References: <4702BA39-7C18-45C3-9920-9E460502B58F@freebsd.org> <4BAA9C32.6040606@delphij.net> <2D129848-8A41-4BB5-A58C-A9A35D5FBD9A@mac.com> <6201873e1003242207m49351c55id94341d872fd8e17@mail.gmail.com> <B473412B-0083-476C-A4F1-413158BBB639@exit2shell.com> <4BAD4078.6000300@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Doug Barton wrote:
> On 03/26/10 14:03, Steven Kreuzer wrote:
>> So, it seems like most of the time python scripts will work with HUGE_STACK_SIZE
>> turned off, but every once and a while some scripts will fail in non obvious ways
>> that could leave a person scratching their head for weeks trying to get to the bottom of it
>>
>> To me, it seems like the best behavior would be to default to compiling with that set. I'll create
>> a patch over the weekend and open a PR
>
>>From the discussion (not speaking from experience or python knowledge)
> it seems like an OPTION is the way to go, with the open question being
> defaults to on or defaults to off.
>
> What is the impact of HUGE_STACK_SIZE when it's compiled in, and how
> will it affect those running python stuff who don't actually need it?
>
> If it turns out that only a few ports need it and the impact is
> undesirable those ports that do need it could be adapted to test for it
> somehow and suggest that the user re-install python with the option.
HUGE_STACK_SIZE actually refers to the stack size used for each thread
other than the primary thread. As I understand it, these stacks are
fully committed while the thread exists, which is different to the
handling of the primary thread's stack (where only the pages used are
committed). On 32 bit systems, careless use of large thread stacks can
cause non-trivial address space wastage.
People don't see the same issues on Linux as it has a much larger
default thread stack size and I'm lead to believe that only the pages in
use in the thread stacks are committed.
Since Python 2.5, there has been a way to change the thread stack size
at runtime: threading.stack_size().
Ports patches using these calls for packages likely to be affected (such
as Zope) could make this easier for a lot of people, if support for them
can't be encouraged upstream.
--
-------------------------------------------------------------------------
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370
andymac@pcug.org.au (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4BAD6D88.1020902>