From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 18 03:16:24 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1855106566B for ; Thu, 18 Sep 2008 03:16:24 +0000 (UTC) (envelope-from andymac@bullseye.apana.org.au) Received: from ipmail05.adl2.internode.on.net (ipmail05.adl2.internode.on.net [203.16.214.145]) by mx1.freebsd.org (Postfix) with ESMTP id E19BC8FC26 for ; Thu, 18 Sep 2008 03:16:18 +0000 (UTC) (envelope-from andymac@bullseye.apana.org.au) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AiQFAAtl0Ug7pzeZ/2dsb2JhbACBZLcxgWc X-IronPort-AV: E=Sophos;i="4.32,419,1217773800"; d="scan'208";a="209009448" Received: from ppp59-167-55-153.lns1.cbr1.internode.on.net (HELO bullseye.apana.org.au) ([59.167.55.153]) by ipmail05.adl2.internode.on.net with ESMTP; 18 Sep 2008 12:46:17 +0930 Received: from [192.168.63.10] (tenring.andymac.org [192.168.63.10]) by bullseye.apana.org.au (8.14.2/8.14.2) with ESMTP id m8I36hD4091081; Thu, 18 Sep 2008 13:06:44 +1000 (EST) (envelope-from andymac@bullseye.andymac.org) Message-ID: <48D1C83A.1010204@bullseye.andymac.org> Date: Thu, 18 Sep 2008 13:17:14 +1000 From: Andrew MacIntyre User-Agent: Thunderbird 2.0.0.14 (OS/2/20080509) MIME-Version: 1.0 To: Jason Evans References: <48D12AD4.1000806@bullseye.andymac.org> <48D13784.2090701@FreeBSD.org> In-Reply-To: <48D13784.2090701@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Andrew MacIntyre , freebsd-hackers@FreeBSD.org Subject: Re: unexpected behaviour of malloc() on 7.0/amd64 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2008 03:16:24 -0000 Jason Evans wrote: > Andrew MacIntyre wrote: >> In investigating a Python 2.6rc1 regression test failure on FreeBSD >> 7.0/amd64, as far as I can tell, malloc() does not return NULL when >> available memory (including swap) is exhausted - the process just gets >> KILLed. >> >> Using ulimit -v to set a virtual memory use limit below the available >> memory does result in malloc() returning NULL when the limit is hit. >> >> The Python regression test concerned does not fail on FreeBSD 7.0/i386, >> however the C program below exhibits the unexpected behaviour on both >> 7.0/amd64 and 7.0/i386. The C program below does behave as >> expected on FreeBSD 6.3/i386; I cannot currently test its behaviour on >> FreeBSD 6.3/amd64. >> >> I can't see this behaviour documented in the malloc() man page. > > From malloc(3): > > === > > IMPLEMENTATION NOTES > Traditionally, allocators have used sbrk(2) to obtain memory, which > is suboptimal for several reasons, including race conditions, increased > fragmentation, and artificial limitations on maximum usable memory. > This allocator uses both sbrk(2) and mmap(2) by default, but it can be > configured at run time to use only one or the other. If resource limits > are not a primary concern, the preferred configuration is > MALLOC_OPTIONS=dM or MALLOC_OPTIONS=DM. When so configured, the > datasize resource limit has little practical effect for typical > applications; use MALLOC_OPTIONS=Dm if that is a concern. Regardless of > allocator configuration, the vmemoryuse resource limit can be used to > bound the total virtual memory used by a process, as described in > limits(1). > > === > > If you want a custom python binary that does not use mmap, you can > define _malloc_options to "d", or just use MALLOC_OPTIONS in the > environment. Thanks for the reply. The malloc(3) man page for 7.0 doesn't include this information, and says sbrk() is only used on i386. From the 7-stable version of the man page, which appears to match your quote above, I infer that these changes will be in 7.1. However my reading of the updated man page suggests that the malloc options string should be "Dm" or "m" to turn off mmap() and only use sbrk() - could you clarify why "d"? Thanks, Andrew. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia