From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 17 17:31:55 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B433106566B for ; Wed, 17 Sep 2008 17:31:55 +0000 (UTC) (envelope-from jasone@FreeBSD.org) Received: from canonware.com (canonware.com [64.183.146.166]) by mx1.freebsd.org (Postfix) with ESMTP id F20AD8FC24 for ; Wed, 17 Sep 2008 17:31:54 +0000 (UTC) (envelope-from jasone@FreeBSD.org) Received: from [192.168.168.201] (unknown [192.168.168.201]) by canonware.com (Postfix) with ESMTP id 20EE912989F; Wed, 17 Sep 2008 10:03:55 -0700 (PDT) Message-ID: <48D13784.2090701@FreeBSD.org> Date: Wed, 17 Sep 2008 09:59:48 -0700 From: Jason Evans User-Agent: Thunderbird 2.0.0.16 (X11/20080724) MIME-Version: 1.0 To: Andrew MacIntyre References: <48D12AD4.1000806@bullseye.andymac.org> In-Reply-To: <48D12AD4.1000806@bullseye.andymac.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: unexpected behaviour of malloc() on 7.0/amd64 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2008 17:31:55 -0000 Andrew MacIntyre wrote: > In investigating a Python 2.6rc1 regression test failure on FreeBSD > 7.0/amd64, as far as I can tell, malloc() does not return NULL when > available memory (including swap) is exhausted - the process just gets > KILLed. > > Using ulimit -v to set a virtual memory use limit below the available > memory does result in malloc() returning NULL when the limit is hit. > > The Python regression test concerned does not fail on FreeBSD 7.0/i386, > however the C program below exhibits the unexpected behaviour on both > 7.0/amd64 and 7.0/i386. The C program below does behave as > expected on FreeBSD 6.3/i386; I cannot currently test its behaviour on > FreeBSD 6.3/amd64. > > I can't see this behaviour documented in the malloc() man page. From malloc(3): === IMPLEMENTATION NOTES Traditionally, allocators have used sbrk(2) to obtain memory, which is suboptimal for several reasons, including race conditions, increased fragmentation, and artificial limitations on maximum usable memory. This allocator uses both sbrk(2) and mmap(2) by default, but it can be configured at run time to use only one or the other. If resource limits are not a primary concern, the preferred configuration is MALLOC_OPTIONS=dM or MALLOC_OPTIONS=DM. When so configured, the datasize resource limit has little practical effect for typical applications; use MALLOC_OPTIONS=Dm if that is a concern. Regardless of allocator configuration, the vmemoryuse resource limit can be used to bound the total virtual memory used by a process, as described in limits(1). === If you want a custom python binary that does not use mmap, you can define _malloc_options to "d", or just use MALLOC_OPTIONS in the environment. Jason