From owner-freebsd-current@FreeBSD.ORG Fri Jan 4 11:19:28 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 760CF16A417; Fri, 4 Jan 2008 11:19:28 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 4527813C46B; Fri, 4 Jan 2008 11:19:28 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id B6BF84D7EC; Fri, 4 Jan 2008 06:19:27 -0500 (EST) Date: Fri, 4 Jan 2008 11:19:27 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Igor Mozolevsky In-Reply-To: Message-ID: <20080104111336.X77222@fledge.watson.org> References: <477C82F0.5060809@freebsd.org> <863ateemw2.fsf@ds4.des.no> <20080104002002.L30578@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , freebsd-current@freebsd.org, Jason Evans Subject: Re: sbrk(2) broken X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2008 11:19:28 -0000 On Fri, 4 Jan 2008, Igor Mozolevsky wrote: > On 04/01/2008, Robert Watson wrote: > >> To be clear, in the new world order, instead of getting NULL back from >> malloc(3), SIGKILL is delivered to large processes. > > Huh??? Again, huh??? FreeBSD allows memory overcommit, both overcommit of physical memory resulting in paging, and overcommit of swap space. For the last few years, resource limits on the data segment size, previously observed by malloc(), have prevented processes from mallocing enough memory individually to exhaust swap on 32-bit systems. This is arguably a bug, because you actually want a single process to be able to allocate enough memory to fill its address space, but because the data segment size is used to make address space layout decisions from the inception of the process, is rather inate to using sbrk(). Jason's new malloc uses mmap() of anonymous memory, which isn't affected by the data segment limit, and hence, as a feature, isn't limited by the resouce limit. This turns out to be awkward if you have a run-away process, as where previously it would simply get back an error when it tried to exceed its resource limit, now it simply consumes all your swap, which then results in overcommit. My hope was that we could re-introduce a resource limit on malloc'd memory without large changes, but that appears to have been more tricky than hoped. The goal is not to prevent overcommit, which is invaluable in UNIX systems due to the fork() model which pretty much pre-supposes it by design, rather, to prevent exhaustion of swap by a single process if not specifically allowed by the administrator (in the same way we limit all sorts of other things, like open files, mbufs, socket buffer memory, etc). The right way to do it is to provide a specifically configurable process limit on swap use, the same way we did for data segment size, only not data segment size, but that was considered likely too risky for 7.0. Robert N M Watson Computer Laboratory University of Cambridge