From owner-freebsd-current@FreeBSD.ORG Tue Jan 8 00:17:08 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACBF516A419 for ; Tue, 8 Jan 2008 00:17:08 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 59BCF13C442 for ; Tue, 8 Jan 2008 00:17:08 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id F39B917105; Tue, 8 Jan 2008 00:17:05 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.2/8.14.2) with ESMTP id m080H4kP015095; Tue, 8 Jan 2008 00:17:04 GMT (envelope-from phk@critter.freebsd.dk) To: "Igor Mozolevsky" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 08 Jan 2008 00:06:33 GMT." Date: Tue, 08 Jan 2008 00:17:04 +0000 Message-ID: <15094.1199751424@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Kostik Belousov , Andrew Reilly , freebsd-current@freebsd.org, Peter Jeremy Subject: Re: sbrk(2) broken X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2008 00:17:08 -0000 In message , "Igor Mozolevsky" writes: >On 07/01/2008, Andrew Reilly wrote: >> On Mon, 07 Jan 2008 13:18:47 +0000 >> "Poul-Henning Kamp" wrote: >> >> > Yes, but you will not see this complication, it will be hidden >> > in the implementation of malloc(3). >> >> How could you hide it inside malloc? Would malloc start >> returning 0 after receiving the "less mem than desirable" >> signal? Would it ever go back to returning non-zero? > >I'm with Andrew on this one. The only (sensible) way I could see it >being hidden behind malloc() is if malloc() blocks until sufficient >memory becomes available. You should read some recent literature on malloc(3), my own and Jasons papers are good places to start. For performance reasons, malloc(3) will hold on to a number of pages that theoretically could be given back to the kernel, simply because it expects to need them shortly. Such parameters and many others of the malloc implementation can be tweaked to "waste" more or less memory, in response to a sensibly granular indication from the kernel about how bad things are. Also, many subsystems in the kernel could adjust their memory use in response to a "memory pressure" indication, if memory is tight, we could cache vnodes and inodes less agressively, if things are going truly bad, we can even ditch all non-active entries from these caches. If one implements this with three states: Green - "all clear" Yellow - "tight" - free one before you allocate one if you can. Red - "all out" - free all that you sensibly can. And implemented strategies like I propose above (and have proposed for the last 10 years), then it is very unlikely that the system would ever get into the red state, because the yellow state will mitigate and reduce the memory pressure. Nothing prevents an intelligent process from listening in and doing sensible things, firefox could ditch the memory cache of pages for instance. But we can't get anywhere until some VM wizard produces the three "lamps" for us to look at in the first place, that's where we have been stuck for the last 10 years. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.