From owner-freebsd-current@FreeBSD.ORG Tue Jan 8 00:28:58 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE80016A417 for ; Tue, 8 Jan 2008 00:28:58 +0000 (UTC) (envelope-from andrew-freebsd@areilly.bpc-users.org) Received: from omta02ps.mx.bigpond.com (omta02ps.mx.bigpond.com [144.140.83.154]) by mx1.freebsd.org (Postfix) with ESMTP id 75C0D13C447 for ; Tue, 8 Jan 2008 00:28:58 +0000 (UTC) (envelope-from andrew-freebsd@areilly.bpc-users.org) Received: from oaamta07ps.mx.bigpond.com ([124.188.162.219]) by omta02ps.mx.bigpond.com with ESMTP id <20080108002857.DAWY28980.omta02ps.mx.bigpond.com@oaamta07ps.mx.bigpond.com> for ; Tue, 8 Jan 2008 00:28:57 +0000 Received: from areilly.bpa.nu ([124.188.162.219]) by oaamta07ps.mx.bigpond.com with ESMTP id <20080108002856.IBKF25013.oaamta07ps.mx.bigpond.com@areilly.bpa.nu> for ; Tue, 8 Jan 2008 00:28:56 +0000 Received: (qmail 8323 invoked from network); 8 Jan 2008 00:28:13 -0000 Received: from localhost (HELO duncan.reilly.home) (127.0.0.1) by localhost with SMTP; 8 Jan 2008 00:28:13 -0000 Date: Tue, 8 Jan 2008 11:28:12 +1100 From: Andrew Reilly To: "Poul-Henning Kamp" Message-ID: <20080108112812.13ffab9f@duncan.reilly.home> In-Reply-To: <15094.1199751424@critter.freebsd.dk> References: <15094.1199751424@critter.freebsd.dk> X-Mailer: Claws Mail 3.0.2 (GTK+ 2.12.3; amd64-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Kostik Belousov , Peter Jeremy , freebsd-current@freebsd.org, Igor Mozolevsky Subject: Re: sbrk(2) broken X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2008 00:28:59 -0000 On Tue, 08 Jan 2008 00:17:04 +0000 "Poul-Henning Kamp" wrote: > For performance reasons, malloc(3) will hold on to a number of pages > that theoretically could be given back to the kernel, simply because > it expects to need them shortly. Aah, OK, so there's some essentially system-level caching going on behind the scenes, and that's readily malleable for this sort of thing. I thought that you were proposing some way to propagate the "yellow" or "red" conditions to user-program activity through malloc, which seems hard, since the only official out-of-band signal there is a zero return. I'll have to track down your papers, though, because I thought that the whole problem revolved around the fact that malloc(3) doesn't hand out physical pages at all: that was left up to the kernel vm pager to do as needed. Is it zeroed (and therefore touched/present) pages that malloc keeps a stash of? > Such parameters and many others of the malloc implementation can > be tweaked to "waste" more or less memory, in response to a sensibly > granular indication from the kernel about how bad things are. > > Also, many subsystems in the kernel could adjust their memory use > in response to a "memory pressure" indication, if memory is tight, > we could cache vnodes and inodes less agressively, if things are > going truly bad, we can even ditch all non-active entries from > these caches. I agree. That sort of auto-tuning of the space/speed trade-off would be extremely cool. > If one implements this with three states: > > Green - "all clear" > > Yellow - "tight" - free one before you allocate one if you can. > > Red - "all out" - free all that you sensibly can. I imagine that even if the accounting can be managed efficiently, the specification of the specific thresholds would be fairly tricky to specify... Cheers, -- Andrew