From owner-freebsd-hackers Wed Jul 14 11: 9: 7 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from alpo.whistle.com (alpo.whistle.com [207.76.204.38]) by hub.freebsd.org (Postfix) with ESMTP id BAA9E14D1C for ; Wed, 14 Jul 1999 11:08:55 -0700 (PDT) (envelope-from julian@whistle.com) Received: from current1.whistle.com (current1.whistle.com [207.76.205.22]) by alpo.whistle.com (8.9.1a/8.9.1) with SMTP id LAA31669; Wed, 14 Jul 1999 11:07:30 -0700 (PDT) Date: Wed, 14 Jul 1999 11:07:29 -0700 (PDT) From: Julian Elischer To: lyndon@orthanc.ab.ca Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Replacement for grep(1) (part 2) In-Reply-To: <199907141755.LAA05231@orthanc.ab.ca> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG If you wanted to fix this, you could add a patch to malloc that touched every page that it handed to the application. (and trapped sig11s) On Wed, 14 Jul 1999 lyndon@orthanc.ab.ca wrote: > > > I mean, jeeze, the reservation for the program stack alone would eat > > up all your available swap space! What is a reasonable stack size? The > > system defaults to 8MB. Do we rewrite every program to specify its own > > stack size? How do we account for architectural differences? > > The alternative is to rewrite every program that assumes the semantics > of malloc() are being followed. The problem I have as an applications > writer is that I tend to believe malloc. To pick a specific example, > our IMAP client takes steps to ensure it won't run out of memory in > critical sections. We maintain a "rainy day" pool block of memory. If > we receive a NULL from malloc, we 1) free up whatever memory we can > in other parts of the client (possibly using the rainy day pool to > stage data out to disk), and 2) if necessary, reduce the size of the > rainy day pool. This whole design is predicated on malloc() telling > the truth. If instead it gives us a bogus block of memory, then > seg faults when we try to use it, the best we can do is try to shut > down without losing any of the users mail (and in fact we don't > even do that, since there are just too many places where this can > happen in third-party libraries that we aren't willing to rewrite). > Sending us a kill signal is even worse. (And extremely unfair, since > we take pains to not waste memory in the first place.) > > Has anyone analyzed all those applications people talk about that > show huge allocation footprints but don't actually use the memory? > That represents the code that needs to be fixed. Breaking malloc() > is not a suitable response IMO. > > As a data point, we routinely disable overcommit on our SGI machines > and it doesn't hurt us one bit. And we aren't allocating gigabytes > of swap space, either. > > --lyndon > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message