From owner-freebsd-hackers Wed Jul 14 21:25:48 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (Postfix) with ESMTP id C2D7214D2F; Wed, 14 Jul 1999 21:25:46 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id VAA06697; Wed, 14 Jul 1999 21:25:31 -0700 (PDT) (envelope-from dillon) Date: Wed, 14 Jul 1999 21:25:31 -0700 (PDT) From: Matthew Dillon Message-Id: <199907150425.VAA06697@apollo.backplane.com> To: lyndon@orthanc.ab.ca Cc: "Brian F. Feldman" , freebsd-hackers@FreeBSD.ORG Subject: Re: Swap overcommit (was Re: Replacement for grep(1) (part 2)) References: <199907141833.MAA05320@orthanc.ab.ca> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :Our IMAP server routinely show a footprint of about 1MB private storage. :This is constant for most operations. However, when you get into doing :SEARCH and SORT, there are certain cases where we need memory, sometimes :a *lot* of memory. : :Your proposal is that my *well behaved* application should be arbitrarily :killed, leaving the client stuck with a) no results and b) no IMAP :connection, in this situation. (And think threaded. That one server :could be handling *hundreds* of clients.) This is preferable to :returning a NULL to the malloc() request, which I can handle :gracefully by simply returning a NO response to the IMAP client? : :What it so evil about having a reasonably intelligent malloc() that :tells the truth, and returns unused memory to the system? Overcommit :is for lazy programmers, plain and simple. At least the SGI documentation :about overcommit admits that (or at least, did at one time). : :--lyndon If you are running an IMAP server that regularly runs out of swap space, you have a configuration problem which needs to be addressed. It's as simple as that. What you are putting forth is an example of something that will never happen on a properly configured server. In regards to the general case where one is running third-party applications. Here you are assuming that you can go in and modify every single piece of software running on the machine to deal with malloc() returning NULL. Because if you don't, the machine isn't going to be very stable. Not only that, you are assuming that you will make the correct decision on what action to take when malloc() *does* return NULL. If you decide to return an error code but not exit, what happens when a potential blowup situation results in thousands of imap processes being run on the system, and NONE of them exit when their malloc() fails? The problem is a whole lot more complex then simply having the OS return NULL from a malloc(). Currently the OS kills processes as a last resort. The idea is that no nominally running system runs out of swap. Now you propose to take away the kernel's ability to recover some memory as a last resort and instead put it into the hands of the very user or root-run processes that are causing the problem in the first place! A much better solution would be to write a simple watchdog script that notices when swap space is low and does the right thing -- e.g. kills the non-essential processes and leaves the essential ones alone. Then the kernel never actually reaches a state of last-resort. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message