From owner-freebsd-hackers Wed Jul 14 16: 3: 4 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (Postfix) with ESMTP id 881CA15454 for ; Wed, 14 Jul 1999 16:02:51 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id QAA97044; Wed, 14 Jul 1999 16:02:43 -0700 (PDT) (envelope-from dillon) Date: Wed, 14 Jul 1999 16:02:43 -0700 (PDT) From: Matthew Dillon Message-Id: <199907142302.QAA97044@apollo.backplane.com> To: Nate Williams Cc: cgd@netbsd.org (Chris G. Demetriou), freebsd-hackers@FreeBSD.ORG, tech-userlevel@netbsd.org Subject: Re: Replacement for grep(1) (part 2) References: <199907132110.OAA23817@lestat.nas.nasa.gov> <199907132114.OAA80781@apollo.backplane.com> <877lo4z0pe.fsf@redmail.redback.com> <199907132212.PAA81234@apollo.backplane.com> <871zecyx0k.fsf@redmail.redback.com> <199907142234.QAA29442@mt.sri.com> <199907142245.PAA96901@apollo.backplane.com> <199907142249.QAA29554@mt.sri.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG : :Most of the work we've done wouldn't allow this, especially if we were :using an OS like FreeBSD with a fairly long bootup time. Especially if :it can be avoided. : :Yes, we could (and did) do our own memory management, but it seems to me :that the kernel has alot more information available to it and would do :it better than I could. Then again, maybe I'm totally confused about :how the VM system 'does its thing', and in reality it's no better at it :than our code, just alot more complex for no reason. :) :) :) : :Nate The kernel just isn't the best place to do this. The level of sophistication required to satisfy the larger set of programming is much greater then anything discussed here. The last thing that the kernel should be doing is returning NULL from malloc() to a program which is operating within its specifications. That doesn't help anyone, least of all the programmer who now has to deal not only with the complexity of the project he is working on, but must also deal with the potential that the OS will step in at any time and give him an error that he must deal with in a sophisticated fashion even when his software is operating properly. The same goes for non-embedded work. Why is it that programs generally exit when they encounter a memory allocation failure? Because it is too damn hard to ensure proper operation at every instance where a program's allocation may fail, especially if the program getting the failure is not responsible for the problem. The glib statement that "programs should deal with memory allocation errors" provides no solution. 98% of the source code in the BSD code base will exit if a memory allocation fails, and I don't know anyone who wants to go through and somehow "fix" it all (and one could argue what the "fix" should be when something like grep or a shell has a memory allocation failure). To require that this code be made twice as sophisticated as it is already in order to deal with a non-overcommit model reliably is a fantasy. Due to the side effects it causes, to put the capability in the kernel in this situation will make things even more unstable then the kernel's current operation. A situation made much worse by the magnitude of the resources necessary to bring such a system up to the same performance levels attainable by a standard overcommit system. The only case where this does not occur is also the case where the programmer can fairly easily manage the memory to avoid the overcommit from occuring in the first place. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message