Date: Tue, 13 Jul 1999 15:29:37 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Noriyuki Soda <soda@sra.co.jp> Cc: Jason Thorpe <thorpej@nas.nasa.gov>, "Brian F. Feldman" <green@FreeBSD.ORG>, bright@rush.net, dcs@newsguy.com, freebsd-hackers@FreeBSD.ORG, jon@oaktree.co.uk, tech-userlevel@netbsd.org Subject: Re: Replacement for grep(1) (part 2) Message-ID: <199907132229.PAA81360@apollo.backplane.com> References: <199907132127.OAA80947@apollo.backplane.com> <199907132139.GAA14890@srapc342.sra.co.jp> <199907132153.OAA81153@apollo.backplane.com> <199907132215.HAA15042@srapc342.sra.co.jp>
next in thread | previous in thread | raw e-mail | index | archive | help
:> a system out of swap, it is fairly easy to do whether the system
:> uses an overcommit model or not. The user has any number of
:> ways of blowing the server up too - for example, by making
:> thousands of connections to it or running many huge queries in
:> parallel.
:
:If the kernel and the application behave properly, critical
:application doesn't lose it's data in such situation on
:non-overcommiting systems.
:Your example doesn't make sense.
:--
:soda
Give me a shell and I can crash any machine. If you are assuming
hostile users, you cannot assume that your magic overcommit protection
will save your server. Saying that the kernel and application behave
properly is a cop-out, because it's virtually impossible to guarentee
that for every situation. The chance of a user blowing up the server
by finding a bug or a hole somewhere is much, much greater then the chance
of a user running the system out of swap. Concentrating on the memory
reservation aspect of the situation means that you miss the more likely
scenario and, in the end, you do not make the end result any more
reliable - in fact, less reliable then the person who doesn't worry
about the 'kernel overcommit problem' and instead engineers the code
such that overcommits cannot occur in the first place.
Saying that the kernel cannot support certain classes of applications
because it does not handle a memory/swap overcommit the way you want
does not change the fact that such critical applications should not allow
themselves to even attempt an overcommit anyway.
A good example of this is sendmail. Before the MaxDaemonChildren and
MaxArticleSize options, it was possible for sendmail to overcommit a
machine. In this case the overcommit that can occur is with I/O, not
swap. As a general performance rule, you have to set MaxDaemonChildren
and MaxArticleSize to prevent the overcommit from occuring. This is a
function of sendmail, not a function of the kernel.
Another good example is a web server. A web server must have specific
limitations on the number of simultanious connections it is allowed
to handle at once and on the number of CGI's or other auxillary programs
that are allowed to be running at any given time. The overcommit issue
here has nothing to do with swap and everything to do with performance.
Specifically, these limitations exist to avoid cascade failures.
In the same manner any truely critical system server must handle the
resource management itself to deal with all sorts of problem situations,
including memory. You do not need to build any of this control into the
kernel.
To say that FreeBSD does not support a certain class of system because
it uses an overcommit model is not correct, because you can trivially
solve the problem by implementing your own management of memory rather
then use the UNIX libc builtins. The UNIX libc bulitins properly assume
a more general machine configuration and it would not be appropriate to
use them for embedded work if memory use is an issue.
-Matt
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199907132229.PAA81360>