Date: Sat, 17 Jul 1999 12:49:28 +0300 (EDT) From: "Valentin Nechayev" <nx@nn.kiev.ua> To: freebsd-hackers@freebsd.org Subject: Re: Replacement for grep(1) (part 2) Message-ID: <ABe65atWdP@nn.kiev.ua> References: <199907132229.PAA81360@apollo.backplane.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Matthew Dillon wrote: > Give me a shell and I can crash any machine. Oh. ;| > A good example of this is sendmail. Before the MaxDaemonChildren and > MaxArticleSize options, it was possible for sendmail to overcommit a > machine. In this case the overcommit that can occur is with I/O, not > swap. As a general performance rule, you have to set MaxDaemonChildren > and MaxArticleSize to prevent the overcommit from occuring. This is a > function of sendmail, not a function of the kernel. Sigh. ((c)you) Sendmail can overcommit a machine with right set of MaxDaemonChildren, MaxArticleSize, QueueLA & RefuseLA options - I have seen such situations. MaxDaemonChildren limits only number of main processes for incoming connections (plus queue run processes). For each connection, after "main from:" and until accepting message, server process for incoming connection forks child which accepts recipient list and letter body. After message accepting, that child can fork delivery process. A queue run process with "O ForkEachJob=true" option, which is default, can create a delivery process for each queue job (in my practice, queue of more than 1000 jobs is ordinary event). All these forks depend only on one test - get current LA and compare it with QueueLA - which fail when high load appeared less than one minute ago. To prevent its overcommit, (I interfere in details with parallel message) the minimal (and possibly not enough) setup set is: 1) patch - insert sm_sleep(1) to server subprocess code before "accepted" reply - limit incoming mail rate; 2) Desrease QueueLA for listening daemon to sub-minimal value (i.e.2); 3) Increase QueueLA for queue running daemon to high values (i.e.50) and set them OForkEachJob=false. But most of these tunings are indirect. A direct tuning invented experimentally on my mail servers is specially hacked pstat program that returns 1 if either swap or file descriptors are used more than 2/3, 0 otherwise; on getting 1, sendmail stops delivering. But, it's pity, this check is unportable. (P.S. Don't tell me change MTA; this is fully another question.) > Another good example is a web server. A web server must have specific > limitations on the number of simultanious connections it is allowed > to handle at once and on the number of CGI's or other auxillary programs > that are allowed to be running at any given time. The overcommit issue > here has nothing to do with swap and everything to do with performance. > Specifically, these limitations exist to avoid cascade failures. As in sendmail case, you propose make some calculations (which are difficult and non-trivial to newbies) to make appreciations of nesessary resources. Another way, which is imho more acceptable, is to provide not hard barriers (SIGKILL on overcommitting), but soft barriers (i.e., stop memory allocating for non-wheel users when memory begins to exhaust). Extra 64M of memory or a disk for swap is commonly quite more cheaper than profitloss on critical service crash. > In the same manner any truely critical system server must handle the > resource management itself to deal with all sorts of problem situations, > including memory. You do not need to build any of this control into the > kernel. No, we need it. Not every server can be patched for such tests (due to loss of sources or another reason), not every admin can make nesessary patches. Kernel must help in it. -- Netch To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ABe65atWdP>