From owner-svn-doc-all@freebsd.org Sat Jan 9 21:30:58 2016 Return-Path: Delivered-To: svn-doc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1DADA6AC21; Sat, 9 Jan 2016 21:30:58 +0000 (UTC) (envelope-from bjk@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BBAB11878; Sat, 9 Jan 2016 21:30:58 +0000 (UTC) (envelope-from bjk@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u09LUvtU033236; Sat, 9 Jan 2016 21:30:57 GMT (envelope-from bjk@FreeBSD.org) Received: (from bjk@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u09LUvoq033235; Sat, 9 Jan 2016 21:30:57 GMT (envelope-from bjk@FreeBSD.org) Message-Id: <201601092130.u09LUvoq033235@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: bjk set sender to bjk@FreeBSD.org using -f From: Benjamin Kaduk Date: Sat, 9 Jan 2016 21:30:57 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r47983 - head/en_US.ISO8859-1/htdocs/news/status X-SVN-Group: doc-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire doc trees \(except for " user" , " projects" , and " translations" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 21:30:59 -0000 Author: bjk Date: Sat Jan 9 21:30:57 2016 New Revision: 47983 URL: https://svnweb.freebsd.org/changeset/doc/47983 Log: Add entry on OOM updates from kib Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml Modified: head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml ============================================================================== --- head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml Sat Jan 9 20:26:45 2016 (r47982) +++ head/en_US.ISO8859-1/htdocs/news/status/report-2015-10-2015-12.xml Sat Jan 9 21:30:57 2016 (r47983) @@ -1101,4 +1101,89 @@ + + + Out of Memory Handler Rewrite + + + + + Konstantin + Belousov + + kib@FreeBSD.org + + + + +

The Out of Memory (OOM) code is intended to handle the + situation where the system needs free memory to make progress, + while no memory can be reused. Most often, the situation is that + to free memory, the system needs more free memory. Consider a + case where the system needs to page-out dirty pages, but needs to + allocate structures to track the writes. OOM "solves" + the problem by killing some selection of user processes. In other + words, it trades away system deadlock by suffering a partial loss + of user data. The assumption is that it is better to kill a + process and recover data in other processes, than lose + everything.

+ +

Free memory in the &os; Virtual Memory (VM) system appears + from two sources. One is the voluntary reclamation of pages used + by a process, for example unmapping private anonymous regions, or + the last unlink of an otherwise unreferenced file with cached + pages. Another source is the pagedaemon, which forcefully frees + pages which carry data, of course, after the data is moved to some + other storage, like swap or file blocks. OOM is triggered when + the pagedaemon definitely cannot free memory to satisfy the + requests.

+ +

The old criteria to trigger OOM action was a combination of + low free swap space and a low count of free pages (the later is + expressed precisely with the paging targets constants, but this is + not relevant to the discussion). That test is mostly incorrect, + e.g., a low free page state might be caused by a greedy consumer + allocating all pages freed by the page daemon in the current pass, + but this does not preclude the page daemon from producing more + pages. Also, since page-outs are asynchronous, the previous page + daemon pass might not immmediately produce any free pages, but + they would appear some short time later.

+ +

More seriously, low swap space does not necessarily indicate + that we are in trouble: lots of pages may not require swap + allocations to freed, e.g., clean pages or pages backed by files. + The last notion is serious, since swap-less systems were + considered as having full swap.

+ +

Instead of trying to deduce the deadlock from looking at + the current VM state, the new OOM handler tracks the history of + page daemon passes. Only if several consequtive passes failed to + meet the paging target is an OOM kill considered neccessary. The + count of consequent failed passes was selected empirically, by + testing on small (32M) and large (512G) machines. Auto-tuning of + the counter is possible, but requires some more architectural + changes to the I/O subsystem.

+ +

Another issue was identified with the algorithm which + selects a victim process for OOM kill. It compared the counts of + pages mapping entries (PTEs) installed into the machine paging + structures. For different reasons, machine-dependent VM code + (pmap) may remove the pte for a memory-resident page. Under some + circumstances, related to other measures to prevent low memory + deadlock, very large processes which consume all system memory, + could have few or no ptes, and the old OOM selector ignored the + process which caused the deadlock, killing unrelated + processes.

+ +

A new function vm_pageout_oom_pagecount() was written which + applies a reasonable heuristic to estimate the number of pages + which would be freed by killing the given process. This + eliminates the effect of selecting small unrelated processes for + OOM kill.

+ +

The rewrite was committed to HEAD in r290917 and r290920.

+ + + The FreeBSD Foundation +