Date: Fri, 9 Nov 2012 19:10:04 +0000 From: "Sears, Steven" <Steven.Sears@netapp.com> To: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Memory reserves or lack thereof Message-ID: <A6DE036C6A90C949A25CE89E844237FB2086970A@SACEXCMBX01-PRD.hq.netapp.com>
next in thread | raw e-mail | index | archive | help
I have a memory subsystem design question that I'm hoping someone can answe=
r.
I've been looking at a machine that is completely out of memory, as in
v_free_count =3D 0,=20
v_cache_count =3D 0,=20
I wondered how a machine could completely run out of memory like this, espe=
cially after finding a lack of interrupt storms or other pathologies that w=
ould tend to overcommit memory. So I started investigating.
Most allocators come down to vm_page_alloc(), which has this guard:
if ((curproc =3D=3D pageproc) && (page_req !=3D VM_ALLOC_INTERRUPT)) {
page_req =3D VM_ALLOC_SYSTEM;
};
if (cnt.v_free_count + cnt.v_cache_count > cnt.v_free_reserved ||
(page_req =3D=3D VM_ALLOC_SYSTEM &&=20
cnt.v_free_count + cnt.v_cache_count > cnt.v_interrupt_free_min) ||
(page_req =3D=3D VM_ALLOC_INTERRUPT &&
cnt.v_free_count + cnt.v_cache_count > 0)) {
The key observation is if VM_ALLOC_INTERRUPT is set, it will allocate every=
last page.
>From the name one might expect VM_ALLOC_INTERRUPT to be somewhat rare, perh=
aps only used from interrupt threads. Not so, see kmem_malloc() or uma_smal=
l_alloc() which both contain this mapping:
if ((flags & (M_NOWAIT|M_USE_RESERVE)) =3D=3D M_NOWAIT)
pflags =3D VM_ALLOC_INTERRUPT | VM_ALLOC_WIRED;
else
pflags =3D VM_ALLOC_SYSTEM | VM_ALLOC_WIRED;
Note that M_USE_RESERVE has been deprecated and is used in just a handful o=
f places. Also note that lots of code paths come through these routines.
What this means is essentially _any_ allocation using M_NOWAIT will bypass =
whatever reserves have been held back and will take every last page availab=
le.
There is no documentation stating M_NOWAIT has this side effect of essentia=
lly being privileged, so any innocuous piece of code that can't block will =
use it. And of course M_NOWAIT is literally used all over.
It looks to me like the design goal of the BSD allocators is on recovery; i=
t will give all pages away knowing it can recover.
Am I missing anything? I would have expected some small number of pages to =
be held in reserve just in case. And I didn't expect M_NOWAIT to be a sort =
of back door for grabbing memory.
Thanks,
-Steve
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A6DE036C6A90C949A25CE89E844237FB2086970A>