Date: Wed, 31 Oct 2018 17:13:23 +0000 From: "Vanco, Juraj" <juraj.vanco@intel.com> To: Konstantin Belousov <kostikbel@gmail.com>, "Bennett, Ciunas" <ciunas.bennett@intel.com> Cc: "freebsd-stable@freebsd.org" <freebsd-stable@freebsd.org> Subject: RE: Possible memory leak in the kernel (contigmalloc) Message-ID: <996B37956731CF40B1A674B085ACCE9440F747CA@IRSMSX103.ger.corp.intel.com> In-Reply-To: <20181030183557.GP5335@kib.kiev.ua> References: <770FD3608C9E864796AB46CB37B561B1BDFCF0CD@IRSMSX101.ger.corp.intel.com> <20181026201230.GV5335@kib.kiev.ua> <770FD3608C9E864796AB46CB37B561B1BE0B4096@irsmsx105.ger.corp.intel.com> <20181030101150.GN5335@kib.kiev.ua> <770FD3608C9E864796AB46CB37B561B1BE0B5562@irsmsx105.ger.corp.intel.com> <20181030183557.GP5335@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Konstatin,
what we can directly see, is that the number of inactive pages increases an=
d this happens after running single loop of contigmalloc -> contigfree even=
with the repeating size of allocated blocks.
The only source of inactive page could come from kernel memory management c=
ontrol structures.
Even if such structure becomes once inactive (cached pages), I do not see a=
ny reason that kernel cannot reclaim these inactive cached pages when again=
the contigmalloc asks for the same space of memory.
I think this cycle should not take all the amount of physical memory withou=
t being able to reclaim back when it's out of memory:
int array0[10] =3D {2097152, 1024, 2101248, 1024, 2097152, 2101248, 210=
1248, 2097152, 2101248, 1024};
int array1[10] =3D {1024, 2101248, 2097152, 2101248, 2101248, 2097152, =
1024, 2101248, 1024, 2097152};
void *mem_blocks[10];
for (i =3D 0; i < 10; i++)
mem_blocks[i] =3D contigmalloc(array0[i], TEST_MEM, 0, 0, ~0, PAGE_=
SIZE, 0);
for (i =3D 0; i < 10; i++)
contigfree(mem_blocks[i], array0[i], TEST_MEM);
for (i =3D 0; i < 10; i++)
mem_blocks[i] =3D contigmalloc(array1[i], TEST_MEM, 0, 0, ~0, PAGE_=
SIZE, 0);
for (i =3D 0; i < 10; i++)
contigfree(mem_blocks[i], array1[i], TEST_MEM);
jv
-----Original Message-----
From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd-stable@freebsd=
.org] On Behalf Of Konstantin Belousov
Sent: Tuesday, October 30, 2018 6:36 PM
To: Bennett, Ciunas <ciunas.bennett@intel.com>
Cc: freebsd-stable@freebsd.org
Subject: Re: Possible memory leak in the kernel (contigmalloc)
On Tue, Oct 30, 2018 at 11:15:47AM +0000, Bennett, Ciunas wrote:
> Hi,
> The only other activity that is running is the csh script that is insert=
ing and removing the kernel module.
> I am not using the VM for any other purpose but to run this test.
> In my tests the correlation between memory allocation and moving to inact=
ive list can be seen.
> I don't think any other process is creating the inactive memory.
But it is created. More, since anon private memory is freed (not deactivat=
ed) on the process exit, something is accumulating the memory.
The other possibility is that the memory is the caching pages from vnodes, =
but for this buffers must be created and then reclaimed, which would sugges=
t even more activity on the system.
> Thanks.
> =
> -----Original Message-----
> From: Konstantin Belousov [mailto:kostikbel@gmail.com]
> Sent: Tuesday, October 30, 2018 10:12 AM
> To: Bennett, Ciunas <ciunas.bennett@intel.com>
> Cc: freebsd-stable@freebsd.org
> Subject: Re: Possible memory leak in the kernel (contigmalloc)
> =
> On Tue, Oct 30, 2018 at 09:46:59AM +0000, Bennett, Ciunas wrote:
> > Hi,
> > =
> > I was debugging the issue by viewing the free ques "sysctl =
> > vm.phys_free" and also using "show page" in ddb. The inactive memory =
> > is never being released back into the free que. I thought that when =
> > inactive memory reaches a certain threshold that the kernel will =
> > start reclaiming and move it to the free list? In my program this is =
> > not happening, the program uses free memory (contigmalloc), and then =
> > it is put into the inactive que (contiigfree) when the program frees it.
> Contigfree() does not release memory into inactive queue. By definition,=
inactive pages have valid content, which is not possible for the pages fre=
ed by contigfree().
> contigfree()->kmem_free()->kmem_unback()->vm_page_free().
> =
> =
> > This inactive memory is never released by the kernel, and the =
> > inactive que grows until all the memory is in this que. I have =
> > attached a xml sheet that shows the memory usage in the system.
> Inactive memory is not freed, it makes no sense as far as there is =
> valid content, which is either not recoverable (anon memory or dirty =
> file
> pages) or high-cost to restore (clean file pages, need to re-read from di=
sk). Inactive is processed by pagedaemon when system has memory deficit, a=
nd either inactive pages are written to swap, or written to their file back=
ing storage.
> =
> I suspect that you have other activities on your system going on, which c=
ause creation of the inactive memory and unrecoverable fragmentation.
> Note that contigmalloc() tries to do defragmentation to satisfy requests,=
but this is not always possible.
> =
> =
> > Ciunas.
> > =
> > -----Original Message-----
> > From: Konstantin Belousov [mailto:kostikbel@gmail.com]
> > Sent: Friday, October 26, 2018 9:13 PM
> > To: Bennett, Ciunas <ciunas.bennett@intel.com>
> > Cc: freebsd-stable@freebsd.org
> > Subject: Re: Possible memory leak in the kernel (contigmalloc)
> > =
> > On Wed, Oct 24, 2018 at 04:27:52PM +0000, Bennett, Ciunas wrote:
> > > Hello,
> > > =
> > > I have encountered an issue with a kernel application that I have =
> > > written, the issue might be caused by a memory leak in the kernel.
> > > The application allocates and deallocates contiguous memory using
> > > contigmalloc() and contigfree(). The application will fail after a =
> > > period of time because there is not enough free contiguous memory =
> > > left. There could be an issue with the freeing of memory when =
> > > using the contigfree() function.
> > >
> > =
> > It is unlikely that there is an issue with a leak, but I would be not s=
urprised if your allocation/free pattern would cause fragmentation on free =
lists that results in contigmalloc(9) failures after.
> > =
> > Look at the vmstat -z/vmstat -m output to see uma and malloc stats.
> > More interesting for your case can be the output from
> > sysctl vm.phys_free
> > which provides information about the free queues and order of free page=
s on them.
> > --------------------------------------------------------------
> > Intel Research and Development Ireland Limited Registered in Ireland =
> > Registered Office: Collinstown Industrial Park, Leixlip, County =
> > Kildare Registered Number: 308263
> > =
> > =
> > This e-mail and any attachments may contain confidential material =
> > for the sole use of the intended recipient(s). Any review or =
> > distribution by others is strictly prohibited. If you are not the =
> > intended recipient, please contact the sender and delete all copies.
> =
> =
> --------------------------------------------------------------
> Intel Research and Development Ireland Limited Registered in Ireland =
> Registered Office: Collinstown Industrial Park, Leixlip, County =
> Kildare Registered Number: 308263
> =
> =
> This e-mail and any attachments may contain confidential material for =
> the sole use of the intended recipient(s). Any review or distribution =
> by others is strictly prohibited. If you are not the intended =
> recipient, please contact the sender and delete all copies.
> =
_______________________________________________
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
This e-mail and any attachments may contain confidential material for the s=
ole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact =
the
sender and delete all copies.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?996B37956731CF40B1A674B085ACCE9440F747CA>