From owner-freebsd-stable@freebsd.org Tue Oct 30 11:17:02 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FDB010DFE4A for ; Tue, 30 Oct 2018 11:17:02 +0000 (UTC) (envelope-from ciunas.bennett@intel.com) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "orsmga106.jf.intel.com", Issuer "COMODO RSA Organization Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7921F7DEA7 for ; Tue, 30 Oct 2018 11:17:01 +0000 (UTC) (envelope-from ciunas.bennett@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 Oct 2018 04:15:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,444,1534834800"; d="scan'208";a="92355724" Received: from irsmsx101.ger.corp.intel.com ([163.33.3.153]) by FMSMGA003.fm.intel.com with ESMTP; 30 Oct 2018 04:15:49 -0700 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.144]) by IRSMSX101.ger.corp.intel.com ([169.254.1.134]) with mapi id 14.03.0415.000; Tue, 30 Oct 2018 11:15:48 +0000 From: "Bennett, Ciunas" To: Konstantin Belousov CC: "freebsd-stable@freebsd.org" Subject: RE: Possible memory leak in the kernel (contigmalloc) Thread-Topic: Possible memory leak in the kernel (contigmalloc) Thread-Index: AdRrtbzKvvQI2iPZToSPdiaEvmHIXQBqhpgAALREjJAAAgPuAAABiKNg Date: Tue, 30 Oct 2018 11:15:47 +0000 Message-ID: <770FD3608C9E864796AB46CB37B561B1BE0B5562@irsmsx105.ger.corp.intel.com> References: <770FD3608C9E864796AB46CB37B561B1BDFCF0CD@IRSMSX101.ger.corp.intel.com> <20181026201230.GV5335@kib.kiev.ua> <770FD3608C9E864796AB46CB37B561B1BE0B4096@irsmsx105.ger.corp.intel.com> <20181030101150.GN5335@kib.kiev.ua> In-Reply-To: <20181030101150.GN5335@kib.kiev.ua> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYzE3YThmNzQtNzc5OC00ZjljLTkzN2EtNDM3ODYyMmRmMWI0IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoidDVJdW1YbllmMmxyaHFZZWJvN2tcLzUwRTgrTm5ZTmVDV0VEXC9OVGdDWlRIMzRod3JncFF6QkMrSDByVGdJNHhsIn0= x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2018 11:17:02 -0000 Hi, The only other activity that is running is the csh script that is insertin= g and removing the kernel module. I am not using the VM for any other purpose but to run this test. In my tests the correlation between memory allocation and moving to inactiv= e list can be seen. I don't think any other process is creating the inactive memory. Thanks. -----Original Message----- From: Konstantin Belousov [mailto:kostikbel@gmail.com] = Sent: Tuesday, October 30, 2018 10:12 AM To: Bennett, Ciunas Cc: freebsd-stable@freebsd.org Subject: Re: Possible memory leak in the kernel (contigmalloc) On Tue, Oct 30, 2018 at 09:46:59AM +0000, Bennett, Ciunas wrote: > Hi, > = > I was debugging the issue by viewing the free ques "sysctl = > vm.phys_free" and also using "show page" in ddb. The inactive memory = > is never being released back into the free que. I thought that when = > inactive memory reaches a certain threshold that the kernel will start = > reclaiming and move it to the free list? In my program this is not = > happening, the program uses free memory (contigmalloc), and then it is = > put into the inactive que (contiigfree) when the program frees it. Contigfree() does not release memory into inactive queue. By definition, i= nactive pages have valid content, which is not possible for the pages freed= by contigfree(). contigfree()->kmem_free()->kmem_unback()->vm_page_free(). > This inactive memory is never released by the kernel, and the inactive = > que grows until all the memory is in this que. I have attached a xml = > sheet that shows the memory usage in the system. Inactive memory is not freed, it makes no sense as far as there is valid co= ntent, which is either not recoverable (anon memory or dirty file pages) or high-cost to restore (clean file pages, need to re-read from disk= ). Inactive is processed by pagedaemon when system has memory deficit, and= either inactive pages are written to swap, or written to their file backin= g storage. I suspect that you have other activities on your system going on, which cau= se creation of the inactive memory and unrecoverable fragmentation. Note that contigmalloc() tries to do defragmentation to satisfy requests, b= ut this is not always possible. > Ciunas. > = > -----Original Message----- > From: Konstantin Belousov [mailto:kostikbel@gmail.com] > Sent: Friday, October 26, 2018 9:13 PM > To: Bennett, Ciunas > Cc: freebsd-stable@freebsd.org > Subject: Re: Possible memory leak in the kernel (contigmalloc) > = > On Wed, Oct 24, 2018 at 04:27:52PM +0000, Bennett, Ciunas wrote: > > Hello, > > = > > I have encountered an issue with a kernel application that I have = > > written, the issue might be caused by a memory leak in the kernel. > > The application allocates and deallocates contiguous memory using > > contigmalloc() and contigfree(). The application will fail after a = > > period of time because there is not enough free contiguous memory = > > left. There could be an issue with the freeing of memory when using = > > the contigfree() function. > > > = > It is unlikely that there is an issue with a leak, but I would be not sur= prised if your allocation/free pattern would cause fragmentation on free li= sts that results in contigmalloc(9) failures after. > = > Look at the vmstat -z/vmstat -m output to see uma and malloc stats. > More interesting for your case can be the output from > sysctl vm.phys_free > which provides information about the free queues and order of free pages = on them. > -------------------------------------------------------------- > Intel Research and Development Ireland Limited Registered in Ireland = > Registered Office: Collinstown Industrial Park, Leixlip, County = > Kildare Registered Number: 308263 > = > = > This e-mail and any attachments may contain confidential material for = > the sole use of the intended recipient(s). Any review or distribution = > by others is strictly prohibited. If you are not the intended = > recipient, please contact the sender and delete all copies. -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the s= ole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact = the sender and delete all copies.