From owner-freebsd-questions@freebsd.org Sun Oct 9 13:12:58 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C912C06A12 for ; Sun, 9 Oct 2016 13:12:58 +0000 (UTC) (envelope-from emorrasg@yahoo.es) Received: from nm36-vm5.bullet.mail.ir2.yahoo.com (nm36-vm5.bullet.mail.ir2.yahoo.com [212.82.97.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DAEA4FF1 for ; Sun, 9 Oct 2016 13:12:57 +0000 (UTC) (envelope-from emorrasg@yahoo.es) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.es; s=s2048; t=1476018599; bh=1z+elOom3IX24caXlPLTUUim/F4y7dvtPH77WUanS+k=; h=Date:From:To:Subject:In-Reply-To:References:From:Subject; b=OeBVAmo1e06w+oMMt1dh7DTIilWFjGXjzXaN7SQvmy9A0cbZ1E8nSVOf38KhQHbNGDpx5GTn/jjr/tNCnd2L6S/tgFi589GMVzPvImEBJfPp3eCawh1Pu9TF5Om6NNtPH9Wm0eWQvrAcQt3Lflxk36voXynPmURV38DGEs+c8lpDmG5x9thQuyCr/YEL17e8lkWWJ8C93D/5LPqSqKH2Snzz1GZSNdoccUSNrNx47YD+0P9pJiEBzHZjHeYjmApE700+G427N8jBoWnlQPc/wLXA/XfApAq1JuRBEnrD6klGe5FSgLq/A2q/IB4dgNBIHw+13AkOzMiAy7AyjPG34w== Received: from [212.82.98.55] by nm36.bullet.mail.ir2.yahoo.com with NNFMP; 09 Oct 2016 13:09:59 -0000 Received: from [46.228.39.73] by tm8.bullet.mail.ir2.yahoo.com with NNFMP; 09 Oct 2016 13:09:52 -0000 Received: from [127.0.0.1] by smtp110.mail.ir2.yahoo.com with NNFMP; 09 Oct 2016 13:09:52 -0000 X-Yahoo-Newman-Id: 554727.60658.bm@smtp110.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 4OM.ziMVM1mvFpH8EYsGFBqXdF.zYEf5.fKAMidS12E6VkQ oBH8C4MfFhiJ5G89GN0HLps4z9QWJ8Hib4d1w7xP1oFSTx9grJveyufbIgJM KRZZOwK3mM8YdUPXi8V30lc76K8RhPVgdGBU9b8NnIfBpzQGR7yeHffTl4xQ BswnkB9aQvMdi5hLXQBwyWQRMSJWU7udNsGubQx.NXe0crBjtzU7ZJDdnV3V 5LNo3s8HZ2RlSiSnMUbqokByUQb2Q5qg6sYLheUjKYhCtnTmsleevO5e4bxr 4E6ZldNh3evmYt1JfrA5fGXTiHykq4LhobsCL1fB9tW_JCCtdLxqipIu4E8I pqDEMQEjI0UAQcsAQppaegr6WxHorX29zyZ_uUeysxw4y7Dhpw9mfDl7o2TX oDC2HdWI5b5.J_Eyf7bdJbes__b7q_uTzEdRzaqOiXSeTqQ7XL9NJF7iwquB FqSjcdG2BKq4bOR7gB4LtBTkA7rRyc9.twqnKBA8SdIuHf3z5fQpm9EiIOmq R1C.sYKZ4ZUgYxUNGo7njlLcixeAvx0RsvmbFEYrx9YZPKkjThaYAcWV46TQ BHBE7OL3qzml33M1YPwS7hcYLYJ1v_u9SoyycrQT_ENSR.ifhARb.GUkvy7c - X-Yahoo-SMTP: mX392iiswBAeJNdO_s.EW62LZDJR Date: Sun, 9 Oct 2016 15:09:53 +0200 From: Eduardo Morras To: freebsd-questions@freebsd.org Subject: Re: Swap exhaustion Message-Id: <20161009150953.8bb8fee6360710c4eb54c945@yahoo.es> In-Reply-To: <9DB5B54C-2AE4-4200-AB78-F86913E1E433@mail.sermon-archive.info> References: <1CD13C1C-5344-4909-A061-F25FBB86AFF9@lafn.org> <9DB5B54C-2AE4-4200-AB78-F86913E1E433@mail.sermon-archive.info> X-Mailer: Sylpheed 3.5.1 Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Oct 2016 13:12:58 -0000 On Sat, 8 Oct 2016 18:33:47 -0700 Doug Hardie wrote: > > > On 27 May 2015, at 16:49, Doug Hardie wrote: > > > > I have a process that is eating up 6 GB of swap space. At that > > point, FreeBSD 9.3 terminates a process. However, occasionally its > > not the one eating up the space. When I manually quit the process > > then the swap space returns to a few KB used. The system runs fine > > after that. > > > > I have very little knowledge of what this process is doing > > internally but would like to know what might be causing this > > issue. There are 5 of these processes running (parent plus 4 > > children). Normally each uses about 90 MB RES/SIZE. However when > > the problem occurs they are about 2GB RES/SIZE each. The system > > has 4 GB memory. I thought that a malloc would not be able to grab > > that much memory, even with swapping as it has to be in memory. > > Could a malloc cause this growth in process size or need I look > > elsewhere? > > I have been trying periodically to resolve this issue. I modified > the application to log (syslog) every memory location it allocates. > Interestingly enough they are all around 0x100F380. However, the > segments that are increasing in number all have the highest order > address bit set, i.e., 0x83ac00000. These are shown by procstat as > type "df". The number of these increases with time. The sizes vary > from what appears to be small to over 5000 resident pages. They > never seem to go away unless I restart the process. Right after a > restart there are 7 "df"s. However, with time there can eventually > be hundreds. Since these do not have any file backing them, they eat > up swap space and thus cause the system to run out of swap and start > killing processes. > > What things allocate memory at the top of the address space? The > application mallocs and mmaps all allocate much lower in the address > space. FreeBSD 9.3 uses (afair) jemalloc 3.4. There are tons of bug fixes in newer versions, last one 4.2.1. Perhaps you were bitten by one. There is not a port :( but compiles nicely under FreeBSD 10.3 (don't tried under 9.3). https://github.com/jemalloc/jemalloc/blob/dev/ChangeLog http://jemalloc.net/ --- --- Eduardo Morras