From owner-freebsd-current@freebsd.org Tue Mar 15 06:40:41 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2714FAD1981 for ; Tue, 15 Mar 2016 06:40:41 +0000 (UTC) (envelope-from ohauer@gmx.de) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0988B767 for ; Tue, 15 Mar 2016 06:40:41 +0000 (UTC) (envelope-from ohauer@gmx.de) Received: by mailman.ysv.freebsd.org (Postfix) id 09C3AAD197B; Tue, 15 Mar 2016 06:40:41 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0957BAD197A for ; Tue, 15 Mar 2016 06:40:41 +0000 (UTC) (envelope-from ohauer@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 720E1764; Tue, 15 Mar 2016 06:40:39 +0000 (UTC) (envelope-from ohauer@gmx.de) Received: from [192.168.100.100] ([87.139.233.65]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MVvi4-1aLxmZ3OXo-00X71V; Tue, 15 Mar 2016 07:40:31 +0100 Subject: Re: how to recycle Inact memory more aggressively? To: "current@freebsd.org" References: <20160312093835.727d7197@ernst.home> <20160314013319.GA68039@wkstn-mjohnston.west.isilon.com> <20160314015104.GB68039@wkstn-mjohnston.west.isilon.com> <1457965197.9986.12.camel@freebsd.org> Cc: Ian Lepore , Adrian Chadd , Mark Johnston , Gary Jennejohn From: olli hauer Reply-To: FreeBSD Current Message-ID: <56E7AF22.1020405@gmx.de> Date: Tue, 15 Mar 2016 07:43:46 +0100 User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <1457965197.9986.12.camel@freebsd.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:KnNh9/TAtM4NJAFkdIJ8qnCrtQ2yXUzhCe5XGIh3SxuYif0r386 2b2qTdOLvqGcmpmNpIO+Xyj3gjPUIsaj+w/gzfsMLNe00CcRahFYRGIjW1GfY9hHdq+nuNF pKboEdNnn69iY1YDVxKJiCiN5AOqv+UQVHeS8LLhAqj7REgIjpHpgT6Kqqr6srkNxDhGtvD VSELXzglEJkIl38+ZnteQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:tI3pihujzIg=:qS7CczRdGOHQ1umzBGdSEt q1qW24o57Z2Mbp4GIYbTESQgWTXxYGeFGBl4pcB9xe34RZSDr3JjlvrUYYtkBgynXVZ1bKq5d sbeKEKItmz1OKvjTAknhdpEw84rElrSnDLBuoHN18ZN67mthlxwuw5y9XTV6rJeh6dsrZBnoO SYo9iaZDnYskKpX/NEClYXWI8q3u5UCmOGDq+eGqHe9yFCbWMPUUnnhq1NNPYJOfmHV3kn/ge aQhMVVcq7rZ7kT1vnvocEAsVVSFQ7KX/d/+mfudUrBmEg/SrmuZOL0sqyEJyvMkfWvLiC04si CMDBlwblj5votBj1i/L8x/H+lV7mGgV1X3ZOMi/XSxEhKcB5gnLoYjpud2tbo2s5gzQvMFdOd gnDbdkJRqkkCPSO2nvzlvCrHcpomkRrIDNTi2K+M67E9HDVvmY8CTtmCzB0m4S0VeWKzraW1l oR2NefnFeTsnQ90wYidPD4nz6Sf71aT1o6dIrdwxpuaGso1gmqRYMl8tucFrisQulqO4afGGo kV59N0ixmzbtOB0/bxF/AK91su/cbq2zWVPtXYdR9NenegNQJSmr8Qu1HhyAzEcLFoAIHIjWQ so33TUUWsEfmUcI1mB4sq+SCA1EAWlqBpaAqwYjmSQcUKbsL/IeTU2PkOsPN3FWlVEO93kAgo JSTth7hWQsyFQHQ1jn4S11WhF7IbdRNaCE8C7WVanRXROIzUUMfXmv6mkPrOdwnglvgZC2Oxn wdcgxpbaK6t/K1Pqmy8ObQGa4dzHAZ32N+tNKnjRTS2obNExEeUyTyg9eoG2l69KjzzMFkrPe eAC9r+Q X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2016 06:40:41 -0000 On 2016-03-14 15:19, Ian Lepore wrote: > On Sun, 2016-03-13 at 19:08 -0700, Adrian Chadd wrote: >> On 13 March 2016 at 18:51, Mark Johnston wrote: >>> On Sun, Mar 13, 2016 at 06:33:46PM -0700, Adrian Chadd wrote: >>>> Hi, >>>> >>>> I can reproduce this by doing a mkimage on a large destination >>>> file >>>> image. it looks like it causes all the desktop processes to get >>>> paged >>>> out whilst it's doing so, and then the whole UI freezes until it >>>> catches up. >>> >>> mkimg(1) maps the destination file with MAP_NOSYNC, so if it's >>> larger >>> than RAM, I think it'll basically force the pagedaemon to write out >>> the >>> image as it tries to reclaim pages from the inactive queue. This >>> can >>> cause stalls if the pagedaemon blocks waiting for some I/O to >>> complete. >>> The user/alc/PQ_LAUNDRY branch helps alleviate this problem by >>> using a >>> different thread to launder dirty pages. I use mkimg on various >>> desktop >>> machines to build bhyve images and have noticed the problem you're >>> describing; PQ_LAUNDRY helps quite a bit in that case. But I don't >>> know >>> why this would be a new problem. >>> >> >> That's why I'm confused. I just know that it didn't used to cause the >> whole UI to hang due to paging. >> > > I've been noticing this too. This machine runs 10-stable and this use > of the swap began happening recently when I updated from 10-stable > around the 10.2 release time to 10-stable right about when the 10.3 > code freeze began. > > In my case I have no zfs anything here. I noticed the problem bigtime > yesterday when rsync was syncing a ufs filesystem of about 500GB from > one disk to another (probably 70-80 GB actually needed copying). My > desktop apps were noticibly unresponsive when I activated a window that > had been idle for a while (like it would take a couple seconds for the > app to begin responding). I could see lots of swap In happening in top > during this unresponsiveness, and noticible amounts of Out activity > when nothing was happening except the rsync. > > This is amd64, 12GB ram, 16GB swap, a tmpfs had about 400MB in it at > the time. Prior to the update around the 10.3 freeze, this machine > would never touch the swap no matter what workload I threw at it (this > rsync stuff happens every day, it's the usual workload). > I'm not sure if it is the same problem, or port related. On two systems without zfs but with many files e.g. svn servers I see now from time to time they are running out of swap. kernel: swap_pager_getswapspace(9): failed kernel: swap_pager_getswapspace(16): failed ... It also happened on one system during the nightly periodic tasks holding only millions of backup files. $ freebsd-version -ku 10.2-RELEASE-p9 10.2-RELEASE-p13