From owner-freebsd-current@freebsd.org Wed Dec 13 11:23:49 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07843E9A6CA for ; Wed, 13 Dec 2017 11:23:49 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C6C5A6C798 for ; Wed, 13 Dec 2017 11:23:48 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.89_1 (FreeBSD)) (envelope-from ) id 1eP58c-0000Yp-WA; Wed, 13 Dec 2017 11:23:47 +0000 Date: Wed, 13 Dec 2017 11:23:46 +0000 From: Gary Palmer To: blubee blubeeme Cc: Mark Millard , FreeBSD Current Subject: Re: get_swap_pager(x) failed Message-ID: <20171213112346.GA27816@in-addr.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 11:23:49 -0000 On Wed, Dec 13, 2017 at 10:34:02AM +0800, blubee blubeeme wrote: > On Wed, Dec 13, 2017 at 5:53 AM, Mark Millard wrote: > > > blubee blubeeme gurenchan at gmail.com wrote on > > Tue Dec 12 15:58:19 UTC 2017 : > > > > > On Tue, Dec 12, 2017 at 3:34 PM, blubee blubeeme > > >wrote: > > > > I am seeing tons of these messages while running tail -f > > /var/log/messages > > > > ============ > > > > Dec 12 15:11:41 blubee kernel: swap_pager_getswapspace(25): failed > > > . . . > > > > 1159 blubee 5 20 0 149M 56876K select 6 1:05 0.00% > > > > ibus-engine-chewing > > > > > > > > =========== > > > > > > > > What's with all the swap errors? I am running ZFS and I have 16GB of > > ram, > > > > how could I be having swap space errors? > > > > > > > > > > Well I added 4GB of extra swap in /var/tmp/swap0 > > > then added that to my /etc/fstab: md99 none swap > > > sw,file=/var/tmp/swap0,late 0 0 > > > > > > and those errors went away. > > > > I recommend reviewing bugzilla 206048 (title in part > > "swapfile usage hangs; swap partition works"): > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206048 > > > > before using file-system based swap spaces. They have > > lots of problems with deadlocks. See especially comments > > #7 and #8 quoting Konstantin Belousov. #8 is just a > > reference to: > > > > https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/ > > kerneldebug-deadlocks.html > > > > Comment #3 shows a way to test for the problematical > > behavior. > > > > Using swap partitions avoids the issue. > > > > === > > Mark Millard > > markmi at dsl-only.net > > > > > Thanks for the info, why would I be getting swap errors like that when I > have 32GB of ram? > sysctl hw.physmem > hw.physmem: 34253692928 > > That really doesn't make any sense to me... Is it Chromium eating up 32GB+ > of ram? Your top output from the first message in the thread gives a hint: Mem: 3442M Active, 293M Inact, 4901M Laundry, 22G Wired, 1057M Free ARC: 18G Total, 1488M MFU, 15G MRU, 4003K Anon, 178M Header, 1171M Other 16G Compressed, 23G Uncompressed, 1.41:1 Ratio Swap: 2048M Total, 2028M Used, 20M Free, 99% Inuse, 36K In You have 22GB RAM used by the kernel, and 18GB of that is used by ZFS. As you can see from the last line, 99% of your small swap space is used and paging was happening at the time of the snapshot ("36K In"). First I would suggest limiting ARC and seeing if that helps. Unless you are doing a lot of fileserver type work or working with lots of files you want to keep around that ARC size is a bit big. My desktop with 32GB RAM has ARC limited to 4GB. Definitely do NOT swap to a file on a filesystem, especially if the filesystem is ZFS. That will eventually lead to a deadlock. An open question would be why ARC is not reducing if the system is under memory pressure. It's meant to, but there have been various bugs in that implementation. Regards, Gary