From owner-freebsd-stable@freebsd.org Sun Feb 4 01:25:51 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FCA5EDBA00 for ; Sun, 4 Feb 2018 01:25:51 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AB7D8532A for ; Sun, 4 Feb 2018 01:25:50 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w141Phmq031245 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 4 Feb 2018 02:25:44 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: mvoorhis@mcvau.net Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w141PXQE079059 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sun, 4 Feb 2018 08:25:33 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 50 percent swap used, but "ps auxww" output shows no processes swapped out To: Michael Voorhis , freebsd-stable@freebsd.org References: <5A763C48.5020907@grosbein.net> <798bb18f-cdd6-f30a-5f21-832032a15d52@mcvau.net> From: Eugene Grosbein Message-ID: <5A766108.7040502@grosbein.net> Date: Sun, 4 Feb 2018 08:25:28 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <798bb18f-cdd6-f30a-5f21-832032a15d52@mcvau.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Feb 2018 01:25:51 -0000 04.02.2018 6:42, Michael Voorhis wrote: > 1 frame of your requested "top" output, sorted as specified: > >> last pid: 47195; load averages: 0.17, 0.37, 0.44 up 99+20:40:41 18:37:07 >> 369 processes: 1 running, 368 sleeping >> CPU: 0.2% user, 0.0% nice, 0.2% system, 0.0% interrupt, 99.7% idle >> Mem: 6989M Active, 79G Inact, 27G Laundry, 10G Wired, 1568M Buf, 1680M Free >> Swap: 21G Total, 11G Used, 10G Free, 50% Inuse >> >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND >> 61839 pgsql 1 21 0 32665M 31349M select 13 3:13 0.00% postgres >> 59354 pgsql 1 20 0 32659M 26625M select 12 288:57 0.00% postgres >> 40354 pgsql 1 20 0 32660M 26182M select 12 220:15 0.00% postgres >> 24708 pgsql 1 20 0 32659M 26144M select 3 204:14 0.00% postgres >> 37664 pgsql 1 20 0 32649M 25967M select 12 48:55 0.00% postgres >> 56094 pgsql 1 25 0 32659M 25734M select 5 189:00 0.00% postgres >> 57255 pgsql 1 20 0 32661M 25414M select 6 143:53 0.00% postgres >> 38662 pgsql 1 20 0 32659M 23669M select 1 229:03 0.00% postgres >> 48035 pgsql 1 20 0 32663M 22195M select 3 149:13 0.00% postgres >> 66831 pgsql 1 20 0 32659M 21944M select 2 61:00 0.13% postgres >> 45832 pgsql 1 20 0 32659M 21427M select 4 158:26 0.00% postgres >> 56976 pgsql 1 20 0 32661M 14110M select 1 56:26 0.00% postgres >> 26207 pgsql 1 20 0 32659M 11620M select 8 34:21 0.00% postgres >> 383 www 64 20 0 34010M 8860M nanslp 14 11:26 0.11% jsvc >> 44067 pgsql 1 20 0 32659M 6194M select 11 15:16 0.00% postgres >> 31431 pgsql 1 20 0 32661M 4928M select 15 15:41 0.00% postgres >> 34641 pgsql 1 20 0 32650M 2383M select 14 2:04 0.00% postgres> [...] Well, you have tens of processes each allocating about 32G of memory. Some part of that memory may be shared but some is not not. No wonder there are tens gigabytes of private pages shown processes have and some of that pages were paged out to swap area in moments when kernel was short of free pages (note small "Free" number). Later when some memory was freed, those pages were NOT get back from the swap because there were no demand for their contents. That's pretty normal. For example, PIDs 56976 or 26207 or 26207 all allocated over 30G and may be touched over 20G (let's assume that) but their RESident size is considerably less, so rest was paged out to swap.