From owner-freebsd-questions@freebsd.org Wed Dec 30 22:53:25 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0A4684D1E8B for ; Wed, 30 Dec 2020 22:53:25 +0000 (UTC) (envelope-from doug@fledge.watson.org) Received: from cyrus.watson.org (cyrus.watson.org [204.107.128.30]) by mx1.freebsd.org (Postfix) with ESMTP id 4D5mm40NSsz4njd for ; Wed, 30 Dec 2020 22:53:24 +0000 (UTC) (envelope-from doug@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [198.74.231.63]) by cyrus.watson.org (Postfix) with ESMTPS id 4199A5FC83 for ; Wed, 30 Dec 2020 22:53:17 +0000 (UTC) Received: from fledge.watson.org (doug@localhost [127.0.0.1]) by fledge.watson.org (8.16.1/8.16.1) with ESMTPS id 0BUMrH6u088178 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 30 Dec 2020 22:53:17 GMT (envelope-from doug@fledge.watson.org) Received: from localhost (doug@localhost) by fledge.watson.org (8.16.1/8.16.1/Submit) with ESMTP id 0BUMrH19088174 for ; Wed, 30 Dec 2020 22:53:17 GMT (envelope-from doug@fledge.watson.org) Date: Wed, 30 Dec 2020 21:46:47 +0000 (UTC) From: doug Reply-To: doug@safeport.com To: "Steve O'Hara-Smith" Subject: Re: Observations on virtual memory operations In-Reply-To: <20201230060647.38938a75f69e6c045802f655@sohara.org> Message-ID: References: <167603f-a82a-7031-6850-2d08f17a36@fledge.watson.org> <8f3a278a-56cd-c732-68a0-cf6fa5d50a3f@nomadlogic.org> <03553c65-c1c2-db2c-6ab9-a0f4d09c3e2d@nomadlogic.org> <20201230060647.38938a75f69e6c045802f655@sohara.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed ReSent-Date: Wed, 30 Dec 2020 22:52:47 +0000 (UTC) ReSent-From: doug ReSent-To: freebsd-questions@FreeBSD.ORG ReSent-Subject: Re: Observations on virtual memory operations ReSent-Message-ID: <1a1be3d6-15fc-d581-292b-da8b47c2d6@fledge.watson.org> X-Rspamd-Queue-Id: 4D5mm40NSsz4njd X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of doug@fledge.watson.org has no SPF policy when checking 204.107.128.30) smtp.mailfrom=doug@fledge.watson.org X-Spamd-Result: default: False [0.63 / 15.00]; HAS_REPLYTO(0.00)[doug@safeport.com]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[204.107.128.30:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; DMARC_NA(0.00)[watson.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[204.107.128.30:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; NEURAL_SPAM_MEDIUM(0.71)[0.707]; NEURAL_HAM_SHORT(-0.08)[-0.079]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11288, ipnet:204.107.128.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-questions] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Dec 2020 22:53:25 -0000 On Wed, 30 Dec 2020, Steve O'Hara-Smith wrote: > On Wed, 30 Dec 2020 01:03:05 +0000 (UTC) > doug@safeport.com wrote: > >> I do not seem to be able to say clearly enough, there is no memory >> problem, the is a problem with the paging subsystem filling up swapspace. > > Are you using tmpfs or swap backed md ? > Neither: swapctl -l Device: 1024-blocks Used: /dev/aacd0p3 4194304 1857780 from vmstat: procs memory page faults cpu r b w avm fre flt re pi po fr sr in sy cs us sy id 2 0 16 13G 1.7G 2557 0 0 0 3044 435 509 5523 875 7 1 93 0 0 16 14G 1.5G 8582 0 0 0 7366 428 491 6410 731 6 1 92 0 0 16 14G 1.5G 2945 0 0 0 2934 480 511 4313 780 6 1 93 0 0 16 14G 1.6G 3795 0 0 0 3950 457 434 4910 716 5 1 94 1 0 16 14G 1.6G 589 0 0 0 1128 431 777 2128 1481 2 1 98 2 0 16 14G 1.6G 3273 0 1 0 3329 446 715 5871 1266 5 1 94 And the perl pgm that develops counts from vm.stats.vm SYSTEM MEMORY INFORMATION: mem_wire: 1520701440 ( 1450MB) [ 18%] Wired: disabled for paging out mem_active: + 957276160 ( 912MB) [ 11%] Active: recently referenced mem_inactive:+ 3454586880 ( 3294MB) [ 41%] Inactive: recently not referenced mem_cache: + 0 ( 0MB) [ 0%] Cached: almost avail. for allocation mem_free: + 1823854592 ( 1739MB) [ 21%] Free: fully available for allocation mem_gap_vm: + 549380096 ( 523MB) [ 6%] Memory gap: UNKNOWN -------------- ------------ ----------- ------ mem_all: = 8305799168 ( 7921MB) [100%] Total real memory managed mem_gap_sys: + 240508928 ( 229MB) Memory gap: Kernel?! -------------- ------------ ----------- mem_phys: = 8546308096 ( 8150MB) Total real memory available mem_gap_hw: + 43626496 ( 41MB) Memory gap: Segment Mappings?! -------------- ------------ ----------- mem_hw: = 8589934592 ( 8192MB) Total real memory installed SYSTEM MEMORY SUMMARY: mem_used: 3311493120 ( 3158MB) [ 38%] Logically used memory mem_avail: + 5278441472 ( 5033MB) [ 61%] Logically available memory -------------- ------------ ----------- ------ mem_total: = 8589934592 ( 8192MB) [100%] Logically total memory The system I have the save a command on is older so an apache/php memory leak is not out of the question. The dipicted here is 11.3, apache24 and php7. I keep this one up by restarting one or more jails with swap space usage >50%. When it so it is almost instantaneous in both cases.