From nobody Mon Jul 18 12:20:07 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Lmgyl193Dz4Syvl for ; Mon, 18 Jul 2022 12:20:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Lmgyk40zKz3Vky for ; Mon, 18 Jul 2022 12:20:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 26ICK7Ut029470 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 18 Jul 2022 15:20:10 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 26ICK7Yg029469; Mon, 18 Jul 2022 15:20:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 18 Jul 2022 15:20:07 +0300 From: Konstantin Belousov To: Arka Sharma Cc: freebsd-hackers@freebsd.org Subject: Re: Private resident count in procstat(1) Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4Lmgyk40zKz3Vky X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-2.07 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; NEURAL_HAM_MEDIUM(-0.08)[-0.079]; MIME_TRACE(0.00)[0:+]; FREEMAIL_TO(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_XAW(0.00)[]; TAGGED_RCPT(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; ARC_NA(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, Jul 18, 2022 at 12:30:59PM +0530, Arka Sharma wrote: > Hi All, > > I tracked the PRES field and observed that 'kve->kve_private_resident' is > the count of vm_page contained by vm_object if the vm_map entry doesn't > contain a shadow object. My question is why this field is called > private resident as I understand the underlying vm_object will also contain > pages which correspond to other map entries for the same process, and for > shared mapping it could be referenced by map entries with other process as > well. > > For the RES field my understanding is, it is the sum of pages of shadow > objects and the tail of the object chain for the range of the given map > entry and we also have 'ki_rssize' which is 'pm_stats.resident_count' plus > the sum of kernel stack pages of the threads. Is there a situation the RES > value will be greater than 'ki_rssize' ? > > Please correct me if above understanding is wrong. The definition of the resident and 'private resident' per-process count are quite arbitrary. I do not think it is possible for Mach VM to have a single definition that would serve all desired uses. Whatever the algorithm to calculate the values is chosen, it should be good enough for common case, while not penalize corner situations by too consuming computations. For instance, if we have a shadow chain behind the top object, should the resident pages in the objects below the top accounted to RSS? Should shadowed pages be accounted, and if not, where their count should go? They do consume memory, leaving less pages available to the rest of the system. On the other hand, if we account them to each address space that this shadow participates in, we would do over-accounting. Another issue is that it is not possible to determine fast if the mapping is shared, or truely shared. Imagine MAP_SHARED anonymous mapping in the process that never forked. Or tmpfs mapped file with no other mappings. In all that cases formally the mappings are shared, but it is very hard to prove that there is no other mappings. More, there are at least three places that look at RSS in kernel: - the pure userspace reporting in kern.proc.vmmap - OOM code that looks for the largest process - swapout code that selects a process to swap out Each place that (slightly) different definition of the residency.