From nobody Fri Mar 25 13:06:55 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3BDD71A463A0 for ; Fri, 25 Mar 2022 13:07:05 +0000 (UTC) (envelope-from markoml@markoturk.info) Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KQ2Rr0sb5z3CwP for ; Fri, 25 Mar 2022 13:07:04 +0000 (UTC) (envelope-from markoml@markoturk.info) Received: by mail-ed1-f45.google.com with SMTP id h4so1407048edr.3 for ; Fri, 25 Mar 2022 06:07:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition; bh=QWCCq5Scpx8q5I3keFBYXkD0i2Z0+YhRAl7yLSXkGMI=; b=Be734bzSZtnGI1E3pf3YAyXJOKe+GqEpzk5K12G6ErjcwQrSQ4q/Q+dLQl1dhSBxkk 2a4MrfSpbxwkgMPYQ9kYE+lEmh7rqOcsCSQwlUeKy186CrhCYVkUSN6HsauqKXU8sXjj 7ldPxVMzn6K9apYoLhLxBBLdbv/3UxsaBlboHBm8Wc3ej2asXtQjinO6OY0LzKdFdSUy T6nkLtYQlL0LUzM5wZLlW3oHz8CMF4WMVIvD/XR4QDL8UYrTTBZNWTzJTh9h0CEgsffP pdCUCbhoFa0VZSZERRlDlWh7nLyf/JiWNTkqf0cuZJbjtooHVXUepFwPm2z8siZhaXTn hGOg== X-Gm-Message-State: AOAM531+0Lv2YWdXzL9b71axlRmrOuUS8MMd6uOUHzOZa7RZseGeLdUI TFijPHE119HUSBztFLHbIwpZ7c3Oa9zUrMEz X-Google-Smtp-Source: ABdhPJxGuQacBS/sNymbfmOkjWUSBy3/s5fSlAVVBGfj0rw/qu/4OVn7eSDDm/TdsxlLt51O/+421g== X-Received: by 2002:a05:6402:42c6:b0:419:276c:4a64 with SMTP id i6-20020a05640242c600b00419276c4a64mr13004765edc.119.1648213617450; Fri, 25 Mar 2022 06:06:57 -0700 (PDT) Received: from vps.markoturk.info (cpe-109-60-8-3.st3.cable.xnet.hr. [109.60.8.3]) by smtp.gmail.com with ESMTPSA id dm8-20020a170907948800b006dfe5b317d3sm2447403ejc.75.2022.03.25.06.06.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Mar 2022 06:06:56 -0700 (PDT) Date: Fri, 25 Mar 2022 14:06:55 +0100 From: Marko Turk To: freebsd-hackers@freebsd.org Subject: Inactive vs ZFS ARC memory priority Message-ID: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="bXArPL3y0jOtpkLG" Content-Disposition: inline X-Rspamd-Queue-Id: 4KQ2Rr0sb5z3CwP X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 209.85.208.45 is neither permitted nor denied by domain of markoml@markoturk.info) smtp.mailfrom=markoml@markoturk.info X-Spamd-Result: default: False [-5.20 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[109.60.8.3:received]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; MID_RHS_MATCH_FROMTLD(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_NA(0.00)[markoturk.info]; RCVD_IN_DNSWL_NONE(0.00)[209.85.208.45:from]; NEURAL_HAM_MEDIUM(-1.00)[-0.996]; MLMMJ_DEST(0.00)[freebsd-hackers]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_TLS_ALL(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.208.45:from] X-ThisMailContainsUnwantedMimeParts: N --bXArPL3y0jOtpkLG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I have a workstation with 32GB or RAM, running recent 12.3-STABLE. vfs.zfs.arc_max is set to "16G". After a few hours of uptime I get this memory usage: Mem: 327M Active, 3007M Inact, 58M Laundry, 20G Wired, 7986M Free ARC: 15G Total, 1723M MFU, 12G MRU, 9032K Anon, 246M Header, 987M Other 13G Compressed, 31G Uncompressed, 2.44:1 Ratio Swap: 4096M Total, 4096M Free All looks good, ZFS ARC is used almost completely up to arc_max. And it stays like this for days. Then I do a make buildworld & buildkernel. And I get this: Mem: 663M Active, 20G Inact, 879M Laundry, 8560M Wired, 1445M Free ARC: 4351M Total, 984M MFU, 2176M MRU, 14M Anon, 37M Header, 1140M Other 2453M Compressed, 5405M Uncompressed, 2.20:1 Ratio Swap: 4096M Total, 4096M Free A lot of memory is in Inactive, and ARC decreased. I think this is still OK. But what I expect to happen eventually is that ARC will grow again and push old data from Inactive. This never happens. I can leave it for days with a lot of disk activity and nothing changes. Seems like old data in Inactive has higher priority than ARC. Why is that so? Can I improve this somehow? If I do something memory intesive, Inacive will decrease. This works as expected, I get something like this: Mem: 659M Active, 16G Inact, 851M Laundry, 9337M Wired, 4343M Free ARC: 5144M Total, 1207M MFU, 2712M MRU, 7275K Anon, 82M Header, 1135M Other 2885M Compressed, 6647M Uncompressed, 2.30:1 Ratio Swap: 4096M Total, 4096M Free After that the ARC will grow and fill the empty memory. But on it's own it will never push Inactive down. Why is old data in Inactive higher priority than new data in ARC? Regards, Marko --bXArPL3y0jOtpkLG Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEhGJv+o0FoHlUFxVJ0ArzBirlGugFAmI9vm4ACgkQ0ArzBirl GugvwQ//fVxQFFpUE9fV0xS1GcwB67vveJvslHTl/pfGRUXJP8AiURNNP9x0nRwI howvNB4/yVxnOCYphQPORN+NQP9Y42+/ga7ZLyQMZ4AgmprjWQyQaY0LUWDQ56dA SZSZPevFGVSblL2tyecmC0usWFV7aT3Hyy0NxmShwxs2d+UC1muWK3zRuGeYcrvq QW7vcKUMtpEXKhoU8+8wT5YqueaezadDq44rEdxBSba1sIFEOxZITsAlMqljiOoY R03dOashQ4SROzNKnfnTX054OwDQMBdSROP00Vg4VpNHzcbE9WLxZ+IiPg8da+Ha MQTIjJJB9rhF3Xy3MvEzkb3LqEJsADOMA1iaFPaF+iNGsiH1fAhRy1/4BHzu7U42 ACqs+eYhdEoSqZqW4pHPPEr1IDw/ADfF/zREK1CT+of4vInDOq1DDbsivK4OT/K8 GMlWIwjPvJ2FSm4UYmxsI95Uz/FVdYf/1qILFEwK3ixm2/GXcSzW1V5OfZWvXkyF 073zGFMOmQbevunjHplRwH6cqtunhoY6UAnYVI4vVCTxqABnEIkr2yJQhu5Ia4EY bUtyzcZhvynkOwushG7Q+1EbVtaocCIxivv6GN8rwznT8qf2JrzRW3hYtUp54deA 7wMrwmFrw3qMudViORovtSYkkONWFMrmOY5B3R7Up6bWm1ghFM8= =k1mG -----END PGP SIGNATURE----- --bXArPL3y0jOtpkLG--