From nobody Mon Nov 15 18:17:28 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2CBD31893E9B for ; Mon, 15 Nov 2021 18:17:36 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HtHV73sxsz3k5Z; Mon, 15 Nov 2021 18:17:35 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-48-165-176.washdc.fios.verizon.net [108.48.165.176]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Mon, 15 Nov 2021 18:17:34 +0000 Received: from smtpclient.apple ( [2001:420:c0c8:1007::50b]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id 47e68299 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Mon, 15 Nov 2021 13:17:32 -0500 (EST) Content-Type: text/plain; charset=utf-8 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: swap_pager: cannot allocate bio From: Chris Ross In-Reply-To: Date: Mon, 15 Nov 2021 13:17:28 -0500 Cc: Mark Johnston , freebsd-fs Content-Transfer-Encoding: quoted-printable Message-Id: <471B80F4-B8F4-4D5A-9DEB-3F1E00F42A68@distal.com> References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> <19A3AAF6-149B-4A3C-8C27-4CFF22382014@distal.com> <6DA63618-F0E9-48EC-AB57-3C3C102BC0C0@distal.com> <35c14795-3b1c-9315-8e9b-a8dfad575a04@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Rspamd-Queue-Id: 4HtHV73sxsz3k5Z X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_FROM(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N > On Nov 15, 2021, at 10:08, Andriy Gapon wrote: > My point was that waiting for the free memory was not strictly needed = yet given 12G free, but that's kind of obvious. I am kind of curious about this. You were noting Andriy that there was = free emory. Is there any way to confirm if this is a NUMA issue, and if = it is, would there be any way around it? Mark, you mentioned making = that less strict as a general improvement, but not a real fix to the = issue that=E2=80=99s hitting me here. I am curious if you thought it = was still worth making. It sounds like it to me. I mean, allocating = memory at all sounds better than failing to allocate memory. Are there = cases where that wouldn=E2=80=99t be true? > Yes, I propose to remove the wait for ARC evictions from arc_lowmem(). >=20 > Another thing that may help a bit is having a greater "slack" between = a threshold where the page daemon starts paging out and a threshold = where memory allocations start to wait (via vm_wait_domain). >=20 > Also, I think that for a long time we had a problem (but not sure if = it's still present) where allocations succeeded without waiting until = the free memory went below certain threshold M, but once a thread = started waiting in vm_wait it would not be woken up until the free = memory went above another threshold N. And the problem was that N >> M. = In other words, a lot of memory had to be freed (and not grabbed by = other threads) before the waiting thread would be woken up. Thank you both for your inputs. Let me know if you=E2=80=99d like me to = try anything, and I=E2=80=99ll kick (reboot) the system and can build a = new kernel when you=E2=80=99d like. I did get another procstat -kka out = of it this morning, and the system has since gone less responsive, but I = assume that new procstat won=E2=80=99t show anything last night=E2=80=99s = didn=E2=80=99t. - Chris=