From nobody Tue Nov 8 07:15:30 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N5zsT4xptz4gcqD for ; Tue, 8 Nov 2022 07:15:57 +0000 (UTC) (envelope-from archimedes.gaviola@gmail.com) Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [IPv6:2607:f8b0:4864:20::1136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4N5zsT38Wqz3Pts for ; Tue, 8 Nov 2022 07:15:57 +0000 (UTC) (envelope-from archimedes.gaviola@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-36cbcda2157so125760327b3.11 for ; Mon, 07 Nov 2022 23:15:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=TZ6yQLoK7oFoICL4VB2EMFNiWktGMA0NhjpWA9MGNPA=; b=ZR/8ZtFi+BdFRIfZ/NA57k9D/V4Tf/wduSWrnvogjeQ1swFzD/W/jxR0jprEO8gFd6 sgiRH51dvXPqggj1Fxbqbaj5neBQmXZv+bv/sTlg/MKkQhjUObqTIWnSpOOUdsksVtGz H3jeSkL7D/w6rzsrAQ/crAyTbe+78eh+KQrNzJEew0X3nFnGdBofotNfbiWUGb0tYBKf aWAuurvmvDesCOTVDEGUlolbR0252OemwbtFX23OQxZW3Z73D1wkmLHuGhP13xWRYO3X NaXCCGZNc2a+bMafDrbmZqj8G14l1NV8bc6AeAanGA8dj8ePM7/As0cl33VEwWox00x3 EL9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TZ6yQLoK7oFoICL4VB2EMFNiWktGMA0NhjpWA9MGNPA=; b=eTq821/KEWmWg1QqVVfqEKsRadgBOzIVXxmT8kWqkipUZaCqwJnQPBVAnNvcU4qv0A r86b3FvvfsQsI+kbtgAOsgVsnW76yzhPTAbaGrYORq/h5uR1V5z2dl8pI1XkhGcYXUEM UFRfskPwdi6vL3M51RPr/4oDRCYm5CtNyzbRucKMgiHO+wC2Fq6PysFD8Nfdcvewq+jc 0XBKphMA5WJ5RBAabBHrUTbDEDuKdFBbRjU0wVDaly0ObcznsCSRqA5wRV1GGFq9r2Fo +ineTXQFh2lVCHb3f+4NxAFztdtSSFf6vgL9Mx1sOA4C07k3L/HWFyEbdGeak0CYJfQm xJUw== X-Gm-Message-State: ACrzQf1Ho8fKBjV7CUkWsS+TLYrQzc7p/xki8dr9PXlIIbJ7/Gfg7x6S w/oSGOMbw53PYbBp2+TDRLxsorO67Co2+rEA5Y8= X-Google-Smtp-Source: AMsMyM6X/iQeK36HkpzdKtZDtNwhDl4JoZ6o+pCpFs2vRvb81EmyzhUTTtVlsUKMMCsiZ1ELh66ddxU+ojsxqyGQrXc= X-Received: by 2002:a81:4656:0:b0:367:6daf:84e4 with SMTP id t83-20020a814656000000b003676daf84e4mr50773260ywa.481.1667891756589; Mon, 07 Nov 2022 23:15:56 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <84376CC9-B991-4BF3-AF5F-0AA09CB28339.ref@yahoo.com> <84376CC9-B991-4BF3-AF5F-0AA09CB28339@yahoo.com> <788B97E7-AA06-4A87-BB4A-CF2602DC1AD3@yahoo.com> In-Reply-To: <788B97E7-AA06-4A87-BB4A-CF2602DC1AD3@yahoo.com> From: Archimedes Gaviola Date: Tue, 8 Nov 2022 15:15:30 +0800 Message-ID: Subject: Re: 14.0-CURRENT failed to reclaim memory error in RPi 3B build To: Mark Millard Cc: Warner Losh , freebsd-current Content-Type: multipart/alternative; boundary="000000000000ace67e05ecf04f5b" X-Rspamd-Queue-Id: 4N5zsT38Wqz3Pts X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-ThisMailContainsUnwantedMimeParts: N --000000000000ace67e05ecf04f5b Content-Type: text/plain; charset="UTF-8" On Tue, Nov 8, 2022 at 11:53 AM Mark Millard wrote: > On Nov 7, 2022, at 19:28, Warner Losh wrote: > > > > . . . > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256929, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3628, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255839, size: 40960 > > pid 46153 (c++), jid 0, uid 0, was killed: a thread waited too long to > allocate a page > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255857, size: 28672 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3634, size: 8192 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256037, size: 4096 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255320, size: 8192 > > > > This means that paging to the swap partition and/or swap file took too > long (> 30 seconds... that's all that indefinite means). > > FYI: I think the "indefinite wait buffer" bound that leads > to those messages is 20 sec (the hz*20 below): > > /* > * Wait for the pages we want to complete. VPO_SWAPINPROG is > always > * cleared on completion. If an I/O error occurs, SWAPBLK_NONE > * is set in the metadata for each page in the request. > */ > VM_OBJECT_WLOCK(object); > /* This could be implemented more efficiently with aflags */ > while ((ma[0]->oflags & VPO_SWAPINPROG) != 0) { > ma[0]->oflags |= VPO_SWAPSLEEP; > VM_CNT_INC(v_intrans); > if (VM_OBJECT_SLEEP(object, &object->handle, PSWP, > "swread", hz * 20)) { > printf( > "swap_pager: indefinite wait buffer: bufobj: %p, blkno: %jd, size: %ld\n", > bp->b_bufobj, (intmax_t)bp->b_blkno, > bp->b_bcount); > } > } > VM_OBJECT_WUNLOCK(object); > > But the "was killed: a thread waited too long to allocate a page" is > tied to a total of 30 sec (3*10sec) from: > > vm.pfault_oom_attempts= 3 > vm.pfault_oom_wait= 10 > > (Presuming that you had defaults at the time.) > Confirming the default values. root@generic:~ # sysctl -a | grep pfault_oom vm.pfault_oom_wait: 10 vm.pfault_oom_attempts: 3 > > > It also means that it can't write to backing store dirty pages to give > to another process... > > > > Typical reason is that the disk / flash is not responsive to writes for > some reason. You'll need to find why... I'd look at trims. > > > > Or.... if you can't change the disk... you need to put less memory > pressure on it.. > > > > > > === > Mark Millard > marklmi at yahoo.com > > --000000000000ace67e05ecf04f5b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Tue, Nov 8, 2022 at 11:53 AM Mark = Millard <marklmi@yahoo.com> = wrote:
On Nov 7,= 2022, at 19:28, Warner Losh <imp@bsdimp.com> wrote:
>
> . . .
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256929, size: 40= 96
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3628, size: 4096=
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255839, size: 40= 960
> pid 46153 (c++), jid 0, uid 0, was killed: a thread waited too long to= allocate a page
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255857, size: 28= 672
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3634, size: 8192=
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 256037, size: 40= 96
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 255320, size: 81= 92
>
> This means that paging to the swap partition and/or swap file took too= long (> 30 seconds... that's all that indefinite means).

FYI: I think the "indefinite wait buffer" bound that leads
to those messages is 20 sec (the hz*20 below):

=C2=A0 =C2=A0 =C2=A0 =C2=A0 /*
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* Wait for the pages we want to complete.= =C2=A0 VPO_SWAPINPROG is always
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* cleared on completion.=C2=A0 If an I/O = error occurs, SWAPBLK_NONE
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* is set in the metadata for each page in= the request.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*/
=C2=A0 =C2=A0 =C2=A0 =C2=A0 VM_OBJECT_WLOCK(object);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 /* This could be implemented more efficiently w= ith aflags */
=C2=A0 =C2=A0 =C2=A0 =C2=A0 while ((ma[0]->oflags & VPO_SWAPINPROG) = !=3D 0) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ma[0]->oflags |= =3D VPO_SWAPSLEEP;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 VM_CNT_INC(v_intran= s);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (VM_OBJECT_SLEEP= (object, &object->handle, PSWP,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "= ;swread", hz * 20)) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 printf(
"swap_pager: indefinite wait buffer: bufobj: %p, blkno: %jd, size: %ld= \n",
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 bp->b_bufobj, (intmax_t)bp->b_blkno, bp->= b_bcount);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 VM_OBJECT_WUNLOCK(object);

But the "was killed: a thread waited too long to allocate a page"= is
tied to a total of 30 sec (3*10sec) from:

vm.pfault_oom_attempts=3D 3
vm.pfault_oom_wait=3D 10

(Presuming that you had defaults at the time.)

Confirming the default values.

root@generi= c:~ # sysctl -a | grep pfault_oom
vm.pfault_oom_wait: 10
vm.pfault_oo= m_attempts: 3
=C2=A0

> It also means that it can't write to backing store dirty pages to = give to another process...
>
> Typical reason is that the disk / flash is not responsive to writes fo= r some reason. You'll need to find why... I'd look at trims.
>
> Or.... if you can't change the disk... you need to put less memory= pressure on it..





=3D=3D=3D
Mark Millard
marklmi at yahoo.com

--000000000000ace67e05ecf04f5b--