Date: Wed, 19 Jul 2023 15:50:44 -0700 From: Scott Gasch <scott.gasch@gmail.com> To: Pete Wright <pete@nomadlogic.org> Cc: freebsd-questions <freebsd-questions@freebsd.org> Subject: Re: Swap filling up, usermode process swap usage doesn't explain Message-ID: <CABYAQkTPBU1hzF4usukMkSq7VumKkfEGXMqb0nOBNpo2g2DsMA@mail.gmail.com> In-Reply-To: <88e0d702-cb86-b922-a5ce-82fe32e66a40@nomadlogic.org> References: <CABYAQkQftAfRXpdSJnqH2Hi=uD-dOiGWdFU8u1XqfeZNBUA35w@mail.gmail.com> <b24efee3-939b-3e20-d07f-8dad92d8e081@nomadlogic.org> <CABYAQkRHbdfbccQa_UqoZ7_YBTKEjwQAR7hv%2BR0Lih60a=vSAw@mail.gmail.com> <88e0d702-cb86-b922-a5ce-82fe32e66a40@nomadlogic.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000003471af0600dedee9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable -cc hackers The laundry is abnormally high and doesn't seem to go down. My understanding of it is that it's pages that are available for reuse and have to be zeroed by some kernel thread that does that job(?). Typically this filling up swap with no clear usermode culprit happens after the machine has been running for ~10 days and I eventually reboot. I can say for sure laundry is nowhere near this full after a reboot for several days. On Wed, Jul 19, 2023 at 3:40=E2=80=AFPM Pete Wright <pete@nomadlogic.org> w= rote: > > > On 7/19/23 15:11, Scott Gasch wrote: > > Yes, I'm using ZFS. Here's what top says: > > > > last pid: 88926; load averages: 1.20, 0.96, 0.87 > > up 5+17:48:34 15:09:58 > > 274 processes: 1 running, 272 sleeping, 1 zombie > > CPU: 1.8% user, 0.0% nice, 0.5% system, 0.0% interrupt, 97.8% idle > > Mem: 1844M Active, 7777M Inact, 77G Laundry, 35G Wired, 750M Buf, 3367M > Free > > ARC: 24G Total, 2878M MFU, 18G MRU, 21M Anon, 119M Header, 2622M Other > > 18G Compressed, 25G Uncompressed, 1.33:1 Ratio > > Swap: 144G Total, 11G Used, 133G Free, 7% Inuse > > > > If I leave this alone it will grow to consume all available swap space. > > I'll try your fix with the sysctl knob and see what happens... I hope > > this is it, I've been fighting this for a while now. > > worth a shot, but 24G of ARC isn't that bad, especially if you are doing > quite a bit of disk i/o. > > i'm more interested in the 77G of Laundry memory, that seems like quite > a bit. but i don't know your workload so not sure... > > -p > > -- > Pete Wright > pete@nomadlogic.org > @nomadlogicLA > --0000000000003471af0600dedee9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>-cc hackers</div><div><br></div>The laundry is abnorm= ally high and doesn't seem to go down.=C2=A0 My understanding of it is = that it's pages that are available for reuse and have to be zeroed by s= ome kernel thread that does that job(?).<div><br></div><div>Typically this = filling up swap with no clear usermode=C2=A0culprit happens after the machi= ne has been running for ~10 days and I eventually reboot.=C2=A0 I can say f= or sure laundry is nowhere near this full after a reboot for several days.<= /div><div><br></div><div><div><br></div><div><br></div></div></div><br><div= class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jul 19= , 2023 at 3:40=E2=80=AFPM Pete Wright <<a href=3D"mailto:pete@nomadlogic= .org">pete@nomadlogic.org</a>> wrote:<br></div><blockquote class=3D"gmai= l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,20= 4,204);padding-left:1ex"><br> <br> On 7/19/23 15:11, Scott Gasch wrote:<br> > Yes, I'm using ZFS.=C2=A0 Here's what top says:<br> > <br> > last pid: 88926; =C2=A0load averages: =C2=A01.20, =C2=A00.96, =C2=A00.= 87=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> >=C2=A0 =C2=A0 =C2=A0up 5+17:48:34 =C2=A015:09:58<br> > 274 processes: 1 running, 272 sleeping, 1 zombie<br> > CPU: =C2=A01.8% user, =C2=A00.0% nice, =C2=A00.5% system, =C2=A00.0% i= nterrupt, 97.8% idle<br> > Mem: 1844M Active, 7777M Inact, 77G Laundry, 35G Wired, 750M Buf, 3367= M Free<br> > ARC: 24G Total, 2878M MFU, 18G MRU, 21M Anon, 119M Header, 2622M Other= <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A018G Compressed, 25G Uncompressed, 1.33:1 Rat= io<br> > Swap: 144G Total, 11G Used, 133G Free, 7% Inuse<br> > <br> > If I leave this alone it will grow to consume all available swap space= .=C2=A0 <br> > I'll try your fix with the sysctl knob and see what happens...=C2= =A0 I hope <br> > this is it, I've been fighting this for a while now.<br> <br> worth a shot, but 24G of ARC isn't that bad, especially if you are doin= g <br> quite a bit of disk i/o.<br> <br> i'm more interested in the 77G of Laundry memory, that seems like quite= <br> a bit.=C2=A0 but i don't know your workload so not sure...<br> <br> -p<br> <br> -- <br> Pete Wright<br> <a href=3D"mailto:pete@nomadlogic.org" target=3D"_blank">pete@nomadlogic.or= g</a><br> @nomadlogicLA<br> </blockquote></div> --0000000000003471af0600dedee9--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CABYAQkTPBU1hzF4usukMkSq7VumKkfEGXMqb0nOBNpo2g2DsMA>