Date: Thu, 1 Feb 2024 17:32:00 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> To: freebsd-hackers@freebsd.org Subject: Re: ... was killed: a thread waited too long to allocate a page Message-ID: <2d2f309b-d27d-46f1-9a04-5d08252c0c0f@quip.cz> In-Reply-To: <29D13BFFCFA5255C07379043@[10.12.30.106]> References: <0C31C8D8-2335-43ED-96B3-21AC46F30C1D@cs.huji.ac.il> <29D13BFFCFA5255C07379043@[10.12.30.106]>
next in thread | previous in thread | raw e-mail | index | archive | help
On 01/02/2024 15:47, Karl Pielorz wrote: [..] > I seem to have run into something similar. I recently upgraded a 12.4 > box to 13.2p9. The box has 32G of RAM, and runs ZFS. We do a lot of > rsync work to it monthly - the first month we've done this with 13.2p9 > we get a lot of processes killed, all with a similar (but not identical) > message, e.g. > > pid 11103 (ssh), jid 0, uid 0, was killed: failed to reclaim memory > pid 10972 (local-unbound), jid 0, uid 59, was killed: failed to reclaim > memory > pid 3223 (snmpd), jid 0, uid 0, was killed: failed to reclaim memory > pid 3243 (mountd), jid 0, uid 0, was killed: failed to reclaim memory > pid 3251 (nfsd), jid 0, uid 0, was killed: failed to reclaim memory > pid 10996 (sshd), jid 0, uid 0, was killed: failed to reclaim memory > pid 3257 (sendmail), jid 0, uid 0, was killed: failed to reclaim memory > pid 8562 (csh), jid 0, uid 0, was killed: failed to reclaim memory > pid 3363 (smartd), jid 0, uid 0, was killed: failed to reclaim memory > pid 8558 (csh), jid 0, uid 0, was killed: failed to reclaim memory > pid 3179 (ntpd), jid 0, uid 0, was killed: failed to reclaim memory > pid 8555 (tcsh), jid 0, uid 1001, was killed: failed to reclaim memory > pid 3260 (sendmail), jid 0, uid 25, was killed: failed to reclaim memory > pid 2806 (devd), jid 0, uid 0, was killed: failed to reclaim memory > pid 3156 (rpcbind), jid 0, uid 0, was killed: failed to reclaim memory > pid 3252 (nfsd), jid 0, uid 0, was killed: failed to reclaim memory > pid 3377 (getty), jid 0, uid 0, was killed: failed to reclaim memory > > This 'looks' like 'out of RAM' type situation - but at the time, top > showed: I remember something similar on our machines after upgrade to 13.x about a year ago. But don't remember what steps we take to walk around this issue (if any). I also see this on my FreeBSD based desktop from time to time... "ad more memory" and limit the ARC in loader.conf is my way. [..] > I've not seen any bug reports along this line, in fact very little > coverage at all of the specific error. > > My only thought is to set a sysctl to limit ZFS ARC usage, i.e. to leave > more free RAM floating around the system. During the rsync it was > 'swapping' occasionally (few K in, few K out) - but it never ran out of > swap that I saw - and it certainly didn't look like an complete out of > memory scenario/box (which is what it felt like with everything getting > killed).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2d2f309b-d27d-46f1-9a04-5d08252c0c0f>