Date: Wed, 1 Aug 2018 06:24:23 +0200 From: =?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?= <gandalf@shopzeus.com> To: freebsd-stable@freebsd.org Subject: Re: All the memory eaten away by ZFS 'solaris' malloc - on 11.1-R amd64 Message-ID: <35c3420a-085a-cb6d-506b-8c0887cd6dc3@shopzeus.com> In-Reply-To: <41a77fe4-8acf-76d5-c5f5-2f60418352a1@ShaneWare.Biz> References: <1a039af7758679ba1085934b4fb81b57@ijs.si> <3e56e4de076111c04c2595068ba71eec@ijs.si> <41a77fe4-8acf-76d5-c5f5-2f60418352a1@ShaneWare.Biz>
next in thread | previous in thread | raw e-mail | index | archive | help
I have experienced a very similar thing. After upgrading my machine from 11.1-R to 11.2-R, the swap space is filled up to about 66% in about every 2 days. First I tought that it was PostgreSQL, and lowered the shared_buffers setting, but it only postponed the problem for another day= =2E The only thing that has changed is the OS version 11.1-R -> 11.2-R. Here is the top of top: last pid: 50425;=C2=A0 load averages:=C2=A0 0.19,=C2=A0 0.16,=C2=A0 0.17=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 up 15+23:02:21=C2=A0 06:18:18 45 processes:=C2=A0 1 running, 43 sleeping, 1 zombie CPU:=C2=A0=C2=A0=C2=A0=C2=A0 % user,=C2=A0=C2=A0=C2=A0=C2=A0 % nice,=C2=A0= =C2=A0=C2=A0=C2=A0 % system,=C2=A0=C2=A0=C2=A0=C2=A0 % interrupt,=C2=A0=C2= =A0=C2=A0=C2=A0 % idle Mem: 81M Active, 91M Inact, 1577M Laundry, 14G Wired, 226M Free ARC: 9598M Total, 90M MFU, 8715M MRU, 105K Anon, 199M Header, 594M Other =C2=A0=C2=A0=C2=A0=C2=A0 8085M Compressed, 15G Uncompressed, 1.84:1 Ratio= Swap: 4096M Total, 3103M Used, 993M Free, 75% Inuse The ARC value seem to be growing for a while, then it starts to use the swap heavily. But this might be unrelated because the swap usage does not go above 80%. (E.g. it does not crash, but it is clearly using swap when it should not.) =C2=A0 Laszlo > On 01/08/2018 07:24, Mark Martinec wrote: >> I have now upgraded this host from 11.1-RELEASE-p11 to 11.2-RELEASE >> and the situation has not improved. Also turned off all services. >> ZFS is still leaking memory about 30 MB per hour, until the host >> runs out of memory and swap space and crashes, unless I reboot it >> first every four days. >> >> Any advise before I try to get rid of that faulted disk with a pool >> (or downgrade to 10.3, which was stable) ? >> >> =C2=A0 Mark >> >> >> 2018-07-23 17:12, myself wrote: >>> After upgrading an older AMD host from FreeBSD 10.3 to 11.1-RELEASE-p= 11 >>> (amd64), ZFS is gradually eating up all memory, so that it crashes ev= ery >>> few days when the memory is completely exhausted (after swapping heav= ily >>> for a couple of hours). >>> >>> This machine has only 4 GB of memory. After capping up the ZFS ARC >>> to 1.8 GB the machine can now stay up a bit longer, but in four days >>> all the memory is used up. The machine is lightly loaded, it runs >>> a bind resolver and a lightly used web server, the ps output >>> does not show any excessive memory use by any process. > When you say all used up - you mean the amount of wired ram goes higher= > than about 90% physical ram? You can watch the wired amount in top, or > calculate it as vm.stats.vm.v_wire_count * hw.pagesize > > ZFS ARC is marked as wired, there is also vm.max_wired which limits how= > much the kernel can wire, this defaults to 30% ram, so about 1.2G for > you. It seems these two wired values don't interact and can add up to > more than physical ram. I have reported this in bug 229764 > > Try the patch at > https://reviews.freebsd.org/D7538 > it has given me the best arc related memory improvements I have seen > since 10.1, I now see arc being released instead of swap being used. >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?35c3420a-085a-cb6d-506b-8c0887cd6dc3>