Date: Sat, 11 Dec 2021 18:12:44 +0000 From: tech-lists <tech-lists@zyxst.net> To: freebsd-questions@freebsd.org, freebsd-pf@freebsd.org Subject: Re: pf cannot allocate memory after a time Message-ID: <YbTqHMrm7i/V/uMn@ceres.zyxst.net> In-Reply-To: <E31F66A5-5014-4A53-B5BD-B0F38A6F18EB@herrbischoff.com> References: <YbTOficBUC8vhklu@ceres.zyxst.net> <E31F66A5-5014-4A53-B5BD-B0F38A6F18EB@herrbischoff.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Y//rnzCKUsuPuT8M Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, On Sat, Dec 11, 2021 at 06:04:17PM +0100, Marcel Bischoff wrote: > I've stumbled into this issue some time ago as well. The usual remedies= =20 > of raising states and table-entries did not help. > > I could resolve this with a combination of increasing the process memory= =20 > limit and lowering ZFS memory usage. I'm guessing that you do use ZFS,=20 > since you have an 8 GB Pi, if not please disregard. You're right, I am using zfs > FreeBSD limits the memory available per process to 512 MB by default.=20 > It appears that large PF tables cause issues with this default.=20 > Raising that limit to 2 GB seems to have done the trick. To attempt this,= =20 > add the "kern.maxdsiz" tunable to /boot/loader.conf and reboot, like so: > >kern.maxdsiz=3D"2147483648" Yes, came to this conclusion shortly before you posted this. I've written= =20 what I've done further in https://forums.freebsd.org/threads/cannot-define-table-cannot-allocate-memo= ry-since-upgrade-to-13-0.80822/#post-546253 I found this thread https://forums.freebsd.org/threads/pf-states-limit-reached.67337/ informative too. But the idea of increasing it came from=20 https://lists.freebsd.org/pipermail/freebsd-pf/2011-May/006139.html I note that on recent -current (main-n251261) on aarch64 kern.maxdsiz=20 seems to be 1073741824 by default. But on amd64 on releng/12.2-p7 and on releng/13.0-p5 and stable/13=20 it defaults to 34359738368 ! On armv6 r366954 12.2 release it's 536870912.=20 I don't know why it's so low on current (arm64.aarch64). It would explain= =20 why i've only seen this issue on this arch. > Sometimes PF still tripped up when replacing one large table with another= =20 > (via pf-badhosts). Usually this happened when the generated list to repla= ce=20 > grew to several MB in size. I've read somewhere that PF needs double the= =20 > memory to replace a table since for an instant it needs to hold both tabl= es=20 > in memory, which makes sense to me. You can verify that this is the case = by=20 > first dropping and then recreating the pf-badhost table with all entries.= =20 > That usually works, while replacing (what pf-badhosts does) usually does = not. Yes, can confirm it's the replacing where the errors arose. > In those cases I found the ZFS ARC cache to be the culprit. By default th= is=20 > grabs most of the available memory to cache file access and frees it on a= pplication=20 > request. Again, PF appears to require the memory quicker than ZFS appears= to be=20 > able to free it. Usually the default ARC settings are fine but in this ca= se you=20 > may want to tune them for testing purposes, again in /boot/loader.conf an= d reboot: > >vfs.zfs.arc_max=3D"2000M" >vfs.zfs.arc_min=3D"500M" Thank you for this, I'll do this if required. For now, I've set kern.maxdsi= z=3D4294967296 and am monitoring the pfbsdhost's mailbox for error reports. --=20 J. --Y//rnzCKUsuPuT8M Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAmG06hIACgkQs8o7QhFz NAXklBAAmiKX9PHyF7rcd0H2SU2HRLOrkdNTX9WkQQN1uoPrc1qoQFqPhMttloPU WofRsG44cWMd+JSDC7IW74NyRVNqdOqxhoC9iryuez6IOrR+ruFMTnY/yv103N6/ yNXZqkSu1C5CRlBuq4cihQBDFTo849lRBU3Wjw/IAHGjSgDpHgLNVTXLl1vXrydi YxgkgpEXs75NduK9lGRTTrguicnFGAn1HxxDbEztw3y9pW5pHm9a2oDR7QnofPk8 sYJ+VogZu+xJVzN/EgWcdrSFxInXvxV8lGSeylpOvReg44xiNfdTEKGvGdSLOAWo Y+WwGzS6cuGPSIv1FufqkM4h4Vx0LFWFWAMyleOQPEHCTV6nbGr2/v671VqoilBd 2urK9Xf7YCRrUkG9Gt77974fjJtK5PANQ60dpTCz1bapbEayvk9+mDVwLG8N/QpO EI9keyPclrNd500XzVjFJNOl0+8e5/DC4GhET6WCQuxd1E16A+CB30qMpQOx/Kmq iz/jZv8dtxetyjt1MmfJXd7KCYkrMhLEQR7FfrPJUfHQ1reWDadhy9xYbGvr0HBF CzAKVMyCHtaamUyahn0g7xNb2zdX6CiQWhZZyVwyXV/lu4qk/xaxw34xPGH9xFlW /+LUk1I3KTcvlikZLRPYVKu6TLMYk5fXJTUTG4k+fzx5JfqlSNw= =ZV8m -----END PGP SIGNATURE----- --Y//rnzCKUsuPuT8M--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YbTqHMrm7i/V/uMn>