Date: Fri, 2 Mar 2012 22:11:23 +1100 From: Peter Jeremy <peterjeremy@acm.org> To: Luke Marsden <luke-lists@hybrid-logic.co.uk> Cc: freebsd-fs@freebsd.org, Alexander Leidinger <Alexander@Leidinger.net> Subject: Re: Another ZFS ARC memory question Message-ID: <20120302111123.GA40708@server.vk2pj.dyndns.org> In-Reply-To: <1330683366.3819.20.camel@pow> References: <1330081612.13430.39.camel@pow> <20120227181436.GA49667@server.vk2pj.dyndns.org> <20120301142826.GG97848@zxy.spb.ru> <20120302102509.Horde.6uPSdpjmRSRPUJH1lHEHc3A@webmail.leidinger.net> <1330683366.3819.20.camel@pow>
next in thread | previous in thread | raw e-mail | index | archive | help
--17pEHd4RhPHOinZp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [Cc list pruned] On 2012-Mar-02 10:16:06 +0000, Luke Marsden <luke-lists@hybrid-logic.co.uk>= wrote: >We are presently working around it by limiting arc_max to 4G on our 24G >RAM production boxes (which seems like a massive waste of performance) >and by doing very careful/aggressive application level management of >memory usage to ensure stability (limits.conf didn't work for us, so we >rolled our own). A better solution would be welcome, though, so that we >can utilise all the free memory we're presently keeping around as a >safety margin - ideally it would be used as ARC cache. Have you tried increasing vm.v_cache_min to cover your spikes? >1. Is it expected that with a 4G limited arc_max value that we should >see wired memory usage around 7-8G? I understand that the kernel has to >use some memory, but really 3-4G of non-ARC data? Yes, that sounds possible. >2. We have some development machines with only 3G of RAM. Previously >they had no arc_max set and were left to tune themselves. They were >quite unstable. Now we've set arc_max to 256M but things have got >worse: we've seen a disk i/o big performance hit (untarring a ports >tarball now takes 20 minutes), and wired memory usage is up around >2.5GB, the machines are swapping a lot, and crashing more frequently. That's stress-testing ZFS more than anything else. You definitely can't use those results as a guide to tune your production boxes (other than what not to do). That said, I have 3.5GB in my $work desktop (running 8.2-stable from about a month ago) and don't have any stability issues with either it or a buildbox with 2GB RAM. >Follows is arc_summary.pl from one of the troubled dev machines showing >the ARC using 500% of the memory it should be. Also uname follows. My >second question is: have there been fixes between 8.2-RELEASE and >8.3-BETA1 or 9.0-RELEASE which solve this ARC over-usage problem? There definitely have been some commits to ensure that arc_max is treated much more as a hard limit but I can't quickly find them so I'm not sure if they pre- or post-date 8.2-RELEASE. --=20 Peter Jeremy --17pEHd4RhPHOinZp Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk9QqtsACgkQ/opHv/APuIfWmgCfW+tqI1+n5qV9b5dWZ4F2aK/M khEAoKqIgtv3M7cmbCQ0RBRrIJpSEfwj =DgVI -----END PGP SIGNATURE----- --17pEHd4RhPHOinZp--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120302111123.GA40708>