Date: Mon, 6 Jul 2015 16:23:16 -0500 From: Dave Duchscher <daved@nostrum.com> To: fs@freebsd.org Subject: ZFS system lockup Message-ID: <1BCFA515-BF3D-4E64-B826-BA475B13E770@nostrum.com>
next in thread | raw e-mail | index | archive | help
In the process of diagnosing an IO performance problem with our virtual = environment, we ran into FreeBSD instances used in testing locking up = and needing to be reset. Moving to real hardware and running the same = tests, we are able to reproduce the lockup. We are testing using fio running a few read and write tests over and = over again. Watching via top, the system locks up and last update from = top is reporting wired memory has taking all the memory (2G in the = system, top shows1947M wired). ARC size at the time of the latest lockup = was around 437M. I can keep the system from locking if I reduce the = maximum ARC size to 512M and wired memory floats around 1G. Setting = maximum ARC 768M or higher and we get consistent lockups after running = for a few hours. What is using this wired memory? Is there a way to keep wired memory under control with ZFS besides = shrinking the ARC cache? Is there any guidance on how much wired memory will be used for various = ARC sizes? Is 2G just too little memory to run ZFS? We understand that the maximum ARC size will need to tuned in some cases = but shrinking it down to 512M seems low. This test hardware has a single 250G disk and 2G of RAM. OS is FreeBSD = 10.1 Release. Upgrading the system to stable and saw similar results. = Currently, the system is running 10.1 Release since that is what is used = elsewhere. We have seen a lockup on one of our database nodes which has 20G of RAM = which we thought was caused by a SAN switch on our VM system. Now we = are not so sure. -- Dave
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1BCFA515-BF3D-4E64-B826-BA475B13E770>