Date: Tue, 10 Dec 2013 18:36:46 +0100 From: Oscar Prieto <oscarmpp@gmail.com> To: Benjamin Lutz <benjamin.lutz@biolab.ch> Cc: freebsd-stable@freebsd.org Subject: Re: Swap on ZFS - bad idea? (swap_pager: indefinite wait buffer) Message-ID: <CAK9wqRrug7G5w3HyT7WiLitS8LJ-E1vYUNsD0X0EjkYfxcqgoQ@mail.gmail.com> In-Reply-To: <OFB3F5370B.79D00F5F-ONC1257C3D.0038AB15-C1257C3D.003AB574@biotronik.com> References: <OFB3F5370B.79D00F5F-ONC1257C3D.0038AB15-C1257C3D.003AB574@biotronik.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Personally I still have to read an announcement telling that swap on zfs is relliable. I'm still using a dedicated swap partition on every install due to that kind of behaviour. On Tue, Dec 10, 2013 at 11:41 AM, Benjamin Lutz <benjamin.lutz@biolab.ch>wrote: > We have storage server that I've found unresponsive this morning. The > console was filled with these messages: > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 17344, size: 61440 > > The lines were all exactly the same, no variation in the numbers. > Ctrl-Alt-Delete had no effect. > > Now, the only thing related to swap that's special about this machine, is > that we use swap on ZFS, because we figured, silent disk corruption is > just as bad when it happens to swapped memory as it is when it happens to > stored data. However, might this be the problem? One thought I was having > was: ZFS hits its memory bounds, a process makes the kernel want to > allocate a swap buffer which causes more ZFS operations which never > complete because ZFS would need to do some housework to free memory, but > the kernel is waiting for swap to become available, some sort of deadlock? > I'm just speculating here, not knowing the internals of this subsystem. > > The machine is a quad core system with 12GiB RAM and a RAIDZ3 consisting > of 12x2TB disks. It's used as a storage system, with accesses mainly > through netatalk (an AFP server, what OS X uses) and a bit of iSCSI. I've > set the following sysctls: > > # > > http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Device_I.2FO_Queue_Size_.28I.2FO_Concurrency.29 > vfs.zfs.vdev.max_pending=20 > # limit kernel size to 10GB to leave some RAM for processes > vm.kmem_size_max="10737418240" > # increase number of mbuf clusters, see igb(4). The default is 25600 > kern.ipc.nmbclusters="102400" > > This sort of hang seems to happen every couple of months. I'd really like > some pointers on how to increase the stability of this system. Maybe there > are some more tunables I need to adjust? Or is swap on ZFS maybe just not > a great idea? > > Cheers, > Benjamin > -- > Benjamin Lutz | Software Engineer | BIOLAB Technology AG > Dufourstr. 80 | CH-8008 Zurich | www.biolab.ch | benjamin.lutz@biolab.ch > PHONE +41 44 295 97 13 | MOBILE +41 79 558 57 13 | FAX +41 44 295 97 19 > > > > This e-mail and the information it contains including attachments are > confidential and meant > only for use by the intended recipient(s); disclosure or copying is > strictly prohibited. If you > are not addressed, but in the possession of this e-mail, please notify the > sender immediately. > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAK9wqRrug7G5w3HyT7WiLitS8LJ-E1vYUNsD0X0EjkYfxcqgoQ>