Date: Wed, 27 Jun 2012 18:34:13 +0200 From: Andreas Nilsson <andrnils@gmail.com> To: Dean Jones <dean.jones@oregonstate.edu> Cc: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: ZFS stalls on Heavy I/O Message-ID: <CAPS9%2BSs1oCh=Szhf_qCam85hPh%2BMFu-XRjTEJZT5hYt12qMhXw@mail.gmail.com> In-Reply-To: <CAMXYB4K%2B9EKPyqdCRZZgLvDQuwK=AAGSZi8%2B-AfkOrnJQzwdUA@mail.gmail.com> References: <CACqg54Si-vHFAVkjpTS40MZt4E1Kn14kDUFKmVb8vx449fCnFw@mail.gmail.com> <CAMXYB4K%2B9EKPyqdCRZZgLvDQuwK=AAGSZi8%2B-AfkOrnJQzwdUA@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
On Wed, Jun 27, 2012 at 5:50 PM, Dean Jones <dean.jones@oregonstate.edu>wrote: > On Wed, Jun 27, 2012 at 2:15 AM, Levent Serinol <lserinol@gmail.com> > wrote: > > Hi, > > > > Under heavy I/O load we face freeze problems on ZFS volumes on both > > Freebsd 9 Release and 10 Current versions. Machines are HP servers > (64bit) > > with HP Smart array 6400 raid controllers (with battery units). Every da > > device is a hardware raid5 where each one includes 9x300GB 10K SCSI hard > > drivers. Main of I/O pattern happens on local system except some small > NFS > > I/O from some other servers (NFS lookup/getattr/ etc.). These servers are > > mail servers (qmail) with small I/O patterns (64K Read/Write). Below you > > can find procstat output on freeze time. write_limit is set to 200MB > > because of the huge amount of txg_wait_opens observed before. Every > process > > stops on D state I think due to txg queue and other 2 queues are full. Is > > there any suggestion to fix the problem ? > > > > btw inject_compress is the main process injecting emails to user inboxes > > (databases). Also, those machines were running without problems on > > Linux/XFS filesystem. For a while ago, we started migration from Linux > to > > Freebsd > > > > > > http://pastebin.com/raw.php?i=ic3YepWQ > > _______________________________________________ > > Looks like you are running dedup with only 12 gigs of ram? > > Dedup is very ram hungry and the dedup tables are probably no longer > fitting entirely in memory and therefore the system is swapping and > thrashing about during writes. > > Also ZFS really prefers to directly address drives instead of RAID > controllers. It can not guarantee or know what the controller is > doing behind the scenes. > You might want to read http://constantin.glez.de/blog/2011/07/zfs-dedupe-or-not-dedupe and see if you need more ram. And yes, having raid below zfs somewhat defeats the point of zfs. Regards Andreashome | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPS9%2BSs1oCh=Szhf_qCam85hPh%2BMFu-XRjTEJZT5hYt12qMhXw>