Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 27 Jun 2012 08:50:22 -0700
From:      Dean Jones <dean.jones@oregonstate.edu>
To:        Levent Serinol <lserinol@gmail.com>
Cc:        "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>
Subject:   Re: ZFS stalls on Heavy I/O
Message-ID:  <CAMXYB4K%2B9EKPyqdCRZZgLvDQuwK=AAGSZi8%2B-AfkOrnJQzwdUA@mail.gmail.com>
In-Reply-To: <CACqg54Si-vHFAVkjpTS40MZt4E1Kn14kDUFKmVb8vx449fCnFw@mail.gmail.com>
References:  <CACqg54Si-vHFAVkjpTS40MZt4E1Kn14kDUFKmVb8vx449fCnFw@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jun 27, 2012 at 2:15 AM, Levent Serinol <lserinol@gmail.com> wrote:
> Hi,
>
> =A0Under heavy I/O load we face freeze problems on ZFS volumes on both
> Freebsd 9 Release and 10 Current versions. Machines are HP servers (64bit=
)
> with HP Smart array 6400 raid controllers (with battery units). Every da
> device is a hardware raid5 where each one includes 9x300GB 10K SCSI hard
> drivers. Main of I/O pattern happens on local system except some small NF=
S
> I/O from some other servers (NFS lookup/getattr/ etc.). These servers are
> mail servers (qmail) with small I/O patterns (64K Read/Write). =A0Below y=
ou
> can =A0find procstat output on freeze time. write_limit is set to 200MB
> because of the huge amount of txg_wait_opens observed before. Every proce=
ss
> stops on D state I think due to txg queue and other 2 queues are full. Is
> there any suggestion to fix the problem ?
>
> btw inject_compress is the main process injecting emails to user inboxes
> (databases). Also, those machines were running without problems on
> Linux/XFS filesystem. For a while ago, we started =A0migration from Linux=
 to
> Freebsd
>
>
> http://pastebin.com/raw.php?i=3Dic3YepWQ
> _______________________________________________

Looks like you are running dedup with only 12 gigs of ram?

Dedup is very ram hungry and the dedup tables are probably no longer
fitting entirely in memory and therefore the system is swapping and
thrashing about during writes.

Also ZFS really prefers to directly address drives instead of RAID
controllers.  It can not guarantee or know what the controller is
doing behind the scenes.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMXYB4K%2B9EKPyqdCRZZgLvDQuwK=AAGSZi8%2B-AfkOrnJQzwdUA>