Date: Wed, 6 Mar 2013 01:08:28 +0200 From: Daniel Kalchev <daniel@digsys.bg> To: Freddie Cash <fjwcash@gmail.com> Cc: Garrett Wollman <wollman@hergotha.csail.mit.edu>, stable@freebsd.org, Steven Hartland <killing@multiplay.co.uk> Subject: Re: ZFS "stalls" -- and maybe we should be talking about defaults? Message-ID: <22720B5D-7BCA-41BC-B1E8-A2ACB2ADB795@digsys.bg> In-Reply-To: <CAOjFWZ7TDYgS=vvcuLRPr%2BfAv4bPk4ZU0CKbgf1BydxG1MeOYw@mail.gmail.com> References: <513524B2.6020600@denninger.net> <1362449266.92708.8.camel@btw.pki2.com> <51355F64.4040409@denninger.net> <201303050540.r255ecEC083742@hergotha.csail.mit.edu> <20130305152252.GA52706@in-addr.com> <CAOjFWZ7TDYgS=vvcuLRPr%2BfAv4bPk4ZU0CKbgf1BydxG1MeOYw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mar 5, 2013, at 8:17 PM, Freddie Cash <fjwcash@gmail.com> wrote: >=20 > ZFS send/recv would eventually complete, but what used to take 15-20 > minutes would take 6-8 hours to complete. >=20 > I've reduced the ARC to only 32 GB, with arc_meta set to 28 GB, and = things > are running much smoother now (50-200 MB/s writes for 3-5 seconds = every > 10s), and send/recv is back down to 10-15 minutes. >=20 > Who would have thought "too much RAM" would be an issue? >=20 > Will play with this over the next couple of days with different ARC = max > settings to see where the problems start. All of our ZFS boxes until = this > one had under 64 GB of RAM. (And we had issues with dedupe enabled on > boxes with too little RAM, as in under 32 GB.) I have an archive box running very similar setup as yours, but with 72GB = of RAM. I have set both arc_max and arc_meta_limit to 64GB, with no = issues. I am still doing a very complex snapshot reordering between two = pools. One of the pools has dedup enabled (which prompted me to add = RAM), with dedup ratio f over 10x and there are still no issues or any = stalling. The other pool has both dedup and compression for some = filesystems.=20 My only issue is that replacing a drive in either pool takes few days = (6-drive vdevs of 3TB drives). Perhaps the memory indexing/search algorithms are inefficient? Daniel=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?22720B5D-7BCA-41BC-B1E8-A2ACB2ADB795>