From owner-freebsd-fs@FreeBSD.ORG  Tue Oct 11 23:25:45 2011
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id E4B511065670
	for <freebsd-fs@freebsd.org>; Tue, 11 Oct 2011 23:25:45 +0000 (UTC)
	(envelope-from freebsd@penx.com)
Received: from Elmer.dco.penx.com (elmer.dco.penx.com [174.46.214.165])
	by mx1.freebsd.org (Postfix) with ESMTP id B8F138FC12
	for <freebsd-fs@freebsd.org>; Tue, 11 Oct 2011 23:25:45 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
	by Elmer.dco.penx.com (8.14.5/8.14.4) with ESMTP id p9BNPfUE015568
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <freebsd-fs@freebsd.org>; Tue, 11 Oct 2011 17:25:44 -0600 (MDT)
	(envelope-from freebsd@penx.com)
Date: Tue, 11 Oct 2011 17:25:41 -0600 (MDT)
From: Dennis Glatting <freebsd@penx.com>
X-X-Sender: dennisg@Elmer.dco.penx.com
To: freebsd-fs@freebsd.org
Message-ID: <alpine.BSF.2.00.1110111710210.12895@Elmer.dco.penx.com>
User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
Subject: ZFS/compression/performance
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 11 Oct 2011 23:25:46 -0000

I would appreciate someone knowledgeable in ZFS point me in the right 
direction.

I have several ZFS arrays, some using gzip for compression. The compressed 
arrays hold very large text documents (10MB->20TB) and are highly 
compressible. Reading the files from a compressed data sets is fast with 
little load. However, writing to the compressed data sets incurs 
substantial load on the order of a load average from 12 to 20.

My questions are:

1) Why such a heavy load on writing?
2) What kind  of limiters can I put into effect to reduce load
    without impacting compressibilty? For example, is there some
    variable to controls the number of parallel compression
    operations?

I have a number of different systems. Memory is 24GB on each of the two 
large data systems, SSD (Revo) for cache, and a SATA II ZIL. One system is 
a 6 core i7 @ 3.33 GHz and the other 4 core ii7 @ 2.93 GHz. The arrays are 
RAIDz using cheap 2TB disks.

Thanks.