From owner-freebsd-questions@FreeBSD.ORG Tue Aug 26 18:00:21 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0E12B51 for ; Tue, 26 Aug 2014 18:00:21 +0000 (UTC) Received: from mail-qc0-f175.google.com (mail-qc0-f175.google.com [209.85.216.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 79C5B3414 for ; Tue, 26 Aug 2014 18:00:20 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id w7so15642786qcr.20 for ; Tue, 26 Aug 2014 11:00:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id:references:to; bh=eY2yuE7i8VnNBUS0/nBEiMmzyQSPeAvI09V66vDBAMQ=; b=KdNH0sh8xGdyGSpf7l4qsJv6GbbvNx9+MGKi12RNM3p2LGcnXsbbhSrUiqHMFrnoYi Qg/kSiacyUk44YfngHuK8ZLEPn65gC38NPJzVNejAci/V4V/GsrH/FtvBO1P89+g6c+M 3odhI1vLgXQj474b6pBJwAQHIh6esfKcxF+bOT0eDEAIwPGhc2lgNWGWQNB3FLXOIkJ4 KMKK3krVYinwmpQloaXYPYHIo4vwk84QdOPPKjU8bnyBcH3EVAZYjWv2HmzsFtE6gSuH RvPz0BSHMbokC69dlRlBDiO47TN8RTQKhckMWQmD9HFQzBnTBXMDU2kaMNPiL8fR9XTr u9QA== X-Gm-Message-State: ALoCoQkKPX1gFI5fxMotAEG+jUhd98VwD1ipxALSm9CsgGxgeDyUZY7AZugcDOESTOGoJYTqRXb3 X-Received: by 10.140.103.75 with SMTP id x69mr29170427qge.17.1409075686059; Tue, 26 Aug 2014 10:54:46 -0700 (PDT) Received: from mbp-1.thecreativeadvantage.com (mail.thecreativeadvantage.com. [96.236.20.34]) by mx.google.com with ESMTPSA id t5sm11453960qat.24.2014.08.26.10.54.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 26 Aug 2014 10:54:45 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: some ZFS questions From: Paul Kraus In-Reply-To: Date: Tue, 26 Aug 2014 13:54:44 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <7432CA21-8F02-4A81-9BD7-CE435EEC992D@kraus-haus.org> References: <201408070816.s778G9ug015988@sdf.org> <27DAA821-0303-4D51-ADA7-7780DB8FE85D@kraus-haus.org> <201408210837.s7L8bm01019230@sdf.org> <9207FB2C-5EDE-49A7-9B0E-7C9839250A7E@kraus-haus.org> <201408241001.s7OA19dZ004925@sdf.org> <5C83C4FD-571B-4557-8AD7-5578276D2ED5@kraus-haus.org> <201408260651.s7Q6pijc023521@sdf.org> To: FreeBSD Questions !!!! X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Aug 2014 18:00:21 -0000 On Aug 26, 2014, at 13:33, Daniel Staal wrote: > Compression can be well worth the CPU cost, actually: In many cases = using light/medium compression under ZFS *improves* performance, as it = takes less time to compress/decompress the data then it does to transfer = the uncompressed data to disk. The issue comes down to three things: 1) How compressible is *your* data? Scott said that his data is already = compressed, so that probably means that ZFS compression will not achieve = much. 2) How much excess CPU capacity do you have and how fast is it? If your = CPUs are idle 80% of the time and they are fast, then they may decrease = the amount of data to be written by enough to make it worthwhile. 3) How fast is your storage? If your storage system is faster than your = *excess* CPU capacity, then compression may not get you any performance = boost. You are balancing the amount by which the compression will reduce the = amount of data to be written (and later read) by the additional CPU = overhead involved in the compression. Also remember that the speed of = your storage will probably be different for writes vs. reads. If your = data access is 95% read, then optimizing writes through compression may = not be the best answer. > Experiment with your data, of course; it depends on how compressible = things are, and your hardware. But don't write it off just because it's = a CPU cost - it might still be a good option. Excellent advice. If I can, I test the compressibility of my data sets, = often with unexpected results. For example, I discovered that raw audio = files (WAV or AIFF) do NOT compress very much using the default ZFS = compression, so I do not enable compression on datasets where those = files reside. On the other hand, I get between 2:1 and 3:1 compression = on virtual machine disk images (even static ones), so I enable = compression on those datasets. One more argument for separate datasets based on the data type. -- Paul Kraus paul@kraus-haus.org