From owner-freebsd-questions@FreeBSD.ORG Tue May 14 13:15:35 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A796DD6 for ; Tue, 14 May 2013 13:15:35 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-ve0-x235.google.com (mail-ve0-x235.google.com [IPv6:2607:f8b0:400c:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id 66664A4D for ; Tue, 14 May 2013 13:15:35 +0000 (UTC) Received: by mail-ve0-f181.google.com with SMTP id pb11so477887veb.40 for ; Tue, 14 May 2013 06:15:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:mime-version:content-type:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=BW1a1s4ktC5sQuEpiHjGXccRiiqy/T80ZlPQkwCVWy0=; b=Y7aJ8gKSx79wcXlv3KbbQMufTApvje6/fw6LNW2Uk93Vibvkxr2ROgHDljsKD1tsxm 7mIKUnyHjAGXv0keZsZcLawuTqxoIxROTl9xnjLUltQ5Yda1MX1EvhcARq8JeaS/nMVP DS5fmOdFkhDiPYCpR5vUfJRe3Ds4Q56TaZlSUEatpVU2539QAXprgWH72nxcUIU6XWyO 71J5v1tZ6qPA1XE01xSV5wTiz2o8nH7TNg2OPtHfkHUmFhI1nXe0esq8T5VABtS9SlIH TW6TS+8ALGLn8UaVh7GrhOhJBs0Tk3t629RC17KzrcYMCJw7eJVoBdikqlbPY6GwGnkU w+8Q== X-Received: by 10.58.200.131 with SMTP id js3mr21704448vec.33.1368537334466; Tue, 14 May 2013 06:15:34 -0700 (PDT) Received: from [192.168.2.66] ([96.236.21.119]) by mx.google.com with ESMTPSA id mw6sm6055678vec.7.2013.05.14.06.15.33 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 14 May 2013 06:15:33 -0700 (PDT) Subject: Re: ZFS mirror install /mnt is empty Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Content-Type: text/plain; charset=windows-1252 From: Paul Kraus In-Reply-To: <5191B950.4040402@ShaneWare.Biz> Date: Tue, 14 May 2013 09:15:32 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5190058D.2030705@micite.net> <472E17AF-B249-4FD3-8F5E-716F8B7867F5@kraus-haus.org> <5191B950.4040402@ShaneWare.Biz> To: Shane Ambler X-Mailer: Apple Mail (2.1503) X-Gm-Message-State: ALoCoQkoau+7YbyX5D6+7Py42y70cpv5NugrMH+veJxsNGNSGoLhi4eWJaSqjs39hWnso/6QaDt2 Cc: "freebsd-questions@freebsd.org List" , =?windows-1252?Q?Trond_Endrest=F8l?= X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 13:15:35 -0000 On May 14, 2013, at 12:10 AM, Shane Ambler = wrote: > When it comes to disk compression I think people overlook the fact = that > it can impact on more than one level. Compression has effects at multiple levels: 1) CPU resources to compress (and decompress) the data 2) Disk space used 3) I/O to/from disks > The size of disks these days means that compression doesn't make a big > difference to storage capacity for most people and 4k blocks mean = little > change in final disk space used. The 4K block issue is *huge* if the majority of your data is = less than 4K files. It is also large when you consider that a 5K file = will not occupy 8K on disk. I am not a UFS on FreeBSD expert, but UFS on = Solaris uses a default block size of 4K but has a fragment size of 1K. = So files are stored on disk with 1K resolution (so to speak). By going = to a 4K minimum block size you are forcing all data up to the next 4K = boundary. Now, if the majority of your data is in large files (1MB or = more), then the 4K minimum black size probably gets lost in the noise. The other factor is the actual compressibility of the data. Most = media files (JPEG, MPEG, GIF, PNG, MP3, AAC, etc.) are already = compressed and trying to compress them again is not likely to garner any = real reduction inn size. In my experience with the default compression = algorithm (lzjb), even uncompressed audio files (.AIFF or .WAV) do not = compress enough to make the CPU overhead worthwhile. > One thing people seem to miss is the fact that compressed files are > going to reduce the amount of data sent through the bottle neck that = is > the wire between motherboard and drive. While a 3k file compressed to = 1k > still uses a 4k block on disk it does (should) reduce the true data > transferred to disk. Given a 9.1 source tree using 865M, if it > compresses to 400M then it is going to reduce the time to read the > entire tree during compilation. This would impact a 32 thread build = more > than a 4 thread build. If the data does not compress well, then you get hit with the = CPU overhead of compression to no bandwidth or space benefit. How = compressible is the source tree ? [Not a loaded question, I haven't = tried to compress it] > While it is said that compression adds little overhead, time wise, Compression most certainly DOES add overhead in terms of time, = based on the speed of your CPU and how busy your system is. My home = server is an HP Proliant Micro with a dual core AMD N36 running at 1.3 = GHz. Turning on compression hurts performance *if* I am getting less = than 1.2:1 compression ratio (5 drive RAIDz2 of 1TB Enterprise disks). = Above that the I/O bandwidth reduction due to the compression makes up = for the lost CPU cycles. I have managed servers where each case = prevailed=85 CPU limited so compression hurt performance and I/O limited = where compression helped performance. > it is > going to take time to compress the data which is going to increase > latency. Going from a 6ms platter disk latency to a 0.2ms SSD latency > gives a noticeable improvement to responsiveness. Adding compression = is > going to bring that back up - possibly higher than 6ms. Interesting point. I am not sure of the data flow through the = code to know if compression has a defined latency component, or is just = throughput limited by CPU cycles to do the compression. > Together these two factors may level out the total time to read a = file. >=20 > One question there is whether the zfs cache uses compressed file data > therefore keeping the latency while eliminating the bandwidth. Data cached in the ZFS ARC or L2ARC is uncompressed. Data sent = via zfs send / zfs receive is uncompressed; there had been talk of an = option to send / receive compressed data, but I do not think it has gone = anywhere. > Personally I have compression turned off (desktop). My thought is that > the latency added for compression would negate the bandwidth savings. >=20 > For a file server I would consider turning it on as network overhead = is > going to hide the latency. Once again, it all depends on the compressibility of the data, = the available CPU resources, the speed of the CPU resources, and the I/O = bandwidth to/from the drives. Note also that RAIDz (RAIDz2, RAIDz3) have their own = computational overhead, so compression may be a bigger advantage in this = case than in the case of a mirror, as the RAID code will have less data = to process after being compressed. -- Paul Kraus Deputy Technical Director, LoneStarCon 3 Sound Coordinator, Schenectady Light Opera Company