From owner-freebsd-fs@freebsd.org Sat Sep 19 14:08:38 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2AE9A02E52 for ; Sat, 19 Sep 2015 14:08:38 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 478471F2E for ; Sat, 19 Sep 2015 14:08:37 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id t8JE4moU015486; Sat, 19 Sep 2015 09:04:48 -0500 (CDT) Date: Sat, 19 Sep 2015 09:04:48 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Sami Halabi cc: freebsd-fs@freebsd.org Subject: Re: ZFS cpu requirements, with/out compression and/or dedup In-Reply-To: Message-ID: References: User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Sat, 19 Sep 2015 09:04:49 -0500 (CDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Sep 2015 14:08:38 -0000 On Sat, 19 Sep 2015, Sami Halabi wrote: > Hi everyone, > I've been searching the documentation, wiki.. etc. but found no rule of > thumb to CPU requirements to run ZFS fs. for memory there are few rules > according to using dedup or not. > > rules of thumb I concluded are so far: > 1. use compression lz4. > 2. use checksum. > 3. disable atime. > > from what i read the status of dedup is not that clear and seems there are > bugs and better to avoid it? I am not so sure about dedup bugs. Usually the problem encountered with dedup is unrealistic expectations about how it will behave. Dedup requires a large amount of RAM and/or L2ARC device which are appropriately sized for the amount of storage and the amount of redundancy encountered. The problem typically encountered is that things work "fine" with dedup until one day they do not and when they do not, it is like going off a cliff. Don't enable dedup unless you have carefully researched it, decided that its value outweighs its cost, and make sure that you have enough RAM or SSD-based L2ARC (preferably RAM) so that the tables it needs are all readily available without accessing rotating media. Depending on what you were planning to use dedup for, there may be other zfs tricks (e.g. cloning) which are amost as effective, but without the risks. > so according to 1-3 above what cpu requirements i need? is ATOM cpu like > supermicro c2750/3/5/8 enough to run system of 20TB /40TB with 1-3 above? > if dedup IS enabled would it still work fine? CPU usage is rarely a problem for zfs except for when compression is involved. Lz4 is supposed to be CPU-efficient on Intel CPUs. Don't use gzip compression if you don't have a powerful CPU. Compression can be changed at any time, although a given file will remain compressed with the compression setting in effect when it was created. CPU use is driven by I/O requirements and selected compression algorithms and not by the size of the storage pool. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/