From owner-freebsd-fs@FreeBSD.ORG Sun Jun 12 04:37:17 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 912E4106564A; Sun, 12 Jun 2011 04:37:17 +0000 (UTC) (envelope-from dmagda@ee.ryerson.ca) Received: from eccles.ee.ryerson.ca (ee.ryerson.ca [141.117.1.2]) by mx1.freebsd.org (Postfix) with ESMTP id 1D9358FC16; Sun, 12 Jun 2011 04:37:16 +0000 (UTC) Received: from [10.0.1.2] (bas2-toronto09-1176443988.dsl.bell.ca [70.31.28.84]) (authenticated bits=0) by eccles.ee.ryerson.ca (8.14.4/8.14.4) with ESMTP id p5C3wY5L015382 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 11 Jun 2011 23:58:35 -0400 (EDT) (envelope-from dmagda@ee.ryerson.ca) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: David Magda In-Reply-To: <4DF28BCF.3060008@gmail.com> Date: Sat, 11 Jun 2011 23:58:39 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <525D503A-240C-49F2-9AAD-EC8E3C1CDB9A@ee.ryerson.ca> References: <4DECB197.8020102@FreeBSD.org> <4DF28BCF.3060008@gmail.com> To: Volodymyr Kostyrko X-Mailer: Apple Mail (2.1084) Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: HEADS UP: ZFS v28 merged to 8-STABLE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 04:37:17 -0000 On Jun 10, 2011, at 17:25, Volodymyr Kostyrko wrote: > Am I missing something? How about using fletcher[24] for dedup? Fletcher is fairly weak as things go, and so even though two checksums = are the same, there's a decent chance that the data is actually = different. At least with recent releases of (Open)Solaris, when you = enable do a 'dedup=3Don' the has used is SHA-256, which has very, very, = very, low odds of having the same value occur from two different blocks = of data. When ZFS dedupe originally came out you could have one of the following = values: . off . on (=3D=3D sha256) . flecther4 with verify/compare . sha256 (without verify/compare) . sha256 with verify There was a long-ish thread on zfs-discuss fairly recently on whether = SHA-256 was "good enough" where you could trust it, or whether one = should do a verify step in addition to SHA-256: = http://mail.opensolaris.org/pipermail/zfs-discuss/2011-January/046875.html= While some people argued that it was prudent to use "verify" (especially = with your data/job on the line), a good portion of folks though said = that it's not worth it (i.e., if you're not worried about being hit by = lightning (2^-17 to 2^-18), you shouldn't be worried about a hash = collision (2^-128)).