From owner-freebsd-fs@freebsd.org Mon Sep 21 21:41:50 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC054A05F2E for ; Mon, 21 Sep 2015 21:41:50 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 924A31CA1 for ; Mon, 21 Sep 2015 21:41:50 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from liminal.local (liminal.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3636:3bff:fed4:b0d6]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.15.2/8.15.2) with ESMTPSA id t8LLfgBv059569 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 21 Sep 2015 22:41:43 +0100 (BST) (envelope-from matthew@FreeBSD.org) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org DKIM-Filter: OpenDKIM Filter v2.10.3 smtp.infracaninophile.co.uk t8LLfgBv059569 Authentication-Results: smtp.infracaninophile.co.uk/t8LLfgBv059569; dkim=none; dkim-atps=neutral X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host liminal.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3636:3bff:fed4:b0d6] claimed to be liminal.local Subject: Re: ZFS cpu requirements, with/out compression and/or dedup To: freebsd-fs@freebsd.org References: <20150921170216.GA98888@blazingdot.com> <20150921211335.GB41102@server.rulingia.com> From: Matthew Seaman X-Enigmail-Draft-Status: N1110 Message-ID: <5600798B.3010208@FreeBSD.org> Date: Mon, 21 Sep 2015 22:41:31 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20150921211335.GB41102@server.rulingia.com> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="O2MrgKL7ShGv9ETuuIKSiFgd0NsfHJUMP" X-Virus-Scanned: clamav-milter 0.98.7 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2015 21:41:51 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --O2MrgKL7ShGv9ETuuIKSiFgd0NsfHJUMP Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 21/09/2015 22:13, Peter Jeremy wrote: > In general, the downsides of dedup outweigh the benefits. If you alrea= dy > have the data in ZFS, you can use 'zdb -S' to see what effect rebuildin= g > the pool with dedup enabled would have - how much disk space you will s= ave > and how big the DDT is (and hence how much RAM you will need). If you = can > afford it, make sure you keep good backups, enable DDT and be ready to = nuke > the pool and restore from backups if dedup doesn't work out. Nuking the entire pool is a little heavy handed. Dedup can be turned on and off on a per-ZFS basis. If you've a ZFS that had dedup enabled, you can remove the effects by zfs send / zfs recv to create a pristine un-deduped copy of the data, destroy the original zfs and rename the new one to take its place. Of course, this depends on your having enough free space in the pool to be able to duplicate (and then some) that ZFS. Failing that, you might be able to 'zpool split' if your pool is composed entirely of mirrors. So long as you're able to do without resilience for a while this basically doubles the space you have available to play with. You can then destroy the contents of one of the split zpools, and zfs send the data over from the other split pool. Unfortunately there isn't a reciprocal 'zfs rejoin' command that undoes the splitting, so you'll have to destroy one of the splits and re-add the constituent devices back to restore the mirroring in the other split. Which is a delicate operations and not one which is forgiving of mistakes. And failing that, you can start pushing data over the network, but that's hardly different to restoring from backup. However, either of the first two choices should be significantly faster if you have large quantities of data to handle. Cheers, Matthew --O2MrgKL7ShGv9ETuuIKSiFgd0NsfHJUMP Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2 iQJ8BAEBCgBmBQJWAHmSXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkATd+wP/2PQGIgUJYI8tn+ygFbe0AHO IL2QcwFy2cE/CTz8ubkqQm6zAvTZ//EYez2tQV+tpf/IWk6kRa194I2fsFG7D0/o Fa9N7P0AQ11Skv5BRI6bjjIPFHfvazg0aDdiEQH5iC4AsJMYT0J9+/fqaY8idjnX VG921iKSbl9EXtRKBJhNz8uJyvMI1Zxl0DorKce+lrpGqHNBIjBLs8jwGPlQ3Fm4 yKGBHPUOrQtT0JDQZzDyCWl7ed1TR4OG3pZRvLIHKTz1tEoWGfB69e25eMsnXXSt 2kGbhbKWHndl3xre7yj1lqxsoqyfBuMmke9TLcRuVAhbYVzOuEeQQAozpWYgfKnJ X9VjvRFN+n9sLhUF9iWmdGNF9GkgIBMPh5XzwDIypuZCAYiZuGn7Dx6QojG8dQSi LQDfA7T1On/m6uFtOn11xhSI5Fg4lytlZtwD+8UWAYenOcM1USzVoyrOayWB+Xvh 6B5kFJMENEvxXPYpwTchoUo5oAyl6T687ytJvyu5/a1rhF/uANF6u1ljP4y4gPnb Glw2FRiWZpWAWrkGYXroxmMvPRtJ97S95D5vDUqC3erah3wchFW/51KU28UVxPM/ x9tyeHmXMLCmQYagecEClnSDBixfGIh1eAR0In4sNqWoANiyLd3uuVn1EYWngMaX 5a8w3Q+oKHI4XlujSDyU =PWfM -----END PGP SIGNATURE----- --O2MrgKL7ShGv9ETuuIKSiFgd0NsfHJUMP--