From owner-freebsd-fs@FreeBSD.ORG Sun May 17 09:17:43 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95C02BF0 for ; Sun, 17 May 2015 09:17:43 +0000 (UTC) Received: from smtp.free.de (smtp.free.de [91.204.6.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 019EA17D4 for ; Sun, 17 May 2015 09:17:42 +0000 (UTC) Received: (qmail 76948 invoked from network); 17 May 2015 11:17:32 +0200 Received: from smtp.free.de (HELO [192.168.178.21]) (k@free.de@[91.204.4.103]) (envelope-sender ) by smtp.free.de (qmail-ldap-1.03) with AES128-SHA encrypted SMTP for ; 17 May 2015 11:17:32 +0200 Message-ID: <55585CAA.6080105@free.de> Date: Sun, 17 May 2015 11:17:30 +0200 From: Kai Gallasch Organization: FREE! User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS RAID 10 capacity expansion and uneven data distribution References: In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="mLJCukBQA4ftFBfQCgHnoluGGbgsecphj" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2015 09:17:43 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --mLJCukBQA4ftFBfQCgHnoluGGbgsecphj Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 14.05.2015 15:59 Daniel Kalchev wrote: > Not a total bs, but.. it could be made simpler/safer. >=20 > skip 2,3,4 and 5 > 7a. zfs snapshot -r zpool.old@send > 7b. zfs send -R zpool.old@send | zfs receive -F zpool > do not skip 8 :) > 11. zpool attach zpool da1 da2 && zpool attach zpool da3 da4 Somehow nifty. I tried this on a test server and found out, that after the zpool split it is safer to do the import with e.g."-o altroot=3D/mnt"= , because if you are doing this on a root pool the import will be mounted over the existing root fs and the zfs installation becomes unusable at this point. (-> reboot) Also the zfs receive should not mount the received Filesystem. > After this operation, you should have the exact same zpool, with evenly= redistributed data. You could use the chance to change ashift etc. Sadly= , this works only for mirrors. In my case this is not true. After completion, data is still not evenly distributed across the mirror pairs and each pair has a differing FRAG value. (Before doing the zfs send I destroyed the old z filesystems on the receiving side..) Although when accessing the data afterwards the situation with one mirror pair being overused and the other almost idle has become better - so this method fixes the problem a bit.. When I recreate the pool and restore the data the picture looks different: Data then is really equal size across the mirrors and they all have the same FRAG value - as expected. My conclusion: Expanding a RAID 10 zpool by adding mirrored vdevs is not really an option if you also want to benefit from the gained IOPS of the new devices. In this case recreating the pool is the cleanest solution. If you cannot recreate the pool you can think about this zpool split hack to reditribute data across all vdevs - althoug you temporarly loose your pool redundancy between zpool split and the end of the resilvering process (risky) Kai. --mLJCukBQA4ftFBfQCgHnoluGGbgsecphj Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJVWFyqAAoJEHBlTXxPsfWIJEcQAIn1U7KXsfPHVnJIXHr/QrY7 dgegqx5zLdDGYZYbgxVvU2N4yxs7YktNcMrdXeXqT/y4kf59/5Q7ByRjldE3d9VI fZSO74F9TETjajH87OjFP7opSScfv/FSxAZtveHXtrGsw6KyQUu2H4i+EUWnvpMA C1RgEnDixyX6JyDPiesgdtAUxa/l7fRINNWoKr+o6GpZJ3Btvtibs/JTDYd+Zyv6 +f7H3NCN79OrAyIhmbR+1S59RPPRg5OgZaGI5EGnzWs9df/0uq/pEVAzDn0qF4OI 4tgxPpYDybHtbVF31WZsii6JiVDOVueoTwoDNhGcumDAt5NFDznrnC7pC+FmNkro e8qjBTonbSI+YQo/RvN+PL4pESxK/lV0aPeLguZpUEgQPKoj4vSWS2GdLOfd2eIX b247G56KDRnMjPtxragaP9AWkNh/AuJdLMRyP+Y4hvpQN2N+zVlmkD5OeffK2cQy J2+jvhH9mWPpNtZFYsV8G72gybFyUre6EMQQnTrkmr4miDgmGIkrRzSFASvmJMwr LtsizhH6vmtLVClI5jbaB+ZSHhwTyclxyJBRZT9zGht2xm2xkdGdM1FhAL86jkTl QglskgFo6bhihBxG9gV5RuHDsGEcgbl8I/Whne4r4qPrLYjIPvewUBB8NbHLuw/a 87NM1iquyGb5kJ3JsZN2 =FQwl -----END PGP SIGNATURE----- --mLJCukBQA4ftFBfQCgHnoluGGbgsecphj--