From owner-freebsd-fs@FreeBSD.ORG Tue May 12 14:05:36 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7CAA61E for ; Tue, 12 May 2015 14:05:36 +0000 (UTC) Received: from smtp.free.de (smtp.free.de [91.204.6.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 155A411D0 for ; Tue, 12 May 2015 14:05:35 +0000 (UTC) Received: (qmail 18859 invoked from network); 12 May 2015 15:58:51 +0200 Received: from smtp.free.de (HELO [192.168.178.21]) (k@free.de@[91.204.4.103]) (envelope-sender ) by smtp.free.de (qmail-ldap-1.03) with AES128-SHA encrypted SMTP for ; 12 May 2015 15:58:51 +0200 Message-ID: <5552071A.40709@free.de> Date: Tue, 12 May 2015 15:58:50 +0200 From: Kai Gallasch Organization: FREE! User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: ZFS RAID 10 capacity expansion and uneven data distribution Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="qtCRT52dH8X4w17M3JFjea7R0G8MGTsqd" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2015 14:05:36 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qtCRT52dH8X4w17M3JFjea7R0G8MGTsqd Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello list. What is the preferred way to expand a mirrored or RAID 10 zpool with additional mirror pairs? On one server I am currently using a four disk RAID 10 zpool: zpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/zpool-da2 ONLINE 0 0 0 gpt/zpool-da3 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 gpt/zpool-da4 ONLINE 0 0 0 gpt/zpool-da5 ONLINE 0 0 0 Originally the pool consisted of only one mirror (zpool-da2 and zpool-da3= ) I then used "zpool add" to add mirror-1 to the pool Directly afer adding the new mirror I had all old data physically sitting on the old mirror and no data on the new disks. So there is much imbalance in the data distribution across the RAID 10. The effect is now, that the IOPS are not evently distributed about all devs of the pool and e.g. "gstat -p" when the server is very busy showed, that the old mirror pair can max out at 100% I/O usage while the other one is almost idle. Also: I also noted that the old mirror-pair had a FRAG about 50%, while the new one only has 3%. So is it generally not a good idea to expand a mirrored pool or RAID 10 pool with new mirror pairs? Or by which procedure can the existing data in the pool be evenly distributed about all devices inside the pool? Any hint appreciated. Regards, Kai. --=20 PGP-KeyID =3D 0x70654D7C4FB1F588 Der Techniker ist informiert. --qtCRT52dH8X4w17M3JFjea7R0G8MGTsqd Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJVUgcaAAoJEHBlTXxPsfWIiMUP/0r48VZDm39vSPQ5l8EtYKqA eJ+M7qc4Qg7Y/Y3kz378/8TOgJr1TPExQmywVmsFdsS3wBE8N53YRj9NW6ux7tOt 97yOtTUXF8uCGZJvDkTgxh4IVFXHYVut7YrBhD25Z237fJj9QAlauDL5ky1b3Ch7 T/iEMiNbzEs6JA016wePXWQjmwf21XDiCS++KOmeK9lGO7Xr7D/M2fuRQwLo8KzF T2YRL+DVEAwd5rItTaPBkkp3gbnkCu0uGLr1DEiSGyh28CvuMXS5f/DuIe4b0QtH 7gRnpn36+jIE2CNjMA5C3ET7JtmW8e/ZLgHVRHw92u5nzrsXA/YEsjObxbRxArmZ m/QsIxpm+cosasLUa9SjpKn5vyW7z1hEezF5SHlckmnkBYT/xKO0t2kAFVh8Xo+Q 7DaGSiLLMQpcZbGwRotwJQI1V9No9fTZNoPGqdnBsx3MCJDpPVH4cNz5iGuNSVJo IsEiDNiTkSmHNpcQ9euku7+sqRpzqQc3KzGFOrKapYAgXFo/9oFH8AP1Ha9RS6Xb zvDNIxH9AEOQ9O2ECBUHHWom5UhioHEol42GVWmLT1BAsB3DMh4BUG8gqNoXIcqj GrOE7scq7hrKpznY0tb7aJrvP3Mv4g9FKhyGRYUIU8z+NxgDg+BKziKLBHKxRZwi bTu6BXUvCKvwFOatOSJ5 =mb50 -----END PGP SIGNATURE----- --qtCRT52dH8X4w17M3JFjea7R0G8MGTsqd--