From owner-freebsd-questions@FreeBSD.ORG Tue Dec 1 17:55:04 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8064106566B for ; Tue, 1 Dec 2009 17:55:04 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email1.allantgroup.com (email1.emsphone.com [199.67.51.115]) by mx1.freebsd.org (Postfix) with ESMTP id 8BA638FC19 for ; Tue, 1 Dec 2009 17:55:04 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email1.allantgroup.com (8.14.0/8.14.0) with ESMTP id nB1Ht1kv019734 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 1 Dec 2009 11:55:02 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.3/8.14.3) with ESMTP id nB1Ht1ah082476 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 1 Dec 2009 11:55:01 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.3/8.14.3/Submit) id nB1Ht11N082471; Tue, 1 Dec 2009 11:55:01 -0600 (CST) (envelope-from dan) Date: Tue, 1 Dec 2009 11:55:01 -0600 From: Dan Nelson To: Rolf Nielsen Message-ID: <20091201175500.GN89004@dan.emsphone.com> References: <4B155562.30109@lazlarlyricon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B155562.30109@lazlarlyricon.com> X-OS: FreeBSD 7.2-STABLE User-Agent: Mutt/1.5.20 (2009-06-14) X-Virus-Scanned: ClamAV version 0.94.1, clamav-milter version 0.94.1 on email1.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (email1.allantgroup.com [199.67.51.78]); Tue, 01 Dec 2009 11:55:02 -0600 (CST) X-Scanned-By: MIMEDefang 2.45 Cc: freebsd-questions@freebsd.org Subject: Re: ZFS pools of consisting of several mirrors X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2009 17:55:04 -0000 In the last episode (Dec 01), Rolf Nielsen said: > In experimenting a bit with ZFS, I, among other things, tried something > like this > > zpool create -R /test test mirror file[01]0 mirror file[01]1 mirror > file[01]2 mirror file[01]3 mirror file[01]4 mirror file[01]5 > > This, according to zpool status, gives me a (file backed) pool consisting > of six mirrors, each mirror consisting of two files. Now for my question. > Exactly how is the pool built? Is it... > > 1. A RAID0 of the six mirrors? > > 2. A mirror of two RAID0 arrays, each array consisting of the six files > file0[0-5] and file1[0-5] respectively? > > 3 and 4. Like 1 and 2 above, but with JBOD instead of RAID0? > > 5. Some other way I haven't thought about? > > I guess it's 1 or 3, as the zpool status output shows me six mirrors, > but which is it? And, provided my guess is correct, is there a way to > implement 2 or 4 without involving geom_stripe or geom_concat? It's 1/3/5. Each mirror is independant, and writes are balanced across the mirrors based on space usage. If you add another mirror to grow the pool, it will get most of the writes until the usages balance out. You usually don't want to build an array with options 2 or 4, since a single drive failure will degrade the entire mirror half. Consider if you have concat00 -> file01 file02 file03 file04 file05 concat01 -> file11 file12 file13 file14 file15 mirror0 -> concat0 concat1 If file01 fails, concat00 fails, causing mirror0 to become degraded. When you replace file01, mirror0 will have to resynch all of concat00 from concat01 since it doesn't know about the subdevices. If you don't replace file01, and then file15 fails, you have lost your entire volume (unless you do some hackery to swap file05 and file15 to create a functioning concat01). -- Dan Nelson dnelson@allantgroup.com