From owner-freebsd-fs@FreeBSD.ORG Mon Jul 9 19:58:20 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BAB1B1065677 for ; Mon, 9 Jul 2012 19:58:20 +0000 (UTC) (envelope-from jusher71@yahoo.com) Received: from nm36-vm6.bullet.mail.ne1.yahoo.com (nm36-vm6.bullet.mail.ne1.yahoo.com [98.138.229.118]) by mx1.freebsd.org (Postfix) with SMTP id 72FC08FC14 for ; Mon, 9 Jul 2012 19:58:20 +0000 (UTC) Received: from [98.138.90.51] by nm36.bullet.mail.ne1.yahoo.com with NNFMP; 09 Jul 2012 19:58:14 -0000 Received: from [98.138.89.194] by tm4.bullet.mail.ne1.yahoo.com with NNFMP; 09 Jul 2012 19:58:14 -0000 Received: from [127.0.0.1] by omp1052.mail.ne1.yahoo.com with NNFMP; 09 Jul 2012 19:58:14 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 464503.16463.bm@omp1052.mail.ne1.yahoo.com Received: (qmail 99150 invoked by uid 60001); 9 Jul 2012 19:58:14 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1341863894; bh=sorSVByBZ/9jDoWKUCrdV1SuDV/GDafRhmhiW52fA40=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=2xsFjC0jrbqMKqmyexPJTWakmkmgGz8ZjDqKCFT05G/JQAe4xvLYW3ZFHqD91/WsHrGPwwBJ5UYVJkfC0uJYdcIa+hjp+Uk5Fm7ZugZZBe2JpEThVCUq/GnoK44ERbTiMV0+9K5WjbYJggT7E3s20a6bnOoEhnRKSGqd7bmHiuM= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=Z45Md+/i1jL7h+4O+BYCY5BN2BGBif4QzXFMVbuTjrpDIIBUcQ7YiaqLF28+V9JzEQcutfmq3Jl9k1igbT3vA9l3SfsTD/WObiKzZe9lV18AENYIBIub62mk/TvnptRlAdOyoCoOcLFV5UBAutWTNrJhBQBYwuS9MIojLoYZ3MA=; X-YMail-OSG: Hjr0voAVM1kSI3ZE4RG9RgxLRW4ex_tzY2bF3T3nX3xn_XV QFX.BK5P3OOp0c16431araxX1PQAgFb39kqD_KCimGlcEK9h_BCXHUR9Kdlz 4BjFCUQ48dfMyr_mmt8qGUGbLRYHBW5FoT_kixjR9d4wcJvfUoNOMv3JN0eD Ul3.uWC6RUJGh766ce2Ipk.33XC.ddrpR0Gfk8OR7U84VC0PAU0jIeyZ.BE2 J471O0Lg1peo1X3_nzmFyBVlwBVQdzyXGntunC800zgo5fVZwZGTu5OcepyQ wM6WTqZlk7N7wl5da0.RJnnVOcr3f5SdFbLylec60Xr4mypFs1gs2E0By9w9 HOGVp6.d0WUUln1l12KiCzYbYJplGVA36_.C7RFOem1e_GD1TT340_bV6KcR b Received: from [66.41.240.246] by web122501.mail.ne1.yahoo.com via HTTP; Mon, 09 Jul 2012 12:58:14 PDT X-Mailer: YahooMailClassic/15.0.8 YahooMailWebService/0.8.120.356233 Message-ID: <1341863894.36655.YahooMailClassic@web122501.mail.ne1.yahoo.com> Date: Mon, 9 Jul 2012 12:58:14 -0700 (PDT) From: Jason Usher To: Zaphod Beeblebrox MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: vdev/pool math with combined raidzX vdevs... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jul 2012 19:58:20 -0000 Hello again,=0A=0A=0A--- On Fri, 7/6/12, Zaphod Beeblebrox wrote:=0A=0A=0A> ... so, again with simplistic assumptions,=0A> =0A> p(= 36drz3 --- 12 drives, 3 groups) =3D p(12drz3) * 3=0A> =0A> A "vanilla" RAID= -Z2 (if I make an assumption to what you're=0A> saying) is:=0A> =0A> p(36dr= z2) =3D 36 * p(f) * 35 * p(f)=0A> =0A> ... but I can't directly answer you = question without knowing=0A> a) the=0A> structure of the RAID-Z2 array and = p(f).=A0 If we use a=0A> 1% figure for=0A> p(f), then P(36drz3,12,3) =3D 0.= 035% and p(36drz2) =3D 4.3%=0A=0A=0A(snip)=0A=0A=0A> Put simply, you add th= e probabilities of things where any=0A> can cause=0A> the failure (either d= rive of R0 failing, any one of the 3=0A> plexes of a=0A> complex array fail= ing) and you multiply things where all=0A> must fail to=0A> produce failure= .=0A=0A=0AOk. So let's start with those numbers from that hardforum link I= posted:=0A=0A(probability of data loss during a rebuild)=0A=0ARAID-10: =0A= F =3D 5%=0A=0ARAID-Z1:=0A1 - (1 - F)^(9 - 1) =3D 33.7%=0AF=3D 33.7%=0A=0ARA= ID-Z2:=0A1 - (1 - F)^(10 - 1) - (10 - 1) F (1 - F)^(10 - 2) =3D 7.1%=0AF=3D= 7.1%=0A=0ARAID-Z3:=0A1 - (1 - F)^(11 - 1) - (11 - 1) F (1 - F)^(11 - 2) - (= 11 - 1)(11 - 2) F^2 (1 - F)^(11 - 3) / 2=0AF =3D 1.15%=0A=0AAgain, doesn't = really matter what F is, since we are only interested in the comparison...= =0A=0AFrom what you said, above, striping 3 different raidz3 arrays togethe= r into one pool is ADDITIVE ... so the 1.15% rises to 3.45%.=0A=0AYes ?=0A= =0ASo we triple our risk by running all three raidz3 arrays in one pool, bu= t we still have less than half the risk of a single raidz2 vdev (with no st= riping) which is 7.1%.=0A=0AAm I on the right track here ? I think I'm mis= sing something because with one raidz3, I have a 1.15% chance of "losing a = drive during rebuild" but I am thinking about competely healthy arrays who = have a larger chance of blowing up because ONE OF THE OTHER vdevs blows fou= r drives simultaneously. =0A=0ASo I am really comparing 0% probability (if= they aren't combined in a zpool, I can take one vdev out and run over it w= ith a train and the other vdev is unharmed) with X% probability, because no= w something happening in the other vdev can ruin the healthy one...=0A=0AAm= I really the only person worrying about the interactive failure properties= of combining vdevs into a pool ?