From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 14:52:07 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3A3A106566B for ; Fri, 6 Jul 2012 14:52:07 +0000 (UTC) (envelope-from jusher71@yahoo.com) Received: from nm22-vm3.bullet.mail.ne1.yahoo.com (nm22-vm3.bullet.mail.ne1.yahoo.com [98.138.91.152]) by mx1.freebsd.org (Postfix) with SMTP id 64B218FC0A for ; Fri, 6 Jul 2012 14:52:07 +0000 (UTC) Received: from [98.138.90.52] by nm22.bullet.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 14:52:01 -0000 Received: from [98.138.89.197] by tm5.bullet.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 14:52:01 -0000 Received: from [127.0.0.1] by omp1055.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 14:52:01 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 114507.84514.bm@omp1055.mail.ne1.yahoo.com Received: (qmail 18995 invoked by uid 60001); 6 Jul 2012 14:52:00 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1341586320; bh=V5T7jxvSy46KOZ3pjyM27IAGVOJWfcd+CDxitWiDBxA=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=aTSVuC2dMf1Cwab1kRfIFo5jG2mFhkNH+F2MTmX65EzBHK9+azKExN2DGnTwmoNiUpam2iagqzHUlm+Jl+O9jPMEuDK6+xuNWGzxlEBQBsx4qrpLhO5a65IfI4kPLlCkPdNkHD3/xuNByE9ZKslCACVZVSD7qjSHwbycC3pHNq8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=hknasFCjvN+dDCY5s9DW3SCYHM+Xh0SYP2IPClXsoT47LqAOPVSndLBDCrzcFUCjX6jdeJ2BcwtUFj5+Ja0s1MYZjOCWjpWRxMw4iJRDT1LyTe79x+jNR11VKi5BnRWeW631SVabihVL+h+wjRw+4fnTIQQls2wHwGYPMjdR5o0=; X-YMail-OSG: TzeQAmkVM1ncNYsn3x2pPL4T4UBUpPN506ZMbV6eMukadQY o3RHedV_IMkse_6HhV60RIvK_wcxS6EmFpGxggVnzgWTK8igwIi0B_JMQiuN vn0f1GWIBiQU2qdfNyVLUhhTjc5yctxJjewTHjxkWzyi35P8r6fW0tMODv0x nBffVRmCWS6Nz7BubzkvYIym74TV6HjGaFhZz2TfDne3gHRMtpqeXv7Odszz FK4r.KPsQHP3oLzjqzWnCcn1VFFnsDDVIGYM4eO0fcTxCoWMmvLJV5sgLba. VN6Uwizrnkouc2IEce3pzfOPDCyqb.2Q7GGHcsXY9XTt._mObFg_l4BTcOc6 dTYmpITMvBVm024D3acknT3WKwJ95gm8oNVk_FS92NBa3uB_LPBOp5FyNzNH Q3OHw8Li3D.8ZV9nFvGau2Vl5e4pwKVg0n6IP.kuO9LJa0C_PnmH24X89lyE if_Po.4OdFQYpbYIT4Qe.selMe5Hij0uQ6qL0zRiVyUJRnbxAMBh1OgUs Received: from [32.178.191.35] by web122502.mail.ne1.yahoo.com via HTTP; Fri, 06 Jul 2012 07:52:00 PDT X-Mailer: YahooMailClassic/15.0.8 YahooMailWebService/0.8.120.356233 Message-ID: <1341586320.93244.YahooMailClassic@web122502.mail.ne1.yahoo.com> Date: Fri, 6 Jul 2012 07:52:00 -0700 (PDT) From: Jason Usher To: Zaphod Beeblebrox In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: vdev/pool math with combined raidzX vdevs... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 14:52:07 -0000 =0A=0A--- On Fri, 7/6/12, Zaphod Beeblebrox wrote:=0A= =0A> Is there some penalty for not=0A> googling some basic stats course?=A0= OK.=0A> This is from memory (hint: you probably should google).=0A=0A=0AAc= tually I spent a few hours googling zfs probabilities, and I cannot find an= y discussion of the changes in failure probability when multiple raidzX are= striped in a single pool.=0A=0AI did not try to google the math and do it = myself, as I wouldn't trust my own results.=0A=0A=0A> Similarly,=0A> =0A> p= (12drz2) =3D 12 * p(f) * 11 * p(f) * 10 * p(f)=0A> p(12drz3) =3D 12 * p(f) = * 11 * p(f) * 10 * p(f) * 9 * p(f)=0A> =0A> ... again with those assumption= s are more complex=0A> probabilities given=0A> your replacement strategy.= =0A> =0A> ... so, again with simplistic assumptions,=0A> =0A> p(36drz3 --- = 12 drives, 3 groups) =3D p(12drz3) * 3=0A> =0A> A "vanilla" RAID-Z2 (if I m= ake an assumption to what you're=0A> saying) is:=0A> =0A> p(36drz2) =3D 36 = * p(f) * 35 * p(f)=0A> =0A> ... but I can't directly answer you question wi= thout knowing=0A> a) the=0A> structure of the RAID-Z2 array and p(f).=A0 If= we use a=0A> 1% figure for=0A> p(f), then P(36drz3,12,3) =3D 0.035% and p(= 36drz2) =3D 4.3%=0A> =0A> ... that is the raid-Z2 case (one group of 36 dri= ves, two=0A> redundant=0A> --- which is crazy) is 4.3% likely to fail where= the 3-group=0A> RAID-Z3=0A> is only 0.035% likely to fail.=A0 As a more sa= ne=0A> comparison,=0A> p(36drz2,12,3) =3D 3.8%=0A=0A=0AOk, you're right - I= did not specify the structure of the raidz2 array.=0A=0AWhat I meant to co= mpare was the failure probabilities of:=0A=0A- a single raidz2 vdev made up= of 12 disks (10 data, 2 parity)=0A=0Avs.=0A=0A- a single raidz3 vdev made = up of 12 disks (9 data, 3 parity)=0A=0Avs.=0A=0A- a single raidz3 vdev made= up of 12 disks (9 data, 3 parity) which ALSO happens to be participating i= n a stripe with two other identical raidz3 vdevs, all in one zpool.=0A=0A= =0AI can see some probabilites for the first two examples here:=0A=0Ahttp:/= /hardforum.com/showthread.php?t=3D1621123=0A=0A(search for first post on pa= ge by "john4200")=0A=0Aas well as your own math, which appears to be the sa= me.=0A=0A=0AAlso, I think we don't need to specify the failure rate, F, sin= ce we are merely comparing three scenarios, and can compare results that st= ill contain an 'F' variable in them ... that is, our answers can contain a = yet undefined 'F', right ?=0A=0A=0AAs for myself, I have decided that raidz= 2 is "not enough" for me, and at the same time, would really, really like t= o combine three raidz3 into a single zpool ... but I don't want to do that = if that configuration brings me back to raidz2-ish failure probabilities...