From owner-freebsd-hackers@FreeBSD.ORG Mon Jun 1 14:15:49 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9712B1065680 for ; Mon, 1 Jun 2009 14:15:49 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-ew0-f212.google.com (mail-ew0-f212.google.com [209.85.219.212]) by mx1.freebsd.org (Postfix) with ESMTP id 132F68FC12 for ; Mon, 1 Jun 2009 14:15:48 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: by ewy8 with SMTP id 8so4383247ewy.43 for ; Mon, 01 Jun 2009 07:15:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:subject:from:to:cc :in-reply-to:references:content-type:date:message-id:mime-version :x-mailer:content-transfer-encoding; bh=BKNoYOaCrkiiOAArDOkBXbL2YgpANgm8YMnw7FHA10M=; b=HTr69orpQMbMDy7Snadrv2aIe5zuynmhWVOB3Hr/Nn3+xprKhm7ZixKhyNcPPw41tj PQ00T+FVg/Y7TFKKQ+4VvML4xCcLn0ddD7sNZQYkm+yRtBFxzi932sm5SCylJnyQHTMF V2bbhf1MleN5eK2HeUfxcuCzVqzI/Ba576QCI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:date :message-id:mime-version:x-mailer:content-transfer-encoding; b=Tx26jsfrRn2eY/CLnt74fTvpU/OCNS/iBowGktEXRk5KnV4WBtn1WCs0U51spxKt3R R50H3w70zPPIhwhivDK4hfTOpiURgvEJ9lp4Pv01k9DljtxyeSYYH57gyZR5VghdAbvi abS+42Eb5heCgQO5iSRD0i3hP/NViscp/E+0U= Received: by 10.210.27.14 with SMTP id a14mr921483eba.9.1243865744741; Mon, 01 Jun 2009 07:15:44 -0700 (PDT) Received: from ?127.0.0.1? (87-194-39-182.bethere.co.uk [87.194.39.182]) by mx.google.com with ESMTPS id 7sm1252459eyg.47.2009.06.01.07.15.43 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Jun 2009 07:15:44 -0700 (PDT) From: Tom Evans To: krad In-Reply-To: <20754E8E3F5940E0875AD0BC76A78BAB@uk.tiscali.intl> References: <20090530175239.GA25604@logik.internal.network> <20090530144354.2255f722@bhuda.mired.org> <20090530191840.GA68514@logik.internal.network> <20090530162744.5d77e9d1@bhuda.mired.org> <20090531201445.GA82420@logik.internal.network> <0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl> <20090531235943.GA77374@logik.internal.network> <3346D963EE2E4D28AE89CB71C2F92939@uk.tiscali.intl> <1243860612.9871.68.camel@strangepork.london.mintel.ad> <20754E8E3F5940E0875AD0BC76A78BAB@uk.tiscali.intl> Content-Type: text/plain Date: Mon, 01 Jun 2009 15:15:43 +0100 Message-Id: <1243865743.9871.75.camel@strangepork.london.mintel.ad> Mime-Version: 1.0 X-Mailer: Evolution 2.26.1.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, xorquewasp@googlemail.com Subject: RE: Request for opinions - gvinum or ccd? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2009 14:15:50 -0000 On Mon, 2009-06-01 at 14:19 +0100, krad wrote: > no you would only loose the data for that block. Zfs also checksums meta > data, but by default keeps multiple copies of it so that's fairly resilient. > If you had the copies set to > 1 then you wouldn't loose the block either, > unless you were real unlucky. > > It's just about pushing the odds back further and further. If you are super > paranoid by all means put in 48 drive, group them into 5 x 8 drive raidz2 > vdevs, have a bunch of hot spares, and enable copies=5 for blocks and > metadata, then duplicate the system and put the other box on another > continent and zfs send all you updates every 15 mins via a private > deadicated. This will all prove very resilient, but you will get very little > % storage from your drives, and have quite a large bandwidth bill 8) > > Oh and don't forget the scrub you disk regularly. BTW that would rebuild any > missing copies as well (eg if you increase the number of copies after data > is stored on the fs) > Well, no you wouldn't, because ZFS would never get to try to recover that error. Since that one block is bad, and you lost a disk, your underlying RAID-5 would not be able to recoverable, and you just lost the entire contents of the RAID-5. ZFS wouldn't be able to recover anything from it. The only time ZFS could recover from this scenario is if you scrubbed before you had your disk failure. Hard to predict disk failures.. What I'm trying to say (badly) is that this is redundancy that ZFS knows nothing about, so it cannot recover from it in the same manner that a 5 disk raidz can. If this happened to a 5 disk raid-z, you would lose just the corrupted block, rather than all your data. PS, top posting is still bad. Thanks for making me cut the context out of all these emails. Cheers Tom