From owner-freebsd-stable@FreeBSD.ORG Wed Jun 29 18:13:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9776106564A for ; Wed, 29 Jun 2011 18:13:33 +0000 (UTC) (envelope-from cpghost@cordula.ws) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id A7FBE8FC08 for ; Wed, 29 Jun 2011 18:13:33 +0000 (UTC) Received: by vws18 with SMTP id 18so1498740vws.13 for ; Wed, 29 Jun 2011 11:13:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.176.3 with SMTP id bc3mr430161vcb.26.1309371212601; Wed, 29 Jun 2011 11:13:32 -0700 (PDT) Received: by 10.220.7.146 with HTTP; Wed, 29 Jun 2011 11:13:32 -0700 (PDT) X-Originating-IP: [93.221.173.99] In-Reply-To: References: Date: Wed, 29 Jun 2011 20:13:32 +0200 Message-ID: From: "C. P. Ghost" To: Todd Wasson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: zfs on geli vs. geli on zfs (via zvol) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2011 18:13:34 -0000 On Tue, Jun 28, 2011 at 8:45 PM, Todd Wasson wrote: > While zfs on geli is less complex (in the sense that geli on zfs involves > two layers of filesystems), I'm concerned as to whether encrypting the > device will somehow affect zfs' ability to detect silent corruption, > self-heal, or in any other way adversely affect zfs' functionality. =A0In= my > mind, if I use geli on zfs, then I've got zfs directly on a device and th= e > zvol it's providing will be transparently gaining the benefits of zfs' > various features, providing a "safety layer" against device failure and > silent corruption that I'm not sure if geli would detect. I'm going out on a limb here, but that's how I see it without intimate knowledge of the geom/geli and zfs code paths involved. Basically, you leave the transparent encryption of sectors to geli, and the integrity checking and repairing to zfs, and you should be *mostly* secure, except for the failure modes below. .eli devices behave just like normal disks, in the sense that they are a block device that transparently encrypts and decrypts sectors when they are accessed. So what could go wrong there: 1. single sectors may be corrupted on disk (e.g. bits flipping). 2. geli metadata (keys etc...) are destroyed. Same for glabel metadata. Case 1.) is probably harmless, because geli would return a corrupted sectors' content to zfs... which zfs will likely detect because it wouldn't checksum correctly. So zfs will correct it out of redundant storage, and write it back through a new encryption. BE CAREFUL: don't enable hmac integrity checks in geli, as that would prevent geli from returning corrupted data and would result in hangs! Case 2.) is a bigger problem. If a sector containing vital geli metadata (perhaps portions of keys?) gets corrupted, and geli had no way to detect and/or correct this (e.g. by using redundant sectors on the same .eli volume!), the whole .eli, or maybe some stripes out of it, could become useless. ZFS couldn't repair this at all... at least not automatically. You'll have to MANUALLY reformat the failed .eli device, and resilver it from zfs redundant storage later. There may be other failure modes involved as well. I don't know. But in most practical day to day uses, with enough redundancy and regular backups, a zfs-over-geli should be good enough. I wouldn't put {zfs,ufs}-over-geli-over-raw-zpool though, as this would involve considerable overhead, IMHO. In this case, I'd rather use a gmirror as a backend, as in a setup: {zfs,ufs}-over-geli-over-{gmirror,graid3} or something similar. But I've never tried this though. -cpghost. --=20 Cordula's Web. http://www.cordula.ws/