From owner-freebsd-fs@FreeBSD.ORG Tue Dec 9 08:53:53 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB44E1065672 for ; Tue, 9 Dec 2008 08:53:53 +0000 (UTC) (envelope-from james-freebsd-fs2@jrv.org) Received: from mail.jrv.org (adsl-70-243-84-13.dsl.austtx.swbell.net [70.243.84.13]) by mx1.freebsd.org (Postfix) with ESMTP id 00ABB8FC1E for ; Tue, 9 Dec 2008 08:53:52 +0000 (UTC) (envelope-from james-freebsd-fs2@jrv.org) Received: from kremvax.housenet.jrv (kremvax.housenet.jrv [192.168.3.124]) by mail.jrv.org (8.14.3/8.13.1) with ESMTP id mB98MgQw017766; Tue, 9 Dec 2008 02:22:43 -0600 (CST) (envelope-from james-freebsd-fs2@jrv.org) Authentication-Results: mail.jrv.org; domainkeys=pass (testing) header.from=james-freebsd-fs2@jrv.org DomainKey-Signature: a=rsa-sha1; s=enigma; d=jrv.org; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:to:cc:subject: references:in-reply-to:x-enigmail-version:content-type:content-transfer-encoding; b=NQZXmH0HhiAIWhrc5VeIkPSMmeT8rvWkYfhuhhJ/59/ERe3Oh7qJ2jzF38hergo9I 05v+sP7dWqNGE3+sGT5Zw58665cloPka87ADf/T30Dgew1Th86Bj8Q3N2pNekFuEmyp KwWFmZE/0UzcmW/I8XrXk/aoV+IN7sgS4iXI5oo= Message-ID: <493E2AD2.8070704@jrv.org> Date: Tue, 09 Dec 2008 02:22:42 -0600 From: "James R. Van Artsdalen" User-Agent: Thunderbird 2.0.0.18 (Macintosh/20081105) MIME-Version: 1.0 To: Bryan Alves References: <92f477740812082155y3365bec7v5574206dd1a98e26@mail.gmail.com> In-Reply-To: <92f477740812082155y3365bec7v5574206dd1a98e26@mail.gmail.com> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: ZFS resize disk vdev X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Dec 2008 08:53:53 -0000 Bryan Alves wrote: > I'm thinking about using a hardware raid array with ZFS, using a single disk > vdev zpool. I want the ability to add/remove disks to an array, and I'm > still unsure of the stability of zfs as a whole. I'm looking for an easy > way to resize and manage disks that are greater than 2 terabytes. > > If I have a single block device, /dev/da0, on my system that is represented > by a zfs disk vdev, and the size of this block device grows (because the > underlying hardware raid expands), will zfs correctly expand? And will it > correctly expand in place? > I see no benefit to using hardware RAID for a vdev. If there is any concern over ZFS stability then you're using a filesystem you suspect on an - at best - really reliable disk: not a step forward! I think best practice is to configure the disk controller to present the disks as JBOD and let ZFS handle things: avoid fancy hardware RAID controllers altogether and use the fastest JBOD controller configuration available. Using a hardware RAID seems likely to hurt performance since the hardware RAID must issue extra reads for partial parity-stripe updates: ZFS never does in-place disk writes and rarely if ever does partial parity-stripe updates. Block allocation will suffer since the filesystem allocator can't know the geometry of the underlying storage array when laying out a file. Parity rebuilds ("resilvering") can be much faster in ZFS since only things that are different need to be recomputed when a disk is reattached to a redundant vdev (and if a disk is replaced free space need not have parity computed). And hardware RAID just adds another layer of processing to slow things down. I'm not sure how ZFS reacts to an existing disk drive suddenly becoming larger. Real disk drives don't do that and ZFS is intended to use real disks. There are some uberblocks (pool superblocks) at the end of the disk and ZFS probably won't be able to find them if the uberblocks at the front of the disk are clobbered and the "end of the disk" has moved out away from the remaining uberblocks. You can replace all of the members of a redundant vdev one-by-one with larger disks and increase the storage capacity of that vdev and hence the pool. I routinely run zpools of 4TB and 5TB, which isn't even warming up for some people. Sun has had customers with ZFS pools in the petabytes. "disks that are greater than 2 terabytes" are pocket change.