From owner-freebsd-fs@FreeBSD.ORG Tue Nov 2 22:29:55 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16637106566C; Tue, 2 Nov 2010 22:29:55 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id AF04F8FC0A; Tue, 2 Nov 2010 22:29:54 +0000 (UTC) Received: by ywh2 with SMTP id 2so4223063ywh.13 for ; Tue, 02 Nov 2010 15:29:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=tIZ0YbyALF4du71yK4RiNb7IiS2NkApV0d2sXDf5K2k=; b=KKAGEFVlPxLqw//UNogV0lYuPcHVMAua5G8xC27U+TtJttWykJtNCZzbu69CvDEv74 IorqZvIkFQYuW6W6cRil3Qg9UH/I4UFrDpX2ipTESZqzv0W/Kcxum1gtHMfLXilGr4IH iozjmosGVMzrBNthM+xsrWTvKvnTo5ZV5aNyg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=axHbckEriDZj2zmR4pYcqMuQyi9dI6i9RcKq4SA5gpgiKNmIHPq8NlZmMSpRIswU86 jCUzrl74940cHy0BzFLpwyYae0izH/MO5RCG1Oa3xPVxDUzR5yqpAlCZIEyYMgcpaxC8 qf0hVWmMP4jq56ojtbYOHYpHV4oKsxJpFJ3w4= MIME-Version: 1.0 Received: by 10.90.4.26 with SMTP id 26mr2088325agd.84.1288736993065; Tue, 02 Nov 2010 15:29:53 -0700 (PDT) Received: by 10.90.97.7 with HTTP; Tue, 2 Nov 2010 15:29:52 -0700 (PDT) In-Reply-To: <20101102220657.GC2051@garage.freebsd.pl> References: <20101016222833.GA6765@garage.freebsd.pl> <20101102220657.GC2051@garage.freebsd.pl> Date: Tue, 2 Nov 2010 15:29:52 -0700 Message-ID: From: Freddie Cash To: Pawel Jakub Dawidek Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: Converting a non-HAST ZFS pool to a HAST pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Nov 2010 22:29:55 -0000 On Tue, Nov 2, 2010 at 3:06 PM, Pawel Jakub Dawidek wrote= : > On Tue, Nov 02, 2010 at 02:52:45PM -0700, Freddie Cash wrote: >> Okay, so converting a non-HAST ZFS setup to a HAST setup using the >> same drives won't work. >> >> Any reason that it wouldn't work when replacing the drives with larger o= nes? >> >> =C2=A0- zpool offline poolname label/disk01 >> =C2=A0- physically replace drive >> =C2=A0- glabel drive as disk01 >> =C2=A0- configure hast to use label/disk01 >> =C2=A0- zpool replace poolname label/drive01 hast/drive01 >> >> I can't think of any reason why it would fail, since the hast device >> will be twice as large as the non-hast device it's replacing. =C2=A0But >> thought I'd double-check, just to be safe. =C2=A0:) > > Yes, this should work. > >> Granted, doing it this way would required a *long* initial sync, as >> there's currently 18 TB of data in the pool. =C2=A0And more going in eve= ry >> day. =C2=A0So it might be better to start fresh. > > If you mean HAST initial sync, then this should be now improved in > r214284: > > =C2=A0 =C2=A0 =C2=A0 =C2=A0Before this change on first connect between pr= imary and > =C2=A0 =C2=A0 =C2=A0 =C2=A0secondary we initialize all the data. This is = huge waste of time > =C2=A0 =C2=A0 =C2=A0 =C2=A0and resources if there were no writes yet, as = there is no real > =C2=A0 =C2=A0 =C2=A0 =C2=A0data to synchronize. > > =C2=A0 =C2=A0 =C2=A0 =C2=A0Optimize this by sending "virgin" argument to = secondary, which > =C2=A0 =C2=A0 =C2=A0 =C2=A0gives it a hint that synchronization is not ne= eded. > > =C2=A0 =C2=A0 =C2=A0 =C2=A0In the common case (where noth nodes are confi= gured at the same > =C2=A0 =C2=A0 =C2=A0 =C2=A0time) instead of synchronizing everything, we = don't synchronize > =C2=A0 =C2=A0 =C2=A0 =C2=A0at all. > > The change is not yet merged to stable/8, AFAIR, but this will happen > today or tomorrow. > > You still need to wait for ZFS to copy the data over to the new vdev. Ah, I see what you mean. I was originally thinking of replacing the drives in the "master" server, then configuring the "slave" server, and then syncing the two. Which would take aeons to do. Instead, it can be done in a piecemeal fashion: - configure slave server using new hardware (all 1 TB drives, for example= ) - replace 1 drive in master server, configure drive as a HAST device - use the HAST device to replace the non-HAST device in the ZFS pool, which causes ZFS to resilver it - this would then start a HAST sync *of just that one drive* - wait for resilver and sync to complete - replace next drive in master server - same process as above - repeat until all drives in master server are replaced with larger drives, using /dev/hast/* devices At that point, the master server should be using all HAST devices to form the ZFS pool, and have a bunch of extra free space as a bonus. *And*, the HAST devices should be all sync'd up with the slave server. During this process, I'd have to make sure that no automatic fail-over is setup, as it needs to sync master-->slave first. But, once the sync is done, then the CARP setup and fail-over can be done. --=20 Freddie Cash fjwcash@gmail.com