From owner-freebsd-fs@FreeBSD.ORG Thu Apr 22 14:52:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B0EB106567B for ; Thu, 22 Apr 2010 14:52:28 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [193.34.186.230]) by mx1.freebsd.org (Postfix) with ESMTP id 0E8208FC13 for ; Thu, 22 Apr 2010 14:52:27 +0000 (UTC) Received: from wratting.cam.lispworks.com (wratting [192.168.1.17]) by lwfs1-cam.cam.lispworks.com (8.12.9p2/8.12.9) with SMTP id o3MEfdQc015214; Thu, 22 Apr 2010 15:41:39 +0100 (BST) (envelope-from martin@lispworks.com) Received: (from martin@localhost) by wratting.cam.lispworks.com (SMI-8.6) id PAA07296; Thu, 22 Apr 2010 15:41:36 +0100 Date: Thu, 22 Apr 2010 15:41:36 +0100 Message-Id: <201004221441.PAA07296@wratting.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org Subject: zfs: unexpected resilver of old disk when attaching a new mirror disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Apr 2010 14:52:28 -0000 When I attach a second disk to a single-disk zpool to make a mirrored pool, the original disk is partially resilvered. Is that expected? It makes me worry that some of the good data is being overwritten. It happens on 8.0-RELEASE (pool version 13) and the 9.0-CURRENT-201004 snapshot (pool version 14). It doesn't happen on Open Solaris snv_111b (also pool version 14). My setup commands for the test, making a 64MB zpool containing a 16MB file are: dd if=/dev/zero of=/tmp/zdisk0 bs=1m count=64 zpool create ztest /tmp/zdisk0 dd if=/dev/zero of=/ztest/a-file bs=1m count=16 The pool looks as expected: pool: ztest state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM ztest ONLINE 0 0 0 /tmp/zdisk0 ONLINE 0 0 0 errors: No known data errors I then attach a second device: dd if=/dev/zero of=/tmp/zdisk1 bs=1m count=64 zpool attach ztest /tmp/zdisk0 /tmp/zdisk1 and the pool looks like this after a while: pool: ztest state: ONLINE scrub: resilver completed after 0h0m with 0 errors on Thu Apr 22 00:29:32 2010 config: NAME STATE READ WRITE CKSUM ztest ONLINE 0 0 0 mirror ONLINE 0 0 0 /tmp/zdisk0 ONLINE 0 0 0 86K resilvered /tmp/zdisk1 ONLINE 0 0 0 16.1M resilvered errors: No known data errors As expected, /tmp/zdisk1 was resilvered with 16MB of data for /ztest/a-file, but why was a small amount of /tmp/zdisk0 also resilvered? The amount varies varies (I've seen up to 220K) but it is never 0. With real disks constaining 5GB, the amount resilvered on the old disk is up to 70MB and it increases gradually from 0 as the resilvering of the new disk proceeds. __Martin