From owner-svn-src-vendor@freebsd.org Wed Sep 20 06:34:49 2017 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B681E25E2D; Wed, 20 Sep 2017 06:34:49 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6789F82446; Wed, 20 Sep 2017 06:34:49 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v8K6Ymgm087166; Wed, 20 Sep 2017 06:34:48 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v8K6Ym39087165; Wed, 20 Sep 2017 06:34:48 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201709200634.v8K6Ym39087165@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 20 Sep 2017 06:34:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323789 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 323789 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2017 06:34:49 -0000 Author: avg Date: Wed Sep 20 06:34:48 2017 New Revision: 323789 URL: https://svnweb.freebsd.org/changeset/base/323789 Log: 8473 scrub does not detect errors on active spares illumos/illumos-gate@554675eee75dd2d7398d960aa5c81083ceb8505a https://github.com/illumos/illumos-gate/commit/554675eee75dd2d7398d960aa5c81083ceb8505a https://www.illumos.org/issues/8473 Scrubbing is supposed to detect and repair all errors in the pool. However, it wrongly ignores active spare devices. The problem can easily be reproduced in OpenZFS at git rev 0ef125d with these commands: truncate -s 64m /tmp/a /tmp/b /tmp/c sudo zpool create testpool mirror /tmp/a /tmp/b spare /tmp/c sudo zpool replace testpool /tmp/a /tmp/c /bin/dd if=/dev/zero bs=1024k count=63 oseek=1 conv=notrunc of=/tmp/c sync sudo zpool scrub testpool zpool status testpool # Will show 0 errors, which is wrong sudo zpool offline testpool /tmp/a sudo zpool scrub testpool zpool status testpool # Will show errors on /tmp/c, which should've already been fixed FreeBSD head is partially affected: the first scrub will detect some errors, but the second scrub will detect more. Reviewed by: Andy Stormont Reviewed by: Matt Ahrens Reviewed by: George Wilson Approved by: Richard Lowe Author: Alan Somers Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Wed Sep 20 06:29:11 2017 (r323788) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Wed Sep 20 06:34:48 2017 (r323789) @@ -29,6 +29,9 @@ #include #include +#include +#include +#include #include #include #include @@ -49,7 +52,7 @@ typedef struct mirror_child { typedef struct mirror_map { int mm_children; - int mm_replacing; + int mm_resilvering; int mm_preferred; int mm_root; mirror_child_t mm_child[1]; @@ -86,7 +89,7 @@ vdev_mirror_map_alloc(zio_t *zio) mm = kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP); mm->mm_children = c; - mm->mm_replacing = B_FALSE; + mm->mm_resilvering = B_FALSE; mm->mm_preferred = spa_get_random(c); mm->mm_root = B_TRUE; @@ -109,13 +112,51 @@ vdev_mirror_map_alloc(zio_t *zio) mc->mc_offset = DVA_GET_OFFSET(&dva[c]); } } else { + int replacing; + c = vd->vdev_children; mm = kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP); mm->mm_children = c; - mm->mm_replacing = (vd->vdev_ops == &vdev_replacing_ops || + /* + * If we are resilvering, then we should handle scrub reads + * differently; we shouldn't issue them to the resilvering + * device because it might not have those blocks. + * + * We are resilvering iff: + * 1) We are a replacing vdev (ie our name is "replacing-1" or + * "spare-1" or something like that), and + * 2) The pool is currently being resilvered. + * + * We cannot simply check vd->vdev_resilver_txg, because it's + * not set in this path. + * + * Nor can we just check our vdev_ops; there are cases (such as + * when a user types "zpool replace pool odev spare_dev" and + * spare_dev is in the spare list, or when a spare device is + * automatically used to replace a DEGRADED device) when + * resilvering is complete but both the original vdev and the + * spare vdev remain in the pool. That behavior is intentional. + * It helps implement the policy that a spare should be + * automatically removed from the pool after the user replaces + * the device that originally failed. + */ + replacing = (vd->vdev_ops == &vdev_replacing_ops || vd->vdev_ops == &vdev_spare_ops); - mm->mm_preferred = mm->mm_replacing ? 0 : + /* + * If a spa load is in progress, then spa_dsl_pool may be + * uninitialized. But we shouldn't be resilvering during a spa + * load anyway. + */ + if (replacing && + (spa_load_state(vd->vdev_spa) == SPA_LOAD_NONE) && + dsl_scan_resilvering(vd->vdev_spa->spa_dsl_pool)) { + mm->mm_resilvering = B_TRUE; + } else { + mm->mm_resilvering = B_FALSE; + } + + mm->mm_preferred = mm->mm_resilvering ? 0 : (zio->io_offset >> vdev_mirror_shift) % c; mm->mm_root = B_FALSE; @@ -271,7 +312,7 @@ vdev_mirror_io_start(zio_t *zio) mm = vdev_mirror_map_alloc(zio); if (zio->io_type == ZIO_TYPE_READ) { - if ((zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_replacing) { + if ((zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_resilvering) { /* * For scrubbing reads we need to allocate a read * buffer for each child and issue reads to all @@ -408,7 +449,7 @@ vdev_mirror_io_done(zio_t *zio) if (good_copies && spa_writeable(zio->io_spa) && (unexpected_errors || (zio->io_flags & ZIO_FLAG_RESILVER) || - ((zio->io_flags & ZIO_FLAG_SCRUB) && mm->mm_replacing))) { + ((zio->io_flags & ZIO_FLAG_SCRUB) && mm->mm_resilvering))) { /* * Use the good data we have in hand to repair damaged children. */