From owner-freebsd-bugs@FreeBSD.ORG Tue Apr 24 18:50:10 2012 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B492F1065670 for ; Tue, 24 Apr 2012 18:50:10 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8DE528FC15 for ; Tue, 24 Apr 2012 18:50:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q3OIoAss052962 for ; Tue, 24 Apr 2012 18:50:10 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q3OIoAoB052961; Tue, 24 Apr 2012 18:50:10 GMT (envelope-from gnats) Resent-Date: Tue, 24 Apr 2012 18:50:10 GMT Resent-Message-Id: <201204241850.q3OIoAoB052961@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, David Alves Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DC234106566B for ; Tue, 24 Apr 2012 18:42:56 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id C6CDF8FC08 for ; Tue, 24 Apr 2012 18:42:56 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q3OIguSC078664 for ; Tue, 24 Apr 2012 18:42:56 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id q3OIgu6n078661; Tue, 24 Apr 2012 18:42:56 GMT (envelope-from nobody) Message-Id: <201204241842.q3OIgu6n078661@red.freebsd.org> Date: Tue, 24 Apr 2012 18:42:56 GMT From: David Alves To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/167272: ZFS Disks reordering causes ZFS to pick the wrong drive X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 18:50:10 -0000 >Number: 167272 >Category: kern >Synopsis: ZFS Disks reordering causes ZFS to pick the wrong drive >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Tue Apr 24 18:50:10 UTC 2012 >Closed-Date: >Last-Modified: >Originator: David Alves >Release: 8.2-RELEASE >Organization: >Environment: FreeBSD xxxxxxxx 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 >Description: Hello, ZFS shows the disks labels when invoking zpool status, if a disk was removed (physically) and the server rebooted, a reordering of the disks is done, and it appears that the old label is used by a valid disk ( the slot of the removed disks don't contains any new disks) ZFS reports it as follows : raidz2 DEGRADED 0 0 0 da16 ONLINE 0 0 0 da17 ONLINE 0 0 0 da18 ONLINE 0 0 0 da19 ONLINE 0 0 0 da20 ONLINE 0 0 0 da21 OFFLINE 0 0 0 da21 ONLINE 0 0 0 da22 ONLINE 0 0 0 raidz2 DEGRADED 0 0 0 da23 ONLINE 0 0 0 da24 ONLINE 0 0 0 da25 ONLINE 0 0 0 da26 ONLINE 0 0 0 da27 ONLINE 0 0 0 da27 OFFLINE 0 0 0 da29 ONLINE 0 0 0 da30 ONLINE 0 0 0 Notice the da21 and da27 drives. the old disks da21/da27 are shown offline (because they were offlined and removed) but the reordering as assigned those labels to others running drives. The problem is when performing a "zpool replace", "zpool replace" will pick the first label when attempting to replace a disk example when replacing da21: It picked up the da21 offline drive to replace because it was the first on the list. raidz2 DEGRADED 0 0 0 da16 ONLINE 0 0 0 da17 ONLINE 0 0 0 da18 ONLINE 0 0 0 da19 ONLINE 0 0 0 da20 ONLINE 0 0 0 replacing DEGRADED 0 0 0 da21 OFFLINE 0 0 0 da31 ONLINE 0 0 0 37.1G resilvered da21 ONLINE 0 0 0 da22 ONLINE 0 0 1 512 resilvered raidz2 DEGRADED 0 0 0 da23 ONLINE 0 0 0 da24 ONLINE 0 0 0 da25 ONLINE 0 0 0 da26 ONLINE 0 0 0 da27 ONLINE 0 0 0 da27 OFFLINE 0 0 0 da29 ONLINE 0 0 0 da30 ONLINE 0 0 0 example when replacing da27: It picked up the da27 online drive to replace because it was the first on the list. raidz2 ONLINE 0 0 0 da16 ONLINE 0 0 0 da17 ONLINE 0 0 0 da18 ONLINE 0 0 0 da19 ONLINE 0 0 0 da20 ONLINE 0 0 0 da31 ONLINE 0 0 0 da21 ONLINE 0 0 0 da22 ONLINE 0 0 1 raidz2 DEGRADED 0 0 0 da23 ONLINE 0 0 0 da24 ONLINE 0 0 0 da25 ONLINE 0 0 0 da26 ONLINE 0 0 0 replacing ONLINE 0 0 0 da27 ONLINE 0 0 0 da28 ONLINE 0 0 0 80.5G resilvered da27 OFFLINE 0 0 0 da29 ONLINE 0 0 0 da30 ONLINE 0 0 0 That would be nice if we can choose exactly what drive from the pool we are going to replace. Thanks you. >How-To-Repeat: To repeat the problem: offline a drive remove the drive reboot >Fix: >Release-Note: >Audit-Trail: >Unformatted: