From owner-freebsd-fs@FreeBSD.ORG Mon Dec 3 15:25:35 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 37CBC2E3 for ; Mon, 3 Dec 2012 15:25:35 +0000 (UTC) (envelope-from gtodd@bellanet.org) Received: from mail-ie0-f176.google.com (mail-ie0-f176.google.com [209.85.223.176]) by mx1.freebsd.org (Postfix) with ESMTP id E38B28FC12 for ; Mon, 3 Dec 2012 15:25:34 +0000 (UTC) Received: by mail-ie0-f176.google.com with SMTP id 13so5186448iea.21 for ; Mon, 03 Dec 2012 07:25:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:x-x-sender:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version:content-type:x-gm-message-state; bh=CRovgCjE5m4aHQMlNN3MyQBxzqDRdrN2T8rpja4xApE=; b=QRf0tyTe3QJcmkVP6kjGUY+3hR4Ew6UjIqxtq5Zbh+94ApREXsGJ//jViX0kmDgdos zzA8pwzoseVJaNj4Mpv7siKCU20WYcv+J7h1qoI4h756SR8fDwJn0OQJkByKKYFYMRmb 5GKgjMdnzr0QN7zS1TRi1h6KnZW/AJyM/uEQYp17h0cW0w+uDOmi3wbi3oAeffcaJf80 bh5YFKPZhUtJcjPQqn4uc75wuhdCkHQJr5cMVjw+MNovoDalaTkLYMJxNHsxBu6x9PMZ PMt4G1uB7Tdn+TLkFCuBMQIzaN6f+dMpeh4BnHkZ8r0CCdASnbAXeH3XqUNtPxCIN7OB tHHg== Received: by 10.50.40.225 with SMTP id a1mr6455581igl.7.1354548333843; Mon, 03 Dec 2012 07:25:33 -0800 (PST) Received: from wawanesa.iciti.ca (CPE0080c8f208a5-CM001371173cf8.cpe.net.cable.rogers.com. [99.246.61.82]) by mx.google.com with ESMTPS id az6sm7209416igb.11.2012.12.03.07.25.32 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 03 Dec 2012 07:25:32 -0800 (PST) Date: Mon, 3 Dec 2012 10:24:15 -0500 (EST) From: Graham Todd X-X-Sender: gtodd@wawanesa.iciti.ca To: nikitosiusis Subject: Re: "zpool add" safety checks are skipped if a pool was created with "-f" In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Gm-Message-State: ALoCoQn/lYJ11GTAZ7D7XjfcqxeUrRNniTxjt2CcICECLCUrP1+8k1L9sIi8cQgG5zrwsCTCzibU Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Dec 2012 15:25:35 -0000 On Tue, 13 Nov 2012, nikitosiusis wrote: > Good day everyone. > I managed to add a single device to a raidz2 pool by a typo(actually I > wanted to add a cache device) without "-f" flag. This can be > reproduced with md devices. > # for a in {1..8}; do dd if=/dev/zero of=$a bs=1M count=96;done > # dd if=/dev/zero of=9 bs=1M count=128 > # ls -la > -rw-r--r-- 1 root wheel 100663296 Nov 12 21:04 1 > -rw-r--r-- 1 root wheel 100663296 Nov 12 21:04 2 > -rw-r--r-- 1 root wheel 100663296 Nov 12 21:04 3 > -rw-r--r-- 1 root wheel 100663296 Nov 12 21:04 4 > -rw-r--r-- 1 root wheel 100663296 Nov 12 21:04 5 > -rw-r--r-- 1 root wheel 100663296 Nov 12 21:04 6 > -rw-r--r-- 1 root wheel 100663296 Nov 12 21:04 7 > -rw-r--r-- 1 root wheel 100663296 Nov 12 21:04 8 > -rw-r--r-- 1 root wheel 134217728 Nov 12 21:04 9 > # for a in {1..9}; do mdconfig -f ~/tmp/$a; done > > Now we create a raidz pool with 8 identical drives. > > # zpool create testpool raidz2 md{1..8} > # zpool add testpool md9 > invalid vdev specification > use '-f' to override the following errors: > mismatched replication level: pool uses raidz and new vdev is disk > > It's ok. It doesn't allow to add a device. > Now we create a new pool, but with devices of different size(I don't > know what is the difference in size allowed, I used 32mb). > > # zpool create testpool raidz2 md{1..7} md9 > invalid vdev specification > use '-f' to override the following errors: > raidz contains devices of different sizes > # zpool create -f testpool raidz2 md{1..7} md9 > # zpool add testpool md8 > And we have no error here - single drive is added to the pool. Since > it is an undoable action - it can ruin your pool and should be > considered as a bug imho. Here is my result in production. > NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT > root 3.63T 1.06T 2.57T 29% 1.00x ONLINE - > raidz2 3.62T 1.06T 2.57T - > ada0 - - - - > ada1 - - - - > ada3 - - - - > ada4 - - - - > ada5 - - - - > ada6 - - - - > ada7 - - - - > ada8 - - - - > da0p1 3.75G 76.5K 3.75G - > > # uname -rv > 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #2: Thu Nov 8 13:50:55 UTC 2012 > root@ex.a.nikitos.name:/usr/obj/usr/src/sys/GENERIC I haven't done it lately, but it was possible to make a similar error with a simple mirror vdev if you were trying to add a second pool or a second separate vdev in an existing pool and "added" instead of "attached", or mistyped the pool name (sometimes pools have similar names). I agree that if this risks creating an undoable mistake in a pool the lack of checks or verification should be considered a "bug". > # zpool get version testpool > NAME PROPERTY VALUE SOURCE > testpool version 28 default > # zfs get version testpool > NAME PROPERTY VALUE SOURCE > testpool version 5 - > > btw is there a chance to remove this device now? I'm not sure if some combination of split, detach, remove, destroy would have helped in the simpler case of a mirror - the split feature was not available when this happened. We had root/boot on a separate device, not much data and extra disks, so we could zfs send/recv things to a new pool and rebuild things that way. Better checks at creation time would have prevented any problems. Is there a PR for these sort of issues at freebsd.org or illumos or elsewhere? cheers