From owner-freebsd-fs@freebsd.org Thu May 18 02:38:39 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24EDDD723D2; Thu, 18 May 2017 02:38:39 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: from mail-pg0-x231.google.com (mail-pg0-x231.google.com [IPv6:2607:f8b0:400e:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5F961D7E; Thu, 18 May 2017 02:38:38 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: by mail-pg0-x231.google.com with SMTP id u28so15660524pgn.1; Wed, 17 May 2017 19:38:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fb817rkaPmR7yPis4e72q48e7yP4C00HB6jBQE5/w60=; b=hoVz6ChGvAM9FEDJU+0tpHWtV8cZ51wyPX434Imyp+Be9EU+wMJoO/BgiqTHlhgyHV ZgtU8p8fdBpWxWJpzNNtExoKrH+L24SiYxPDthcuo1DdT5padZNey9n6wD/jpI+DGaxK G9OW6CN8xYo8S0AarmuSp1TiOHX4r94yaXy89XLZhRR1WdIUDyq6kDOqhzUr5qsaD64D qDyE4+SiyYiVIj23gj52L3+xRp8rjPN2l6IfklQlC2f55nR+ZaGXBNvY77bsPMW4khAu Jy3ZnaxAM4op3qpjkbBtI1gR/ZKODexRY6RN+6tBS4qMnnr40KzH/bHok2ehSasyMCUW vzcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fb817rkaPmR7yPis4e72q48e7yP4C00HB6jBQE5/w60=; b=jkCQ6X0MMock6S9ey5liK0bYGjxNf7LBVXe9oqq5ENFzHXFgLkxNaHxujqAFoxQD1L BT/tNqhTCDd+Xb9tNMXdth6QueZ5UTqltPVMn2NlP/RHx3Ik2D4OpjRvcYnTPdoVlflt NKE1ScbJmHiNg9vuiTRWQJIgQrUpx7M9kHa/TncEmzCGQBbFeloK3NNONcAgU49lc31/ GhurwUGQHbh7xo7EM4Es6KT6/O5CHNIsVXdqMlij0NFQQ15/rQznZFZ5ARP6qkFejy6z erQ1d4L9PopEVdpwbanY4gZLeNyY0m9CGS8VjC7k/d0+vmV2h1c4a6JaDR+x1ziA1cPb RlTw== X-Gm-Message-State: AODbwcAEJW5lfBdSJy0VNMnUSK0idKhhJQONiIOD1CmPa4LRr1a/dHXH pe8tnzp8U2n0rhC0qc1Lfx4PffCANSttDo0= X-Received: by 10.84.193.129 with SMTP id f1mr2043238pld.129.1495075118396; Wed, 17 May 2017 19:38:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.155.47 with HTTP; Wed, 17 May 2017 19:38:37 -0700 (PDT) In-Reply-To: <81774CB0-9E06-43AF-B58B-259234111D61@kraus-haus.org> References: <7c059678-4af4-f0c9-ff3b-c6266e02fb7a@gmx.com> <2318D9EB-4271-42C8-AEDB-4BB86607BB3D@kraus-haus.org> <81774CB0-9E06-43AF-B58B-259234111D61@kraus-haus.org> From: "Eric A. Borisch" Date: Wed, 17 May 2017 21:38:37 -0500 Message-ID: Subject: Re: zpool imported twice with different names (was Re: Fwd: ZFS) To: Paul Kraus Cc: FreeBSD Stable , =?UTF-8?Q?Trond_Endrest=C3=B8l?= , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 May 2017 02:38:39 -0000 On Wed, May 17, 2017 at 8:27 PM Paul Kraus wrote: > > > On May 17, 2017, at 5:12 PM, Eric A. Borisch wrote= : > > > > Short version: > > > > It's not the -f causing the problem, it's the parsing and (double) > > importing via /boot/zfs/zpool.cache that is. > > I was unclear=E2=80=A6 _how_ do you get the double entry in zpool.cache _= except_ > by using the -f option with a zpool that is already imported ? > To be clear, I'm using "already imported" to me =3D=3D shows up in 'zpool l= ist' of some running OS, not described in a config file somewhere. The Oracle description [1] that goes along with 'import -f' is: "Do not attempt to import a pool that is active on one system to another system. ZFS is not a native cluster, distributed, or parallel file system and cannot provide concurrent access from multiple, different hosts." This sounds much more like like my description (shows in a running zpool list) than described by a config/cache file. Steps to recreate: (I did it in < 5 minutes in a VM.) Arabic numerals are user actions, Roman numerals are zpool state. 1) Install FreeBSD root-on-zfs onto 'Alpha' I. Alpha exists on disk, and is active whenever your computer is running 1a) Decide at some point later you would like to rename your root pool... so you 2) Boot off live CD II. Alpha exists on disk, but is not active (imported) 3) zpool import alpha beta (because that's how you rename pools) 3a) zpool helpfully points out that it *may be in use* on another system, and use -f to import it anyway. Realize you are at the system that was using it, but in a completely different environment, and decide that it is safe to import. III. Alpha still exists on disk, and is still not active 4) zpool import -f alpha beta succeeds without any complaints. IV. Alpha no longer exists, Beta exists (with the same GUID as Alpha) AND is active. This is an 'atomic' operation from user's point of view, and at no point does zpool list show more than one pool 5) zpool list shows only beta; scrubs fine, life is good.. V. Beta exists on disk and is active and healthy. Alpha is no more. (Except as a ghost in a cache file) 6) Reboot into OS. Kernel messages mention booting off of beta/. Good, good... VI. Beta exists at this point certainly, as boot progresses. The ghost of Alpha is still lurking, however. Before we drop to a prompt, the zpool.cache is parsed, and Alpha springs back into existence beside Beta. NOTE: This is where I'm suggesting a cache file shouldn't trump the on-disk metadata (name mismatch) and running state (GUID collision) and lead to corruption. VII. Alpha and Beta exist together violating the Pauli exclusion principle as it applies to zpools. 7) Despair. Try to export one or the other. No good. Errors start to show up in zpool status. Rebooting fails in various colorful ways. Trying to import the pool again via Live CD is equally fruitless. VIII. Your zpool is cooked. By a config file. Hope you weren't using ZFS for integrity. I would maintain that it is the combination of the import of an already > imported zpool which causes the double entry in the zpool.cache in > combination with a recent rename of the zpool. > > Would (did) the import without the force fail ? The last host this zpool > was imported on was _this_ host, so the =E2=80=9Cis this zpool used by an= other > host=E2=80=9D check would not stop such an import. Yes, it fails without the -f, and helpfully suggests using the -f to import it. I don't think it's already imported in a way that most people (including Oracle) describe it. Or, is the zpool rename not updating the name in the zpool.cache file ? As the zpool rename is occurring in the live cd system; the cache file (living on the filesystems of the pool to be renamed) isn't accessible during the import/rename step, and isn't known about by the active OS. I suspect using a temporary and updating the cache file post-rename pre-reboot would work, just like deleting the cache file before reboot worked. I still, however, maintain an errant cache file shouldn't trump both on-disk state (my name... is Beta) and in-memory state (but I already have that GUID) and bring down a pool. - Eric [1] http://docs.oracle.com/cd/E19253-01/819-5461/gazuf/index.html >