From owner-freebsd-fs@FreeBSD.ORG Mon Dec 16 09:38:20 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 090D4FD6 for ; Mon, 16 Dec 2013 09:38:20 +0000 (UTC) Received: from mx.tetcu.info (mx.tetcu.info [217.19.15.179]) by mx1.freebsd.org (Postfix) with ESMTP id 7A1771B20 for ; Mon, 16 Dec 2013 09:38:19 +0000 (UTC) Received: from F2 (f1e.forteasig.com [81.181.146.226]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx.tetcu.info (Postfix) with ESMTPSA id 83F103A2D9E for ; Mon, 16 Dec 2013 11:38:18 +0200 (EET) Date: Mon, 16 Dec 2013 11:38:18 +0200 From: Ion-Mihai Tetcu To: freebsd-fs@freebsd.org Subject: GTP ZFS boot failed after upgrading to 9.2-STABLE (can't read MOS) Message-Id: <20131216113818.b108196769e1fd1dd3b7e67d@FreeBSD.org> X-Mailer: Sylpheed 3.3.0 (GTK+ 2.10.14; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 09:38:20 -0000 Hi, After upgrading to 9.2-STABLE #8 r259396: Sun Dec 15 01:20:44 EET 2013 GENERIC amd64 the system in question didn't come up: ZFS: i/o error - all block copies unavailable ZFS: can't read MOS of pool zroot gptszfsboot: failed to mount default pool zroot GPTZFSBoot setup like in the wiki, except I didn't bothered with gnop. At the time of the upgrade the boot disk was ada0. Luckly I can boot without problems from any of the other 2 disks. Pool history: 1. started as a mirror over partitions on a 500GB HDD and a 1TB HDD (still present in the system - ada1 bellow, can boot from it) 2. the 500GB was replaced by ada0, the system was able to boot from the new disk; the pool extended via zfs online -e 3. ada1 (1TB) was replaced by ada2, the pool not extended yet --- At this point I could boot from any of the disks. 4. zfs scrub the mirror without any error 5. upgrade (svn up, buildworld, buildkernel, installkernel, mergemaster -p, installworld, mergemaster -iU, delete-old-libs, update ports .ko modules, reboot) 6. The error above. 7. Boot from ada2, gpart bootcode ... ada0, still the same error. I have snapshots of the pool from before the upgrade, so I could try to rollback and see if it makes any difference. (The machine is in production so I can't do it on the spot). # zpool list -v NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT zroot 896G 608G 288G 67% 1.00x ONLINE - mirror 896G 608G 288G 901G gpt/z_ES3_2T - - - - gpt/z_wd2T - - - - # gpart show -l -p => 34 3907029101 ada0 GPT (1.8T) 34 6 - free - (3.0k) 40 216 ada0p1 boot_wd2T (108k) 256 67108864 ada0p2 swap_wd2T (32G) 67109120 3774873600 ada0p3 z_wd2T (1.8T) 3841982720 65046415 - free - (31G) => 34 1953525101 ada1 GPT (931G) 34 6 - free - (3.0k) 40 216 ada1p1 boot1 (108k) 256 67108864 ada1p2 swap1 (32G) 67109120 1885339648 ada1p3 disk1 (899G) 1952448768 1076367 - free - (525M) => 34 3907029101 ada2 GPT (1.8T) 34 216 ada2p1 boot_ES3_2T (108k) 250 67108864 ada2p2 swap_ES3_2T (32G) 67109114 3837788160 ada2p3 z_ES3_2T (1.8T) 3904897274 2131861 - free - (1.0G) # zfs list -t snapshot | wc -l 16025 /dev/ada0: Device Model: WDC WD2000F9YZ-09N20L0 Serial Number: WD-WCC1P0590651 LU WWN Device Id: 5 0014ee 25e1b7330 Firmware Version: 01.01A01 User Capacity: 2,000,398,934,016 bytes [2.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical /dev/ada1: Model Family: Western Digital RE4 Device Model: WDC WD1003FBYX-01Y7B1 Serial Number: WD-WCAW35154447 LU WWN Device Id: 5 0014ee 2b2a4da9a Firmware Version: 01.01V02 User Capacity: 1,000,204,886,016 bytes [1.00 TB] /dev/ada2: Model Family: Seagate Constellation ES.3 Device Model: ST2000NM0033-9ZM175 Serial Number: Z1X0W9SP LU WWN Device Id: 5 000c50 064bc4572 Firmware Version: SN03 User Capacity: 2,000,398,934,016 bytes [2.00 TB] Similar to this, it seems: From: Łukasz Wąsikowski To: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: ZFS: can't read MOS of pool Date: Mon, 22 Jul 2013 18:18:49 +0200 Any help is greatly appreciated. -- Ion-Mihai Tetcu