From owner-freebsd-fs@FreeBSD.ORG Mon Dec 16 14:37:01 2013 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D9B6BF9; Mon, 16 Dec 2013 14:37:01 +0000 (UTC) Received: from mx.tetcu.info (mx.tetcu.info [217.19.15.179]) by mx1.freebsd.org (Postfix) with ESMTP id 334FD14FB; Mon, 16 Dec 2013 14:36:59 +0000 (UTC) Received: from F2 (f1e.forteasig.com [81.181.146.226]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx.tetcu.info (Postfix) with ESMTPSA id E85BE3A2878; Mon, 16 Dec 2013 16:36:58 +0200 (EET) Date: Mon, 16 Dec 2013 16:36:59 +0200 From: Ion-Mihai Tetcu To: Andriy Gapon Subject: Re: GTP ZFS boot failed after upgrading to 9.2-STABLE (can't read MOS) Message-Id: <20131216163659.fdcd9cad35bb479f40122aeb@FreeBSD.org> In-Reply-To: <52AEF02A.6020108@FreeBSD.org> References: <20131216113818.b108196769e1fd1dd3b7e67d@FreeBSD.org> <52AEE884.5000307@FreeBSD.org> <20131216135546.7ceb65c5991344d32303b64b@FreeBSD.org> <52AEF02A.6020108@FreeBSD.org> X-Mailer: Sylpheed 3.3.0 (GTK+ 2.10.14; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 14:37:01 -0000 On Mon, 16 Dec 2013 14:20:58 +0200 Andriy Gapon wrote: > on 16/12/2013 13:55 Ion-Mihai Tetcu said the following: > > On Mon, 16 Dec 2013 13:48:20 +0200 > > Andriy Gapon wrote: > > > >> on 16/12/2013 11:38 Ion-Mihai Tetcu said the following: > >>> Hi, > >>> > >>> > >>> After upgrading to > >>> 9.2-STABLE #8 r259396: Sun Dec 15 01:20:44 EET 2013 GENERIC amd64 > >>> the system in question didn't come up: > >>> ZFS: i/o error - all block copies unavailable > >>> ZFS: can't read MOS of pool zroot > >>> gptszfsboot: failed to mount default pool zroot > >>> > >>> GPTZFSBoot setup like in the wiki, except I didn't bothered with > >>> gnop. > >> > >> Could you please build zfsboottest utility in > >> tools/tools/zfsboottest and then run it like this? > >> zfsboottest /dev/gpt/z_ES3_2T /dev/gpt/z_wd2T - /boot/zfsloader > > > > # /root/bin/zfsboottest /dev/gpt/z_ES3_2T /dev/gpt/z_wd2T > > # - /boot/zfsloader > > pool: zroot > > bootfs: zroot/ROOT/default > > config: > > > > NAME STATE > > zroot ONLINE > > mirror ONLINE > > gpt/z_ES3_2T ONLINE > > gpt/z_wd2T ONLINE > > > > 809b79a8e78d637dddc618d992b37004 /boot/zfsloader > > > > > > Okay, so ZFS boot code is able to read the pool in general. > Could you please also do the following as well? > > zdb -l /dev/gpt/z_wd2T # zdb -l /dev/gpt/z_wd2T -------------------------------------------- LABEL 0 -------------------------------------------- version: 5000 name: 'zroot' state: 0 txg: 321793 pool_guid: 6514863746620611513 hostid: 2982265512 hostname: '' top_guid: 859717930543015389 guid: 15465645118630601473 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 859717930543015389 metaslab_array: 33 metaslab_shift: 32 ashift: 9 asize: 965289181184 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12917042815582263639 path: '/dev/gpt/z_ES3_2T' phys_path: '/dev/gpt/z_ES3_2T' whole_disk: 1 DTL: 66201 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 15465645118630601473 path: '/dev/gpt/z_wd2T' phys_path: '/dev/gpt/z_wd2T' whole_disk: 1 DTL: 1032 create_txg: 4 features_for_read: -------------------------------------------- LABEL 1 -------------------------------------------- version: 5000 name: 'zroot' state: 0 txg: 321793 pool_guid: 6514863746620611513 hostid: 2982265512 hostname: 'f1.c.forteasig.com' top_guid: 859717930543015389 guid: 15465645118630601473 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 859717930543015389 metaslab_array: 33 metaslab_shift: 32 ashift: 9 asize: 965289181184 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12917042815582263639 path: '/dev/gpt/z_ES3_2T' phys_path: '/dev/gpt/z_ES3_2T' whole_disk: 1 DTL: 66201 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 15465645118630601473 path: '/dev/gpt/z_wd2T' phys_path: '/dev/gpt/z_wd2T' whole_disk: 1 DTL: 1032 create_txg: 4 features_for_read: -------------------------------------------- LABEL 2 -------------------------------------------- version: 5000 name: 'zroot' state: 0 txg: 321793 pool_guid: 6514863746620611513 hostid: 2982265512 hostname: '' top_guid: 859717930543015389 guid: 15465645118630601473 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 859717930543015389 metaslab_array: 33 metaslab_shift: 32 ashift: 9 asize: 965289181184 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12917042815582263639 path: '/dev/gpt/z_ES3_2T' phys_path: '/dev/gpt/z_ES3_2T' whole_disk: 1 DTL: 66201 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 15465645118630601473 path: '/dev/gpt/z_wd2T' phys_path: '/dev/gpt/z_wd2T' whole_disk: 1 DTL: 1032 create_txg: 4 features_for_read: -------------------------------------------- LABEL 3 -------------------------------------------- version: 5000 name: 'zroot' state: 0 txg: 321793 pool_guid: 6514863746620611513 hostid: 2982265512 hostname: 'f1.c.forteasig.com' top_guid: 859717930543015389 guid: 15465645118630601473 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 859717930543015389 metaslab_array: 33 metaslab_shift: 32 ashift: 9 asize: 965289181184 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12917042815582263639 path: '/dev/gpt/z_ES3_2T' phys_path: '/dev/gpt/z_ES3_2T' whole_disk: 1 DTL: 66201 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 15465645118630601473 path: '/dev/gpt/z_wd2T' phys_path: '/dev/gpt/z_wd2T' whole_disk: 1 DTL: 1032 create_txg: 4 features_for_read: root@f1:/usr/src/tools/tools > zdb -l /dev/gpt/z_ES3_2T # zdb -l /dev/gpt/z_ES3_2T -------------------------------------------- LABEL 0 -------------------------------------------- version: 5000 name: 'zroot' state: 0 txg: 321793 pool_guid: 6514863746620611513 hostid: 2982265512 hostname: '' top_guid: 859717930543015389 guid: 12917042815582263639 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 859717930543015389 metaslab_array: 33 metaslab_shift: 32 ashift: 9 asize: 965289181184 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12917042815582263639 path: '/dev/gpt/z_ES3_2T' phys_path: '/dev/gpt/z_ES3_2T' whole_disk: 1 DTL: 66201 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 15465645118630601473 path: '/dev/gpt/z_wd2T' phys_path: '/dev/gpt/z_wd2T' whole_disk: 1 DTL: 1032 create_txg: 4 features_for_read: -------------------------------------------- LABEL 1 -------------------------------------------- version: 5000 name: 'zroot' state: 0 txg: 321793 pool_guid: 6514863746620611513 hostid: 2982265512 hostname: 'f1.c.forteasig.com' top_guid: 859717930543015389 guid: 12917042815582263639 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 859717930543015389 metaslab_array: 33 metaslab_shift: 32 ashift: 9 asize: 965289181184 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12917042815582263639 path: '/dev/gpt/z_ES3_2T' phys_path: '/dev/gpt/z_ES3_2T' whole_disk: 1 DTL: 66201 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 15465645118630601473 path: '/dev/gpt/z_wd2T' phys_path: '/dev/gpt/z_wd2T' whole_disk: 1 DTL: 1032 create_txg: 4 features_for_read: -------------------------------------------- LABEL 2 -------------------------------------------- version: 5000 name: 'zroot' state: 0 txg: 321793 pool_guid: 6514863746620611513 hostid: 2982265512 hostname: '' top_guid: 859717930543015389 guid: 12917042815582263639 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 859717930543015389 metaslab_array: 33 metaslab_shift: 32 ashift: 9 asize: 965289181184 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12917042815582263639 path: '/dev/gpt/z_ES3_2T' phys_path: '/dev/gpt/z_ES3_2T' whole_disk: 1 DTL: 66201 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 15465645118630601473 path: '/dev/gpt/z_wd2T' phys_path: '/dev/gpt/z_wd2T' whole_disk: 1 DTL: 1032 create_txg: 4 features_for_read: -------------------------------------------- LABEL 3 -------------------------------------------- version: 5000 name: 'zroot' state: 0 txg: 321793 pool_guid: 6514863746620611513 hostid: 2982265512 hostname: 'f1.c.forteasig.com' top_guid: 859717930543015389 guid: 12917042815582263639 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 859717930543015389 metaslab_array: 33 metaslab_shift: 32 ashift: 9 asize: 965289181184 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12917042815582263639 path: '/dev/gpt/z_ES3_2T' phys_path: '/dev/gpt/z_ES3_2T' whole_disk: 1 DTL: 66201 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 15465645118630601473 path: '/dev/gpt/z_wd2T' phys_path: '/dev/gpt/z_wd2T' whole_disk: 1 DTL: 1032 create_txg: 4 features_for_read: > zdb -dddd zroot 1 # zdb -dddd zroot 1 Dataset mos [META], ID 0, cr_txg 4, 350M, 46112 objects, rootbp DVA[0]=<0:a868cb3200:200> DVA[1]=<0:5972804200:200> DVA[2]=<0:1709271a00:200> [L0 DMU objset] fletcher4 lzjb LE contiguous uni que triple size=800L/200P birth=340049L/340049P fill=46112 cksum=10f34cf819:6a938097d4b:156f1629bb5f4:2f0fcf1ec9220d Object lvl iblk dblk dsize lsize %full type 1 1 16K 16K 12.0K 32K 100.00 object directory dnode flags: USED_BYTES dnode maxblkid: 1 Fat ZAP stats: Pointer table: 1024 elements zt_blk: 0 zt_numblks: 0 zt_shift: 10 zt_blks_copied: 0 zt_nextblk: 0 ZAP entries: 15 Leaf blocks: 1 Total blocks: 2 zap_block_type: 0x8000000000000001 zap_magic: 0x2f52ab2ab zap_salt: 0xbf9403 Leafs with 2^n pointers: 9: 1 * Blocks with n*5 entries: 3: 1 * Blocks n/10 full: 2: 1 * Entries with n chunks: 3: 14 ************** 4: 0 5: 0 6: 0 7: 0 8: 0 9: 1 * Buckets with n entries: 0: 497 **************************************** 1: 15 ** history = 32 scan = 2 2 0 3 293592 294507 293592 1386927167 1386933313 609892873216 609933173760 0 609933173760 0 1 3 0 0 0 0 0 0 0 0 pool_props = 158 root_dataset = 2 errlog_last = 0 errlog_scrub = 0 features_for_write = 29 config = 27 empty_bpobj = 43 sync_bplist = 31 free_bpobj = 11 feature_descriptions = 30 features_for_read = 28 creation_version = 5000 deflate = 1 -- Ion-Mihai Tetcu