From owner-freebsd-fs@freebsd.org Thu Jul 2 10:02:40 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54AE499207D for ; Thu, 2 Jul 2015 10:02:40 +0000 (UTC) (envelope-from felipemonteiro.carvalho@gmail.com) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2542421DD; Thu, 2 Jul 2015 10:02:40 +0000 (UTC) (envelope-from felipemonteiro.carvalho@gmail.com) Received: by paceq1 with SMTP id eq1so38199740pac.3; Thu, 02 Jul 2015 03:02:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=uuzSutuE6OIWJ+nQSYZMDoDIg8aCpLZZ402odZw4JXI=; b=QhpRDQ316wwzIAc66Rz9aY8OHatf459MRiOsQeo0DMEm/mVPp3+IRMNKEmW1trgOsH eKFSecMrpjcpelE0Nn5aITQeW9ui2eZUh/4wnUePhFVmkNb/xCeMRH1iutZCKuURyRVZ J7Wbt9HqnBEUfRGBO9jwqb2oZx0t/ZpSes9v4CXDyuTSKn6EOy6OaXnE7GIzpdUfEyI/ hmtmIqTACQLqb3aGgW+4BUrhfBImrZHVegj8ZCF7+CKBWDH3IrJ66v5VXhvwlu2hHgZq tUUUL3ah+lU+RTnxR30LbsiCIIUHnp2zOvrPlsjU/O/eye4OEvgtgrVOhMETUMjeyt3Q fOqg== MIME-Version: 1.0 X-Received: by 10.67.30.102 with SMTP id kd6mr64326714pad.132.1435831359605; Thu, 02 Jul 2015 03:02:39 -0700 (PDT) Received: by 10.66.147.4 with HTTP; Thu, 2 Jul 2015 03:02:39 -0700 (PDT) In-Reply-To: <557C282D.8060809@freebsd.org> References: <557B0255.8060809@freebsd.org> <01184F08-1C6B-4282-9203-1BF98F07A05A@gmail.com> <557C282D.8060809@freebsd.org> Date: Thu, 2 Jul 2015 12:02:39 +0200 Message-ID: Subject: Re: Uberblock location From: Felipe Monteiro de Carvalho To: Julian Elischer Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jul 2015 10:02:40 -0000 Hello, Ok, thanks a lot =) I am now working based on the FreeBSD source code, and I made a lot of progress at adapting it to my needs. My use is very different, so its not a simple "use as is", so I'm trying to understand what goes on in every step to adapt it, and I have arrived at a point where I got lost. Ok, so far I have the following working: Part 1> Read the disk label: based on function vdev_probe, reads from the disk the type vdev_phys_t This issues the following data: [0] DATA_TYPE_UINT64 name="version" val=5000 [1] DATA_TYPE_STRING name="name" val="zfs_felipe_2" [2] DATA_TYPE_UINT64 name="state" val=1 [3] DATA_TYPE_UINT64 name="txg" val=164 [4] DATA_TYPE_UINT64 name="pool_guid" val=8752563210577670337 [5] DATA_TYPE_UINT64 name="errata" val=0 [6] DATA_TYPE_UINT64 name="hostid" val=8323329 [7] DATA_TYPE_STRING name="hostname" val="felipe-VirtualBox" [8] DATA_TYPE_UINT64 name="top_guid" val=2451523895343473151 [9] DATA_TYPE_UINT64 name="guid" val=2451523895343473151 [10] DATA_TYPE_UINT64 name="vdev_children" val=1 [11] DATA_TYPE_NVLIST name="vdev_tree" >=[0] data_type_string 'name'="type" val="disk" >=[1] data_type_uint64 'name'="id" val=0 >=[2] data_type_uint64 'name'="guid" val=2,45152389534347E+018 >=[3] data_type_string 'name'="path" val="/dev/loop7" >=[4] data_type_uint64 'name'="whole_disk" val=0 >=[5] data_type_uint64 'name'="metaslab_array" val=34 >=[6] data_type_uint64 'name'="metaslab_shift" val=21 >=[7] data_type_uint64 'name'="ashift" val=9 >=[8] data_type_uint64 'name'="asize" val=257425408 >=[9] data_type_uint64 'name'="is_log" val=0 >=[10] data_type_uint64 'name'="create_txg" val=4 [12] DATA_TYPE_NVLIST name="features_for_read" >=[0] data_type_boolean 'name'="com.delphix:hole_birth" >=[1] data_type_boolean 'name'="com.delphix:embedded_data" This part is OK, although I was upset that in a pool with 2 disks, there is no info in each disk about the other one, except that vdev_children=2 =( It would have been great if there was the path to the second disk, but anyway, this is no huge problem. Part 2> Read the uberblocks and figure out which is the most current one. I select the uberblock with the largest ub_timestamp as the best one, and then I try to read the data pointed to by the uberblock.ub_rootbp of type blkptr_t The best uberblock has the following ub_rootbp: [0] VolumePos=$28000 DiskPos=$28000 IsVolumePosValid=1 IsSBValid=1 OriginNr=4 SB.ub_version=5000 SB.ub_timestamp=555CA4DF SB.ub_software_version=5000 BlockPtr=DVA[0]= DVA[1]= DVA[2]= LEVEL=0 TYPE=B LSIZE=800 PSIZE=200 COMP=F BIRTH=A0 PHYS_BIRTH=A0 read using the functions such as BP_GET_COMPRESS, etc. And this part is where I am stuck, because: 1> The offset 0x325D400 counting from disk image start is filled with CC CC CC CC .... it doesn't look like at all that it is something valid. Maybe the offset should not be read from disk image start, but instead counting from somewhere else? 2> comp=F wow, so even basic blocks use compression? Or my reading of the data somehow wrong? It will be a lot of work to get decompression working. any ideas? Attached is the "best" uberblock, with block pointer data with blue background. thanks, -- Felipe Monteiro de Carvalho