From owner-freebsd-fs@FreeBSD.ORG Thu Oct 13 14:27:54 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7655E1065670 for ; Thu, 13 Oct 2011 14:27:54 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (lrosenman-1-pt.tunnel.tserv8.dal1.ipv6.he.net [IPv6:2001:470:1f0e:3ad::2]) by mx1.freebsd.org (Postfix) with ESMTP id 2254C8FC08 for ; Thu, 13 Oct 2011 14:27:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Content-Type:MIME-Version:References:Message-ID:In-Reply-To:Subject:cc:To:From:Date; bh=qxJXPoisfqO2Hs5JtviXAQDBJSW7IPYYmr7sqKSEbwQ=; b=FdmuvIWWvTlBcWp5hmY+d76S19Yh0MyRfjA8j9M/HEvmJzx1eytf+MlzG/SUzlQ+Fbs9oMkouLV7jvf7cfwtt4uR0dHKe4w3gYiV7ee6zreaHNz8jBdac3Q2qYlFgRbtb6wEJSCsja6kH5oGfNU8RE3KNGIwvuYbn6Jv9socejs=; Received: from lrosenman-1-pt.tunnel.tserv8.dal1.ipv6.he.net ([2001:470:1f0e:3ad::2]:37335) by thebighonker.lerctr.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1REMG6-0002KP-Us; Thu, 13 Oct 2011 09:27:50 -0500 Date: Thu, 13 Oct 2011 09:27:38 -0500 (CDT) From: Larry Rosenman To: Johannes Totz In-Reply-To: Message-ID: References: <4E95AE08.7030105@lerctr.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Spam-Score: -2.7 (--) X-LERCTR-Spam-Score: -2.7 (--) X-Spam-Report: SpamScore (-2.7/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, SARE_SUB_OBFU_OTHER=0.135, TW_ZF=0.077 X-LERCTR-Spam-Report: SpamScore (-2.7/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, SARE_SUB_OBFU_OTHER=0.135, TW_ZF=0.077 Cc: freebsd-fs@freebsd.org Subject: Re: AF (4096 byte sector) drives: Can you mix/match in a ZFS pool? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2011 14:27:54 -0000 On Wed, 12 Oct 2011, Johannes Totz wrote: > On 12/10/2011 16:11, Larry Rosenman wrote: >> I have a root on ZFS box with 6 drives, all 400G (except one 500G) in a >> pool. >> >> I want to upgrade to 2T or 3T drives, but was wondering if you can >> mix/match while doing the drive by drive >> replacement. >> >> This is on 9.0-BETA3 if that matters. > > Not sure if this applies to you... > I tried to add a 4k-gnop'd drive to an existing 512-byte-sector pool. > And zpool tool complained about alignment mismatch. > This was on a recent 8-stable. It didn't work. I just rebuilt the entire pool with one gnop'd drive, and then removed the gnop, and it's now ashift=12. I also made sure that the first partition started at block 36. I think I'll be ok now. Here's what it looks like: pool: zroot state: ONLINE scan: scrub repaired 0 in 0h3m with 0 errors on Thu Oct 13 02:16:23 2011 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 gptid/dab78d7a-f17f-11e0-a060-0030488e9ff3 ONLINE 0 0 0 gptid/54f70329-f180-11e0-a060-0030488e9ff3 ONLINE 0 0 0 gptid/551f4215-f180-11e0-a060-0030488e9ff3 ONLINE 0 0 0 gptid/554fbc57-f180-11e0-a060-0030488e9ff3 ONLINE 0 0 0 gptid/557e84df-f180-11e0-a060-0030488e9ff3 ONLINE 0 0 0 gptid/55ab4aad-f180-11e0-a060-0030488e9ff3 ONLINE 0 0 0 errors: No known data errors zroot: version: 28 name: 'zroot' state: 0 txg: 558 pool_guid: 5966906085647800520 hostid: 4114256494 hostname: '' vdev_children: 1 vdev_tree: type: 'root' id: 0 guid: 5966906085647800520 children[0]: type: 'raidz' id: 0 guid: 15251804863163772249 nparity: 1 metaslab_array: 30 metaslab_shift: 34 ashift: 12 asize: 2374730514432 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 13248457090896416694 path: '/dev/gptid/dab78d7a-f17f-11e0-a060-0030488e9ff3' phys_path: '/dev/gptid/dab78d7a-f17f-11e0-a060-0030488e9ff3' whole_disk: 1 DTL: 153 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 14267002476812000053 path: '/dev/gptid/54f70329-f180-11e0-a060-0030488e9ff3' phys_path: '/dev/gptid/54f70329-f180-11e0-a060-0030488e9ff3' whole_disk: 1 DTL: 152 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 4813683986967800595 path: '/dev/gptid/551f4215-f180-11e0-a060-0030488e9ff3' phys_path: '/dev/gptid/551f4215-f180-11e0-a060-0030488e9ff3' whole_disk: 1 DTL: 151 create_txg: 4 children[3]: type: 'disk' id: 3 guid: 6938997802835048973 path: '/dev/gptid/554fbc57-f180-11e0-a060-0030488e9ff3' phys_path: '/dev/gptid/554fbc57-f180-11e0-a060-0030488e9ff3' whole_disk: 1 DTL: 150 create_txg: 4 children[4]: type: 'disk' id: 4 guid: 18091841386475062099 path: '/dev/gptid/557e84df-f180-11e0-a060-0030488e9ff3' phys_path: '/dev/gptid/557e84df-f180-11e0-a060-0030488e9ff3' whole_disk: 1 DTL: 149 create_txg: 4 children[5]: type: 'disk' id: 5 guid: 5543901141375635781 path: '/dev/gptid/55ab4aad-f180-11e0-a060-0030488e9ff3' phys_path: '/dev/gptid/55ab4aad-f180-11e0-a060-0030488e9ff3' whole_disk: 1 DTL: 148 create_txg: 4 I think something(tm) should be put in the handbook about this. (oh, here's the partitions: Geom name: ada0 modified: false state: OK fwheads: 16 fwsectors: 63 last: 781422734 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada0p1 Mediasize: 65536 (64k) Sectorsize: 512 Stripesize: 0 Stripeoffset: 18432 Mode: r0w0e0 rawuuid: bc4c1c42-f17f-11e0-a060-0030488e9ff3 rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 18432 type: freebsd-boot index: 1 end: 163 start: 36 2. Name: ada0p2 Mediasize: 4294967296 (4.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e1 rawuuid: ca30afcf-f17f-11e0-a060-0030488e9ff3 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: swap0 length: 4294967296 offset: 83968 type: freebsd-swap index: 2 end: 8388771 start: 164 3. Name: ada0p3 Mediasize: 395793389056 (368G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e2 rawuuid: dab78d7a-f17f-11e0-a060-0030488e9ff3 rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b label: disk0 length: 395793389056 offset: 4295051264 type: freebsd-zfs index: 3 end: 781422734 start: 8388772 Consumers: 1. Name: ada0 Mediasize: 400088457216 (372G) Sectorsize: 512 Mode: r2w2e5 Geom name: ada1 modified: false state: OK fwheads: 16 fwsectors: 63 last: 781422734 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada1p1 Mediasize: 65536 (64k) Sectorsize: 512 Stripesize: 0 Stripeoffset: 18432 Mode: r0w0e0 rawuuid: 54e701ff-f180-11e0-a060-0030488e9ff3 rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 18432 type: freebsd-boot index: 1 end: 163 start: 36 2. Name: ada1p2 Mediasize: 4294967296 (4.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e1 rawuuid: 54eebd95-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: swap1 length: 4294967296 offset: 83968 type: freebsd-swap index: 2 end: 8388771 start: 164 3. Name: ada1p3 Mediasize: 395793389056 (368G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e2 rawuuid: 54f70329-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b label: disk1 length: 395793389056 offset: 4295051264 type: freebsd-zfs index: 3 end: 781422734 start: 8388772 Consumers: 1. Name: ada1 Mediasize: 400088457216 (372G) Sectorsize: 512 Mode: r2w2e5 Geom name: ada2 modified: false state: OK fwheads: 16 fwsectors: 63 last: 976773134 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada2p1 Mediasize: 65536 (64k) Sectorsize: 512 Stripesize: 0 Stripeoffset: 18432 Mode: r0w0e0 rawuuid: 550e22c7-f180-11e0-a060-0030488e9ff3 rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 18432 type: freebsd-boot index: 1 end: 163 start: 36 2. Name: ada2p2 Mediasize: 4294967296 (4.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e1 rawuuid: 5515f1c4-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: swap2 length: 4294967296 offset: 83968 type: freebsd-swap index: 2 end: 8388771 start: 164 3. Name: ada2p3 Mediasize: 495812793856 (461G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e2 rawuuid: 551f4215-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b label: disk2 length: 495812793856 offset: 4295051264 type: freebsd-zfs index: 3 end: 976773134 start: 8388772 Consumers: 1. Name: ada2 Mediasize: 500107862016 (465G) Sectorsize: 512 Mode: r2w2e5 Geom name: ada3 modified: false state: OK fwheads: 16 fwsectors: 63 last: 781422734 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada3p1 Mediasize: 65536 (64k) Sectorsize: 512 Stripesize: 0 Stripeoffset: 18432 Mode: r0w0e0 rawuuid: 553d6d5f-f180-11e0-a060-0030488e9ff3 rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 18432 type: freebsd-boot index: 1 end: 163 start: 36 2. Name: ada3p2 Mediasize: 4294967296 (4.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e1 rawuuid: 554677f6-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: swap3 length: 4294967296 offset: 83968 type: freebsd-swap index: 2 end: 8388771 start: 164 3. Name: ada3p3 Mediasize: 395793389056 (368G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e2 rawuuid: 554fbc57-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b label: disk3 length: 395793389056 offset: 4295051264 type: freebsd-zfs index: 3 end: 781422734 start: 8388772 Consumers: 1. Name: ada3 Mediasize: 400088457216 (372G) Sectorsize: 512 Mode: r2w2e5 Geom name: ada4 modified: false state: OK fwheads: 16 fwsectors: 63 last: 781422734 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada4p1 Mediasize: 65536 (64k) Sectorsize: 512 Stripesize: 0 Stripeoffset: 18432 Mode: r0w0e0 rawuuid: 556c3554-f180-11e0-a060-0030488e9ff3 rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 18432 type: freebsd-boot index: 1 end: 163 start: 36 2. Name: ada4p2 Mediasize: 4294967296 (4.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e1 rawuuid: 55754618-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: swap4 length: 4294967296 offset: 83968 type: freebsd-swap index: 2 end: 8388771 start: 164 3. Name: ada4p3 Mediasize: 395793389056 (368G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e2 rawuuid: 557e84df-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b label: disk4 length: 395793389056 offset: 4295051264 type: freebsd-zfs index: 3 end: 781422734 start: 8388772 Consumers: 1. Name: ada4 Mediasize: 400088457216 (372G) Sectorsize: 512 Mode: r2w2e5 Geom name: ada5 modified: false state: OK fwheads: 16 fwsectors: 63 last: 781422734 first: 34 entries: 128 scheme: GPT Providers: 1. Name: ada5p1 Mediasize: 65536 (64k) Sectorsize: 512 Stripesize: 0 Stripeoffset: 18432 Mode: r0w0e0 rawuuid: 5598ebf4-f180-11e0-a060-0030488e9ff3 rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: (null) length: 65536 offset: 18432 type: freebsd-boot index: 1 end: 163 start: 36 2. Name: ada5p2 Mediasize: 4294967296 (4.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e1 rawuuid: 55a1f6a0-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: swap5 length: 4294967296 offset: 83968 type: freebsd-swap index: 2 end: 8388771 start: 164 3. Name: ada5p3 Mediasize: 395793389056 (368G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 83968 Mode: r1w1e2 rawuuid: 55ab4aad-f180-11e0-a060-0030488e9ff3 rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b label: disk5 length: 395793389056 offset: 4295051264 type: freebsd-zfs index: 3 end: 781422734 start: 8388772 Consumers: 1. Name: ada5 Mediasize: 400088457216 (372G) Sectorsize: 512 Mode: r2w2e5