From owner-freebsd-questions@freebsd.org Sat Nov 21 05:16:19 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 04B5A47B8A9 for ; Sat, 21 Nov 2020 05:16:19 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [IPv6:2001:470:0:19b::b869:801b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.holgerdanske.com", Issuer "www.holgerdanske.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CdM8J6nhzz3Nl9 for ; Sat, 21 Nov 2020 05:16:16 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from 99.100.19.101 (99-100-19-101.lightspeed.frokca.sbcglobal.net [99.100.19.101]) by holgerdanske.com with ESMTPSA (TLS_AES_128_GCM_SHA256:TLSv1.3:Kx=any:Au=any:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Fri, 20 Nov 2020 21:16:07 -0800 Subject: Re: "zpool attach" problem To: freebsd-questions@freebsd.org References: <202011200659.0AK6xlWb009163@sdf.org> From: David Christensen Message-ID: <6658aa78-6f02-e97e-15f9-5a08c36236d7@holgerdanske.com> Date: Fri, 20 Nov 2020 21:16:06 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <202011200659.0AK6xlWb009163@sdf.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CdM8J6nhzz3Nl9 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of dpchrist@holgerdanske.com has no SPF policy when checking 2001:470:0:19b::b869:801b) smtp.mailfrom=dpchrist@holgerdanske.com X-Spamd-Result: default: False [-0.10 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:0:19b::b869:801b:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2001:470:0:19b::b869:801b:from:127.0.2.255]; ARC_NA(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; DMARC_NA(0.00)[holgerdanske.com]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-questions] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2020 05:16:19 -0000 On 2020-11-19 22:59, Scott Bennett via freebsd-questions wrote: > I had a pool with two two-way mirrors as the top-level vdevs. I needed > to shift some of those partitions by a short distance on the drives, so I > detached and deleted and rebuilt them one at a time until I hit a snag. Here > is the situation. > > Script started on Fri Nov 20 00:40:36 2020 > hellas# gpart show -l ada2 da0 da1 da2 > => 40 5860533088 ada2 GPT (2.7T) > 40 4294967296 1 WD-WMC130F2V1RN (2.0T) > 4294967336 31457496 - free - (15G) > 4326424832 125829120 11 zmisc mirror-0 1 (60G) > 4452253952 209715200 15 bw2-0 (100G) > 4661969152 1198563976 - free - (572G) > > => 34 3907029101 da0 GPT (1.8T) > 34 14 - free - (7.0K) > 48 3749709824 1 WD WCC4MH1P7LYS (1.7T) > 3749709872 73400320 5 bw1-0 (35G) > 3823110192 2000 - free - (1.0M) > 3823112192 83886080 8 zmisc mirror-1 1 (40G) > 3906998272 30863 - free - (15M) > > => 34 3907029100 da1 GPT (1.8T) > 34 14 - free - (7.0K) > 48 3749709824 1 Seagate NA5KYLVM (1.7T) > 3749709872 16 - free - (8.0K) > 3749709888 73400320 5 bw1-1 (35G) > 3823110208 1984 - free - (992K) > 3823112192 83886080 8 zmisc mirror-1 0 (40G) > 3906998272 30862 - free - (15M) > > => 40 3907029088 da2 GPT (1.8T) > 40 8 - free - (4.0K) > 48 3749709824 1 WD-WCC6N7KD2YAK (1.7T) > 3749709872 16 - free - (8.0K) > 3749709888 31457280 5 bw0-0 (15G) > 3781167168 1984 - free - (992K) > 3781169152 125829120 8 zmisc mirror-0 0 (60G) > 3906998272 30856 - free - (15M) > > hellas# zpool status zmisc > pool: zmisc > state: ONLINE > scan: resilvered 25.8G in 0 days 00:16:07 with 0 errors on Fri Nov 20 00:10:19 2020 > config: > > NAME STATE READ WRITE CKSUM > zmisc ONLINE 0 0 0 > ada2p11 ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > da0p8 ONLINE 0 0 0 > da1p8 ONLINE 0 0 0 > > errors: No known data errors > hellas# zpool attach zmisc ada2p11 da2p8 > cannot attach da2p8 to ada2p11: no such pool or dataset > hellas# exit > exit > > Script done on Fri Nov 20 00:42:33 2020 > > Would somebody please tell me what I am doing wrong here? Many thanks in > advance to whoever can help. It looks like you added the slice ada2p11 to zmisc, rather than the mirror ada2p11 da2p8. If so, these commands could fix things: # zpool remove zmisc ada2p11 # zpool add zmisc mirror ada2p11 da2p8 But, I am confused by your storage architecture. Why one internal "3 TB" drive and three external "2 TB" drives? What is the 2.0T internal slice for? What are the three 1.7 GiB external slices for? What are the bw?-? slices for, and why are they different sizes? Why are the zmisc slices different sizes? What about ada0 and ada1? And, do you have spaces in your GPT labels? David [1] https://docs.oracle.com/cd/E37838_01/html/E61017/remove-devices.html [2] https://docs.oracle.com/cd/E19253-01/819-5461/gazgw/index.html