From nobody Sat Dec 2 00:56:41 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Shs2R6P21z52nlZ for <freebsd-cloud@mlmmj.nyi.freebsd.org>; Sat, 2 Dec 2023 00:56:47 +0000 (UTC) (envelope-from postmaster@shaneware.biz) Received: from mail-relay3.dca2.superb.net (mail-relay3.dca2.superb.net [66.148.95.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Shs2R2BBKz3FHx for <freebsd-cloud@FreeBSD.org>; Sat, 2 Dec 2023 00:56:47 +0000 (UTC) (envelope-from postmaster@shaneware.biz) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of postmaster@shaneware.biz has no SPF policy when checking 66.148.95.60) smtp.mailfrom=postmaster@shaneware.biz; dmarc=none Received: from ppp230-11.static.internode.on.net ([203.122.230.11] helo=[192.168.9.155]) by mail-relay3.dca2.superb.net with esmtpsa (TLS1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (envelope-from <postmaster@shaneware.biz>) id 1r9EJI-0000gR-Pv for freebsd-cloud@FreeBSD.org; Fri, 01 Dec 2023 19:56:46 -0500 Message-ID: <9f04355d-4657-7781-0797-3518a164119d@ShaneWare.Biz> Date: Sat, 2 Dec 2023 11:26:41 +1030 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) <freebsd-cloud.freebsd.org> List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: <mailto:freebsd-cloud+help@freebsd.org> List-Post: <mailto:freebsd-cloud@freebsd.org> List-Subscribe: <mailto:freebsd-cloud+subscribe@freebsd.org> List-Unsubscribe: <mailto:freebsd-cloud+unsubscribe@freebsd.org> Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Content-Language: en-US To: FreeBSD Cloud <freebsd-cloud@FreeBSD.org> From: Shane Ambler <FreeBSD@ShaneWare.Biz> Subject: Resizing EBS with ZFS on AWS Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FORGED_SENDER(0.30)[FreeBSD@ShaneWare.Biz,postmaster@shaneware.biz]; RWL_MAILSPIKE_VERYGOOD(-0.20)[66.148.95.60:from]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org]; ASN(0.00)[asn:14361, ipnet:66.148.95.0/24, country:US]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEFALL_USER(0.00)[postmaster]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[FreeBSD@ShaneWare.Biz,postmaster@shaneware.biz]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[shaneware.biz]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4Shs2R2BBKz3FHx X-Spamd-Bar: - Hi, I recently started an EC2 with the 14.0 ZFS AMI After filling the drive, I expanded it, recovered the gpart partition and expanded the partition and zpool size and kept going. Then I saw that I would need more space, so I have expanded the EBS vol again, while the EC2 was shutdown. This time the EC2 will not boot. Using the EC2 console I get to... nvme0: SET_FEATURES (09) sqid:0 cid:15 nsid:0 cdw10:0000000b cdw11:0000031f nvme0: INVALID_FIELD (00/02) crd:0 m:1 dnr:0 p:1 sqid:0 cid:15 cdw0:0 nda0 at nvme0 bus 0 scbus0 target 0 lun 1 nda0: <Amazon Elastic Block Store 2.0 vol0f445f0597836b567> nda0: Serial Number vol0f445f0597836b567 nda0: nvme version 1.4 nda0: 153600MB (314572800 512 byte sectors) Trying to mount root from zfs:zroot/ROOT/default []... Mounting from zfs:zroot/ROOT/default failed with error 2: unknown file system. GEOM: nda0: the secondary GPT header is not in the last LBA. What can I do at mountroot> to recover gpart? or boot from here? -- FreeBSD - the place to B...Serving Data Shane Ambler From nobody Sat Dec 2 04:30:37 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ShxnQ4D0Pz52XtV for <freebsd-cloud@mlmmj.nyi.freebsd.org>; Sat, 2 Dec 2023 04:30:50 +0000 (UTC) (envelope-from postmaster@shaneware.biz) Received: from mail-relay2.dca2.superb.net (mail-relay2.dca2.superb.net [66.148.95.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ShxnP5xKFz4JV0 for <freebsd-cloud@FreeBSD.org>; Sat, 2 Dec 2023 04:30:49 +0000 (UTC) (envelope-from postmaster@shaneware.biz) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of postmaster@shaneware.biz has no SPF policy when checking 66.148.95.42) smtp.mailfrom=postmaster@shaneware.biz; dmarc=none Received: from ppp230-11.static.internode.on.net ([203.122.230.11] helo=[192.168.9.155]) by mail-relay2.dca2.superb.net with esmtpsa (TLS1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (envelope-from <postmaster@shaneware.biz>) id 1r9HeM-00006A-V0 for freebsd-cloud@FreeBSD.org; Fri, 01 Dec 2023 23:30:43 -0500 Message-ID: <ee8385fc-83d4-fd53-96f0-c2475941fa7b@ShaneWare.Biz> Date: Sat, 2 Dec 2023 15:00:37 +1030 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) <freebsd-cloud.freebsd.org> List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: <mailto:freebsd-cloud+help@freebsd.org> List-Post: <mailto:freebsd-cloud@freebsd.org> List-Subscribe: <mailto:freebsd-cloud+subscribe@freebsd.org> List-Unsubscribe: <mailto:freebsd-cloud+unsubscribe@freebsd.org> Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: Resizing EBS with ZFS on AWS Content-Language: en-US From: Shane Ambler <FreeBSD@ShaneWare.Biz> To: FreeBSD Cloud <freebsd-cloud@FreeBSD.org> References: <9f04355d-4657-7781-0797-3518a164119d@ShaneWare.Biz> In-Reply-To: <9f04355d-4657-7781-0797-3518a164119d@ShaneWare.Biz> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-0.53 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; FORGED_SENDER(0.30)[FreeBSD@ShaneWare.Biz,postmaster@shaneware.biz]; NEURAL_SPAM_SHORT(0.27)[0.275]; MIME_GOOD(-0.10)[text/plain]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14361, ipnet:66.148.95.0/24, country:US]; R_SPF_NA(0.00)[no SPF record]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEFALL_USER(0.00)[postmaster]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[FreeBSD@ShaneWare.Biz,postmaster@shaneware.biz]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[shaneware.biz]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4ShxnP5xKFz4JV0 X-Spamd-Bar: / On 2/12/23 11:26, Shane Ambler wrote: > Hi, > > I recently started an EC2 with the 14.0 ZFS AMI > > After filling the drive, I expanded it, recovered the gpart partition > and expanded the partition and zpool size and kept going. > > Then I saw that I would need more space, so I have expanded the EBS vol > again, while the EC2 was shutdown. This time the EC2 will not boot. > > Using the EC2 console I get to... > > Trying to mount root from zfs:zroot/ROOT/default []... > Mounting from zfs:zroot/ROOT/default failed with error 2: unknown file > system. > GEOM: nda0: the secondary GPT header is not in the last LBA. > > > What can I do at mountroot> to recover gpart? or boot from here? > OK so it seems the expanding disk was a red herring, I had altered loader.conf to break boot. I came up with steps to attach the unbootable EBS to another instance, import the pool, fix loader.conf, then re-attached to the original instance and got it to boot. Might need to document these cloud recovery steps somewhere. -- FreeBSD - the place to B...Scaring Daemons Shane Ambler