Date: Fri, 6 Sep 2024 08:56:25 -0700 From: Freddie Cash <fjwcash@gmail.com> To: mike tancsa <mike@sentex.net> Cc: Chris Ross <cross+freebsd@distal.com>, FreeBSD Filesystems <freebsd-fs@freebsd.org> Subject: Re: Unable to replace drive in raidz1 Message-ID: <CAOjFWZ7aSy3HtBUVMvo6U8JETTYGWLgszt-FmgbfGDq-qC-mgg@mail.gmail.com> In-Reply-To: <69093f39-f673-416e-aac9-59a80b936ab7@sentex.net> References: <5ED5CB56-2E2A-4D83-8CDA-6D6A0719ED19@distal.com> <69093f39-f673-416e-aac9-59a80b936ab7@sentex.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000b70c9506217573fa
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On Fri, Sep 6, 2024 at 8:44=E2=80=AFAM mike tancsa <mike@sentex.net> wrote:
> On 9/6/2024 10:24 AM, Chris Ross wrote:
>
> NAME STATE READ WRITE CKSUM
> tank DEGRADED 0 0 0
> raidz1-0 DEGRADED 0 0 0
> da3 FAULTED 0 0 0 external device fa=
ult
> da1 ONLINE 0 0 0
> da2 ONLINE 0 0 0
> raidz1-1 ONLINE 0 0 0
> diskid/DISK-K1GMBN9D ONLINE 0 0 0
> diskid/DISK-K1GMEDMD ONLINE 0 0 0
> diskid/DISK-K1GMAX1D ONLINE 0 0 0
> raidz1-2 ONLINE 0 0 0
> diskid/DISK-3WJDHJ2J ONLINE 0 0 0
> diskid/DISK-3WK3G1KJ ONLINE 0 0 0
> diskid/DISK-3WJ7ZMMJ ONLINE 0 0 0
>
>
> I would triple check to see what the devices are that are part of the
> pool. I wish there was a way to tell zfs to only display one or the
> other. So list out what diskid/DISK-K1GMBN9D, diskid/DISK-K1GMEDMD... to
> diskid/DISK-3WJ7ZMMJ are in terms of /dev/da* actually are. I have some
> controllers that will re-order the disks on every reboot. glabel status
> and camcontrol devlist should help verify
>
You can't tell ZFS specifically to use one form of GEOM ID vs another, but
you can tell the whole system which GEOM IDs to not use. Add the following
to /boot/loader.conf:
kern.geom.label.disk_ident.enable=3D"0" # Disable the
auto-generated Disk IDs for disks
kern.geom.label.gptid.enable=3D"0" # Disable the auto-generated GPT UUIDs f=
or
disks
kern.geom.label.ufsid.enable=3D"0" # Disable the auto-generated UFS UUIDs f=
or
filesystems
The first line will remove the diskid/DISK-* entries and show the device
nodes (daX). The other two lines remove GPT and UFS UUIDs as well.
All my ZFS systems have those entries in loader.conf, as I prefer to use
GPT partition labels in my pools (gpt/label-name) where I list which
specific JBOD chassis and drive bay the HD is located. That way, it
doesn't matter if the device nodes are renumbered, as the labels don't
change. Makes it much easier to find the specific drive to be replaced,
whether in my home server with 6 drives or my backups servers at work with
multiple JBODs and 92 drives.
--=20
Freddie Cash
fjwcash@gmail.com
--000000000000b70c9506217573fa
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div dir=3D"ltr">On Fri, Sep 6, 2024 at 8:44=E2=80=AFAM mi=
ke tancsa <<a href=3D"mailto:mike@sentex.net">mike@sentex.net</a>> wr=
ote:<br></div><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pa=
dding-left:1ex"><u></u>
=20
=20
=20
<div>
<div>On 9/6/2024 10:24 AM, Chris Ross wrote:<br>
</div>
<blockquote type=3D"cite">
<pre> NAME STATE READ WRITE CKSUM
tank DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
da3 FAULTED 0 0 0 external device faul=
t
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
diskid/DISK-K1GMBN9D ONLINE 0 0 0
diskid/DISK-K1GMEDMD ONLINE 0 0 0
diskid/DISK-K1GMAX1D ONLINE 0 0 0
raidz1-2 ONLINE 0 0 0
diskid/DISK-3WJDHJ2J ONLINE 0 0 0
diskid/DISK-3WK3G1KJ ONLINE 0 0 0
diskid/DISK-3WJ7ZMMJ ONLINE 0 0 0
</pre>
</blockquote>
<br>
<p>I would triple check to see what the devices are that are part of
the pool.=C2=A0 I wish there was a way to tell zfs to only display on=
e
or the other.=C2=A0 So list out what diskid/DISK-K1GMBN9D,
diskid/DISK-K1GMEDMD... to diskid/DISK-3WJ7ZMMJ are in terms of
/dev/da* actually are.=C2=A0 I have some controllers that will re-ord=
er
the disks on every reboot.=C2=A0 glabel status and camcontrol devlist
should help verify</p></div></blockquote><div>You can't tell ZFS =
specifically to use one form of GEOM ID vs another, but you can tell the wh=
ole system which GEOM IDs to not use.=C2=A0 Add the following to /boot/load=
er.conf:<br><br></div><div>kern.geom.label.disk_ident.enable=3D"0"=
; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Disable the auto-generated Disk IDs =
=C2=A0for disks<br>kern.geom.label.gptid.enable=3D"0" # Disable =
the auto-generated GPT UUIDs for disks<br>kern.geom.label.ufsid.enable=3D&q=
uot;0" # Disable the auto-generated UFS UUIDs for filesystems</div><d=
iv><br></div><div>The first line will remove the diskid/DISK-* entries and =
show the device nodes (daX).=C2=A0 The other two lines remove GPT and UFS U=
UIDs as well.</div><div><br></div><div>All my ZFS systems have those entrie=
s in loader.conf, as I prefer to use GPT partition labels in my pools (gpt/=
label-name) where I list which specific JBOD chassis and drive bay the HD i=
s located.=C2=A0 That way, it doesn't matter if the device nodes are re=
numbered, as the labels don't change.=C2=A0 Makes it much easier to fin=
d the specific drive to be replaced, whether in my home server with 6 drive=
s or my backups servers at work with multiple JBODs and 92 drives.</div></d=
iv><div><br></div><span class=3D"gmail_signature_prefix">-- </span><br><div=
dir=3D"ltr" class=3D"gmail_signature">Freddie Cash<br><a href=3D"mailto:fj=
wcash@gmail.com" target=3D"_blank">fjwcash@gmail.com</a></div></div>
--000000000000b70c9506217573fa--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOjFWZ7aSy3HtBUVMvo6U8JETTYGWLgszt-FmgbfGDq-qC-mgg>