Date: Fri, 6 Sep 2024 08:56:25 -0700 From: Freddie Cash <fjwcash@gmail.com> To: mike tancsa <mike@sentex.net> Cc: Chris Ross <cross+freebsd@distal.com>, FreeBSD Filesystems <freebsd-fs@freebsd.org> Subject: Re: Unable to replace drive in raidz1 Message-ID: <CAOjFWZ7aSy3HtBUVMvo6U8JETTYGWLgszt-FmgbfGDq-qC-mgg@mail.gmail.com> In-Reply-To: <69093f39-f673-416e-aac9-59a80b936ab7@sentex.net> References: <5ED5CB56-2E2A-4D83-8CDA-6D6A0719ED19@distal.com> <69093f39-f673-416e-aac9-59a80b936ab7@sentex.net>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
On Fri, Sep 6, 2024 at 8:44 AM mike tancsa <mike@sentex.net> wrote:
> On 9/6/2024 10:24 AM, Chris Ross wrote:
>
> NAME STATE READ WRITE CKSUM
> tank DEGRADED 0 0 0
> raidz1-0 DEGRADED 0 0 0
> da3 FAULTED 0 0 0 external device fault
> da1 ONLINE 0 0 0
> da2 ONLINE 0 0 0
> raidz1-1 ONLINE 0 0 0
> diskid/DISK-K1GMBN9D ONLINE 0 0 0
> diskid/DISK-K1GMEDMD ONLINE 0 0 0
> diskid/DISK-K1GMAX1D ONLINE 0 0 0
> raidz1-2 ONLINE 0 0 0
> diskid/DISK-3WJDHJ2J ONLINE 0 0 0
> diskid/DISK-3WK3G1KJ ONLINE 0 0 0
> diskid/DISK-3WJ7ZMMJ ONLINE 0 0 0
>
>
> I would triple check to see what the devices are that are part of the
> pool. I wish there was a way to tell zfs to only display one or the
> other. So list out what diskid/DISK-K1GMBN9D, diskid/DISK-K1GMEDMD... to
> diskid/DISK-3WJ7ZMMJ are in terms of /dev/da* actually are. I have some
> controllers that will re-order the disks on every reboot. glabel status
> and camcontrol devlist should help verify
>
You can't tell ZFS specifically to use one form of GEOM ID vs another, but
you can tell the whole system which GEOM IDs to not use. Add the following
to /boot/loader.conf:
kern.geom.label.disk_ident.enable="0" # Disable the
auto-generated Disk IDs for disks
kern.geom.label.gptid.enable="0" # Disable the auto-generated GPT UUIDs for
disks
kern.geom.label.ufsid.enable="0" # Disable the auto-generated UFS UUIDs for
filesystems
The first line will remove the diskid/DISK-* entries and show the device
nodes (daX). The other two lines remove GPT and UFS UUIDs as well.
All my ZFS systems have those entries in loader.conf, as I prefer to use
GPT partition labels in my pools (gpt/label-name) where I list which
specific JBOD chassis and drive bay the HD is located. That way, it
doesn't matter if the device nodes are renumbered, as the labels don't
change. Makes it much easier to find the specific drive to be replaced,
whether in my home server with 6 drives or my backups servers at work with
multiple JBODs and 92 drives.
--
Freddie Cash
fjwcash@gmail.com
[-- Attachment #2 --]
<div dir="ltr"><div dir="ltr">On Fri, Sep 6, 2024 at 8:44 AM mike tancsa <<a href="mailto:mike@sentex.net">mike@sentex.net</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div>
<div>On 9/6/2024 10:24 AM, Chris Ross wrote:<br>
</div>
<blockquote type="cite">
<pre> NAME STATE READ WRITE CKSUM
tank DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
da3 FAULTED 0 0 0 external device fault
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
diskid/DISK-K1GMBN9D ONLINE 0 0 0
diskid/DISK-K1GMEDMD ONLINE 0 0 0
diskid/DISK-K1GMAX1D ONLINE 0 0 0
raidz1-2 ONLINE 0 0 0
diskid/DISK-3WJDHJ2J ONLINE 0 0 0
diskid/DISK-3WK3G1KJ ONLINE 0 0 0
diskid/DISK-3WJ7ZMMJ ONLINE 0 0 0
</pre>
</blockquote>
<br>
<p>I would triple check to see what the devices are that are part of
the pool. I wish there was a way to tell zfs to only display one
or the other. So list out what diskid/DISK-K1GMBN9D,
diskid/DISK-K1GMEDMD... to diskid/DISK-3WJ7ZMMJ are in terms of
/dev/da* actually are. I have some controllers that will re-order
the disks on every reboot. glabel status and camcontrol devlist
should help verify</p></div></blockquote><div>You can't tell ZFS specifically to use one form of GEOM ID vs another, but you can tell the whole system which GEOM IDs to not use. Add the following to /boot/loader.conf:<br><br></div><div>kern.geom.label.disk_ident.enable="0" # Disable the auto-generated Disk IDs for disks<br>kern.geom.label.gptid.enable="0" # Disable the auto-generated GPT UUIDs for disks<br>kern.geom.label.ufsid.enable="0" # Disable the auto-generated UFS UUIDs for filesystems</div><div><br></div><div>The first line will remove the diskid/DISK-* entries and show the device nodes (daX). The other two lines remove GPT and UFS UUIDs as well.</div><div><br></div><div>All my ZFS systems have those entries in loader.conf, as I prefer to use GPT partition labels in my pools (gpt/label-name) where I list which specific JBOD chassis and drive bay the HD is located. That way, it doesn't matter if the device nodes are renumbered, as the labels don't change. Makes it much easier to find the specific drive to be replaced, whether in my home server with 6 drives or my backups servers at work with multiple JBODs and 92 drives.</div></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature">Freddie Cash<br><a href="mailto:fjwcash@gmail.com" target="_blank">fjwcash@gmail.com</a></div></div>
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOjFWZ7aSy3HtBUVMvo6U8JETTYGWLgszt-FmgbfGDq-qC-mgg>