Date: Thu, 22 Sep 2016 13:14:50 -0700 From: Aaron Miller <aaronkmiller@gmail.com> To: freebsd-fs@freebsd.org Subject: Logical unit not ready, manual intervention required Message-ID: <CALSvhybZxmYTH%2Br0FZ775p6rSDYM2aVMs=5e%2B=FzGVbOstf4xg@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello all, I hope I'm posting to the right list and someone can help me.
After rebooting my FreeBSD 10.3 machine the iSCSI LUN it is hosting is no
longer accessible on my SAN.
The underlying zpool and zvol seem fine:
root@freebsd:~ # zpool status
pool: spool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
spool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
mirror-3 ONLINE 0 0 0
da6 ONLINE 0 0 0
da7 ONLINE 0 0 0
mirror-4 ONLINE 0 0 0
da8 ONLINE 0 0 0
da9 ONLINE 0 0 0
mirror-5 ONLINE 0 0 0
da10 ONLINE 0 0 0
da11 ONLINE 0 0 0
root@freebsd:~ # zfs list
NAME USED AVAIL REFER MOUNTPOINT
spool 3.16T 0 19K /spool
spool/volume1 3.16T 3.15T 5.18G -
root@freebsd:~ # zfs get all spool/volume1
NAME PROPERTY VALUE SOURCE
spool/volume1 type volume -
spool/volume1 creation Tue Sep 6 8:23 2016 -
spool/volume1 used 3.16T -
spool/volume1 available 3.15T -
spool/volume1 referenced 5.18G -
spool/volume1 compressratio 1.00x -
spool/volume1 reservation none default
spool/volume1 volsize 3.06T local
spool/volume1 volblocksize 8K -
spool/volume1 checksum on default
spool/volume1 compression off default
spool/volume1 readonly off default
spool/volume1 copies 1 default
spool/volume1 refreservation 3.16T local
spool/volume1 primarycache all default
spool/volume1 secondarycache all default
spool/volume1 usedbysnapshots 0 -
spool/volume1 usedbydataset 5.18G -
spool/volume1 usedbychildren 0 -
spool/volume1 usedbyrefreservation 3.15T -
spool/volume1 logbias latency default
spool/volume1 dedup off default
spool/volume1 mlslabel -
spool/volume1 sync standard default
spool/volume1 refcompressratio 1.00x -
spool/volume1 written 5.18G -
spool/volume1 logicalused 5.15G -
spool/volume1 logicalreferenced 5.15G -
spool/volume1 volmode default default
spool/volume1 snapshot_limit none default
spool/volume1 snapshot_count none default
spool/volume1 redundant_metadata all default
The LUN seems to start okay?
root@freebsd:/var/log # ctladm start 0
(7:0:0/0): LUN started successfully
I do have some errors in the log but I'm not sure if they are related:
Sep 22 19:12:03 freebsd kernel: GEOM: da0: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da0: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da1: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da1: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da2: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da2: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da3: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da3: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da4: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da4: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da5: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da5: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da6: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da6: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da7: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da7: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da8: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da8: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da9: the primary GPT table is corrupt
or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da9: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:03 freebsd kernel: GEOM: da10: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:03 freebsd kernel: GEOM: da10: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM: da11: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM: da11: using the secondary instead --
recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZY8NW1L%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZY8NW1L%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-J9WV8UEL%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-J9WV8UEL%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-J9WRVTEL%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-J9WRVTEL%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZXZN71J%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZXZN71J%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZXYHAKJ%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZXYHAKJ%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZXZB4PJ%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZXZB4PJ%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZXZN7HJ%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZXZN7HJ%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZWKHP0J%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZWKHP0J%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZWKJB9J%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZWKJB9J%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-J9WT1S8L%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-J9WT1S8L%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-J9WRD6PL%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-J9WRD6PL%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZYD0VWM%20%20%20%20%20%20%20%20: the primary GPT table is
corrupt or invalid.
Sep 22 19:12:04 freebsd kernel: GEOM:
diskid/DISK-JZYD0VWM%20%20%20%20%20%20%20%20: using the secondary instead
-- recovery strongly advised.
There's a lot of this noise when an initiator is online and trying to
connect:
Sep 22 19:40:49 freebsd ctld[6983]: 10.33.80.62: read: connection lost
Sep 22 19:40:49 freebsd ctld[659]: child process 6983 terminated with exit
status 1
Sep 22 19:40:49 freebsd ctld[6984]: 10.33.80.62: read: connection lost
Sep 22 19:40:49 freebsd ctld[659]: child process 6984 terminated with exit
status 1
Sep 22 19:40:49 freebsd ctld[6985]: 10.33.80.62
(iqn.1993-08.org.debian:01:46952f23d3e): read: connection lost
Sep 22 19:40:49 freebsd ctld[659]: child process 6985 terminated with exit
status 1
Sep 22 19:40:57 freebsd ctld[6987]: 10.33.80.62
(iqn.1993-08.org.debian:01:46952f23d3e): read: connection lost
On the initiator which is running debian-based proxmox there is a lot of
this in the log:
Sep 22 12:49:53 proxmox kernel: [ 136.579835] sd 1:0:0:0: [sdb] Add.
Sense: Logical unit not ready, manual intervention required
Sep 22 12:49:53 proxmox kernel: [ 136.580246] sd 1:0:0:0: [sdb] Read
Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Sep 22 12:49:53 proxmox kernel: [ 136.580250] sd 1:0:0:0: [sdb] Sense Key
: Not Ready [current]
Which after some googling sounds like I need to run 'ctladm start 0 -o',
right? Well for some reason that doesn't work:
root@freebsd:~ # ctladm start 0 -o
ctladm: illegal option -- o
ctladm: illegal option -- o
(7:0:0/0): LUN started successfully
No idea why it's saying illegal option here? It's listed in the command
help:
root@freebsd:~ # ctladm
Usage:
Primary commands:
<output snipped>
ctladm start [dev_id][general options] [-i] [-o]
But that option is missing from 'man ctladm'? Folks on #freebsd have
suggested that maybe it was added after freebsd 10.3 but I've found man
pages online from 9.x that have it.
Any assistance greatly appreciated! Thanks!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALSvhybZxmYTH%2Br0FZ775p6rSDYM2aVMs=5e%2B=FzGVbOstf4xg>