Date: Fri, 23 Sep 2016 15:48:02 -0700 From: Aaron Miller <aaronkmiller@gmail.com> To: freebsd-fs@freebsd.org Subject: Re: Logical unit not ready, manual intervention required Message-ID: <CALSvhyZ7po4vZtY7MO32n4sg_1rTfmf2Lgzkp5PV12v%2BjzT52g@mail.gmail.com> In-Reply-To: <CALSvhybZxmYTH%2Br0FZ775p6rSDYM2aVMs=5e%2B=FzGVbOstf4xg@mail.gmail.com> References: <CALSvhybZxmYTH%2Br0FZ775p6rSDYM2aVMs=5e%2B=FzGVbOstf4xg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I figured it out. I did not have a size defined for my LUN in /etc/ctl.conf. Added that, restarted ctld and now it's working. On Thu, Sep 22, 2016 at 1:14 PM, Aaron Miller <aaronkmiller@gmail.com> wrote: > Hello all, I hope I'm posting to the right list and someone can help me. > After rebooting my FreeBSD 10.3 machine the iSCSI LUN it is hosting is no > longer accessible on my SAN. > > The underlying zpool and zvol seem fine: > > root@freebsd:~ # zpool status > pool: spool > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > spool ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > da0 ONLINE 0 0 0 > da1 ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > da2 ONLINE 0 0 0 > da3 ONLINE 0 0 0 > mirror-2 ONLINE 0 0 0 > da4 ONLINE 0 0 0 > da5 ONLINE 0 0 0 > mirror-3 ONLINE 0 0 0 > da6 ONLINE 0 0 0 > da7 ONLINE 0 0 0 > mirror-4 ONLINE 0 0 0 > da8 ONLINE 0 0 0 > da9 ONLINE 0 0 0 > mirror-5 ONLINE 0 0 0 > da10 ONLINE 0 0 0 > da11 ONLINE 0 0 0 > > root@freebsd:~ # zfs list > NAME USED AVAIL REFER MOUNTPOINT > spool 3.16T 0 19K /spool > spool/volume1 3.16T 3.15T 5.18G - > > root@freebsd:~ # zfs get all spool/volume1 > NAME PROPERTY VALUE SOURCE > spool/volume1 type volume - > spool/volume1 creation Tue Sep 6 8:23 2016 - > spool/volume1 used 3.16T - > spool/volume1 available 3.15T - > spool/volume1 referenced 5.18G - > spool/volume1 compressratio 1.00x - > spool/volume1 reservation none default > spool/volume1 volsize 3.06T local > spool/volume1 volblocksize 8K - > spool/volume1 checksum on default > spool/volume1 compression off default > spool/volume1 readonly off default > spool/volume1 copies 1 default > spool/volume1 refreservation 3.16T local > spool/volume1 primarycache all default > spool/volume1 secondarycache all default > spool/volume1 usedbysnapshots 0 - > spool/volume1 usedbydataset 5.18G - > spool/volume1 usedbychildren 0 - > spool/volume1 usedbyrefreservation 3.15T - > spool/volume1 logbias latency default > spool/volume1 dedup off default > spool/volume1 mlslabel - > spool/volume1 sync standard default > spool/volume1 refcompressratio 1.00x - > spool/volume1 written 5.18G - > spool/volume1 logicalused 5.15G - > spool/volume1 logicalreferenced 5.15G - > spool/volume1 volmode default default > spool/volume1 snapshot_limit none default > spool/volume1 snapshot_count none default > spool/volume1 redundant_metadata all default > > > The LUN seems to start okay? > > root@freebsd:/var/log # ctladm start 0 > (7:0:0/0): LUN started successfully > > > I do have some errors in the log but I'm not sure if they are related: > > Sep 22 19:12:03 freebsd kernel: GEOM: da0: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da0: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da1: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da1: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da2: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da2: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da3: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da3: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da4: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da4: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da5: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da5: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da6: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da6: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da7: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da7: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da8: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da8: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da9: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da9: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:03 freebsd kernel: GEOM: da10: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:03 freebsd kernel: GEOM: da10: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: da11: the primary GPT table is > corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: da11: using the secondary instead -- > recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZY8NW1L%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZY8NW1L%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-J9WV8UEL%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-J9WV8UEL%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-J9WRVTEL%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-J9WRVTEL%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZXZN71J%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZXZN71J%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZXYHAKJ%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZXYHAKJ%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZXZB4PJ%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZXZB4PJ%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZXZN7HJ%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZXZN7HJ%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZWKHP0J%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZWKHP0J%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZWKJB9J%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZWKJB9J%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-J9WT1S8L%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-J9WT1S8L%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-J9WRD6PL%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-J9WRD6PL%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZYD0VWM%20%20%20%20%20%20%20%20: > the primary GPT table is corrupt or invalid. > Sep 22 19:12:04 freebsd kernel: GEOM: diskid/DISK-JZYD0VWM%20%20%20%20%20%20%20%20: > using the secondary instead -- recovery strongly advised. > > > There's a lot of this noise when an initiator is online and trying to > connect: > > Sep 22 19:40:49 freebsd ctld[6983]: 10.33.80.62: read: connection lost > Sep 22 19:40:49 freebsd ctld[659]: child process 6983 terminated with exit > status 1 > Sep 22 19:40:49 freebsd ctld[6984]: 10.33.80.62: read: connection lost > Sep 22 19:40:49 freebsd ctld[659]: child process 6984 terminated with exit > status 1 > Sep 22 19:40:49 freebsd ctld[6985]: 10.33.80.62 (iqn.1993-08.org.debian:01:46952f23d3e): > read: connection lost > Sep 22 19:40:49 freebsd ctld[659]: child process 6985 terminated with exit > status 1 > Sep 22 19:40:57 freebsd ctld[6987]: 10.33.80.62 (iqn.1993-08.org.debian:01:46952f23d3e): > read: connection lost > > > On the initiator which is running debian-based proxmox there is a lot of > this in the log: > > Sep 22 12:49:53 proxmox kernel: [ 136.579835] sd 1:0:0:0: [sdb] Add. > Sense: Logical unit not ready, manual intervention required > Sep 22 12:49:53 proxmox kernel: [ 136.580246] sd 1:0:0:0: [sdb] Read > Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE > Sep 22 12:49:53 proxmox kernel: [ 136.580250] sd 1:0:0:0: [sdb] Sense Key > : Not Ready [current] > > > Which after some googling sounds like I need to run 'ctladm start 0 -o', > right? Well for some reason that doesn't work: > > root@freebsd:~ # ctladm start 0 -o > ctladm: illegal option -- o > ctladm: illegal option -- o > (7:0:0/0): LUN started successfully > > > No idea why it's saying illegal option here? It's listed in the command > help: > > root@freebsd:~ # ctladm > Usage: > Primary commands: > <output snipped> > ctladm start [dev_id][general options] [-i] [-o] > > > But that option is missing from 'man ctladm'? Folks on #freebsd have > suggested that maybe it was added after freebsd 10.3 but I've found man > pages online from 9.x that have it. > > > Any assistance greatly appreciated! Thanks! >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALSvhyZ7po4vZtY7MO32n4sg_1rTfmf2Lgzkp5PV12v%2BjzT52g>