From owner-freebsd-stable@freebsd.org Wed Oct 4 10:18:45 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 688DCE3510C for ; Wed, 4 Oct 2017 10:18:45 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E5AD07F36F for ; Wed, 4 Oct 2017 10:18:44 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. (net206-94.perm.ertelecom.ru [46.146.206.94] (may be forged)) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id v94AIdpp026100 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 4 Oct 2017 15:18:39 +0500 (YEKT) (envelope-from emz@norma.perm.ru) To: FreeBSD Stable From: "Eugene M. Zheganin" Subject: iSCSI: LUN modification error: LUN XXX is not managed by the block backend and LUN device confusion Message-ID: <2ed81698-394e-9d62-a346-c244ed92c01a@norma.perm.ru> Date: Wed, 4 Oct 2017 15:18:39 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spamd-Result: default: False [1.64 / 25.00] BAYES_HAM(-2.86)[99.39%] HFILTER_HOSTNAME_UNKNOWN(2.50)[] RBL_SPAMHAUS_PBL(2.00)[94.206.146.46.zen.spamhaus.org : 127.0.0.10] MID_RHS_MATCH_FROM(0.00)[] RCVD_COUNT_1(0.00)[] FROM_EQ_ENVFROM(0.00)[] RCPT_COUNT_1(0.00)[] ONCE_RECEIVED(0.10)[] MIME_GOOD(-0.10)[text/plain] DMARC_NA(0.00)[norma.perm.ru] RECEIVED_SPAMHAUS(0.00)[94.206.146.46.zen.spamhaus.org] R_DKIM_NA(0.00)[] FROM_HAS_DN(0.00)[] TO_MATCH_ENVRCPT_ALL(0.00)[] TO_DN_ALL(0.00)[] R_SPF_SOFTFAIL(0.00)[~all] X-Rspamd-Server: localhost X-Rspamd-Scan-Time: 0.73 X-Rspamd-Queue-ID: v94AIdpp026100 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Oct 2017 10:18:45 -0000 Hi, I got one more problem while dealing iSCSI targets in the production (yeah, I'm boring and stubborn). The environment is as in previous questions (a production site, hundreds of VMs and hundreds of disks). I've encountered this issue before, but this time i decided to ask whether it's possible that the reason is my inadequate actions. We have two types of disks in production, one is called "userdata[number]" and another is called "games[something+number]". The iSCSI targets are named appropriately, plus userdata disks have the scsiname[number] option. Number simply indicates the VM it should be attached to. But sometimes some weird confusion happens, and I have two sorts of things, let me show them using one LUN as example (in reality right now I have 6 LUNs like this). So from now we are considering the 310 as a VM tag, and two disks, userdata310 and games disk. So imagine a piece of ctl.conf like this: ===Cut=== # # worker310 # target iqn.2016-04.net.playkey.iscsi:userdata-worker310 { initiator-portal 10.0.3.142/32 portal-group playkey auth-type none lun 0 { option scsiname userdata310 path /dev/zvol/data/userdata/worker310 } } # # worker310 # target iqn.2016-04.net.playkey.iscsi:gamestop-worker310 { initiator-portal 10.0.3.142/32 portal-group playkey auth-type none lun 0 { path /dev/zvol/data/reference-ver13_1233-worker310 } } ===Cut=== When the issue happens, I got the following line in the log: Oct 4 12:00:55 san1 ctld[777]: LUN modification error: LUN 547 is not managed by the block backend Oct 4 12:00:55 san1 ctld[777]: failed to modify lun "iqn.2016-04.net.playkey.iscsi:userdata-worker310,lun,0", CTL lun 547 In the "ctladm devlist -v" I see this about the LUN 547: 547 block 10737418240 512 MYSERIAL 738 MYDEVID 738 lun_type=0 num_threads=14 file=/dev/zvol/data/reference-ver13_1233-worker228 ctld_name=iqn.2016-04.net.playkey.iscsi:gamestop-worker228,lun,0 scsiname=userdata310 So, notice, that the userdata disk for VM310 has the devices for completely different VM (according to their names). Weird ! One may think that this is simply the misconfiguration and the games disk for worker228 VM simply has the erroneous scsiname option tag. But no, it hasn't: # # worker228 # target iqn.2016-04.net.playkey.iscsi:gamestop-worker228 { initiator-portal [...obfuscated...]/32 portal-group playkey auth-type none lun 0 { path /dev/zvol/data/reference-ver13_1233-worker228 } } The workaround to this is simply to comment the troublesome LUNs/targets in the ctl.conf, reload, uncomment and reload again. Am I doing something wrong ? Thanks. Eugene.