From owner-freebsd-stable@FreeBSD.ORG Wed Sep 26 08:33:59 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 96023106566C for ; Wed, 26 Sep 2012 08:33:59 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id 130588FC16 for ; Wed, 26 Sep 2012 08:33:58 +0000 (UTC) Received: by wibhq12 with SMTP id hq12so3519885wib.13 for ; Wed, 26 Sep 2012 01:33:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:date:subject:to:message-id:mime-version:x-mailer; bh=9VCmrLmYyvkaGzDd/ZXHrPiShQawLVXouWLcYC/SL9g=; b=crZctoSsH54qSb+itJolhcft5Eg9ElmwHSJfU+SgFlfyXZ/wJ48QJnspPog9Vv2EYP O86q6+83lZUEYJnmVzKesF9w2qsm5QbANCDn8lFLvng+aYJlmd3NCbZpIM9Bu8BZ30n4 7wByQmMkLahZdOEloX3iV1iLn1dJlslUm15SedKrZ5e7Zcs9H4PZxTS/gxNg7gYEf/vQ N7q1QaRTc/5vyH5jIrZVJwLuL/DhFyeXeonYirsINvepRtrnAXM01wsHeDWmNw2/vO/R g/LjKrGIO3j0kFUZdQ9stLiyqqngxpdYxuXSUnToKEHQqdi93dILt+gCKxXnHeTE0eNP GRGQ== Received: by 10.180.93.33 with SMTP id cr1mr27776070wib.8.1348648437901; Wed, 26 Sep 2012 01:33:57 -0700 (PDT) Received: from ndenevsa.sf.moneybookers.net (g1.moneybookers.com. [217.18.249.148]) by mx.google.com with ESMTPS id fb20sm5516459wid.1.2012.09.26.01.33.56 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 26 Sep 2012 01:33:57 -0700 (PDT) From: Nikolay Denev Date: Wed, 26 Sep 2012 11:33:57 +0300 To: "freebsd-stable@freebsd.org" Message-Id: <057A94AC-3CBC-4E91-8816-74EE2E4C8420@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) X-Mailer: Apple Mail (2.1498) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: CAM Target Layer, Linux and camcontrol readcap X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Sep 2012 08:33:59 -0000 Hi, I'm running RELENG_9 and I'm trying to play with CTL. Initially I've setup an isp(4) interface in TARGET mode and tried to = export a LUN to a directly connected Linux RHEL host, but for some reason that failed with the block backend = (ramdisk was exported properly) : This is how I export the volume : zfs create -V1000G tank/oracle ctladm create -b block -o file=3D/dev/zvol/tank/oracle -S ZFSSERIAL001 = -d ZFSLUN001 ctladm port -o on ctladm realsync off=20 camcontrol and ctladm show the device correctly : [10:25]root@goliath:/home/ndenev# ctladm port -l =20 Port Online Type Name pp vp WWNN WWPN = =20 0 YES IOCTL CTL ioctl 0 0 0 0 = =20 1 YES INTERNAL ctl2cam 0 0 0x500000091a1f4700 = 0x500000091a1f4702 2 YES INTERNAL CTL internal 0 0 0 0 = =20 3 YES FC isp0 0 0 0x20000024ff376b98 = 0x21000024ff376b98 4 YES FC isp1 1 0 0x20000024ff376b99 = 0x21000024ff376b99 [10:26]root@goliath:/home/ndenev# ctladm devlist -v LUN Backend Size (Blocks) BS Serial Number Device ID =20 0 block 2097152000 512 ZFSSERIAL001 ZFSLUN001 =20 lun_type=3D0 num_threads=3D14 file=3D/dev/zvol/tank/oracle [10:26]root@goliath:/home/ndenev# camcontrol devlist at scbus2 target 1 lun 0 (da0,pass0) This is what I see on the Linux host for the exported zvol: qla2xxx 0000:0a:00.1: LOOP UP detected (8 Gbps). qla2xxx 0000:0a:00.1: qla2xxx_eh_host_reset: reset succeeded qla2xxx 0000:0a:00.1: scsi(4:0:0): Abort command issued -- 1 c 2002. sd 4:0:0:0: scsi: Device offlined - not ready after error recovery sd 4:0:0:0: rejecting I/O to offline device sd 4:0:0:0: rejecting I/O to offline device sd 4:0:0:0: rejecting I/O to offline device sdq : READ CAPACITY failed. sdq : status=3D0, message=3D00, host=3D1, driver=3D00=20 sdq : sense not available.=20 sd 4:0:0:0: rejecting I/O to offline device sdq: Write Protect is off sdq: Mode Sense: 00 00 00 00 sd 4:0:0:0: rejecting I/O to offline device sdq: asking for cache data failed sdq: assuming drive cache: write through sd 4:0:0:0: Attached scsi disk sdq sd 4:0:0:0: Attached scsi generic sg18 type 0 sd 4:0:0:0: rejecting I/O to offline device sd 4:0:0:0: rejecting I/O to offline device sd 4:0:0:0: rejecting I/O to offline device sd 4:0:0:0: rejecting I/O to offline device sd 4:0:0:0: rejecting I/O to offline device sd 4:0:0:0: rejecting I/O to offline device I've noticed the READ CAPACITY failed message and tried to issue a = several "camcontrol readcap" commands, but they got stuck and are now unkillable : [10:21]root@goliath:/home/ndenev# ps axuw|grep cam root 29739 0.0 0.0 16300 1628 0- DL+ 8:09AM 0:00.00 = camcontrol readcap da0 root 30033 0.0 0.0 16300 1628 1- DL+ 8:12AM 0:00.00 = camcontrol readcap 2:1:0 root 30135 0.0 0.0 16300 1632 2 D+ 8:21AM 0:00.00 = camcontrol start pass0 procstat shows the same kernel stack for them : [10:23]root@goliath.:/home/ndenev# procstat -kk 30033 PID TID COMM TDNAME KSTACK = =20 30033 101161 camcontrol - mi_switch+0x186 = sleepq_wait+0x42 _sleep+0x390 cam_periph_runccb+0x5a passioctl+0x171 = devfs_ioctl_f+0x7b kern_ioctl+0x115 sys_ioctl+0xfd amd64_syscall+0x546 = Xfast_syscall+0xf7=20 Also there is this suspicious message on the FreeBSD machine : cfcs_action: unsupported CCB type 0x918 Then I've tried to remove the volume from CTL but the command also got = stuck : [10:29]root@goliath:/home/ndenev# ps axuww|grep ctladm root 54269 0.0 0.0 18484 1692 3 I 8:24AM 0:00.00 = ctladm remove -b block -l 0 root 20969 0.0 0.0 16280 1720 5 S+ 10:30AM 0:00.00 grep = ctladm [10:30]root@goliath:/home/ndenev# procstat -kk 54269 PID TID COMM TDNAME KSTACK = =20 54269 101580 ctladm - mi_switch+0x186 = sleepq_catch_signals+0x2cc sleepq_wait_sig+0xc _sleep+0x2b9 = ctl_be_block_ioctl+0x9aa ctl_ioctl+0x9e6 devfs_ioctl_f+0x7b = kern_ioctl+0x115 sys_ioctl+0xfd amd64_syscall+0x546 Xfast_syscall+0xf7=20= Some Linux mailing lists suggest that maybe a firmware upgrade on the = Linux box can help, but this does not explain the stuck readcap and ctladm commands. Any ideas on how to debug this further are welcome.=