From owner-freebsd-scsi@freebsd.org Thu Aug 11 15:33:52 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D04DBB5EE0 for ; Thu, 11 Aug 2016 15:33:52 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9BC4C1C27 for ; Thu, 11 Aug 2016 15:33:51 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x232.google.com with SMTP id o80so3705525wme.1 for ; Thu, 11 Aug 2016 08:33:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=aqMf4i3111HSw+umlUyM1UI7FdDRnPfYbmHW4p0U3T0=; b=yB2VjnSsLrouhCkhRjiOvy3uLz+ajp+4hpOjYwWZE1cokUt9X/hzyXSGwwF8vKBW1K gH8/HdGNZ5Gzvx4V47z5Oh5JJ1nsU4FrnBpVYa1izsbJjqw2FPpG3g4/KGMtcdUzFkfm +ePt5uZLX0IYLqwQFBLukT7S/W2D2SqUOu62js45uZqCeFtBw/4eNQwHheKhdVkxkn/X aHB/ARN96xz+fm0Sa1QKKEzYS4/jvM0Ma41mMZUs7dL1/hmnLoTf2uR78PQT+yUnyDdi kVD4IHEq0mXsVQPHbZi5FL6oWt7+YkTGCyziIASUMGrBhlBZ6DzzAzUk9Og+hm5ix+1p ZTpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=aqMf4i3111HSw+umlUyM1UI7FdDRnPfYbmHW4p0U3T0=; b=RbA0s6TiLs0vcotNMS369UKB4zOSOzbRVDtS9Gu9YMAEzx8iqBLCOeZuto/VZFCkw2 qhrMKZZcnaB+25pJM9lEf4/aXaxYZ3IGoVy3xAll0Nwf2kLZyWSj+26zCnQPTJt7YGBY zdAtGhmW3KICgHEFG3LCH7vgflwFWn6N6GiLEsfzpIH+mxdgHdLZdbzTajK0UnqWup5X 56Yaf2Hle5mkT/TaIrWm3ltJdzkFWSyA/JAZ320ZOF+1DzSQKNWU4rDqkj48WqSKm0mp jSTPLo3l62DUmzILPHtpf+cABSoFk8lF/ltfm4sWZTZPTw7nkBA44wXugBYrnv+QsysY 2yGA== X-Gm-Message-State: AEkoousinpqwowxNHuSlGWy+gQTIp1EP4B8JAuBjCN9wadP5T/ysz4ID9OjClkSvy8bJYQ== X-Received: by 10.194.176.165 with SMTP id cj5mr12489288wjc.82.1470929629359; Thu, 11 Aug 2016 08:33:49 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id r67sm337186wmb.14.2016.08.11.08.33.48 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 11 Aug 2016 08:33:48 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Help to make iSER working From: Ben RUBSON In-Reply-To: <6a464f2c-c048-d29c-e602-c129b35a5d4a@mellanox.com> Date: Thu, 11 Aug 2016 17:33:47 +0200 Cc: Roy Shterman , Oren Duer , Oded Shanoon , Hans Petter Selasky , Meny Yossefi Content-Transfer-Encoding: quoted-printable Message-Id: References: <71E4886B-61C3-4091-A7FF-E5A4F59AEACE@gmail.com> <64d8bc44-b6a3-4c0a-6d13-f919bf26685a@mellanox.com> <41CCC244-A781-4087-A811-B33847EDEC01@gmail.com> <6a464f2c-c048-d29c-e602-c129b35a5d4a@mellanox.com> To: freebsd-scsi@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 15:33:52 -0000 Thank you very much Max for your help. I managed to compile the kernel with iSER support doing the 3 = modifications below. I added the following to /usr/src/sys/conf/options : ICL_KERNEL_PROXY opt_global.h And I added "options ICL_KERNEL_PROXY" to my kernel configuration file. I then faced this compilation error : /usr/src/sys/modules/ctl/../../cam/ctl/ctl_frontend_iscsi.c:1358:2: = error: implicit declaration of function 'icl_conn_handoff_sock' icl_conn_handoff_sock(cs->cs_conn, so); /usr/src/sys/modules/ctl/../../cam/ctl/ctl_frontend_iscsi.c:1358:2: = note: did you mean 'icl_soft_handoff_sock'? /usr/src/sys/dev/iscsi/icl.h:158:7: note: 'icl_soft_handoff_sock' = declared here int icl_soft_handoff_sock(struct icl_conn *ic, struct socket *so); So I replaced icl_conn_handoff_sock by icl_soft_handoff_sock (sounds = like it is the needed correction). iscsictl -r now does not produce previous "invalid argument" error. @trasz : perhaps these modifications should be done in FreeBSD 11 source = tree ? Now I'm facing the following issue : # iscsictl -A -r -t iqn.2012-06.com.test:target1 -p 192.168.2.2 iscsid[8743]: 192.168.2.2 (iqn.2012-06.com.test:target1): failed to = connect to 192.168.2.2 using ICL kernel proxy: ISCSIDCONNECT: = Input/output error kernel: ERROR: iser_connect_error: conn 0xfffff8096786f000 iscsid[1078]: child process 8743 terminated with exit status 1 If I remove the "-r" option from the iscsictl command, it correctly = connects to the target. On target side, Lio, targetcli : /> ls iscsi/ o- iscsi ............................................. [1 Target] o- iqn.2012-06.com.test:target1 ....................... [1 TPG] o- tpg1 ........................................... [enabled] o- acls .......................................... [0 ACLs] o- luns ........................................... [1 LUN] | o- lun0 ..................... [iblock/block1 (/dev/sdc7)] o- portals ..................................... [1 Portal] o- 192.168.2.2:3260 .................. [OK, iser enabled] I wanted to make some RDMA connection tests with udaddy for example, but = I did not found the FreeBSD version. Any advice ? Many thanks ! Ben > On 11 Aug 2016, at 15:44, Max Gurtovoy wrote: >=20 > These are the instructions I get from iscsi maintainer that merged our = code to FreeBSD-11: > 1. set WITH_OFED=3Dyes in /etc/make.conf > 2. "make buildkernel buildworld installkernel installworld" > 3. reboot > 4. kldload iser >=20 > After that, make sure you configured all the interfaces correctly and = that you have ping between the 2 servers. > later you need to check basic RDMA stuff on your servers (Added = Hans/Oded/Meny to help you if needed). >=20 > Only if the above are working you can proceed to check iser. >=20 > Max. >=20 > On 8/11/2016 4:22 PM, Ben RUBSON wrote: >> I just tested to connect to a Linux iSER enabled target, but I still = get this error message at initiator side : >> iscsid[1187]: 192.168.2.2 (iqn.2012-06.com.test:target1): failed to = connect to 192.168.2.2 using ICL kernel proxy: ISCSIDCONNECT: Invalid = argument >>=20 >>> as if my initiator did not understand the iSER command = "ISCSIDCONNECT". >>> How to be sure everything is correctly compiled with = ICL_KERNEL_PROXY enabled ? >>=20 >> Ben >>=20 >>=20 >>> On 10 Aug 2016, at 22:51, Max Gurtovoy wrote: >>>=20 >>> Hi Ben, >>>=20 >>> iSER initiator is supported in FreeBSD-11 but not the iser target. >>> for the target you should use Linux target such as TGT (user space), >>> Lio or SCST. >>>=20 >>> thanks, >>> Max. >>>=20 >>>=20 >>> On 8/10/2016 8:13 PM, Ben RUBSON wrote: >>>> Hello, >>>>=20 >>>> I have some difficulties to make iSER (FreeBSD 11 BETA3) working. >>>>=20 >>>>=20 >>>>=20 >>>> ### My configuration : >>>>=20 >>>> 2 servers with Mellanox ConnectX-3 NICs. >>>>=20 >>>>=20 >>>>=20 >>>> ### What I did : >>>>=20 >>>> I rebuilt kernel adding the following to GENERIC : >>>> options OFED >>>> device mlxen >>>> options COMPAT_LINUXKPI >>>>=20 >>>> I built iser : >>>> cd /usr/src/sys/modules/iser/ ; make ; make install ; kldload iser >>>> (I saw later-on that I could have used "device iser" in kernel = configuration) >>>>=20 >>>> My /etc/ctl.conf : >>>> portal-group pg0 { >>>> discovery-auth-group no-authentication >>>> listen 192.168.2.2 >>>> offload iser >>>> } >>>> target iqn.2012-06.com.test:target1 { >>>> auth-group no-authentication >>>> portal-group pg0 >>>> alias my-iser-disk >>>> lun 0 { >>>> path /dev/da8 >>>> } >>>> } >>>>=20 >>>>=20 >>>>=20 >>>> ### Error : >>>>=20 >>>> As soon as I do the following : >>>> iscsictl -A -r -t iqn.2012-06.com.test:target1 -p 192.168.2.2 >>>> I get this error : >>>> iscsid[1187]: 192.168.2.2 (iqn.2012-06.com.test:target1): failed to = connect to 192.168.2.2 using ICL kernel proxy: ISCSIDCONNECT: Invalid = argument >>>> iscsid[655]: child process 1187 terminated with exit status 1 >>>>=20 >>>> "sysctl hw.mlxen1.stat.tx_packets" on initiator shows that no = packet is sent at all when trying to connect. >>>>=20 >>>> Did I miss something ? >>>>=20 >>>> Many thanks ! >>>>=20 >>>> Ben >>>>=20 >>>> _______________________________________________ >>>> freebsd-scsi@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >>>> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" >>>>=20 >>=20 >> _______________________________________________ >> freebsd-scsi@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" >>=20