Date: Thu, 11 Aug 2016 19:10:58 +0300 From: Max Gurtovoy <maxg@mellanox.com> To: Ben RUBSON <ben.rubson@gmail.com>, <freebsd-scsi@freebsd.org> Cc: Hans Petter Selasky <hanss@mellanox.com>, Oren Duer <oren@mellanox.com>, Oded Shanoon <odeds@mellanox.com>, Meny Yossefi <menyy@mellanox.com>, "Roy Shterman" <roysh@mellanox.com> Subject: Re: Help to make iSER working Message-ID: <a8b1cd70-3e96-2dd6-20ae-1bf3b315ff89@mellanox.com> In-Reply-To: <B3B47B30-D253-4543-A813-1AEDD3EE2656@gmail.com> References: <71E4886B-61C3-4091-A7FF-E5A4F59AEACE@gmail.com> <64d8bc44-b6a3-4c0a-6d13-f919bf26685a@mellanox.com> <41CCC244-A781-4087-A811-B33847EDEC01@gmail.com> <6a464f2c-c048-d29c-e602-c129b35a5d4a@mellanox.com> <B3B47B30-D253-4543-A813-1AEDD3EE2656@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
On 8/11/2016 6:33 PM, Ben RUBSON wrote:
> Thank you very much Max for your help.
>
>
>
> I managed to compile the kernel with iSER support doing the 3 modifications below.
>
> I added the following to /usr/src/sys/conf/options :
> ICL_KERNEL_PROXY opt_global.h
>
> And I added "options ICL_KERNEL_PROXY" to my kernel configuration file.
>
> I then faced this compilation error :
> /usr/src/sys/modules/ctl/../../cam/ctl/ctl_frontend_iscsi.c:1358:2: error: implicit declaration of function 'icl_conn_handoff_sock'
> icl_conn_handoff_sock(cs->cs_conn, so);
> /usr/src/sys/modules/ctl/../../cam/ctl/ctl_frontend_iscsi.c:1358:2: note: did you mean 'icl_soft_handoff_sock'?
> /usr/src/sys/dev/iscsi/icl.h:158:7: note: 'icl_soft_handoff_sock' declared here
> int icl_soft_handoff_sock(struct icl_conn *ic, struct socket *so);
>
> So I replaced icl_conn_handoff_sock by icl_soft_handoff_sock (sounds like it is the needed correction).
>
> iscsictl -r now does not produce previous "invalid argument" error.
>
> @trasz : perhaps these modifications should be done in FreeBSD 11 source tree ?
>
>
>
> Now I'm facing the following issue :
>
> # iscsictl -A -r -t iqn.2012-06.com.test:target1 -p 192.168.2.2
> iscsid[8743]: 192.168.2.2 (iqn.2012-06.com.test:target1): failed to connect to 192.168.2.2 using ICL kernel proxy: ISCSIDCONNECT: Input/output error
> kernel: ERROR: iser_connect_error: conn 0xfffff8096786f000
> iscsid[1078]: child process 8743 terminated with exit status 1
please increase the debug level in both initiator and target and sent it
out.
From first look I think you need to configure the LIO target to work
with demo mode.
Another thing you can try is the user space tgt target (compile with
make ISCSI_RDMA=1) that I work with mostly.
>
> If I remove the "-r" option from the iscsictl command, it correctly connects to the target.
>
> On target side, Lio, targetcli :
> /> ls iscsi/
> o- iscsi ............................................. [1 Target]
> o- iqn.2012-06.com.test:target1 ....................... [1 TPG]
> o- tpg1 ........................................... [enabled]
> o- acls .......................................... [0 ACLs]
> o- luns ........................................... [1 LUN]
> | o- lun0 ..................... [iblock/block1 (/dev/sdc7)]
> o- portals ..................................... [1 Portal]
> o- 192.168.2.2:3260 .................. [OK, iser enabled]
>
> I wanted to make some RDMA connection tests with udaddy for example, but I did not found the FreeBSD version.
>
> Any advice ?
>
> Many thanks !
>
> Ben
>
>
>
>> On 11 Aug 2016, at 15:44, Max Gurtovoy <maxg@mellanox.com> wrote:
>>
>> These are the instructions I get from iscsi maintainer that merged our code to FreeBSD-11:
>> 1. set WITH_OFED=yes in /etc/make.conf
>> 2. "make buildkernel buildworld installkernel installworld"
>> 3. reboot
>> 4. kldload iser
>>
>> After that, make sure you configured all the interfaces correctly and that you have ping between the 2 servers.
>> later you need to check basic RDMA stuff on your servers (Added Hans/Oded/Meny to help you if needed).
>>
>> Only if the above are working you can proceed to check iser.
>>
>> Max.
>>
>> On 8/11/2016 4:22 PM, Ben RUBSON wrote:
>>> I just tested to connect to a Linux iSER enabled target, but I still get this error message at initiator side :
>>> iscsid[1187]: 192.168.2.2 (iqn.2012-06.com.test:target1): failed to connect to 192.168.2.2 using ICL kernel proxy: ISCSIDCONNECT: Invalid argument
>>>
>>>> as if my initiator did not understand the iSER command "ISCSIDCONNECT".
>>>> How to be sure everything is correctly compiled with ICL_KERNEL_PROXY enabled ?
>>>
>>> Ben
>>>
>>>
>>>> On 10 Aug 2016, at 22:51, Max Gurtovoy <maxg@mellanox.com> wrote:
>>>>
>>>> Hi Ben,
>>>>
>>>> iSER initiator is supported in FreeBSD-11 but not the iser target.
>>>> for the target you should use Linux target such as TGT (user space),
>>>> Lio or SCST.
>>>>
>>>> thanks,
>>>> Max.
>>>>
>>>>
>>>> On 8/10/2016 8:13 PM, Ben RUBSON wrote:
>>>>> Hello,
>>>>>
>>>>> I have some difficulties to make iSER (FreeBSD 11 BETA3) working.
>>>>>
>>>>>
>>>>>
>>>>> ### My configuration :
>>>>>
>>>>> 2 servers with Mellanox ConnectX-3 NICs.
>>>>>
>>>>>
>>>>>
>>>>> ### What I did :
>>>>>
>>>>> I rebuilt kernel adding the following to GENERIC :
>>>>> options OFED
>>>>> device mlxen
>>>>> options COMPAT_LINUXKPI
>>>>>
>>>>> I built iser :
>>>>> cd /usr/src/sys/modules/iser/ ; make ; make install ; kldload iser
>>>>> (I saw later-on that I could have used "device iser" in kernel configuration)
>>>>>
>>>>> My /etc/ctl.conf :
>>>>> portal-group pg0 {
>>>>> discovery-auth-group no-authentication
>>>>> listen 192.168.2.2
>>>>> offload iser
>>>>> }
>>>>> target iqn.2012-06.com.test:target1 {
>>>>> auth-group no-authentication
>>>>> portal-group pg0
>>>>> alias my-iser-disk
>>>>> lun 0 {
>>>>> path /dev/da8
>>>>> }
>>>>> }
>>>>>
>>>>>
>>>>>
>>>>> ### Error :
>>>>>
>>>>> As soon as I do the following :
>>>>> iscsictl -A -r -t iqn.2012-06.com.test:target1 -p 192.168.2.2
>>>>> I get this error :
>>>>> iscsid[1187]: 192.168.2.2 (iqn.2012-06.com.test:target1): failed to connect to 192.168.2.2 using ICL kernel proxy: ISCSIDCONNECT: Invalid argument
>>>>> iscsid[655]: child process 1187 terminated with exit status 1
>>>>>
>>>>> "sysctl hw.mlxen1.stat.tx_packets" on initiator shows that no packet is sent at all when trying to connect.
>>>>>
>>>>> Did I miss something ?
>>>>>
>>>>> Many thanks !
>>>>>
>>>>> Ben
>>>>>
>>>>> _______________________________________________
>>>>> freebsd-scsi@freebsd.org mailing list
>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
>>>>> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org"
>>>>>
>>>
>>> _______________________________________________
>>> freebsd-scsi@freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
>>> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org"
>>>
>
> _______________________________________________
> freebsd-scsi@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi
> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a8b1cd70-3e96-2dd6-20ae-1bf3b315ff89>