From owner-freebsd-scsi@freebsd.org Thu Aug 11 13:45:47 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA842BB59F5 for ; Thu, 11 Aug 2016 13:45:47 +0000 (UTC) (envelope-from maxg@mellanox.com) Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30071.outbound.protection.outlook.com [40.107.3.71]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 330D41645 for ; Thu, 11 Aug 2016 13:45:46 +0000 (UTC) (envelope-from maxg@mellanox.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+KkWESNawUqAcCa1OmYb06Tq3dh4SXZHAu3LRUKcoZ0=; b=qwwLhMB+NzCEDkj9X2fl89b8TwuLws2IZlZFuStLt4XcwWxzLr4dIvxjIpOVzMQnXI52ox7wBbXcODh5PNu+CVTQ8JRcSI/Op/vqiwWg7aWRx1TN0eCXOc3uxH+Hzzmolupk+f9iNe18AMmrLs5lBDO5xYzWd6M9QPoOD87jBeU= Received: from AMSPR05CA0048.eurprd05.prod.outlook.com (2a01:111:e400:8000::48) by AM3PR05MB0773.eurprd05.prod.outlook.com (2a01:111:e400:8849::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Thu, 11 Aug 2016 13:45:37 +0000 Received: from AM1FFO11FD005.protection.gbl (2a01:111:f400:7e00::198) by AMSPR05CA0048.outlook.office365.com (2a01:111:e400:8000::48) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.549.15 via Frontend Transport; Thu, 11 Aug 2016 13:45:37 +0000 Authentication-Results: spf=pass (sender IP is 193.47.165.134) smtp.mailfrom=mellanox.com; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=pass action=none header.from=mellanox.com; Received-SPF: Pass (protection.outlook.com: domain of mellanox.com designates 193.47.165.134 as permitted sender) receiver=protection.outlook.com; client-ip=193.47.165.134; helo=mtlcas13.mtl.com; Received: from mtlcas13.mtl.com (193.47.165.134) by AM1FFO11FD005.mail.protection.outlook.com (10.174.64.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.567.7 via Frontend Transport; Thu, 11 Aug 2016 13:45:38 +0000 Received: from MTLCAS13.mtl.com (10.0.8.78) by mtlcas13.mtl.com (10.0.8.78) with Microsoft SMTP Server (TLS) id 15.0.1178.4; Thu, 11 Aug 2016 16:45:13 +0300 Received: from MTLCAS01.mtl.com (10.0.8.71) by MTLCAS13.mtl.com (10.0.8.78) with Microsoft SMTP Server (TLS) id 15.0.1178.4 via Frontend Transport; Thu, 11 Aug 2016 16:45:13 +0300 Received: from [10.223.3.133] (10.223.3.133) by MTLCAS01.mtl.com (10.0.8.71) with Microsoft SMTP Server (TLS) id 14.3.301.0; Thu, 11 Aug 2016 16:44:35 +0300 Subject: Re: Help to make iSER working To: Ben RUBSON , References: <71E4886B-61C3-4091-A7FF-E5A4F59AEACE@gmail.com> <64d8bc44-b6a3-4c0a-6d13-f919bf26685a@mellanox.com> <41CCC244-A781-4087-A811-B33847EDEC01@gmail.com> CC: Roy Shterman , Oren Duer , "Oded Shanoon" , Hans Petter Selasky , Meny Yossefi From: Max Gurtovoy Message-ID: <6a464f2c-c048-d29c-e602-c129b35a5d4a@mellanox.com> Date: Thu, 11 Aug 2016 16:44:34 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <41CCC244-A781-4087-A811-B33847EDEC01@gmail.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.223.3.133] X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:193.47.165.134; IPV:NLI; CTRY:IL; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(7916002)(2980300002)(438002)(199003)(189002)(66654002)(377454003)(24454002)(5001770100001)(107886002)(7846002)(7736002)(4001350100001)(3846002)(83506001)(6116002)(586003)(189998001)(31686004)(2950100001)(626004)(23746002)(356003)(87936001)(65806001)(36756003)(65956001)(15975445007)(2906002)(54356999)(81156014)(4001430100002)(11100500001)(8676002)(86362001)(47776003)(8936002)(50986999)(76176999)(81166006)(4326007)(64126003)(230700001)(19580405001)(106466001)(92566002)(33646002)(31696002)(19580395003)(305945005)(77096005)(50466002)(65826006)(3940600001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM3PR05MB0773; H:mtlcas13.mtl.com; FPR:; SPF:Pass; PTR:mail13.mellanox.com; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; AM1FFO11FD005; 1:u3a/PwsbZMqfDNzCic4LjCqgTiSeGLluI+c3cuEh/VSLmlWZ4v4sa8p8AiF8tCDkGEnccFGQxwtov3ixSNv7UWEYoAAPSKt4kILRhS6Fqia4XDsNxM0wGOVcxcoSwP62IFWMb5SBUv8ACIdqj0YTTSnXG+cuxr/d4cZvZy4fj6UUOMOB1tHS0gJMVOhPtQyQ8PmdzbVtVGfbd69uUCvvcx3BzCEutK2BPUwyC8zCg0orcUYCyLAV/vP5rt9XXWa9Go/trzsTx61avD7BlzY7A7GXgVQn0/7VIMBWe1UsNZ2+7XvLJdnGKTaiOCbKitNKlRtHGfQwFu/K/B1Es1Yba2Y3aGalACJDMq1JvNro7ocuIt6LAdXk0fNutt8QH/WVrdXsxjpoQNs7ro+gW2TJEYNmFQLFHF4Yf0QyLamsCYLCHI5SlL1Maa6nLaA/koyINPHfNQhMeB78irz+hheW5hydIFs5XM7TUsL01MoihahxHGnJn8KEtlbMJrTwPPYW2gMEoZX6cMk0j1GKM1yiJ06Dvtd8c7QQJy1xN3qFUjk= X-MS-Office365-Filtering-Correlation-Id: 3d90222b-1f7b-406f-c1fc-08d3c1edc772 X-Microsoft-Exchange-Diagnostics: 1; AM3PR05MB0773; 2:0cqVdiqII/lt5uttgld1fm1qd9zG1nfHRX1AC02h88CQT8E/yF+DATWF1ZLK+6QiPZBy6UcUQORVrxBD/yZWU2+JY3quKz1hQqNTZP3h9xM+pViSO6oDyo+Tk8DcwZAtPQD5NRP0AgTz6TNAQJkIluV4mnMrd6Gpi1lAjNy2hZb73aInMM9UdJW58vHUqD8g; 3:3+9QmJNKtf7Z4vnG1ukCzZ3ENLwmE+XlEbOXP0ZHJHyR5pSRqbaxmjgqTRKUer0q27QpAngZ14JqGjnmm9wm/ItkTrG9fAh8vd7mb/em4rwwK0dZfkh6CARbmfCdBqcpJdSwQfHiHJxMh93cVRM6J15zjNeqvwUixX9Ih56BiEELT9RI+C8V4a7CkZujWxBZ4N5NgxdR6ihcBpIPa3PcMaq3EViyNk7aIxSHKdwTO7MD2eyHBs1DG/VGxFgrekbOf6bA8LMtVdcj7vbj3t20cQ==; 25:isUxd+sR+7WlbC9uomHONfrJ4UuKwkGurkqD8Wk3zoLVwa258eGMRCDcHT8tm+5KxxwzFuAuTnFW70mRApY/CNbUejhOhLCxIRFWOf26id167vMpjYOwwxMxE2t5hdOVqm608+RhVO91+SY+Rpbk0sRodN2N+QuOAI8qGd3el9Ynfa76tvX7DfUta5on50SqQt+lua7YoHcZrIQ8vKdfcAMr6GBkaD79LwEys1FidSgE04tsTs/yjC2/iryP2861UNFQgzPfR2m5shI6z4c9Mly1ddlOWaM+JM3aEs6lTTAD95tMRSCT7Zb0EgaQ7Mv46/X6jgCYuPShm3lecIVLIAFJak/i/NBrUyAD6oL8ZIb6adg+A7AbA8Lc6D03P6jelcvUxk8ZJW8y5WQZI7xUq05HEXPRJmoFIgErcpGluBo= X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501002); SRVR:AM3PR05MB0773; X-Microsoft-Exchange-Diagnostics: 1; AM3PR05MB0773; 31:22NpVYtpCIZ0615UwtcPtQWbEKpByB/P3cqpqKyx6QBPYpvFefLATBPiJZ2Kw6x22BM6LzbIdWpCY0rbf/eGPejraSmxenWkaK2yvr6Mmm7m2a1VcCHlWNJrDc3Nny2Z+bbLNJZRc86vl3Z0e+r4xNybL4kq0nWxdlJ/Db2lkV8I+Gia1TNOCVwork5OY4CD857JPtjYNZScaSwBBubgsO3SUdoEJySv2V3KaMd7zCA=; 20:9A9ohcwd69L5Z1h92zCqNLi+grdUGoX3vSWRYJZqqD1sbtBGx4m1cYg+YdeIq2khF+R1/qRSCFAQ9Cig0TlEzGyQ4C3s4WTqEEgZQknVaUR2iwTZAx+qA7lvczbELNAEVPB6EBFom/pPZIoBfsjLuaeVV4Ia+ig+8KjtSJx1jux2hJLvD2dQbhLpovK23qZSwp+/62JCAdgYu58sxF+NzNz1m5bC0wnhQ46/8urSKKpkQ2iYh5mJY+2HbrNwzgB29wksjsJyzC5fqLb0qK+7JLdPJ3HPY6yjSyJKOlN/HyksryBtxBMctGVBSTRWPgBmnJmmB9G1JOw1zIUu/wrShfCtyv/iJ7/LrpwP8A/hwxEm3LpsOMnHu1psqiLCg2GncW6W0fIPEjz6Pg8ExfNcoi6FYmP1FQcfyilbohUKBFvcDjBIoAByWYiteol34A3QaH4IvVOQ3fBH4Ud+aNYf/mjJkvV71j+QJ3byge9OoQufvyNETnzQ/lPlx479ASRR X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(258220229709018)(75325880899374)(21532816269658); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(13018025)(13016025)(3002001)(10201501046)(6055026); SRVR:AM3PR05MB0773; BCL:0; PCL:0; RULEID:; SRVR:AM3PR05MB0773; X-Microsoft-Exchange-Diagnostics: 1; AM3PR05MB0773; 4:4MpeK96UNRmTcqNoCT7E+UjQ37NgtOcpuhixoLUP4fmPCXIrplrK6K4DN5/iRx2rwlglsvCNWm2YXxEtxcRjVpi5jiWCRiB3uyPULmJTKOcwF8I3kDU5oy8Q/EY08vFEjBPi6AlHvcRluB2I9im0S5XQUVB+dSbNQvqvQAghPmf2heXfJd1kDfsqQyPwvfnEh8x4jNE/NTnirU0Kq9m8NWEg8CqRRymQQfGPPHbEYnddFJONM+2UPT4MDGJzW4FQ2ibaDm5LxzMEVN38vjGIPA359cxo0oBZsTi7vGNaoz0MuBaF4rEvt8B3msxOL+B9xcjqAFMv7eni4ILK/54oBWK4zXtlGETfJrRaVrcXEmpQdl4aV/VTTuVVRp8ZB/OGqnSpqSiAdymzHnNbq8fbN80NWkWnz7y9XszhErkUnIK0rbcrsreRqhS7tGZvk+aB7psLxr4crHpwoC5s59G/tFNHPUFDxi8WgOExQordMgnshhUuLu37EIyX5mCzFSnqMxPelhh8e1y67lnet4qM2MKqCnycXVsUhXaQmg5beQY= X-Forefront-PRVS: 0031A0FFAF X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; AM3PR05MB0773; 23:C6E4MVq/1a3tqcXgytGcBvn5jCMT4dBYoHV0o?= =?Windows-1252?Q?4o9HOpWeJMdV4vLNbguYW+tBVGcAY6ZGtIFSESHxUMMdjgScksWZAbnV?= =?Windows-1252?Q?XbAgOOo/6hHq0zypS26kOmNvisgMSLWjSj1TAwqnvdjK2ZfNUvpIWXUb?= =?Windows-1252?Q?ar9cFf0AoNBZlN+uwxflSoSTkWB31gNZCQNHGlhPJkp+HYnbl3oQxX1D?= =?Windows-1252?Q?Xl8MOLYddZeUH2eevJY4TesitIEYFyrzOVKCveTtAqH/E1RIYve1G+z0?= =?Windows-1252?Q?av3V/NTB/Mtd+5rhNLGQd/ao/vHtNmAPMvQuounQPJJz9SnGkZBF7W8e?= =?Windows-1252?Q?Wo6F8Xyhc8NCzjzm1WUmIDGnVoUNLaEN8+IcwquIAo5WVv05cyfCaj8B?= =?Windows-1252?Q?zII5uggP2QMuaZO50MIUp0rq6bWOfQ2XwCVdWwOOgqprqlBJ/A+/Vpa/?= =?Windows-1252?Q?ChFWCK2VWMnR7/iAZDpguaHvXGa7EI+Mc4jZdqw1sWnqvJIkxlxoU9On?= =?Windows-1252?Q?2kYTR9d/dImi01pEsvnRFlARqI0qjWuSTQS8YU2aMkw+HJs0+oFvhHOp?= =?Windows-1252?Q?mtEF5djwQUL74A70Yf86nlvglMA5bz2N89TafAhiyD63ENFHFm66mYrN?= =?Windows-1252?Q?dFEtIZJ/2C8VdK2K5seJmreuJBiFkvV+88EjKXVZ7YTV/RacRJi45tQ4?= =?Windows-1252?Q?NFxR0E0o/Xxd0/Ndw0Cn1jCfwKnqb99qJK3W2hRBDyuVejPRa01jKZk7?= =?Windows-1252?Q?BnV84lvNv1wu7KN6Fe/9UxPPRUQG1SJ+wZMXLbzsthIeoZCx1+oIcEzV?= =?Windows-1252?Q?jfC7kSX7ujnyjDYvGZU0ssOFiVzjAsgT42tswi1xbkf1ZJ+x4YchX8FI?= =?Windows-1252?Q?yuvdrdUsnm1FseH7Kb+ClBiMtMHouRxiu8xdkofU/pm0U1YGYtOzt+eZ?= =?Windows-1252?Q?W0uy/ee89rCC3guc96WSQVuMZKJK1OTgg8cwPgRDLT7GM5DI+ciaewiW?= =?Windows-1252?Q?TKE9CmTY3Gk7FlLWT60DTX048TpkvWSGYH6ujoK+t0Dj4cxMWeOxWPWZ?= =?Windows-1252?Q?gjlhGrue+u2JbK1inb8yvzjEKE9U7dYVbtcPa5B6NwMgQ4/nbV5b70nf?= =?Windows-1252?Q?A7Xvn57vVU+QQ0vD/GoMYO6OtCx2NZN9KuhsltrGmTCVqpuJg5fOnZzR?= =?Windows-1252?Q?9vPrDTj/ht8SwmZv87HZ9n2JTn/S5vFT88tiAwJUv9fzaIahGP0bAQV/?= =?Windows-1252?Q?hDFTyDabtBUlNcG6lpxsQ85RuVDk0rYOLNFdkD14ZRBcw6uydxbPKMSC?= =?Windows-1252?Q?ADKc0h/ifd2ZL5hTstiV+lHzjXathyTyBW9qgfRD7CpPOFMJvLniB1He?= =?Windows-1252?Q?WM2lFNEF0h4euzruim4UoPB2L9UW/GkxJJooqF4YJruHDeMcWR4CjSzG?= =?Windows-1252?Q?aSwUmjgCmWIkWFrRrslL4aL+DpuZIVn24dwGqZFSg=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; AM3PR05MB0773; 6:RwVKa32vxJLQgJX5EYfgYwGtWfCFPJRE7Qs6/O3uwf2Hp6ElmW9YULjkZk1nRQ21rfuqNJSqL6GLWUzwojN3+24kWe1nHQeRGoTnCGgzavJtKwKCkA8uk4F20hdOpWBlq6g8kP+zQwBMnZMAsmqPH6TP7LdMfJK4kn+Nk0ErOcOMXbqRZpBFINRNvyQUoA6gZurqLC7+WzkM92+MJrUMJDpBEwvmS9eJpwNJQX3GGpDl3ub4pQGdu20RlmydAUTh6E5mqkTwiaeqjz8qH+tSM+tKSxUJQTEGYReQsjWNtgq+B6UpB3wIuPpTvTRfwFapenwSEVC6vDgAmjO0UKsb/Q==; 5:0+cpsWBJWua3KvU+m+jvz3rnZCP2Qe8LU9oUMsGCXhDHC02HfeuaEYjST2wgjDrOmjf3sXvKwQjc/nNB84BTJm5+sy6vWHa5XBNmD0GbADJmhoIggObd0IHODN6AdcNWDnUXzW3JB1lheWxts+S2vw==; 24:+jbY/l+79wjCAvSthjPWP327XADapyrexAR2ol42l+NNO2riaq9Jj1+Wx2m1HSxPBChNM8CKjMCfEZd+2bCacoOLIB/MWITb8DJ6I+JOs3Q=; 7:PnCCPCe/4nUO0I1o+EBk8PLIfzZWANKJx2hpgezpGhVp0qIb76hYXOc1I2X56Krz9anlx714FTIsvFXua/7BsuPGjTix2gFs+Rmm3Yn6qB99LKIWHOxekJ/aLbqyPRat5DrRss/YdwJtmuxhPvBYUY2gPD5GSeVwVAeeUad/7/2VwKhtCo2N3Qhmc81vDgOHQwnLwl4ygtX95MhyahGsxegLvBQ0nY5JX0qrkDTNe4jX6a+VQ8wx7ZdjjKBcFUVE SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2016 13:45:38.5408 (UTC) X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a652971c-7d2e-4d9b-a6a4-d149256f461b; Ip=[193.47.165.134]; Helo=[mtlcas13.mtl.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM3PR05MB0773 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 13:45:48 -0000 These are the instructions I get from iscsi maintainer that merged our code to FreeBSD-11: 1. set WITH_OFED=yes in /etc/make.conf 2. "make buildkernel buildworld installkernel installworld" 3. reboot 4. kldload iser After that, make sure you configured all the interfaces correctly and that you have ping between the 2 servers. later you need to check basic RDMA stuff on your servers (Added Hans/Oded/Meny to help you if needed). Only if the above are working you can proceed to check iser. Max. On 8/11/2016 4:22 PM, Ben RUBSON wrote: > I just tested to connect to a Linux iSER enabled target, but I still get this error message at initiator side : > iscsid[1187]: 192.168.2.2 (iqn.2012-06.com.test:target1): failed to connect to 192.168.2.2 using ICL kernel proxy: ISCSIDCONNECT: Invalid argument > >> as if my initiator did not understand the iSER command "ISCSIDCONNECT". >> How to be sure everything is correctly compiled with ICL_KERNEL_PROXY enabled ? > > Ben > > >> On 10 Aug 2016, at 22:51, Max Gurtovoy wrote: >> >> Hi Ben, >> >> iSER initiator is supported in FreeBSD-11 but not the iser target. >> for the target you should use Linux target such as TGT (user space), >> Lio or SCST. >> >> thanks, >> Max. >> >> >> On 8/10/2016 8:13 PM, Ben RUBSON wrote: >>> Hello, >>> >>> I have some difficulties to make iSER (FreeBSD 11 BETA3) working. >>> >>> >>> >>> ### My configuration : >>> >>> 2 servers with Mellanox ConnectX-3 NICs. >>> >>> >>> >>> ### What I did : >>> >>> I rebuilt kernel adding the following to GENERIC : >>> options OFED >>> device mlxen >>> options COMPAT_LINUXKPI >>> >>> I built iser : >>> cd /usr/src/sys/modules/iser/ ; make ; make install ; kldload iser >>> (I saw later-on that I could have used "device iser" in kernel configuration) >>> >>> My /etc/ctl.conf : >>> portal-group pg0 { >>> discovery-auth-group no-authentication >>> listen 192.168.2.2 >>> offload iser >>> } >>> target iqn.2012-06.com.test:target1 { >>> auth-group no-authentication >>> portal-group pg0 >>> alias my-iser-disk >>> lun 0 { >>> path /dev/da8 >>> } >>> } >>> >>> >>> >>> ### Error : >>> >>> As soon as I do the following : >>> iscsictl -A -r -t iqn.2012-06.com.test:target1 -p 192.168.2.2 >>> I get this error : >>> iscsid[1187]: 192.168.2.2 (iqn.2012-06.com.test:target1): failed to connect to 192.168.2.2 using ICL kernel proxy: ISCSIDCONNECT: Invalid argument >>> iscsid[655]: child process 1187 terminated with exit status 1 >>> >>> "sysctl hw.mlxen1.stat.tx_packets" on initiator shows that no packet is sent at all when trying to connect. >>> >>> Did I miss something ? >>> >>> Many thanks ! >>> >>> Ben >>> >>> _______________________________________________ >>> freebsd-scsi@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >>> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" >>> > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" >