From owner-freebsd-scsi@FreeBSD.ORG Thu May 7 14:03:30 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C817C1D for ; Thu, 7 May 2015 14:03:30 +0000 (UTC) Received: from mail.tds.net (smtp-fo.tds.net [64.8.70.105]) by mx1.freebsd.org (Postfix) with ESMTP id 417471A52 for ; Thu, 7 May 2015 14:03:30 +0000 (UTC) X_CMAE_Category: , , X-CNFS-Analysis: v=2.1 cv=TdqCtHgh c=1 sm=1 tr=0 a=ZUodfxLL0dFkpt6uoqiZUQ==:117 a=ZUodfxLL0dFkpt6uoqiZUQ==:17 a=K-v-2zaBAAAA:8 a=vC3ST6aIoYIA:10 a=N659UExz7-8A:10 a=5Bwo4ec7AAAA:8 a=NEAV23lmAAAA:8 a=6I5d2MoRAAAA:8 a=-BM0Rm_KmpwObZHXfjcA:9 a=pILNOxqGKmIA:10 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: andiYWNvbkB0ZHMubmV0 Authentication-Results: smtp01.tds.cmh.synacor.com smtp.user=jwbacon@tds.net; auth=pass (LOGIN) Received: from [96.60.148.194] ([96.60.148.194:50413] helo=pro.local) by mail.tds.net (envelope-from ) (ecelerity 3.6.5.45644 r(Core:3.6.5.0)) with ESMTPSA (cipher=AES128-SHA) id 84/56-13657-9A07B455; Thu, 07 May 2015 10:03:22 -0400 Message-ID: <554B70A8.9050605@tds.net> Date: Thu, 07 May 2015 09:03:20 -0500 From: Jason Bacon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Sagi Grimberg , freebsd-scsi@freebsd.org, freebsd-infiniband@freebsd.org CC: Max Gurtovoy , Or Gerlitz , Oren Duer Subject: Re: FreeBSD iSER initiator References: <554B4CDC.3090900@dev.mellanox.co.il> In-Reply-To: <554B4CDC.3090900@dev.mellanox.co.il> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2015 14:03:30 -0000 Many thanks to Mellanox for supporting FreeBSD! Having the capabilities of FreeBSD available to Infiniband HPC clusters, etc. will be a great advantage for the entire industry... On 5/7/15 6:30 AM, Sagi Grimberg wrote: > Hello everyone, > > Mellanox Technologies is pleased to announce the availability of a new > iSCSI Extensions for RDMA (iSER) initiator mode support for FreeBSD. > > What is it ? > > iSER is a computer network protocol that extends the iSCSI storage > networking protocol to use Remote Direct Memory Access (RDMA). RDMA is > provided by either RoCE (RDMA over converged Ethernet), iWARP or > InfiniBand to enable efficient data movement using hardware offload > capabilities. > > Why doing this ? > > RDMA infrastructure provides benefits such as Zero-Copy, CPU offload, > Reliable transport, Fabric consolidation and many more. The iSER > protocol eliminates some of the bottlenecks in the traditional > iSCSI/TCP stack, provides low latency and high throughput and is well > suited for latency aware workloads. > > What was added ? > > The iSER transport was added as another icl provider module, and it > opens and RDMA channel to the target (icl_conn_connect). Other than > that the addition of the iser module is pretty straight-forward. > > Some minor additions and changes were added to the iscsi layer for > iser support: > - Not relying on io_received when handling a scsi response since > io_received is not maintained in iser (data is transferred via RDMA). > - Added a PDU pointer to task_setup so iser can reference the scsiio in > its private pdu context. > - Added a -T flag to user-space to indicate using iser. > - Added iSER relevant information to the login request. > > > The code is available at https://github.com/sagigrimberg/iser-freebsd > > The code maturity level is considered as alpha. It has gone through > limited testing on top of 11-current using Mellanox ConnectX-3 adapters > (Login, Logout, I/O). We tested against Linux LIO and TGT targets and > we have several known issues in error flows and session recovery areas > (Documented in release_notes.txt). > > We also did some initial performance measurements and we can see that > iser achieves x20 BW (~5.4 GB/s) and x2 IOPs (~100 KIOPs) compared to > iscsi/tcp for a single LUN device. > > We would like to get some feedback on the code in general, and > specifically on some of the framework changes that we did in the iscsi > common layers (and user-space). Moreover, feel free to try out the > code, report BUGs (to Max or me at this point) and even provide fixes. > > Cheers, > > Max Gurtovoy & Sagi Grimberg > > Mellanox Technologies > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to > "freebsd-infiniband-unsubscribe@freebsd.org" -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Jason W. Bacon jwbacon@tds.net If a problem can be solved, there's no need to worry. If it cannot be solved, then worrying will do no good. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~