Date: Thu, 07 May 2015 14:30:36 +0300 From: Sagi Grimberg <sagig@dev.mellanox.co.il> To: freebsd-scsi@freebsd.org, freebsd-infiniband@freebsd.org Cc: Max Gurtovoy <maxg@mellanox.com>, Oren Duer <oren@mellanox.com>, Hans Petter Selasky <hps@selasky.org>, Or Gerlitz <ogerlitz@mellanox.com>, Oded Shanoon <odeds@mellanox.com> Subject: FreeBSD iSER initiator Message-ID: <554B4CDC.3090900@dev.mellanox.co.il>
next in thread | raw e-mail | index | archive | help
Hello everyone, Mellanox Technologies is pleased to announce the availability of a new iSCSI Extensions for RDMA (iSER) initiator mode support for FreeBSD. What is it ? iSER is a computer network protocol that extends the iSCSI storage networking protocol to use Remote Direct Memory Access (RDMA). RDMA is provided by either RoCE (RDMA over converged Ethernet), iWARP or InfiniBand to enable efficient data movement using hardware offload capabilities. Why doing this ? RDMA infrastructure provides benefits such as Zero-Copy, CPU offload, Reliable transport, Fabric consolidation and many more. The iSER protocol eliminates some of the bottlenecks in the traditional iSCSI/TCP stack, provides low latency and high throughput and is well suited for latency aware workloads. What was added ? The iSER transport was added as another icl provider module, and it opens and RDMA channel to the target (icl_conn_connect). Other than that the addition of the iser module is pretty straight-forward. Some minor additions and changes were added to the iscsi layer for iser support: - Not relying on io_received when handling a scsi response since io_received is not maintained in iser (data is transferred via RDMA). - Added a PDU pointer to task_setup so iser can reference the scsiio in its private pdu context. - Added a -T <transport> flag to user-space to indicate using iser. - Added iSER relevant information to the login request. The code is available at https://github.com/sagigrimberg/iser-freebsd The code maturity level is considered as alpha. It has gone through limited testing on top of 11-current using Mellanox ConnectX-3 adapters (Login, Logout, I/O). We tested against Linux LIO and TGT targets and we have several known issues in error flows and session recovery areas (Documented in release_notes.txt). We also did some initial performance measurements and we can see that iser achieves x20 BW (~5.4 GB/s) and x2 IOPs (~100 KIOPs) compared to iscsi/tcp for a single LUN device. We would like to get some feedback on the code in general, and specifically on some of the framework changes that we did in the iscsi common layers (and user-space). Moreover, feel free to try out the code, report BUGs (to Max or me at this point) and even provide fixes. Cheers, Max Gurtovoy & Sagi Grimberg Mellanox Technologies
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?554B4CDC.3090900>