From owner-freebsd-infiniband@FreeBSD.ORG Tue Feb 10 18:01:09 2015 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35C47218 for ; Tue, 10 Feb 2015 18:01:09 +0000 (UTC) Received: from proofpoint5.lanl.gov (proofpoint5.lanl.gov [IPv6:2001:400:4210:400::a5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E5823E8C for ; Tue, 10 Feb 2015 18:01:08 +0000 (UTC) Received: from mailrelay1.lanl.gov (mailrelay1.lanl.gov [128.165.4.101]) by mailgate5.lanl.gov (8.14.7/8.14.7) with ESMTP id t1AI171I009198; Tue, 10 Feb 2015 11:01:07 -0700 Received: from localhost (localhost.localdomain [127.0.0.1]) by mailrelay1.lanl.gov (Postfix) with ESMTP id C416713777A9; Tue, 10 Feb 2015 11:01:07 -0700 (MST) X-NIE-2-Virus-Scanner: amavisd-new at mailrelay1.lanl.gov Received: from tykhe.lanl.gov (tykhe.lanl.gov [128.165.248.149]) by mailrelay1.lanl.gov (Postfix) with ESMTP id B1BC113777A0; Tue, 10 Feb 2015 11:01:07 -0700 (MST) Message-ID: <54DA4763.6070203@lanl.gov> Date: Tue, 10 Feb 2015 11:01:07 -0700 From: Ron Croonenberg User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: John Nielsen , Ron Croonenberg Subject: Re: IBSRP and switches References: <54DA2F75.3000307@lanl.gov> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68, 1.0.33, 0.0.0000 definitions=2015-02-10_06:2015-02-10,2015-02-10,1970-01-01 signatures=0 Cc: freebsd-infiniband@freebsd.org X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Feb 2015 18:01:09 -0000 On 02/10/2015 10:39 AM, John Nielsen wrote: > On Feb 10, 2015, at 9:19 AM, Ron Croonenberg wrote: > >> From what I understand IBSRP is a point to point protocol and cannot be switched over an IB switch, correct? (and the presented devices cannot be see by initiators 'through' a switch. > > At $work we use SRP over InfiniBand with lots of switches in the mix. Never had an issue. > Ok, I'll try some of that. I assume you were able to see all drives on all ports. What I am trying to do is see all drives on all hosts (connected to a switch). IBSRP seems to be 'point to point' So I wonder if it (ibsrp) will let me mount a drive multiple times. (I am doing this in a 'not so posix' kind of way) >> Is there a 'scsi protocol' that I can run over IB that can be switched? > > Unfortunately, there isn't currently SRP support on FreeBSD. There is RDMA support, so it might be feasible to port the SRP code from Linux, but I haven't really looked in to it. That is why I am looking into it this way, because of RDMA. There is also something called ISER (iSCSI extensions over RDMA) > There's always iSCSI. FreeBSD 10.0 saw the addition of the kernel-based target and initiator (with some improvements since), but I don't think it takes advantage of RDMA or other acceleration techniques when used with InfiniBand. I'd love to be wrong. :) right, but I am trying to go for the highest performance that is reliable. > JN > Ron