From owner-freebsd-stable@FreeBSD.ORG Mon Jul 2 18:14:13 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71CA1106566B; Mon, 2 Jul 2012 18:14:13 +0000 (UTC) (envelope-from phoemix@harmless.hu) Received: from marvin.harmless.hu (marvin.harmless.hu [195.56.55.204]) by mx1.freebsd.org (Postfix) with ESMTP id 2D6BD8FC14; Mon, 2 Jul 2012 18:14:12 +0000 (UTC) Received: from 94-21-194-68.pool.digikabel.hu ([94.21.194.68] helo=unknown) by marvin.harmless.hu with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.75 (FreeBSD)) (envelope-from ) id 1Sll8V-000PCS-VH; Mon, 02 Jul 2012 20:14:12 +0200 Date: Mon, 2 Jul 2012 20:14:10 +0200 From: Gergely CZUCZY To: John Baldwin Message-ID: <20120702201410.00005b38@unknown> In-Reply-To: <201207021352.26014.jhb@freebsd.org> References: <201207021055.04326.jhb@freebsd.org> <20120702192329.000074c0@unknown> <201207021352.26014.jhb@freebsd.org> Organization: Harmless Digital X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: Why Are You NOT Using FreeBSD ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 18:14:13 -0000 On Mon, 2 Jul 2012 13:52:25 -0400 John Baldwin wrote: > On Monday, July 02, 2012 1:23:29 pm Gergely CZUCZY wrote: > > On Mon, 2 Jul 2012 10:55:04 -0400 > > John Baldwin wrote: > > > > > On Friday, June 29, 2012 5:50:05 am Gergely CZUCZY wrote: > > > > Hello, > > > > > > > > I'm mostly using freebsd, but there are a few cases where it's > > > > impossible to do, and because of these, i'm not using fbsd > > > > there. > > > > > > > > These reasons are mostly are: > > > > - Lack of a working infiniband/OFED stack, with all its utils, > > > > mellanox connectX3 drivers, RDMA, iscsi-over-RDMA, > > > > nfs-over-RDMA, and such things. > > > > > > FYI, OFED is present in 9.0 and later including mellanox drivers. > > > RDMA should be present, but I've no idea about iscsi-over-RDMA or > > > nfs-over-RDMA. > > > > Agreed, it's present. However, it's not in a functional state. Half > > of the toolkit is missing, and we couldn't get out of the system any > > better than icmp responses. any other means of communication failed > > to work. Yup, present, but not functional :) > > Hmm, in my local testing we've been able to use the 40G mlxen(4) > adapters fine with the OFED stack. I believe we have also done a bit > more involved testing on the IB side than just ping as well (at least > RX and TX of UDP packets). > Well, it didn't work for us. We have the connext3 cards. And our goal was making iscsi-over-RDMA and NFS-over-RDMA work. Though at the end we've settled with linux and not using RDMA, because that's kinda messy. When we've tried to do some testin with netcat, no packets were transmitted really. They've got somewhere lost, there had been some error message in the syslog, which i can't recall now. We felt like the packets are getting lost somewhere between the OFED and the IP stack.