From owner-freebsd-infiniband@FreeBSD.ORG Thu May 15 16:50:13 2014 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E404CB58 for ; Thu, 15 May 2014 16:50:13 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 8BF8E2ACD for ; Thu, 15 May 2014 16:50:12 +0000 (UTC) Received: from study64.tdx.co.uk (study64.tdx.co.uk [62.13.130.231]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s4FGo3mp059805 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 15 May 2014 17:50:04 +0100 (BST) Date: Thu, 15 May 2014 17:50:03 +0100 From: Karl Pielorz To: "Janky Jay, III" , freebsd-infiniband@freebsd.org Subject: Re: FBSD to FBSD NFS Mounts over IB. Message-ID: <55BC554716A7EA5C54F5DD02@study64.tdx.co.uk> In-Reply-To: <5374D431.5020501@unfs.us> References: <5374D431.5020501@unfs.us> X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 16:50:14 -0000 --On 15 May 2014 08:50:25 -0600 "Janky Jay, III" wrote: > I have set up one of the FBSD systems to run OpenSM and also be an NFS > server which all the systems seem to be able to mount over the IB devices > without any issue at all. Small reads and writes to and from the NFS > server to all the other nodes also seems to work without any issue. > However, if I try to dump large amounts of data using "dd" (in order to > test speeds and stability), the FBSD NFS client craps out immediately. I > just get the following message(s) over and over: > > newnfs server 10.11.1.1:/data: not responding > newnfs server 10.11.1.1:/data: not responding Can both sides 'ping' each other when this happens? The reason I ask is I've hit a similar issue setting up ZFS over iSCSI on IPOIB (I'm not running connected mode). At my end it looks like an ARP expires or something so the sides 'lose site' of each other. If 'A' can't see 'B' - a ping from 'B' to 'A' usually restores the connection. Maybe make sure they can both still see each other outside of nfs - I can temporarily 'fix' the issue here by leaving both sides pinging each other - I've not really had a chance to look at it much recently... -Karl