From owner-freebsd-fs@freebsd.org Tue Aug 22 09:26:42 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECD0DDDC05C for ; Tue, 22 Aug 2017 09:26:42 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B1FF57DBAB for ; Tue, 22 Aug 2017 09:26:42 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1dk5Cf-0000Ag-NO; Tue, 22 Aug 2017 11:10:30 +0200 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "freebsd-fs@freebsd.org" , "Rick Macklem" Subject: Re: when has a pNFS data server failed? References: Date: Tue, 22 Aug 2017 11:10:29 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 autolearn=disabled version=3.4.0 X-Scan-Signature: 1f72ff50073f138f9668c095d6f579a1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 09:26:43 -0000 On Fri, 18 Aug 2017 23:52:12 +0200, Rick Macklem wrote: > This is kind of a "big picture" question that I thought I 'd throw out. > > As a brief background, I now have the code for running mirrored pNFS > Data Servers > working for normal operation. You can look at: > http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt > if you are interested in details related to the pNFS server code/testing. > > So, now I am facing the interesting part: > 1 - The Metadata Server (MDS) needs to decide that a mirrored DS has > failed at some > point. Once that happens, it stops using the DS, etc. > --> This brings me to the question of "when should the MDS decide that > the DS has > failed and should be taken offline?". > - I'm not up to date w.r.t. the TCP stack, so I'm not sure how > long it will take for the > TCP connection to decide that a DS server is no longer working > and fail the TCP > connection. I think it takes a fair amount of time, so I'm not > sure if TCP connection > loss is a good indicator of DS server failure or not? > - It seems to me that the MDS should wait a fairly long time before > failing the DS, > since this will have a major impact on the pNFS server, requiring > repair/resilvering > by a sysadmin once it happens. > So, any comments or thoughts on this? rick Hi, This is a quite common problem for all clustered/connected systems. I think there is no general answer. And there are a lot of papers written about it. For example: in NFS you have the 'soft' option. It is recommended not to use it. I can imagine that if your home-dir or /usr is mounted over NFS, but at work I want my http-servers to not hang and just give an IO-error when the backend fileserver with data is gone. Something similar happens here. Doesn't the protocol definition say something about this? Or what do other implemenations do? Regards, Ronald.