From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 20 16:02:54 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 904B337B401; Fri, 20 Jun 2003 16:02:54 -0700 (PDT) Received: from bsdone.bsdwins.com (www.bsdwins.com [192.58.184.33]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A99443F85; Fri, 20 Jun 2003 16:02:53 -0700 (PDT) (envelope-from jwd@bsdwins.com) Received: from bsdone.bsdwins.com (localhost [127.0.0.1]) by bsdone.bsdwins.com (8.12.9/8.12.9) with ESMTP id h5KMugsH029758; Fri, 20 Jun 2003 18:56:43 -0400 (EDT) (envelope-from jwd@www.bsdwins.com) Received: (from jwd@localhost) by bsdone.bsdwins.com (8.12.9/8.12.9/Submit) id h5KMuW9L029748; Fri, 20 Jun 2003 18:56:32 -0400 (EDT) Date: Fri, 20 Jun 2003 18:56:32 -0400 From: John To: Terry Lambert Message-ID: <20030620225632.GA29485@BSDWins.Com> References: <200306200617.h5K6HaM7058935@gw.catspoiler.org> <3EF2CF89.3E5542F5@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3EF2CF89.3E5542F5@mindspring.com> User-Agent: Mutt/1.4.1i cc: freebsd-hackers@freebsd.org cc: Don Lewis cc: uitm@blackflag.ru Subject: Re: open() and ESTALE error X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2003 23:02:54 -0000 ----- Terry Lambert's Original Message ----- > Specifically, see the underline part of: > > > > + if (error == ESTALE && stale++ == 0) > --------------- > > ...he exits it after retrying it fails, and falls into the > standard ESTALE return case. > > If this gets committed (which I think it shouldn't because I > can see a genuinely bad handle getting converted to a good one > in a couple of cases), that line should probably be rewritten > to be more obvious (e.g. move the "stale++" before the "if" > statement and adjust the compare to compensate for the difference > so no one else reads it the way we did). hi folks, After looking at his original patch, I suggested modifying it for clarity to be of the form: error = vn_open(&nd, flags, cmode); if (error == ESTALE) error = vn_open(&nd, flags, cmode); /* single retry */ While I understand a number of you have reservations against this change, I think it worth serious consideration. Unless someone is willing to go into each of the individual fs layers and deal with ESTALE, this appears to be a relatively straight forward and easy to understand approach. Most of the main applications I run on clusters have all had their open routines recoded similar to the following (this from ftpd): int try = 0; while ((fin = fopen(name,"r")) == NULL && errno == ESTALE && try < 3 ) { if (logging > 1) syslog(LOG_INFO,"fopen(\"%s\"): %m: attempting retry",name); } if (fin == NULL && logging > 1) syslog(LOG_INFO,"get fopen(\"%s\"): %m",name); This is a real problem when using fbsd in high load / high throughput situations where highly sequenced operations are performed on a common set of data files from multiple machines. An example of this environment can be seen here: http://www.freebsd.org/~jwd/images/cluster.jpg If no one has any patches which can provide a better solution for handling ESTALE I would like to see Andreys' patch given a chance. Of course, if we don't want to do this, then I think it is high time we documented that open(2) can return ESTALE and provide a library routine that wraps open() with a retry :-) -John