Date: Mon, 16 Oct 2006 11:32:36 -0400 (EDT) From: rick@snowhite.cis.uoguelph.ca To: fs@freebsd.org Cc: mohan_srinivasan@yahoo.com Subject: Re: lost dotdot caching pessimizes nfs especially Message-ID: <200610161532.LAA59652@snowhite.cis.uoguelph.ca>
next in thread | raw e-mail | index | archive | help
> SunOS and Solaris have provided close-to-open consistency for a very long time (for > at least 15 years now). I believe you (and I understand the principal "if Solaris does X, that's the way it needs to be"). Until very recently, Solaris sources weren't open and I didn't have access to them. That was important in the "bad old days", since Sun's Legal Beagles did once send a threatening letter w.r.t. my NFS violating their proprietary... I'd argue that these days it's "if Linux does X, that's the way it needs to be".:-) > Not having the NFS client enforce close-to-open consistency will break a heck of a lot > of applications. Since the other NFS clients (that matter) Solaris and Linux support it, > I would argue that not supporting cto consistency is not really an option. I am a bit surprised that a lot of applications break. To do so, they must be running on multiple clients, read/write sharing the same NFS mounted file(s) and use some back-end protocol that says something like "I've just closed it, so you can now open it" so inconsistencies < 1 minute, causes problems. I wish those applications had been common a decade ago becuase the NFS community might have cared about cache consistency and something along the lines of my experimental NQNFS might have happenned. NFSv4 simply says that clients that care about cache consistency should use byte range locking. Here's what RFC3530 (the NFSv4 RFC) says: at top of Page 14: If an application wants to serialize access to file data, file locking of the file data ranges in question should be used. Admittedly the paragraph that preceeds this almost says what Solaris is doing and seems to contradict Sec 9: 9. Client-Side Caching Client-side caching of data, of file attributes, and of file names is essential to providing good performance with the NFS protocol. Providing distributed cache coherence is a difficult problem and previous versions of the NFS protocol have not attempted it. Instead, several NFS client implementation techniques have been used to reduce the problems that a lack of coherence poses for users. These techniques have not been clearly defined by earlier protocol specifications and it is often unclear what is valid or invalid client behavior. The NFS version 4 protocol uses many techniques similar to those that have been used in previous protocol versions. The NFS version 4 protocol does not provide distributed cache coherence. However, it defines a more limited set of caching guarantees to allow locks and share reservations to be used without destructive interference from client side caching. In addition, the NFS version 4 protocol introduces a delegation mechanism which allows many decisions normally made by the server to be made locally by clients. This mechanism provides efficient support of the common cases where sharing is infrequent or where sharing is read-only. Once clients figure out how to effectively use Delegations, I think there will be significant performance improvements. Unfortunately, we haven't gotten there yet. For NFSv2/3, maintaining close-to-open consistency may be appropriate (and necessary), but it can result in a big performance hit. For example, try an experiment where you turn off "push writes on close" in the code and see what effect that has on performance for the common, non-read/write shared files case. (nb: I don't think you can get away with doing this without some other cache consistency guarantees, but it would sure be nice from a performance point of view. That was what nqnfs was all about. If you are really bored, you can go to http://www.cis.uoguelph.ca/~nfsv4 and read my ancient nqnfs paper.) rick
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200610161532.LAA59652>