From owner-freebsd-net  Fri Apr 26 10:15:44 2002
Delivered-To: freebsd-net@freebsd.org
Received: from mk-smarthost-1.mail.uk.tiscali.com (mk-smarthost-1.mail.uk.tiscali.com [212.74.112.71])
	by hub.freebsd.org (Postfix) with ESMTP
	id 7D69037B404; Fri, 26 Apr 2002 10:15:38 -0700 (PDT)
Received: from [212.139.129.126] (helo=bloodhound.uk.worldonline.com)
	by mk-smarthost-1.mail.uk.tiscali.com with esmtp (Exim 3.35 #1)
	id 1719JS-0000bE-00; Fri, 26 Apr 2002 18:15:14 +0100
Received: from brian by bloodhound.uk.worldonline.com with local (Exim 3.22 #1)
	id 1719Jn-0000jD-00; Fri, 26 Apr 2002 18:15:35 +0100
Date: Fri, 26 Apr 2002 18:15:35 +0100
From: Brian Candler <B.Candler@pobox.com>
To: freebsd-fs@freebsd.org, freebsd-net@freebsd.org
Subject: NFS clearing attribute cache in nfs_open
Message-ID: <20020426181535.B2748@linnet.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
Sender: owner-freebsd-net@FreeBSD.ORG
Precedence: bulk
List-ID: <freebsd-net.FreeBSD.ORG>
List-Archive: <http://docs.freebsd.org/mail/> (Web Archive)
List-Help: <mailto:majordomo@FreeBSD.ORG?subject=help> (List Instructions)
List-Subscribe: <mailto:majordomo@FreeBSD.ORG?subject=subscribe%20freebsd-net>
List-Unsubscribe: <mailto:majordomo@FreeBSD.ORG?subject=unsubscribe%20freebsd-net>
X-Loop: FreeBSD.org

I have been tracing some performance problems on diskless systems. In
particular, the following test program:

   perl -e 'for ($i=0;$i<1000;$i++) { open F,"</bin/rm"; $x=<F>; close F;}'

generates 1000 'access' transactions, and hence an exchange of 2000 UDP
packets, just from repeatedly opening the same file.

After some digging around, I have found the cause: the NFS attribute cache
for a file is explicitly invalidated every time the file is opened.

[sys/nfs/nfs_vnops.c]

static int
nfs_open(ap)
...
        if ((nmp->nm_flag & NFSMNT_NQNFS) == 0)
                np->n_attrstamp = 0; /* For Open/Close consistency */
        return (0);
}

For test purposes I changed this to

        if ((nmp->nm_flag & NFSMNT_NQNFS) == 0 && (np->n_flag & NMODIFIED))
                np->n_attrstamp = 0; /* For Open/Close consistency */
        return (0);

and suddenly the problem went away - attributes were being cached happily,
and NFS traffic dropped to virtually zero.

Now, there is obviously some "consistency" issue to beware of. Are there any
NFS gurus out there who can say why it is necessary to clear the attribute
cache on _every_ open? Could it safely be made less restrictive, e.g. don't
clear the cache when opening a file for read? I think there are some
potentially large performance gains for diskless server clusters, where the
same file is being repeatedly exec()'d.

I see that the code in nfs_close doesn't invalidate the cache unless the
file has been modified. However that doesn't help much if the cache is going
to get invalidated anyway the next time the file is opened :-)

Many thanks,

Brian Candler.

P.S. I am working to FreeBSD-4.5-STABLE-20020426

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message