Date: Sat, 13 Dec 1997 10:23:42 +0000 (GMT) From: Doug Rabson <dfr@nlsystems.com> To: Bill Paul <wpaul@skynet.ctr.columbia.edu> Cc: current@freebsd.org, toor@dyson.iquest.net Subject: Re: mmap() + NFS problems persist Message-ID: <Pine.BSF.3.95q.971213101402.514A-100000@herring.nlsystems.com> In-Reply-To: <199712122239.RAA08542@skynet.ctr.columbia.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 12 Dec 1997, Bill Paul wrote:
> Yes, I'm still here.
>
> I'm still seeing problems with FreeBSD-current, mmap() and NFS. I've
> upgraded to a 3.0 SNAP from Dec, 9th and the trouble is still there.
> Again, there are two possible failure modes: in the first case, the
> process becomes wedged and unkillable with ps -alx showing wait channel
> to be "vmopar", and in the second case, the whole system wedges because
> nfs_bioread() gets caught in an endless loop.
>
> I've been trying to investigate the latter problem since it's more of
> a show-stopper, but my RPC clue isn't enough to help me understand the
> inner workings of the VM system, which I think is partly where the
> problem lies (inasmuch as it relates to NFS anyway).
I think I understand what might be happening. I can't easily check since
my FreeBSD hacking box is at work though. What I think happens is that
when brelse is called in this code fragment,
> if (not_readin && n > 0) {
> if (on < bp->b_validoff || (on + n) >
> bp->b_validend) {
> bp->b_flags |= B_NOCACHE;
> bp->b_flags |= B_INVAFTERWRITE;
> if (bp->b_dirtyend > 0) {
> if ((bp->b_flags & B_DELWRI) == 0)
> panic("nfsbioread");
> if (VOP_BWRITE(bp) == EINTR)
> return (EINTR);
> } else
> brelse(bp);
> goto again; <----- LOOPS HERE!!
the 8k buffer has exactly one VM page associated with it. The NFS code is
attempting to throw the buffer away since it is only partially valid and
it wants to read from the invalid section of the buf. It does this by
setting the B_NOCACHE flag before calling brelse. Unfortunately the
underlying VM page is still valid, so when getblk is called, the code in
allocbuf which tracks down the underlying VM pages carefully resets
b_validoff and b_validend causing the loop.
Basically, the VMIO system has managed to ignore the NFS code's request
for a cache flush, which the NFS code relied on to break the loop in
nfs_bioread. As I see it, the problem can be fixed in two ways. The
first would be for brelse() on a B_NOCACHE buffer to invalidate the VM
pages in the buffer, restoring the old behaviour which NFS expected and
the second would be to rewrite that section of the NFS client to cope
differently with partially valid buffers.
--
Doug Rabson Mail: dfr@nlsystems.com
Nonlinear Systems Ltd. Phone: +44 181 951 1891
Fax: +44 181 381 1039
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.95q.971213101402.514A-100000>