Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 15 Oct 2012 16:58:16 -0400 (EDT)
From:      Rick Macklem <rmacklem@uoguelph.ca>
To:        Ivan Voras <ivoras@freebsd.org>
Cc:        freebsd-hackers@freebsd.org
Subject:   Re: NFS server bottlenecks
Message-ID:  <1516511249.2287339.1350334696127.JavaMail.root@erie.cs.uoguelph.ca>
In-Reply-To: <k5gtdh$nc0$1@ger.gmane.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Ivan Voras wrote:
> On 13/10/2012 17:22, Nikolay Denev wrote:
> 
> > drc3.patch applied and build cleanly and shows nice improvement!
> >
> > I've done a quick benchmark using iozone over the NFS mount from the
> > Linux host.
> >
> 
> Hi,
> 
> If you are already testing, could you please also test this patch:
> 
> http://people.freebsd.org/~ivoras/diffs/nfscache_lock.patch
> 
I don't think (it is hard to test this) your trim cache algorithm
will choose the correct entries to delete.

The problem is that UDP entries very seldom time out (unless the
NFS server isn't seeing hardly any load) and are mostly trimmed
because the size exceeds the highwater mark.

With your code, it will clear out all of the entries in the first
hash buckets that aren't currently busy, until the total count
drops below the high water mark. (If you monitor a busy server
with "nfsstat -e -s", you'll see the cache never goes below the
high water mark, which is 500 by default.) This would delete
entries of fairly recent requests.

If you are going to replace the global LRU list with ones for
each hash bucket, then you'll have to compare the time stamps
on the least recently used entries of all the hash buckets and
then delete those. If you keep the timestamp of the least recent
one for that hash bucket in the hash bucket head, you could at least
use that to select which bucket to delete from next, but you'll still
need to:
  - lock that hash bucket
    - delete a few entries from that bucket's lru list
  - unlock hash bucket
- repeat for various buckets until the count is beloew the high
  water mark
Or something like that. I think you'll find it a lot more work that
one LRU list and one mutex. Remember that mutex isn't held for long.

Btw, the code looks very nice. (If I was being a style(9) zealot,
I'd remind you that it likes "return (X);" and not "return X;".

rick

> It should apply to HEAD without Rick's patches.
> 
> It's a bit different approach than Rick's, breaking down locks even
> more.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1516511249.2287339.1350334696127.JavaMail.root>