Date: Fri, 20 Aug 1999 22:44:35 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Terry Lambert <tlambert@primenet.com> Cc: fullermd@futuresouth.com, steve@cioe.com, freebsd-hackers@FreeBSD.ORG Subject: Re: Async NFS exports? Message-ID: <199908210544.WAA69698@apollo.backplane.com> References: <199908202252.PAA17363@usr09.primenet.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:> The reason is due to the way NFSv3 issues writes. NFSv3 issues a
:> write but no longer assumes that the write has been synced to the
:> server's disk as of when the reply comes back. Instead it keeps the
:..
:
:If you are looking for more optimizations, you can delay NFS write
:operations for some interval X, on all write buffers, entirely.
:
:Sun does this (it is technically called "write gathering" in the
:literature).
The client-side pushes a buffer out the moment it fills up. Since
this corresponds to a maximally-sized NFS data packet the client will
not be more efficient by delaying it.
The server-side in the FreeBSD implementation *DOES* do write-clustering,
as I demonstrate below. The server does a better job of write-clustering
when the client is a FreeBSD-CURRENT box, though, because FreeBSD-CURRENT
clients generate a more uniform data flow.
The server is able to do write-clustering w/ NFSv3 mounts because data
writes are not required to be synched to disk until the client requests
a commit.
Note that KB/t fields below.
CURRENT server, CURRENT client. iostat on server while client writes
apollo:/usr/src/sys/nfs# iostat da0 1
tty da0 cpu
tin tout KB/t tps MB/s us ni sy in id
0 20 0.00 0 0.00 1 0 1 0 98
0 43 0.00 0 0.00 0 0 1 0 99
0 43 1.00 1 0.00 0 0 0 0100
0 42 6.86 7 0.05 0 0 10 0 90
0 43 61.80 68 4.12 1 0 25 21 53
0 43 64.00 72 4.52 0 0 18 28 54
0 43 64.00 73 4.58 0 0 21 25 54
0 42 62.29 74 4.51 0 0 24 32 45
0 43 63.23 72 4.46 0 0 22 23 54
0 43 64.00 32 1.98 0 0 9 9 82
CURRENT server, STABLE client. iostat on server while client writes
apollo:/usr/src/sys/nfs# iostat da0 1
tty da0 cpu
tin tout KB/t tps MB/s us ni sy in id
0 20 0.00 0 0.00 1 0 1 0 98
0 43 43.43 98 4.15 0 0 15 21 64
0 42 17.48 107 1.82 0 0 9 7 84
0 43 62.49 73 4.47 0 0 19 33 48
0 43 19.93 121 2.35 0 0 8 9 84
0 43 43.07 85 3.58 0 0 17 22 61
0 42 46.77 90 4.11 0 0 21 22 57
0 43 15.63 108 1.65 0 0 8 7 85
0 43 64.00 70 4.39 2 0 17 27 54
However, even a FreeBSD-CURRENT breaks down when its buffer cache
saturates. The example below demonstrates this:
CURRENT server, CURRENT client, iostat of server disks while client writes
(client buffer cache not saturated)
0 43 0.00 0 0.00 0 0 0 0100
0 43 56.00 26 1.41 0 0 14 5 81
0 42 64.00 72 4.52 0 0 21 26 53
0 43 64.00 73 4.58 0 0 25 19 57
11 54 64.00 71 4.46 2 0 17 28 53
0 42 62.27 73 4.45 0 0 26 22 52
5 48 64.00 72 4.52 0 0 22 24 53
5 48 52.72 82 4.23 0 0 20 26 53
(client buffer cache saturates)
11 54 18.76 163 2.99 2 0 12 16 71
1 43 19.20 129 2.41 1 0 13 11 75
7 70 20.65 146 2.95 2 0 14 16 67
4 47 17.53 130 2.22 0 0 9 15 77
6 51 24.63 113 2.72 1 0 10 13 76
4 73 15.17 153 2.27 0 0 11 7 82
The problem that occurs on the FreeBSD server is simply that the
nfsrv_commit() procedure calls fsync() on the file... on the *ENTIRE*
file, for every commit rpc, rather then syncing just the offset/range
requested. I am looking into ways to fix this.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199908210544.WAA69698>