Date: Wed, 02 Jul 2014 16:50:56 -0700 From: "Russell L. Carter" <rcarter@pinyon.org> To: freebsd-net@freebsd.org Subject: NFS client READ performance on -current Message-ID: <53B49AE0.4030902@pinyon.org>
next in thread | raw e-mail | index | archive | help
Greetings! It's been 14 years. OMG do I love poudriere and zfs. But apropos of this post from last January: http://lists.freebsd.org/pipermail/freebsd-net/2014-January/037547.html I am going to capitalize READ to emphasize that all I am looking at here is reading on the client. In some of posts I have read people have been confusing read vs. write performance and losing the thread. I have fresh current server and client, each with otherwise quiescent em NICs. I also have debian kernel 3.14-1 server and client. The freebsd hardware is large enough, 16GB RAM + vishera 8 cores. (The debian systems are much smaller/slower) Upshot is that with a -current NFSV4 (4.1) serving a -current client and a linux client, I see today client READ performance of: -current -> -current: 2.7MB/s READ nice and steady -current -> linux: 80MB/s READ very bursty And for the weird (to me) result: linux -> -current: 55.74MB/s READ So to be very clear, the freebsd -current server seems capable of of supporting excellent client READ bandwidth. But something is dramatically awry with the freebsd client. (scp, rsync over ssh, all are near wire speed when I get rid of disk overhead). The -current server has got an 11TB raidz2 behind it but I'm only READing, not writing, so all of the discussion about ZFS + NFS sync write problems seems immaterial, possibly. Here's the mount info: -current client's output from nfsstat -m (terp is current server, ari is linux server): terp:/raid/library on /mnt/terp/library nfsv4,minorversion=1,tcp,resvport,hard,cto,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=1,wcommitsize=2798255,timeout=120,retrans=2147483647 ari:/d1/library on /mnt/ari nfsv4,minorversion=0,tcp,resvport,hard,cto,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=1,wcommitsize=2798255,timeout=120,retrans=2147483647 Linux client's excerpt from mount output: terp:/raid/library on /mnt/terp/library type nfs4 (rw,relatime,vers=4.0,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.0.10.3,local_lock=none,addr=10.0.10.4) BTW, linux -> linux is about 80MB/s I tried nfsv3 -current -> -current and still got 2.7MB/s READ. I have spent about 5 hours strolling through the archives of the last several years of discussion about this issue in various forums, but I didn't see a reference once since Jaunary's thread on freebsd-net, and I didn't find a bug entry, so I thought I'd keep it alive, so to speak. I've tried all of the various sysctl tweaks that have been suggested over time, that would make sense for reads, to no effect. Russell
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53B49AE0.4030902>