Date: Wed, 3 Aug 2016 07:59:54 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= <gerrit.kuehn@aei.mpg.de> To: Alan Somers <asomers@freebsd.org> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: NFS on 10G interfaces still painfully slow Message-ID: <20160803075954.35914021394d6c3f9b3802f3@aei.mpg.de> In-Reply-To: <CAOtMX2hTGkm68rY7=hWDNG6YeuE-xd4%2BrV7%2BLoaywaPBo5yjdQ@mail.gmail.com> References: <20160802104929.a10602a4786c68b4547a45b9@aei.mpg.de> <CAOtMX2hTGkm68rY7=hWDNG6YeuE-xd4%2BrV7%2BLoaywaPBo5yjdQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 2 Aug 2016 08:45:07 -0600 Alan Somers <asomers@freebsd.org> wrote about Re: NFS on 10G interfaces still painfully slow: > > Is there anyone around here who can confirm that nfs can go faster > > over 10G links? > > Any hints for further tuning/debugging are greatly appreciated. AS> I can get 1GB/s over NFS on a 10G link, so it's not always slow. AS> There's probably something about your setup that's slowing it down. AS> What is your NFS client? This time, FreeBSD 10.3 on both client and server (to make debugging easier). AS> What benchmark are you using to measure that speed? Right now only very simple things like using dd from /dev/zero or copying large files. In my experience it is useless to go for more sophisticated benchmarks, if these simple things already don't work as expected. AS> Did you remember to start lockd and statd? Yes. AS> If you post your /etc/exports and the client's /etc/fstab, that might AS> reveal something. exports on the server side: V4: /mt-rear -sec=sys 192.168.1.11 /mt-right 192.168.1.11 -maproot=root /mt-rear -maproot=root 192.168.1.11 /mt-left 192.168.1.11 -maproot=root fstab on the client does not tell you anything, I still use commandline mounts during testing. This is what nfsstat -m will tell (V4 is not mounted right now): tom:/mt-rear on /net/mt-rear nfsv3,tcp,resvport,hard,cto,lockd,rdirplus,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=4,wcommitsize=50000000,timeout=120,retrans=2 tom:/mt-right on /net/mt-right nfsv3,tcp,resvport,hard,cto,lockd,rdirplus,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=1,wcommitsize=16777216,timeout=120,retrans=2 tom:/mt-left on /net/mt-left nfsv3,tcp,resvport,hard,cto,lockd,rdirplus,sec=sys,acdirmin=3,acdirmax=60,acregmin=5,acregmax=60,nametimeo=60,negnametimeo=60,rsize=65536,wsize=65536,readdirsize=65536,readahead=1,wcommitsize=16777216,timeout=120,retrans=2 This is what, e.g., dd gives: root@crest:~ # dd if=/dev/zero of=/net/mt-rear/Z bs=1024k count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 1.403620 secs (747051194 bytes/sec) root@crest:~ # dd if=/dev/zero of=/net/mt-right/Z bs=1024k count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 1.380546 secs (759537249 bytes/sec) And yes (before that question pops up :-), I'm using zfs on the server side, but I disabled syncing for testing purposes. cu Gerrit
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160803075954.35914021394d6c3f9b3802f3>