Date: Thu, 25 Jun 2015 20:49:11 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Gerrit =?utf-8?B?S8O8aG4=?= <gerrit.kuehn@aei.mpg.de> Cc: freebsd-net@freebsd.org, carsten aulbert <carsten.aulbert@aei.mpg.de> Subject: Re: NFS on 10G interface terribly slow Message-ID: <623856025.328424.1435279751389.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20150625145238.12cf9da3b368ef0b9a30f193@aei.mpg.de> References: <20150625145238.12cf9da3b368ef0b9a30f193@aei.mpg.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Gerrit Kuhn wrote: > Hi all, > > We have a recent FreeBSD 10.1 installation here that is supposed to act as > nfs (v3) client to an Oracle x4-2l server running Soalris 11.2. > We have Intel 10-Gigabit X540-AT2 NICs on both ends, iperf is showing > plenty of bandwidth (9.xGB/s) in both directions. > However, nfs appears to be terribly slow, especially for writing: > > root@crest:~ # dd if=/dev/zero of=/net/hellpool/Z bs=1024k count=1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes transferred in 20.263190 secs (51747824 bytes/sec) > Recent commits to stable/10 (not in 10.1) done by Alexander Motin (mav@) might help w.r.t. write performance (it avoids large writes doing synchronous writes when the wcommitsize is exceeded). If you can try stable/10, that might be worth it. Otherwise, the main mount option you can try is "wcommitsize", which you probably want to make larger. (It sounds like you already tried most of what I could suggest.) > > Reading appears to be faster, but still far away from full bandwidth: > > root@crest:~ # dd of=/dev/null if=/net/hellpool/Z bs=1024k > 1000+0 records in > 1000+0 records out > 1048576000 bytes transferred in 5.129869 secs (204406000 bytes/sec) > You could try increasing readahead. Look for the mount option and try cranking it up to 8 or 16. Good luck with it, rick > > We have already tried to tune rsize/wsize parameters, but they appear to > have little (if any) impact on these results. Also, neither stripping down > rxsum, txsum, tso etc. from the interface nor increasing MTU to 9000 for > jumbo frames did improve anything. > It is quite embarrassing to achieve way less than 1GBE performance with > 10GBE equipment. Are there any hints what else might be causing this (and > how to fix it)? > > > cu > Gerrit > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?623856025.328424.1435279751389.JavaMail.zimbra>