Date: Mon, 28 Jun 2010 10:18:35 -0500 From: "Rick C. Petty" <rick-freebsd2009@kiwi-computer.com> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: Rick Macklem <rmacklem@uoguelph.ca>, freebsd-stable@freebsd.org Subject: Re: Why is NFSv4 so slow? Message-ID: <20100628151835.GA53315@kay.kiwi-computer.com> In-Reply-To: <20100628145600.GA92144@icarus.home.lan> References: <20100627221607.GA31646@kay.kiwi-computer.com> <Pine.GSO.4.63.1006271949220.3233@muncher.cs.uoguelph.ca> <20100628031401.GA45282@kay.kiwi-computer.com> <20100628034741.GA45748@kay.kiwi-computer.com> <20100628045852.GA75380@icarus.home.lan> <20100628142025.GB52174@kay.kiwi-computer.com> <20100628145600.GA92144@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jun 28, 2010 at 07:56:00AM -0700, Jeremy Chadwick wrote: > > Three other things to provide output from if you could (you can X out IPs > and MACs too), from both client and server: > > 6) netstat -idn server: Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop nfe0 1500 <Link#1> 00:22:15:b4:2d:XX 1767890778 0 0 872169302 0 0 0 nfe0 1500 172.XX.XX.0/2 172.XX.XX.4 1767882158 - - 1964274616 - - - lo0 16384 <Link#2> 3728 0 0 3728 0 0 0 lo0 16384 (28)00:00:00:00:00:00:fe:80:00:02:00:00:00:00:00:00:00:00:00:00:00:01 3728 0 0 3728 0 0 0 lo0 16384 (28)00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:01 3728 0 0 3728 0 0 0 lo0 16384 127.0.0.0/8 127.0.0.1 3648 - - 3664 - - - client: Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop re0 1500 <Link#1> e0:cb:4e:cd:d3:XX 955288523 0 0 696819089 0 0 0 re0 1500 172.XX.XX.0/2 172.XX.XX.2 955279721 - - 696814499 - - - lo0 16384 <Link#2> 3148 0 0 3148 0 0 0 lo0 16384 (28)00:00:00:00:00:00:fe:80:00:02:00:00:00:00:00:00:00:00:00:00:00:01 3148 0 0 3148 0 0 0 lo0 16384 (28)00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:01 3148 0 0 3148 0 0 0 lo0 16384 127.0.0.0/8 127.0.0.1 3112 - - 3112 - - - > 7) sysctl hw.pci | grep msi both server and client: hw.pci.honor_msi_blacklist: 1 hw.pci.enable_msix: 1 hw.pci.enable_msi: 1 > 8) Contents of /etc/sysctl.conf server and client: # 4 virtual channels dev.pcm.0.play.vchans=4 # Read modules from /usr/local/modules kern.module_path=/boot/kernel;/boot/modules;/usr/local/modules # Remove those annoying ARP moved messages: net.link.ether.inet.log_arp_movements=0 # 32MB write cache on disk controllers system-wide vfs.hirunningspace=33554432 # Allow users to mount file systems vfs.usermount=1 # misc net.link.tap.user_open=1 net.inet.ip.forwarding=1 compat.linux.osrelease=2.6.16 debug.ddb.textdump.pending=1 # for NFSv4 kern.ipc.maxsockbuf=524288 > > server, immediately after restarting all of nfs scripts (rpcbind > > nfsclient nfsuserd nfsserver mountd nfsd statd lockd nfscbd): > > > > Jun 27 18:04:44 rpcbind: cannot get information for udp6 > > Jun 27 18:04:44 rpcbind: cannot get information for tcp6 > > These two usually indicate you removed IPv6 support from the kernel, > except your ifconfig output (I've remove it) on the server shows you do > have IPv6 support. I've been trying to get these warnings removed for > quite some time (PR kern/96242). They're harmless, but the > inconsistency here is a little weird -- are you explicitly disabling > IPv6 on nfe0? I have WITHOUT_IPV6= in my make.conf on all my machines (or I have problems with jdk1.6) and WITHOUT_INET6= in my src.conf. I'm not sure why the rpcbind/ifconfig binaries have a different concept than the kernel since I always "make buildworld kernel" and keep things in sync with mergemaster when I reboot. I'm building new worlds/kernels now to see if that makes any difference. -- Rick C. Petty
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100628151835.GA53315>