Date: Thu, 29 Dec 2005 11:19:14 +0100 From: Angel Blazquez <boisan@gmail.com> To: freebsd-isp@freebsd.org Subject: nfs server overload (nfsd) Message-ID: <b698c5950512290219v7febf9f9l450ac821ad927d44@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello, We are expecting incredible overload in a NFS server. A top shows nfsd consuming most of the CPU: PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMA= ND 6000 root -8 0 1204K 660K biord 1 124:15 27.88% 27.88% nfsd 6002 root 4 0 1204K 660K *Giant 0 124:18 17.58% 17.58% nfsd 6006 root 4 0 1204K 660K *Giant 0 123:38 10.21% 10.21% nfsd 6005 root 4 0 1204K 660K *Giant 0 123:36 7.47% 7.47% nfsd 6003 root 4 0 1204K 660K *Giant 0 123:08 4.15% 4.15% nfsd 6001 root 4 0 1204K 660K *Giant 0 123:16 2.83% 2.83% nfsd Memory looks fine: Mem: 27M Active, 910M Inact, 136M Wired, 51M Cache, 112M Buf, 1828K Free Swap: 2048M Total, 72K Used, 2048M Free Typing in the nfs server (console/ssh) becomes terrible, the server does not reply well. We are running this nfs server in FreeBSD 5.3-RELEASE-p23 on a Compaq Proliant server with a Compaq Smart Array 5300 that comunicates with a array of disks: /dev/da0s1d 164G 124G 27G 82% /data0 /dev/da1s1d 131G 80G 41G 66% /data1 We have /data0 and /data1 exported: /data0 -maproot=3Droot -alldirs -network 192.168.62.0 -mask 255.255.255.0 /data1 -maproot=3Droot -alldirs -network 192.168.62.0 -mask 255.255.255.0 so a couple of incoming SMTP servers we have can deliver e-mail to those filesystems. We are running exim 4.60.0 in those other servers, 4.10-RELEASE-p5 in one of them, and FreeBSD 6.0-RELEASE #0 in the other one. If we stop exim delivering e-mail, nfs server does well, the cpu gets free, and the nfs server works fine (replies to user interaction, etc). FreeBSD 6.0 sysctl output (nfs related): vfs.nfs4.access_cache_timeout: 60 vfs.nfs4.nfsv3_commit_on_close: 0 vfs.nfs.downdelayinitial: 12 vfs.nfs.downdelayinterval: 30 vfs.nfs.realign_test: 1294030 vfs.nfs.realign_count: 0 vfs.nfs.bufpackets: 4 vfs.nfs.reconnects: 2 vfs.nfs.iodmaxidle: 120 vfs.nfs.iodmin: 4 vfs.nfs.iodmax: 20 vfs.nfs.defect: 0 vfs.nfs.nfs_ip_paranoia: 1 vfs.nfs.diskless_valid: 0 vfs.nfs.diskless_rootpath: vfs.nfs.access_cache_timeout: 2 vfs.nfs.nfsv3_commit_on_close: 0 vfs.nfs.clean_pages_on_close: 1 vfs.nfs.nfs_directio_enable: 0 vfs.nfs.nfs_directio_allow_mmap: 1 vfs.nfsrv.nfs_privport: 0 vfs.nfsrv.async: 0 vfs.nfsrv.commit_blks: 0 vfs.nfsrv.commit_miss: 0 vfs.nfsrv.realign_test: 0 vfs.nfsrv.realign_count: 0 vfs.nfsrv.gatherdelay: 10000 vfs.nfsrv.gatherdelay_v3: 0 FreeBSD 4.10 sysctl output (nfs related): vfs.nfs.nfs_privport: 0 vfs.nfs.async: 0 vfs.nfs.commit_blks: 0 vfs.nfs.commit_miss: 0 vfs.nfs.realign_test: 84602323 vfs.nfs.realign_count: 99713 vfs.nfs.bufpackets: 4 vfs.nfs.gatherdelay: 10000 vfs.nfs.gatherdelay_v3: 0 vfs.nfs.defect: 0 vfs.nfs.nfs_ip_paranoia: 1 vfs.nfs.diskless_valid: 0 vfs.nfs.diskless_rootpath: vfs.nfs.diskless_swappath: vfs.nfs.access_cache_timeout: 2 vfs.nfs.nfsv3_commit_on_close: 0 This couple of servers mounts the filesystems with this options: 192.168.62.54:/data1 /mail nfs =20 rw,nfsv3,intr,dumbtimer,rdirplus,nosuid,nodev 0 0 192.168.62.54:/data0 /data0 nfs =20 rw,nfsv3,intr,dumbtimer,rdirplus,nosuid,nodev 0 0 On the server, sysctl nfs related output looks like this: vfs.nfs.downdelayinitial: 12 vfs.nfs.downdelayinterval: 30 vfs.nfs.realign_test: 2694 vfs.nfs.realign_count: 0 vfs.nfs.bufpackets: 4 vfs.nfs.reconnects: 2 vfs.nfs.iodmaxidle: 120 vfs.nfs.iodmin: 4 vfs.nfs.iodmax: 20 vfs.nfs.defect: 0 vfs.nfs.nfs_ip_paranoia: 1 vfs.nfs.diskless_valid: 0 vfs.nfs.diskless_rootpath: vfs.nfs.access_cache_timeout: 2 vfs.nfs.nfsv3_commit_on_close: 0 vfs.nfs4.access_cache_timeout: 60 vfs.nfs4.nfsv3_commit_on_close: 0 vfs.nfsrv.nfs_privport: 0 vfs.nfsrv.async: 1 vfs.nfsrv.commit_blks: 579238 vfs.nfsrv.commit_miss: 413059 vfs.nfsrv.realign_test: 88269083 vfs.nfsrv.realign_count: 11961 vfs.nfsrv.gatherdelay: 10000 vfs.nfsrv.gatherdelay_v3: 0 debug.hashstat.nfsnode: 65536 5 1 0 Thanks in advance, Best regards, Angel Blazquez
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?b698c5950512290219v7febf9f9l450ac821ad927d44>