Date: Wed, 31 Dec 2008 10:31:42 +0100 From: Bernard Dugas <bernard@dugas-family.org> To: Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> Cc: freebsd-questions@freebsd.org, Vincent Hoffman <vince@unsane.co.uk> Subject: Re: Optimising NFS for system files Message-ID: <495B3BFE.1040404@dugas-family.org> In-Reply-To: <20081231002557.D45571@wojtek.tensor.gdynia.pl> References: <20081216094719.EDCEE1065675@hub.freebsd.org> <495930E4.1030501@dugas-family.org> <20081229230115.F68805@wojtek.tensor.gdynia.pl> <4959DFEF.5090407@dugas-family.org> <20081230150507.U87347@wojtek.tensor.gdynia.pl> <495A3957.9050002@dugas-family.org> <495A3C72.5070802@unsane.co.uk> <20081230201351.L12391@wojtek.tensor.gdynia.pl> <495A9595.6080201@dugas-family.org> <20081231002557.D45571@wojtek.tensor.gdynia.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
I am trying a memory disk on server to see the effect of hard drive
performances, and also discovering the function :-)
The conclusion is that memory disk is faster that this drive ;-)
45MB/s vs 10Mb/s
But the NFS access to the memory drive is still 5MB/s :-(
As there is no more hard drive involved, we know that there is a
bottleneck at 5MB in NFS layer on this system... Where ?
Thanks a lot for any help on the method to find/diagnose this !
----------------------------------------Details are below :
nfsserver# uname -a
FreeBSD nfsserver 7.1-RC1 FreeBSD 7.1-RC1 #0: Sun Dec 7 00:38:13 UTC
2008 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
nfsserver# mdconfig -a -t swap -s 600m -o reserve -u 7
nfsserver# ls /dev/md*
/dev/md7 /dev/mdctl
nfsserver# newfs -i 2048 /dev/md7
Reduced frags per cylinder group from 60864 to 59928 to enlarge last cyl
group
/dev/md7: 600.0MB (1228800 sectors) block size 16384, fragment size 2048
using 6 cylinder groups of 117.05MB, 7491 blks, 59968 inodes.
super-block backups (for fsck -b #) at:
160, 239872, 479584, 719296, 959008, 1198720
nfsserver# mkdir /tstnfs
nfsserver# mount /dev/md7 /tstnfs
nfsserver# date ; time tar -cf -
/nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports > /dev/null ; date ;
Wed Dec 31 09:11:08 CET 2008
tar: Removing leading '/' from member names
3.794u 8.766s 0:46.40 27.0% 71+1406k 123375+0io 0pf+0w
Wed Dec 31 09:11:54 CET 2008
That makes 498MB / 46s = 10.8MB/s for disk drive.
nfsserver# date ; cp -r -p
/nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports /tstnfs/ports ; date
Wed Dec 31 09:33:09 CET 2008
Wed Dec 31 09:34:46 CET 2008
df -h
/dev/md7 512M 498M -27M 106% /tstnfs
nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:36:59 CET 2008
tar: Removing leading '/' from member names
2.947u 6.218s 0:10.61 86.2% 74+1463k 104885+0io 0pf+0w
Wed Dec 31 09:37:10 CET 2008
nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:37:12 CET 2008
tar: Removing leading '/' from member names
2.895u 6.487s 0:11.01 85.1% 74+1466k 112838+0io 0pf+0w
Wed Dec 31 09:37:23 CET 2008
nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:40:22 CET 2008
tar: Removing leading '/' from member names
2.902u 6.610s 0:11.10 85.6% 75+1483k 113393+0io 0pf+0w
Wed Dec 31 09:40:33 CET 2008
That makes 498MB / 11s = 45MB/s : better that 10MB/s for disk, but not
exceptional.
Now on the client :
client9# uname -a
FreeBSD client9 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 11:42:13 UTC 2008
root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
client9# mount -o ro nfsserver:/tstnfs /tstnfs
client9# df -h
nfsserver:/tstnfs 512M 498M
0B 100% /tstnfs
client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:50:39 CET 2008
tar: Removing leading '/' from member names
2.896u 13.020s 1:35.22 16.7% 75+1483k 0+0io 2pf+0w
Wed Dec 31 09:52:14 CET 2008
client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:52:22 CET 2008
tar: Removing leading '/' from member names
2.700u 12.755s 1:27.78 17.6% 76+1498k 0+0io 0pf+0w
Wed Dec 31 09:53:50 CET 2008
client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ;
Wed Dec 31 09:55:02 CET 2008
tar: Removing leading '/' from member names
2.681u 12.688s 1:28.15 17.4% 74+1464k 0+0io 0pf+0w
Wed Dec 31 09:56:30 CET 2008
That makes between 95s and 87s, then 498MB / 95s = 5,2MB/s and 5.7MB/s,
like previous test from hard drive NFS export.
Top is showing around 100MB of free memory while taring on client9, so i
don't think tar is paging on network :
last pid: 3318; load averages: 0.17, 0.11, 0.04 up 0+11:14:27
10:08:10
30 processes: 1 running, 29 sleeping
CPU: 0.8% user, 0.0% nice, 9.0% system, 0.0% interrupt, 90.2% idle
Mem: 19M Active, 720M Inact, 136M Wired, 240K Cache, 110M Buf, 98M Free
Best regards,
--
Bernard DUGAS Mobile +33 615 333 770
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?495B3BFE.1040404>