From owner-freebsd-questions@FreeBSD.ORG Wed Dec 31 09:29:54 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF610106566B for ; Wed, 31 Dec 2008 09:29:54 +0000 (UTC) (envelope-from bernard@dugas-family.org) Received: from smtp11.ispronet.com (smtp11.ispronet.com [81.28.196.233]) by mx1.freebsd.org (Postfix) with ESMTP id 557608FC1F for ; Wed, 31 Dec 2008 09:29:54 +0000 (UTC) (envelope-from bernard@dugas-family.org) Received: from localhost (unknown [127.0.0.1]) by smtp11.ispronet.com (Postfix) with ESMTP id 058F1B82F; Wed, 31 Dec 2008 10:29:53 +0100 (CET) Received: from smtp11.ispronet.com ([127.0.0.1]) by localhost (smtp11.ispronet.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 58059-02; Wed, 31 Dec 2008 10:29:43 +0100 (CET) Received: from [192.168.1.50] (unknown [81.28.194.61]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp11.ispronet.com (Postfix) with ESMTP; Wed, 31 Dec 2008 10:29:43 +0100 (CET) Message-ID: <495B3BFE.1040404@dugas-family.org> Date: Wed, 31 Dec 2008 10:31:42 +0100 From: Bernard Dugas User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7) Gecko/20040616 X-Accept-Language: fr,en MIME-Version: 1.0 To: Wojciech Puchar References: <20081216094719.EDCEE1065675@hub.freebsd.org> <495930E4.1030501@dugas-family.org> <20081229230115.F68805@wojtek.tensor.gdynia.pl> <4959DFEF.5090407@dugas-family.org> <20081230150507.U87347@wojtek.tensor.gdynia.pl> <495A3957.9050002@dugas-family.org> <495A3C72.5070802@unsane.co.uk> <20081230201351.L12391@wojtek.tensor.gdynia.pl> <495A9595.6080201@dugas-family.org> <20081231002557.D45571@wojtek.tensor.gdynia.pl> In-Reply-To: <20081231002557.D45571@wojtek.tensor.gdynia.pl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at ispronet.com Cc: freebsd-questions@freebsd.org, Vincent Hoffman Subject: Re: Optimising NFS for system files X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Dec 2008 09:29:55 -0000 I am trying a memory disk on server to see the effect of hard drive performances, and also discovering the function :-) The conclusion is that memory disk is faster that this drive ;-) 45MB/s vs 10Mb/s But the NFS access to the memory drive is still 5MB/s :-( As there is no more hard drive involved, we know that there is a bottleneck at 5MB in NFS layer on this system... Where ? Thanks a lot for any help on the method to find/diagnose this ! ----------------------------------------Details are below : nfsserver# uname -a FreeBSD nfsserver 7.1-RC1 FreeBSD 7.1-RC1 #0: Sun Dec 7 00:38:13 UTC 2008 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 nfsserver# mdconfig -a -t swap -s 600m -o reserve -u 7 nfsserver# ls /dev/md* /dev/md7 /dev/mdctl nfsserver# newfs -i 2048 /dev/md7 Reduced frags per cylinder group from 60864 to 59928 to enlarge last cyl group /dev/md7: 600.0MB (1228800 sectors) block size 16384, fragment size 2048 using 6 cylinder groups of 117.05MB, 7491 blks, 59968 inodes. super-block backups (for fsck -b #) at: 160, 239872, 479584, 719296, 959008, 1198720 nfsserver# mkdir /tstnfs nfsserver# mount /dev/md7 /tstnfs nfsserver# date ; time tar -cf - /nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports > /dev/null ; date ; Wed Dec 31 09:11:08 CET 2008 tar: Removing leading '/' from member names 3.794u 8.766s 0:46.40 27.0% 71+1406k 123375+0io 0pf+0w Wed Dec 31 09:11:54 CET 2008 That makes 498MB / 46s = 10.8MB/s for disk drive. nfsserver# date ; cp -r -p /nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports /tstnfs/ports ; date Wed Dec 31 09:33:09 CET 2008 Wed Dec 31 09:34:46 CET 2008 df -h /dev/md7 512M 498M -27M 106% /tstnfs nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:36:59 CET 2008 tar: Removing leading '/' from member names 2.947u 6.218s 0:10.61 86.2% 74+1463k 104885+0io 0pf+0w Wed Dec 31 09:37:10 CET 2008 nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:37:12 CET 2008 tar: Removing leading '/' from member names 2.895u 6.487s 0:11.01 85.1% 74+1466k 112838+0io 0pf+0w Wed Dec 31 09:37:23 CET 2008 nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:40:22 CET 2008 tar: Removing leading '/' from member names 2.902u 6.610s 0:11.10 85.6% 75+1483k 113393+0io 0pf+0w Wed Dec 31 09:40:33 CET 2008 That makes 498MB / 11s = 45MB/s : better that 10MB/s for disk, but not exceptional. Now on the client : client9# uname -a FreeBSD client9 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 11:42:13 UTC 2008 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 client9# mount -o ro nfsserver:/tstnfs /tstnfs client9# df -h nfsserver:/tstnfs 512M 498M 0B 100% /tstnfs client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:50:39 CET 2008 tar: Removing leading '/' from member names 2.896u 13.020s 1:35.22 16.7% 75+1483k 0+0io 2pf+0w Wed Dec 31 09:52:14 CET 2008 client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:52:22 CET 2008 tar: Removing leading '/' from member names 2.700u 12.755s 1:27.78 17.6% 76+1498k 0+0io 0pf+0w Wed Dec 31 09:53:50 CET 2008 client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:55:02 CET 2008 tar: Removing leading '/' from member names 2.681u 12.688s 1:28.15 17.4% 74+1464k 0+0io 0pf+0w Wed Dec 31 09:56:30 CET 2008 That makes between 95s and 87s, then 498MB / 95s = 5,2MB/s and 5.7MB/s, like previous test from hard drive NFS export. Top is showing around 100MB of free memory while taring on client9, so i don't think tar is paging on network : last pid: 3318; load averages: 0.17, 0.11, 0.04 up 0+11:14:27 10:08:10 30 processes: 1 running, 29 sleeping CPU: 0.8% user, 0.0% nice, 9.0% system, 0.0% interrupt, 90.2% idle Mem: 19M Active, 720M Inact, 136M Wired, 240K Cache, 110M Buf, 98M Free Best regards, -- Bernard DUGAS Mobile +33 615 333 770