From owner-freebsd-stable Wed Jul 24 10:15:13 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B4E4637B400 for ; Wed, 24 Jul 2002 10:15:09 -0700 (PDT) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 677EB43E97 for ; Wed, 24 Jul 2002 10:15:09 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.5/8.12.4) with ESMTP id g6OHF9CV028676; Wed, 24 Jul 2002 10:15:09 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.5/8.12.4/Submit) id g6OHF9Ne028675; Wed, 24 Jul 2002 10:15:09 -0700 (PDT) (envelope-from dillon) Date: Wed, 24 Jul 2002 10:15:09 -0700 (PDT) From: Matthew Dillon Message-Id: <200207241715.g6OHF9Ne028675@apollo.backplane.com> To: "Jaime Bozza" Cc: , "'Olaf R'" Subject: Re: RE: Abominable NFSv3 read performance / FreeBSD server / Solaris client References: <022101c2332c$729a5610$6401010a@bozza> Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :.. :FreeBSD 4.6-STABLE server, default nfsd settings, tcp.sendspace=32768 :Solaris 8 client (SparcStation 20), default settings, default options :for the mount. :bigfile is 64953143 bytes : :(Solaris 8 with NFSv3 defaults to a 32768 read buffer size, rsize=32768) : :(on Solaris system) :mount server:/export /mnt/server :time dd if=/mnt/server/bigfile of=/dev/null bs=64k : :returns: :991+1 records in :991+1 records out :0.05u 1.68s 2:29.76 1.1% : : :mount -o rsize=16384 server:/export /mnt/server :time dd if=/mnt/server/bigfile of=/dev/null bs=64k : :returns: :991+1 records in :991+1 records out :0.03u 3.11s 0:19.61 16.0% : : :rsize=8192 (0.00u 2.50s 0:33.37 7.4%) :rsize=12288 (0.07u 3.00s 0:28.95 10.6%) Well, Solaris is issuing four NFS read rpc's. With the original trace and a buffer size of 8K this will cause FreeBSD to push out a little over 32K worth of TCP. This implies that a 32K TCP send and receive window is not sufficient. : :Increasing tcp.sendspace on the server did not seem to change the times :all that much. (the default 32768 read buffer would have the same 780 :bytes in the Send-Q you talked about below) Increase sendspace on the server and recvspace on the client, and make sure they've actually increased. A tcpdump should show much larger window advertisements from the solaris client, and netstat -tn on the FreeBSD box during the transfer (over and over again) should show that the FreeBSD box is able to queue up more then 32K. -Matt : :If you want me to run additional tests or outputs, I'd be happy to. : :Jaime Bozza To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message