From owner-freebsd-stable Thu Jul 25 16:14:28 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C561737B400 for ; Thu, 25 Jul 2002 16:14:26 -0700 (PDT) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 785C943E5E for ; Thu, 25 Jul 2002 16:14:26 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.5/8.12.4) with ESMTP id g6PNEPCV035517; Thu, 25 Jul 2002 16:14:25 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.5/8.12.4/Submit) id g6PNEPUR035516; Thu, 25 Jul 2002 16:14:25 -0700 (PDT) (envelope-from dillon) Date: Thu, 25 Jul 2002 16:14:25 -0700 (PDT) From: Matthew Dillon Message-Id: <200207252314.g6PNEPUR035516@apollo.backplane.com> To: "Jaime Bozza" Cc: "'Barney Wolff'" , "'Olaf R'" , Subject: Re: RE: Abominable NFSv3 read performance / FreeBSD server / Solaris client References: <027d01c2341f$908b1620$6401010a@bozza> Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Ok, I've now been (almost) successful in getting Solaris to FreeBSD to :work the same way as Solaris to Solaris. : :Solaris has a parameter tcp_deferred_ack_interval that is set to a :default of 100. This sounded like is was causing the 1/10 second :delays. I dropped it down to 10. Suddenly, NFS was tons faster at the :default rsize. (I started out at 150 seconds for a 64MB file or so, and :this single change moved it up to 20 seconds) : :I see that FreeBSD (net.inet.tcp.delacktime?) uses 100 as well. :Searching Google, I came up with a few discussions on the deferred_ack :parameter and in almost all cases people were lowering it to 10. : :The question still is why do the delays happen across platforms and not :when connecting on the same platform? : : :Jaime From my read it's because Solaris is not implementing the TCP protocol spec correctly. When data is streaming the TCP protocol is supposed to ack every other packet, even with delayed acks turned on. Solaris is not doing this. Your solution seems reasonable though it will not completely solve the problem. A permanent solution, yielding maximal streaming, would be to increase solaris's TCP receive window. With properly sized TCP buffers I can run NFS over TCP between two FreeBSD boxes at the line rate (11-12 MBytes/sec) test1# mount_nfs -T apollo:/FreeBSD /mnt test1# nfsiod -n 4 test1# dd if=test of=/dev/null bs=32k 2048+0 records in 2048+0 records out 67108864 bytes transferred in 5.795125 secs (11580227 bytes/sec) -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message