From owner-freebsd-stable Wed Jul 24 14:24:37 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 044C837B400 for ; Wed, 24 Jul 2002 14:24:34 -0700 (PDT) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 90EF643E75 for ; Wed, 24 Jul 2002 14:24:33 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.5/8.12.4) with ESMTP id g6OLOXCV029617; Wed, 24 Jul 2002 14:24:33 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.5/8.12.4/Submit) id g6OLOXaM029616; Wed, 24 Jul 2002 14:24:33 -0700 (PDT) (envelope-from dillon) Date: Wed, 24 Jul 2002 14:24:33 -0700 (PDT) From: Matthew Dillon Message-Id: <200207242124.g6OLOXaM029616@apollo.backplane.com> To: "Jaime Bozza" Cc: , "'Olaf R'" Subject: Re: RE: RE: Abominable NFSv3 read performance / FreeBSD server / Solaris client References: <025c01c23353$f75cc730$6401010a@bozza> Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :Ok, I fiddled with the tcp_recv_hiwat parameter on Solaris 8, along :with the tcp.sendspace parameter on FreeBSD. No matter what, anything :above 32767 would continually get the delays. : : :Unfortunately, I don't think I can change them with a running system. :(Have to set them in /etc/system and reboot to make the changes) : :But regardless, both are set to 32768. I wouldn't be surprised if :rsize=32768 :would work fine if I increased both of these values. Here are the :beginnings :of a couple of tcpdumps. : :FreeBSD to FreeBSD: :14:52:36.435205 freebsd-client.1700858811 > freebsd-server.nfs: 132 :lookup [|nfs] :14:52:36.435334 freebsd-server.nfs > freebsd-client.1700858811: reply ok :... Umm. Jaime, these tcp dumps show a UDP NFS mount, not a TCP NFS mount. The first set of TCP dumps showed a TCP NFS mount. :... :that the first :couple of lines are looking up the file. After that, I'm at a loss with :the Solaris :dump, other than the fact that I'm pretty sure the "reply ERR" are :causing delays :somewhere. (The FreeBSD one is so clean!) : :Does FreeBSD attempt to send packets out using a size of NFS_MAXPACKET :(33172?) all :the time? Could the "win 33176" be centered around that? From what I'm :seeing A UDP NFS mount will send packets that are approximately rsize or wsize bytes for full block data reads or writes, depending on the direction. A TCP NFS mount will not send packets larger then the MTU (i.e. there will be no fragments). :(correct me if I'm wrong), FreeBSD is sending 8328 bytes in each TCP :packet when :Sending to a FreeBSD client. This most recent TCP dump you made was a UDP mount, not a TCP mount. :figure :out what's really going on here. Any pointers on where to research next :would be :helpful. I'm pretty sure that it would have to be something on Solaris :that's :causing the problem, but I'm at a loss as to what. : :Jaime Well, I really don't think the rsize or wsize really has anything to do with it. I still feel that the problem is a window-sizing issue with TCP and the rsize/wsize changes I will also note that window sizing is irrelevant with a UDP NFS mount. It is only relevant on a TCP NFS mount. You should be getting nearly full speed over a UDP mount (using a rsize and wsize of 8K). If you aren't then there's likely a network problem somewhere. What I really need is a tcpdump from the FreeBSD side showing a few seconds worth of a large transfer for a TCP NFS mount from the FreeBSD box to a Solaris client. On the FreeBSD it would be something like: tcpdump -s 4096 -n -i -l > dumpfile -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message