From owner-freebsd-net@FreeBSD.ORG Sun Feb 9 18:38:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 133A1305 for ; Sun, 9 Feb 2014 18:38:43 +0000 (UTC) Received: from mail-in-09.arcor-online.net (mail-in-09.arcor-online.net [151.189.21.49]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ADA821B0C for ; Sun, 9 Feb 2014 18:38:42 +0000 (UTC) Received: from mail-in-05-z2.arcor-online.net (mail-in-05-z2.arcor-online.net [151.189.8.17]) by mx.arcor.de (Postfix) with ESMTP id 1F0C4198D27 for ; Sun, 9 Feb 2014 19:03:22 +0100 (CET) Received: from mail-in-15.arcor-online.net (mail-in-15.arcor-online.net [151.189.21.55]) by mail-in-05-z2.arcor-online.net (Postfix) with ESMTP id D768C116007 for ; Sun, 9 Feb 2014 19:03:22 +0100 (CET) X-Greylist: Passed host: 88.67.113.179 X-DKIM: Sendmail DKIM Filter v2.8.2 mail-in-15.arcor-online.net C014C1AB52F Received: from lorvorc.mips.inka.de (dslb-088-067-113-179.pools.arcor-ip.net [88.67.113.179]) by mail-in-15.arcor-online.net (Postfix) with ESMTPS id C014C1AB52F for ; Sun, 9 Feb 2014 19:03:22 +0100 (CET) Received: from lorvorc.mips.inka.de (localhost [127.0.0.1]) by lorvorc.mips.inka.de (8.14.7/8.14.7) with ESMTP id s19I3Mk6071693 for ; Sun, 9 Feb 2014 19:03:22 +0100 (CET) (envelope-from mailnull@lorvorc.mips.inka.de) Received: (from mailnull@localhost) by lorvorc.mips.inka.de (8.14.7/8.14.7/Submit) id s19I3M4L071692 for freebsd-net@freebsd.org; Sun, 9 Feb 2014 19:03:22 +0100 (CET) (envelope-from mailnull) From: naddy@mips.inka.de (Christian Weisgerber) Subject: Re: Terrible NFS performance under 9.2-RELEASE? Date: Sun, 9 Feb 2014 18:03:21 +0000 (UTC) Message-ID: References: <52DC1241.7010004@egr.msu.edu> <1629593139.16590858.1390789014324.JavaMail.root@uoguelph.ca> Originator: naddy@mips.inka.de (Christian Weisgerber) To: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Feb 2014 18:38:43 -0000 Rick Macklem wrote: > I have a "hunch" that might explain why 64K NFS reads/writes perform > poorly for some network environments. > A 64K NFS read reply/write request consists of a list of 34 mbufs when > passed to TCP via sosend() and a total data length of around 65680bytes. > Looking at a couple of drivers (virtio and ixgbe), they seem to expect > no more than 32-33 mbufs in a list for a 65535 byte TSO xmit. I think > (I don't have anything that does TSO to confirm this) that NFS will pass > a list that is longer (34 plus a TCP/IP header). This may or may not be the same problem: When I switched my desktop box from FreeBSD 7 to 9, NFS read performance from my media server (running OpenBSD) became extremely poor. I couldn't even stream a movie any longer. Disabling TSO on the nfe(4) interface had no effect. My workaround was to switch from a TCP mount to a UDP one. The problem has persisted to FreeBSD 10. I can now report that switching to [rw]size=32768 with a TCP mount also works fine. -- Christian "naddy" Weisgerber naddy@mips.inka.de