From owner-freebsd-net@FreeBSD.ORG  Sat Feb  1 01:41:06 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id CC1C0E2D;
 Sat,  1 Feb 2014 01:41:06 +0000 (UTC)
Received: from mail-ig0-x229.google.com (mail-ig0-x229.google.com
 [IPv6:2607:f8b0:4001:c05::229])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 910071E27;
 Sat,  1 Feb 2014 01:41:06 +0000 (UTC)
Received: by mail-ig0-f169.google.com with SMTP id uq10so2217954igb.0
 for <multiple recipients>; Fri, 31 Jan 2014 17:41:06 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:sender:in-reply-to:references:date:message-id:subject
 :from:to:cc:content-type;
 bh=809WyS4AowujPVBOoSqcmfMWqqLhGWjfV5hkWBXl1Zg=;
 b=H0NuY9jQ63yowaFL2gZKDpBBw0wl2OAyHHRhJhZv1PIn75iQEIlbNHD6yjLj6L0pxt
 pllsj2XNL3HXdCGItxjOqR1bHvM2sRfnmv8EoJnqnEq3tihDuHsqsnebNMSN8SI1hcq5
 VbG1jwVFUX1n4dvCtIR22S7OithFqNHZQ2yHQbr8pRvUpSHiOf+RWiEoxGGnYEE9/1h7
 rCAK1X05pqpFqA/WvnhOffXhfqBuuCTjB2QFkxvCicKQGU1MhP3jr2ewfAtAm1ssHL7v
 uE0AHSkKon2LS+FJyxFO7Ip+QFYR6bHTzOz8Afu2RP4u+/aBNfS94sIVngi5x/lQVGJI
 WHdg==
MIME-Version: 1.0
X-Received: by 10.42.52.209 with SMTP id k17mr17146362icg.1.1391218866028;
 Fri, 31 Jan 2014 17:41:06 -0800 (PST)
Sender: jdavidlists@gmail.com
Received: by 10.42.170.8 with HTTP; Fri, 31 Jan 2014 17:41:05 -0800 (PST)
In-Reply-To: <1609454808.1083115.1391210456671.JavaMail.root@uoguelph.ca>
References: <CABXB=RQ6LdeoNi4vNZGCaM2C_up_JCf2SpWPzm2S_M_+pTnzsQ@mail.gmail.com>
 <1609454808.1083115.1391210456671.JavaMail.root@uoguelph.ca>
Date: Fri, 31 Jan 2014 20:41:05 -0500
X-Google-Sender-Auth: B8v5sRIw6xBSVZsi1DwpfAzKP5Y
Message-ID: <CABXB=RSCHTg662Fj2CBZHD=8-GEaZyYdVRcsCTxBbaJ-f8ZwuA@mail.gmail.com>
Subject: Re: Terrible NFS performance under 9.2-RELEASE?
From: J David <j.david.lists@gmail.com>
To: Rick Macklem <rmacklem@uoguelph.ca>
Content-Type: text/plain; charset=ISO-8859-1
Cc: freebsd-net@freebsd.org, Garrett Wollman <wollman@freebsd.org>
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 01 Feb 2014 01:41:06 -0000

On Fri, Jan 31, 2014 at 6:20 PM, Rick Macklem <rmacklem@uoguelph.ca> wrote:
> Oh, and remember to try setting readahead=8 in your mounts, too. NFS will
> do a read + N readaheads (where N == 1 by default) and then wait for
> replies to those before continuing on.

Predictably, this has no effect on anything but sequential reads.

No tuning is going to change the fact that writing 14MiB/sec from the
client to the server results in 200+ MiB/sec of wasted traffic being
sent from the server back to the client.

This is from the client's interface during a write-only test:

      Interface           Traffic               Peak                Total


        vtnet1  in    202.838 MB/s        219.467 MB/s          359.898 GB

                 out    14.127 MB/s         14.346 MB/s           96.503 GB


If write performance did get to wire speed on this workload, the most
it could ever do would be <128MiB/sec, because the unused backflow of
2GiB/sec would max out the interface.

Thanks!