From owner-freebsd-net@freebsd.org Tue Aug 2 14:45:09 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D9FBBACB78 for ; Tue, 2 Aug 2016 14:45:09 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi0-x22b.google.com (mail-oi0-x22b.google.com [IPv6:2607:f8b0:4003:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF0CE1383 for ; Tue, 2 Aug 2016 14:45:08 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi0-x22b.google.com with SMTP id w18so239614483oiw.3 for ; Tue, 02 Aug 2016 07:45:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=Zs19BeAik0lxWCbo/045u+0halLZ2k23niWRGsqZD6M=; b=A42v3dfMuLjFAOJlG6CbdxsthvDDW5zEyqsOYEGzU8H8z6djwjDbC855GYMt/5nlbF o5YZF7qfERTVsNKj6usCZrNaP7uE9c1OPc0eUnB/iIcSDFX9wkDnfTCGfTQ0OU5YJx4D xd2YdTzr6xUaEVyeuS0KeM5Rh5jfwItqNUK1qZrL77OhbRRzbz37aIWiNNzN4Dy+lTrz jWFu8E7MINt9Zl0EkORnHzOrMS4emqMsfGhkuBPPVtmrkfM+8Sp8Gal2RKVUhjAUj7xE xLSyQBKG3R/T/etaenGeGuvUREL6zwrNBEjra5i2nh9yKu2tzPniaLUa8vY2G/eScfRp jLPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=Zs19BeAik0lxWCbo/045u+0halLZ2k23niWRGsqZD6M=; b=Bi6iCrBA7JS9wspBhtJSe67iGRjecXj+Y5wb8NmYyc8F+/bX+KDsbRKtM0IJUBbe+g YYb11aaUIKTfOUF0LaHNSaWrwOaz/91xaGR5UnzoXrivyABDHKEc32TFHrze7A22Pfu7 xEbAPFUtZNOsC3Pg5pfubDw/OLPVqEigRj09456lMS8dUQFL/xXxhQ+GAFIwaAfPwx/7 L7UWn6jhGTbwYmHcBrypfMY7Vsmiu0dKGbgjybXp63lB8EZjXoqqkzB1frBIGAe1jEZN pjS/9FPy9rBvAZGr90PzDiR6Ze05gLSRQNi0WoQWj4seoxJxMsGSn1OcMPI9FBaF9nYs Sotw== X-Gm-Message-State: AEkoouvIAwNgEZHG9Z5Os9ItZia/9wLT2ZPMP9XD1iUkA2HSV+xXCB3RgSNYrrL2MNynW+cHzhv2p/sw8fwL8g== X-Received: by 10.202.230.141 with SMTP id d135mr37910728oih.182.1470149107993; Tue, 02 Aug 2016 07:45:07 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.202.196.149 with HTTP; Tue, 2 Aug 2016 07:45:07 -0700 (PDT) In-Reply-To: <20160802104929.a10602a4786c68b4547a45b9@aei.mpg.de> References: <20160802104929.a10602a4786c68b4547a45b9@aei.mpg.de> From: Alan Somers Date: Tue, 2 Aug 2016 08:45:07 -0600 X-Google-Sender-Auth: PskSLoEnl5UZ1dvDAMG43IyLgnI Message-ID: Subject: Re: NFS on 10G interfaces still painfully slow To: =?UTF-8?B?R2Vycml0IEvDvGhu?= Cc: FreeBSD Net Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 14:45:09 -0000 On Tue, Aug 2, 2016 at 2:49 AM, Gerrit K=C3=BChn = wrote: > Hi all, > > I already reported this issue here a year ago and unfortunately was not > able to fix it back then. Now I had another run at it, using two recent > 10.3-machines with a direct 10G link. I still see nfs is painfully > slow (around 20-80MB/s). I tried both nfsv3 and nfsv4, with almost the sa= me > results. Everything I tried so far (mtu size, wcommitsize, readahead...) > only makes things worse or at least not much better. > Moving data in different ways (scp, ggate) is much faster, so plain > network speed should not be an issue. > > Is there anyone around here who can confirm that nfs can go faster over > 10G links? > Any hints for further tuning/debugging are greatly appreciated. > > > cu > Gerrit I can get 1GB/s over NFS on a 10G link, so it's not always slow. There's probably something about your setup that's slowing it down. What is your NFS client? If Linux, make sure that you're using the "async" mount option instead of "sync". What benchmark are you using to measure that speed? Did you remember to start lockd and statd? If you post your /etc/exports and the client's /etc/fstab, that might reveal something.