From owner-freebsd-net@freebsd.org Thu Jun 25 12:58:10 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6EE598DD0B for ; Thu, 25 Jun 2015 12:58:10 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E1AA1FB7 for ; Thu, 25 Jun 2015 12:58:09 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id 06B929AE00A for ; Thu, 25 Jun 2015 14:52:49 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id E4D0D405881; Thu, 25 Jun 2015 14:52:48 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id C3BB2405880; Thu, 25 Jun 2015 14:52:48 +0200 (CEST) Received: from arc.aei.uni-hannover.de ([10.117.15.110]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6HF1691) with ESMTP id 2015062514523841-65082 ; Thu, 25 Jun 2015 14:52:38 +0200 Date: Thu, 25 Jun 2015 14:52:38 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: freebsd-net@freebsd.org Cc: "carsten.aulbert@aei.mpg.de" Subject: NFS on 10G interface terribly slow Message-Id: <20150625145238.12cf9da3b368ef0b9a30f193@aei.mpg.de> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6HF1691 | May 7, 2015) at 25.06.2015 14:52:38, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6HF1691 | May 7, 2015) at 25.06.2015 14:52:48, Serialize complete at 25.06.2015 14:52:48 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2015.6.25.124516 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1200_1299 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, NO_URI_FOUND 0, NO_URI_HTTPS 0, __C230066_P5 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __SUBJ_ALPHA_END 0, __TO_MALFORMED_2 0, __TO_NO_NAME 0' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2015 12:58:10 -0000 Hi all, We have a recent FreeBSD 10.1 installation here that is supposed to act as nfs (v3) client to an Oracle x4-2l server running Soalris 11.2. We have Intel 10-Gigabit X540-AT2 NICs on both ends, iperf is showing plenty of bandwidth (9.xGB/s) in both directions. However, nfs appears to be terribly slow, especially for writing: root@crest:~ # dd if=/dev/zero of=/net/hellpool/Z bs=1024k count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 20.263190 secs (51747824 bytes/sec) Reading appears to be faster, but still far away from full bandwidth: root@crest:~ # dd of=/dev/null if=/net/hellpool/Z bs=1024k 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 5.129869 secs (204406000 bytes/sec) We have already tried to tune rsize/wsize parameters, but they appear to have little (if any) impact on these results. Also, neither stripping down rxsum, txsum, tso etc. from the interface nor increasing MTU to 9000 for jumbo frames did improve anything. It is quite embarrassing to achieve way less than 1GBE performance with 10GBE equipment. Are there any hints what else might be causing this (and how to fix it)? cu Gerrit