From owner-freebsd-questions@FreeBSD.ORG Mon Aug 23 06:04:38 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B7921065675 for ; Mon, 23 Aug 2010 06:04:38 +0000 (UTC) (envelope-from nikolai@net24.co.nz) Received: from omr.mail.isx.net.nz (omr.mail.isx.net.nz [210.5.49.98]) by mx1.freebsd.org (Postfix) with ESMTP id EA1608FC16 for ; Mon, 23 Aug 2010 06:04:37 +0000 (UTC) Received: from 210.55.30.56 (EHLO _192.168.1.111_) ([210.55.30.56]) by omr.mail.isx.net.nz (MOS 3.8.3-GA FastPath queued) with ESMTP id APX28271 (AUTH nikolai@net24.co.nz); Mon, 23 Aug 2010 17:53:30 +1200 (NZST) From: Nikolai Schupbach Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Mon, 23 Aug 2010 17:53:29 +1200 Message-Id: To: FreeBSD Mailing List Mime-Version: 1.0 (Apple Message framework v1081) X-Mailer: Apple Mail (2.1081) Subject: NFSv3 and v4 slow with newnfs code X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 06:04:38 -0000 We are doing some performance testing on a new system. We have a = OpenSolaris NFS server sharing a folder on a ZFS filesystem and a = FreeBSD 8.1 NFS client. The machines are directly connected using 10GbE = (no switch in-between). Below are the performance figures we attained when doing simple 10GB dd = write (dd if=3D/dev/zero of=3D/mnt/file.tmp bs=3D1M count=3D10240) and = read (dd of=3D/dev/null if=3D/mnt/file.tmp) tests over NFS from the = FreeBSD client using various mount options. We performed these tests numerous times and all results are roughly the = same for each test. We have tuned kern.ipc.maxsockbuf, = net.inet.tcp.recvspace and net.inet.tcp.sendspace. This didn't result in = any significant differences in the test results. Both NFS client and = server NICs have MTU set to 9000; this improves performance noticeable. Currently it appears sticking with the stable NFSv3 code yields the best = results. Both NFSv3 and NFSv4 with the newnfs code has disappointing = performance. We installed Linux on the client machine as a test and = unfortunately Linux has the best performance by far.=20 Are there any other options we can use to improve the performance of = NFSv4 for large sequential writes and reads?=20 FreeBSD NFSv4 \w newnfs --- mount -t newnfs -o nfsv4 10.0.0.2:/volumes/datpool/test /mnt Writing: 10737418240 bytes transferred in 236.627936 secs, 43 MB/s Reading: 10737418240 bytes transferred in 131.385064 secs, 95 MB/s mount -t newnfs -o nfsv4,wsize=3D32768,rsize=3D32768 = 10.0.0.2:/volumes/datpool/test /mnt Writing: 10737418240 bytes transferred in 258.567077 secs, 39 MB/s Reading: 10737418240 bytes transferred in 120.646962 secs, 85 MB/s FreeBSD NFSv3 \w newnfs --- mount -t newnfs -o nfsv4 10.0.0.2:/volumes/datpool/test /mnt Writing 10737418240 bytes transferred in 215.963140 secs, 47 MB/s Reading 10737418240 bytes transferred in 101.110872 secs, 101 MB/s FreeBSD NFSv3=20 --- mount -t nfs -o nfsv3 10.0.0.2:/volumes/datpool/test /mnt Writing: 10737418240 bytes transferred in 161.739156 secs 63 MB/s Reading: 10737418240 bytes transferred in 72.327859 secs, 129 MB/s Linux NFSv4 (CentOS 5.5) --- Writing: 10737418240 bytes (11 GB) copied, 143.776 seconds, 75 MB/s Reading: 10737418240 bytes (11 GB) copied, 63.7012 seconds, 170 MB/s --Nik