From owner-freebsd-fs@FreeBSD.ORG Sun Jul 7 22:54:14 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9FD47CF8 for ; Sun, 7 Jul 2013 22:54:14 +0000 (UTC) (envelope-from berend@pobox.com) Received: from smtp.pobox.com (b-pb-sasl-quonix.pobox.com [208.72.237.35]) by mx1.freebsd.org (Postfix) with ESMTP id 5CB231A91 for ; Sun, 7 Jul 2013 22:54:13 +0000 (UTC) Received: from smtp.pobox.com (unknown [127.0.0.1]) by b-sasl-quonix.pobox.com (Postfix) with ESMTP id 7C87D2F6E2 for ; Sun, 7 Jul 2013 22:54:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=date :message-id:from:to:subject:mime-version:content-type :content-transfer-encoding; s=sasl; bh=DAQ3ZXlLe3+p+AZsyB3AblmDJ vs=; b=waIkvqt1QjFp+jeeDwPRtJ7jKtksQgdp2zLw634X59b0xFq11Z7HJfGCw i3jX1YuVtu/3Mx/LbZctNz3vgNnqBaH/SQdWZVVtpvHJfqJjAYlpgrmc2tfgJWQk P4nTisezXmdPfIJqUsazzyCxd54DTXRnNEeyPn0D38zVYbNIPU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=date:message-id :from:to:subject:mime-version:content-type :content-transfer-encoding; q=dns; s=sasl; b=d4dOyuwDTKNW7yN+LkW xnOD/w8cjKE0dMeC3LUpD2NGAOgDPto0oGC4tXTFr6TYdvlj7EmfKO8N0+pJno2a SJuPHtTIrm8mZlSZHf7/SPPFKtciyCdQA4uJyXERr5pvhz3NEM9wGVK71+xDFmYI cBik2HNSjHXvOZvLtpcwlVX4= Received: from b-pb-sasl-quonix.pobox.com (unknown [127.0.0.1]) by b-sasl-quonix.pobox.com (Postfix) with ESMTP id 710842F6E1 for ; Sun, 7 Jul 2013 22:54:12 +0000 (UTC) Received: from bmach.nederware.nl (unknown [27.252.169.66]) by b-sasl-quonix.pobox.com (Postfix) with ESMTPA id 00B3A2F6DE for ; Sun, 7 Jul 2013 22:54:12 +0000 (UTC) Received: from quadrio.nederware.nl (quadrio.nederware.nl [192.168.33.13]) by bmach.nederware.nl (Postfix) with ESMTP id 1A9085C84 for ; Mon, 8 Jul 2013 10:54:05 +1200 (NZST) Received: from quadrio.nederware.nl (quadrio.nederware.nl [127.0.0.1]) by quadrio.nederware.nl (Postfix) with ESMTP id 7203B49FB979 for ; Mon, 8 Jul 2013 10:54:09 +1200 (NZST) Date: Mon, 08 Jul 2013 10:54:09 +1200 Message-ID: <87y59i0yni.wl%berend@pobox.com> From: Berend de Boer To: freebsd-fs Subject: Terrible NFS4 performance: FreeBSD 9.1 + ZFS + AWS EC2 User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/24.3 (i686-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: Xplain Technology Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/signed; boundary="pgp-sign-Multipart_Mon_Jul__8_10:54:09_2013-1"; micalg=pgp-sha256; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit X-Pobox-Relay-ID: 23E84B2A-E758-11E2-B4B1-E84251E3A03C-48001098!b-pb-sasl-quonix.pobox.com X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jul 2013 22:54:14 -0000 --pgp-sign-Multipart_Mon_Jul__8_10:54:09_2013-1 Content-Type: text/plain; charset=US-ASCII Hi All, I've just completed a round of NFS testing on FreeBSD 9.1 on AWS. The underlying file system is ZFS. I have a real nfs killer test: doing an "svn update" of a directory of 3541 files. Performance on the NFS server itself is good: checking this out takes 11 seconds (doing the same on a not really comparable Linux NFS server is 43 seconds). Doing this on a client writing to an NFS mounted home directory is however terrible. Really terrible. This takes 25 minutes! With "sync=disable" it's 16 minutes. (doing this against the underpowered Linux NFS server is about 4.5 minutes). The problem might be that the NFS server (nfsd) runs at 70-80% CPU. So my writing speed is cpu bound (go figure). Repeating this with nfs4 + udp: doesn't work at all, get input/output error after a few files. nfs3: 1m55s (nfsd cpu is in the 6%-8% range) Varying tcp/udp, lock/nolock, sync/async, or enabling zfs sync=standard doesn't make any particular difference. All timing were within the 1m49s - 2m3s range. So what's up with NFS4 on FreeBSD? -- All the best, Berend de Boer ------------------------------------------------------ Awesome Drupal hosting: https://www.xplainhosting.com/ --pgp-sign-Multipart_Mon_Jul__8_10:54:09_2013-1 Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit Content-Description: OpenPGP Digital Signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iQIcBAABCAAGBQJR2fGRAAoJEKOfeD48G3g5NBsP/2p6Di1zrqbEtUASd81QfoaM 598UTjg44oDr7HZor02JiTbwdTtVpvN94sEb8jZa5ZNhcLnOvL6uBdzTSS+Bz7Tb X+ZQte2qfjtFLShBb8IwWt+L1hpVYiT/Hxg37hgnHtzzuW4Z3bR2mNUtAi5Bjdv6 HPNpZ0tjTY60huCLMI9tR67VBcGeo2yCBf/k32oM/wGoQoZQLgZlAxUdpk52KiQ1 jxeCY/fgjXyHA0MlC5H+q3e55Q9pS+sDK+0c6op3Z3QbCGstncWXSBPtWxWHMwuq Fg+rhaM8A1tbAEMIUHgN2pI8YqTrGOokrG2MH4YHPy+i6QwupZr5PksbGE6VW+EK 3k8v8rxS9t7DErBZWZjQMclaFnswcVg95v7gCCyrwS2LCPEv+y6nvGQvKeyLnkp8 Vdq+WaEoPrOhR/Wv2dmHStO+9628t0A8aNCoUXjGUV+w9yg+Sf2RL33AwY++6pEX frHzSIts+UjBf/H/l4FLY5OC/6TG+pnMG7pEWeNv069HR5NQ564rEMPnPLB++kqo ySOicFdudetqCeS8gsF8LE2Z2SwT+U/mOwnDyUq/ukpclu9biMIwRauA6NpNLIaR ywjZsVnTA9Ip4NlmV1dCYN3EPWx6zAroiXctAlshEfm4Ud6Tu3ubvj0LbJuCI7v2 KKnIZiqgv+F9AWaswJhL =gJek -----END PGP SIGNATURE----- --pgp-sign-Multipart_Mon_Jul__8_10:54:09_2013-1--