From owner-freebsd-fs@FreeBSD.ORG Tue Jul 9 23:57:55 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 002A64B7 for ; Tue, 9 Jul 2013 23:57:54 +0000 (UTC) (envelope-from berend@pobox.com) Received: from smtp.pobox.com (b-pb-sasl-quonix.pobox.com [208.72.237.35]) by mx1.freebsd.org (Postfix) with ESMTP id B2F0D17AF for ; Tue, 9 Jul 2013 23:57:54 +0000 (UTC) Received: from smtp.pobox.com (unknown [127.0.0.1]) by b-sasl-quonix.pobox.com (Postfix) with ESMTP id D97F62F4C1; Tue, 9 Jul 2013 23:57:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=date :message-id:from:to:cc:subject:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=sasl; bh=JZE5LshtvPP2EyFZbH0IKLyIyqc=; b=eLmbrYZuCC1pWXd9HoStx3t8IZEI 2M2Iwd+J2WW6iQoGexJ/Su8c4hubk2TiOw9kwYrk96jbvRf6H9jPjG3qnjntSBbO XyWaCzKwkZZQk2Uryz9wvFzpoGICbOV5EJsY4ueb4Ezo4m9pQwBpMsc+eTSV0zWK FPDg6PeVffi6gCQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=date:message-id :from:to:cc:subject:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=sasl; b=uUGM1k 639Y6UrC8sHNj7Z5GtH7Mzp25K99ELUfs4ULfmbLtl3izv5FGiyjQNF5FUgezlU0 RZCbPxXgfzPtCjHBw0Z6MxyDBfa7oe5sFw+o5YbDkw7P921KJmAoARqA5lI6SOo0 0UsvgqSekBT2t/+zgOpOf4KMpNuldi7WLwdzY= Received: from b-pb-sasl-quonix.pobox.com (unknown [127.0.0.1]) by b-sasl-quonix.pobox.com (Postfix) with ESMTP id CF9422F4BF; Tue, 9 Jul 2013 23:57:47 +0000 (UTC) Received: from bmach.nederware.nl (unknown [27.252.169.66]) by b-sasl-quonix.pobox.com (Postfix) with ESMTPA id 56EFA2F4BE; Tue, 9 Jul 2013 23:57:47 +0000 (UTC) Received: from quadrio.nederware.nl (quadrio.nederware.nl [192.168.33.13]) by bmach.nederware.nl (Postfix) with ESMTP id 8031D5C89; Wed, 10 Jul 2013 11:57:40 +1200 (NZST) Received: from quadrio.nederware.nl (quadrio.nederware.nl [127.0.0.1]) by quadrio.nederware.nl (Postfix) with ESMTP id C74E849FB97C; Wed, 10 Jul 2013 11:57:44 +1200 (NZST) Date: Wed, 10 Jul 2013 11:57:44 +1200 Message-ID: <87k3kzxp53.wl%berend@pobox.com> From: Berend de Boer To: Rick Macklem Subject: Re: Terrible NFS4 performance: FreeBSD 9.1 + ZFS + AWS EC2 In-Reply-To: <818900293.3878290.1373413081112.JavaMail.root@uoguelph.ca> References: <877gh0yvhm.wl%berend@pobox.com> <818900293.3878290.1373413081112.JavaMail.root@uoguelph.ca> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/24.3 (i686-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: Xplain Technology Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/signed; boundary="pgp-sign-Multipart_Wed_Jul_10_11:57:44_2013-1"; micalg=pgp-sha256; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit X-Pobox-Relay-ID: 5ADCB16A-E8F3-11E2-BEBA-E84251E3A03C-48001098!b-pb-sasl-quonix.pobox.com Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jul 2013 23:57:55 -0000 --pgp-sign-Multipart_Wed_Jul_10_11:57:44_2013-1 Content-Type: text/plain; charset=US-ASCII >>>>> "Rick" == Rick Macklem writes: Rick> Hmm, this is interesting. ken@'s file handle affinity patch Rick> works for NFSv3, but not NFSv4. If I understood his posts Rick> correctly, the fh affinity patch was needed, so ZFS's Rick> heuristic for recognizing sequential reading would function Rick> correctly. (A file handle affinity patch for NFSv4 will take Rick> some time, since all RPCs in NFSv4 are compounds, with Rick> reads/writes imbedded in them, along with other ops.) These are all very small files, not much large, so it's not some kind of sequential reading/writing test. Just thousands of small files being written, and a fair amount of reads. Rick> Also, I can't remember if you've bumped up the # of nfsd Rick> threads, but I'd go for 256. nfs_server_flags="-u -t -n Rick> 256" - in your /etc/rc.conf Didn't. But only one client was writing, so I figured that shouldn't matter. -- All the best, Berend de Boer ------------------------------------------------------ Awesome Drupal hosting: https://www.xplainhosting.com/ --pgp-sign-Multipart_Wed_Jul_10_11:57:44_2013-1 Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit Content-Description: OpenPGP Digital Signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iQIcBAABCAAGBQJR3KN4AAoJEKOfeD48G3g576cP/3A0slrBgVVBqJ82XaNTOSd7 oMWOXe5RyK3YxrlbpfjfziGZkeCIVQSp/ip1gqZKABYHaGi+UD0Hw6apZNuyeuDa miHlYLBMijzqh8O1E0rJ72rrbdhlD4t+9J0jCjCwoGIz69WToz1ckUyW/y0iL+lq nkwMzi3JwXo0vO3RtIY4x0DBMyqjNH0LR3ROHLemz7CxoixJibWnRpLnrJO7Zx/y tUQo5l3CI4/EQQ6qCK0y9Hb27MJ3u4fjIJ3XgRpHdsekFDk32JpCzQ2mPeC8XmQx /Ftu59lk+hf5V2e93vmdV7mzLpsrW4IIu1A9WVXsVCkbzxaNX76f7Oo3/6tjvGUV R1JenQUtj5SKhPa4o80KSN24OEdaLhM5XPa1BisCy8un+4lgFZe6OBGLjZB0+MQ8 W1H8tWgZYHcPA4KMWLWWF5uNibZTSp8FzJyg6Z+H+NOwVagvokhdy45Q1UXa0J1q DnJsoQGpWCa6nDCEmLMBg5AezNYVPbLENtGn20uov0yCIMtSp0N4Yv8G6Dt4ntPL pRnkaL3+cmN5lrXT28BqIZBT2VeO5UczbitSpe/kRHbJKCfuox8/Qa2q/B6SFpXh Br+MEAWCkPru/w3mCqs17AhUpAnkoIjOirT+Pa40Hw/dl6GW4CFRynbpZGKkT0IC bFyq80l8ZjjaFsNFq2QZ =g89k -----END PGP SIGNATURE----- --pgp-sign-Multipart_Wed_Jul_10_11:57:44_2013-1--