From owner-freebsd-fs@FreeBSD.ORG Sat Jul 13 05:48:10 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7BBE9399 for ; Sat, 13 Jul 2013 05:48:10 +0000 (UTC) (envelope-from berend@pobox.com) Received: from smtp.pobox.com (b-pb-sasl-quonix.pobox.com [208.72.237.35]) by mx1.freebsd.org (Postfix) with ESMTP id 397C711FD for ; Sat, 13 Jul 2013 05:48:09 +0000 (UTC) Received: from smtp.pobox.com (unknown [127.0.0.1]) by b-sasl-quonix.pobox.com (Postfix) with ESMTP id E302929059; Sat, 13 Jul 2013 05:48:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=date :message-id:from:to:cc:subject:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=sasl; bh=hO+HMgauLC5l/3vjFK4WU64SqRI=; b=QDgD6ol/U0r6A7SCWEtRmflXKQej Z0JvxHlTqr10YUwng9aaMK+I3J6DIM0fGXCYePI+pPQZkJHpe7l5Bcz0Kw0Dne7I nvPC08EiasiiEwbBz/6oAuBJE1YGaboOoRkokYcZTQy8QeJTjQPDMyh6FJBSdprw C8DGA8kZiRioito= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=date:message-id :from:to:cc:subject:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=sasl; b=gBv09m UJ+dacvAMWomC4/xVsbFLp1A8SMrqxrZj8vdA7jJ3L9dk4DOTc696sh+vb/KKUct SnA8Cb8kIjCgN2ZN1qjp55EHukEIeriL9pRSUFiwu7WgH4H6CrANolkdJrCcpLnh 0QqyeNjNwbQW9i9WbU3bvsOVAmM1iJn0YgHUM= Received: from b-pb-sasl-quonix.pobox.com (unknown [127.0.0.1]) by b-sasl-quonix.pobox.com (Postfix) with ESMTP id DA53729052; Sat, 13 Jul 2013 05:48:07 +0000 (UTC) Received: from bmach.nederware.nl (unknown [27.252.169.66]) by b-sasl-quonix.pobox.com (Postfix) with ESMTPA id 5400D29050; Sat, 13 Jul 2013 05:48:07 +0000 (UTC) Received: from quadrio.nederware.nl (quadrio.nederware.nl [192.168.33.13]) by bmach.nederware.nl (Postfix) with ESMTP id 228725C57; Sat, 13 Jul 2013 17:47:59 +1200 (NZST) Received: from quadrio.nederware.nl (quadrio.nederware.nl [127.0.0.1]) by quadrio.nederware.nl (Postfix) with ESMTP id 57B4A4355BBC; Sat, 13 Jul 2013 17:48:02 +1200 (NZST) Date: Sat, 13 Jul 2013 17:48:02 +1200 Message-ID: <8761wfvwml.wl%berend@pobox.com> From: Berend de Boer To: Rick Macklem Subject: Re: Terrible NFS4 performance: FreeBSD 9.1 + UFS/ZFS + AWS EC2 In-Reply-To: <818900293.3878290.1373413081112.JavaMail.root@uoguelph.ca> References: <877gh0yvhm.wl%berend@pobox.com> <818900293.3878290.1373413081112.JavaMail.root@uoguelph.ca> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/24.3 (i686-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: Xplain Technology Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/signed; boundary="pgp-sign-Multipart_Sat_Jul_13_17:48:02_2013-1"; micalg=pgp-sha256; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit X-Pobox-Relay-ID: CAFDDF08-EB7F-11E2-A8CD-E84251E3A03C-48001098!b-pb-sasl-quonix.pobox.com Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jul 2013 05:48:10 -0000 --pgp-sign-Multipart_Sat_Jul_13_17:48:02_2013-1 Content-Type: text/plain; charset=US-ASCII >>>>> "Rick" == Rick Macklem writes: Rick> If you could do some testing where you export a UFS volume, Rick> the results might help to isolate the issue to ZFS vs nfsd. Indeed! Have changed subject, as indeed ZFS is a red herring. Issue shows up with UFS as well. Very high cpu for nfds, about the same time to do the operation. Tried it with these settings: vfs.nfsd.tcphighwater=5000 vfs.nfsd.tcpcachetimeo=300 nfs_server_flags="-u -t -n 256" On nfs3 + ufs everything back to normal. I.e. nfs4 was about 15 minutes, same operation was 241s minutes with nfs3 and nfsd using no cpu at all basically. Per other reply, tried this too: vfs.nfsd.issue_delegations=1 Locks up the client at first write access. Ctrl+C doesn't work, need to explicitly send a KILL signal from other terminal. I think it locks up the server in some way as well. Doing an ls on an exported path locks up. Ctrl+C won't work anymore. Process doesn't react to any signal. In the end I rebooted the server to get rid of this. -- All the best, Berend de Boer ------------------------------------------------------ Awesome Drupal hosting: https://www.xplainhosting.com/ --pgp-sign-Multipart_Sat_Jul_13_17:48:02_2013-1 Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit Content-Description: OpenPGP Digital Signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iQIcBAABCAAGBQJR4OoSAAoJEKOfeD48G3g5dvkP/jXVqkXYVWlbpdrw5LLOkLua QQ55z7jvPzQ4oRGCZXAhgX2Xy3h9T2rQcbO6jYWQ+XM93aziBEi6n0nOGeRgQIm/ n/E6s9Vrdy1kaowL5DsdmovYweqnjEEPI0hH4rp5XfJVUCUwkFzZhNeXQ6uvlmmV jtAUsrcrS5k/xJ1vJ8dANcacFkKKGOQ51/ctNiGEK6j29yk6/fmXHEtnDZPm/Bs5 YINoGpP0QoUegb0SzsQI/7TY1e1ezAvaMZenmQHhKCdNpwyz4AP9XxnIFPUPITFf dKamuwxHT5RXcHCJYD5G/wYvuBDHY25pGp6bss5vvCaULNnyTZ4M5XFhYnT8OG47 q0vq44LjU7tzFuXFWovh/nEHkvZjBP82AHDD2d7nn974ju6RSrE4t5HnVeUKLa0Y m9nyPBOZL6CewiBKN8y7Tyn1SgWv8Vw8xeenoVleNcjaWb89dXTmA1m1GpR+pbrH M9VYSVoMYh9EalSSrPPlkiqtEB61tqNTrvKlykjewkQ0N2IWIM976URyvtvy621p 5wpGXU0pcKudhJXI5laXv3LbGPwye7+ipk64CIQRfjnCXKkjZCWLuQNvbw6RWj95 k8HPJc2EZa+m/XCkA06qrS3UMDAXG7RVwajW2jeV+LbgAHuPcvglRdLeh7C1zHBQ qa+wHBP6pcZ4TWBjKqN9 =AehI -----END PGP SIGNATURE----- --pgp-sign-Multipart_Sat_Jul_13_17:48:02_2013-1--