From owner-freebsd-fs@FreeBSD.ORG Sun Sep 12 22:16:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F8E0106566C for ; Sun, 12 Sep 2010 22:16:21 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from out2.smtp.messagingengine.com (out2.smtp.messagingengine.com [66.111.4.26]) by mx1.freebsd.org (Postfix) with ESMTP id 672FF8FC12 for ; Sun, 12 Sep 2010 22:16:21 +0000 (UTC) Received: from compute1.internal (compute1.nyi.mail.srv.osa [10.202.2.41]) by gateway1.messagingengine.com (Postfix) with ESMTP id DDD212C1; Sun, 12 Sep 2010 18:16:20 -0400 (EDT) Received: from frontend2.messagingengine.com ([10.202.2.161]) by compute1.internal (MEProxy); Sun, 12 Sep 2010 18:16:20 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=messagingengine.com; h=from:to:subject:date:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:message-id; s=smtpout; bh=kzMF+HTMttRM6TLvY2TsI3WoWSw=; b=HnWCxXBTpREEASvKcuQzDnzZ+AnyMBhXCNomFYKYuR3QDV14wlwfnPHtgAXuL0BEvbzdytEyRCuy/cSjsUplATFjfDmGZffy1WSBFvUxKhPEUuyNr6D5o/7yNZ0ZcIC4zQU2YnczUY2jfb7qPQdnCj6E1/JzMWJsieh0kDV7spo= X-Sasl-enc: B//KrRj8QOlAg5LLLTTARIWJrCuCJh/Z6QargQ0Nt0Pm 1284329780 Received: from tcbug.ixsystems.com (173-123-10-0.pools.spcsdns.net [173.123.10.0]) by mail.messagingengine.com (Postfix) with ESMTPSA id 11CCB5E6CE8; Sun, 12 Sep 2010 18:16:20 -0400 (EDT) From: Josh Paetzel To: freebsd-fs@freebsd.org Date: Sun, 12 Sep 2010 17:15:51 -0500 User-Agent: KMail/1.13.5 (FreeBSD/9.0-CURRENT; KDE/4.5.1; amd64; ; ) References: <954605288.782335.1284305288639.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <954605288.782335.1284305288639.JavaMail.root@erie.cs.uoguelph.ca> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3409528.XkxCAgStvQ"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201009121716.17813.josh@tcbug.org> Cc: Terry Kennedy Subject: Re: Weird Linux - FreeBSD/ZFS NFSv4 interoperability problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 22:16:21 -0000 --nextPart3409528.XkxCAgStvQ Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Sunday 12 September 2010 10:28:08 Rick Macklem wrote: > > > A couple of people have reported very slow read rates for the NFSv4 > > > client (actually the experimental client, since they see it for > > > NFSv3 too). If you could easily do the following, using a FreeBSD8.1 > > > or newer client: > > > # mount -t nfs -o nfsv4 :/path > > > - cd to anywhere in the mount that has a 100Mbyte+ file > > > # dd if=3D<100Mbyte+ file> of=3D/dev/null bs=3D1m > > >=20 > > > and then report what read rate you see along with the client's > > > machine-arch/# of cores/ram size/network driver used by the mount > > >=20 > > > rick > > > ps: Btw, anyone else who can do this test, it would be appreciated. > > >=20 > > > If you aren't set up for NFSv4, you can do an NFSv3 mount using > > > the exp. client instead. > > > # mount -t newnfs -o nfsv3 :/path > >=20 > > On 8-STABLE (both client and server). First test is NFSv3 on the > > standard > > client: > >=20 > > (0:842) new-gate:~terry# mount -t nfs -o nfsv4 new-rz1:/data /foo > > [tcp6] new-rz1:/data: NFSPROC_NULL: RPC: Program/version mismatch; low > > version =3D 2, high version =3D 3 > > [tcp] new-rz1:/data: NFSPROC_NULL: RPC: Program/version mismatch; low > > version =3D 2, high version =3D 3 > >=20 > > ^C > > (1:843) new-gate:~terry# mount -t nfs -o nfsv3 new-rz1:/data /foo > > [...] > > (0:869) new-gate:/foo/Backups/Suzanne VAIO# dd if=3D0cff3d7b_VOL.spf > > of=3D/dev/null bs=3D1m > > 6010+1 records in > > 6010+1 records out > > 6301945344 bytes transferred in 69.730064 secs (90376302 bytes/sec) > >=20 > > Now, let's try the newnfs client (cache should have been primed by the >=20 > > first run, so we'd expect this to be faster): > Just thought I'd mention that, since it is a different mount, the caches > won't be primed, which is good, because that would mask differences. >=20 > > (0:879) new-gate:/tmp# umount /foo > > (0:880) new-gate:/tmp# mount -t newnfs -o nfsv3 new-rz1:/data /foo > > (0:881) new-gate:/tmp# cd /foo/Backups/Suzanne\ VAIO/ > > (0:882) new-gate:/foo/Backups/Suzanne VAIO# dd if=3D0cff3d7b_VOL.spf > > of=3D/dev/null bs=3D1m > > 6010+1 records in > > 6010+1 records out > > 6301945344 bytes transferred in 135.927222 secs (46362644 bytes/sec) > >=20 > > Hmmm. Half the performance. The problem isn't the disk speed on the >=20 > > server: > Ok, good. You aren't seeing what the two guys reported (they were really > slow, at less than 2Mbytes/sec). If you would like to, you could try the > following, since the two clients use different default r/w sizes. >=20 > # mount -t newnfs -o nfsv3,rsize=3D32768,wsize=3D32768 new-rz1:/data /foo >=20 > and see how it changes the read rate. I don't know why there is a > factor of 2 difference (if it isn't the different r/w size), but it > will probably get resolved as I bring the experimental client up to date. >=20 > Thanks a lot for doing the test and giving me a data point, rick root@jester1d / ->mount -t nfs -o wsize=3D65536,rsize=3D65536=20 servant.ixsystems.com:/a/isos /mnt root@jester1d / ->cd /mnt root@jester1d /mnt ->dd if=3DPCBSD8-STABLE-20100420-x64-DVD.iso of=3D/dev/n= ull=20 bs=3D1m 3344+1 records in 3344+1 records out 3507386368 bytes transferred in 34.562502 secs (101479528 bytes/sec) root@jester1d /mnt ->cd .. root@jester1d / ->umount /mnt root@jester1d / ->mount -t newnfs -o nfsv3,rsize=3D65536,wsize=3D65536=20 servant.ixsystems.com:/a/isos /mnt root@jester1d / ->cd /mnt root@jester1d /mnt ->dd if=3DPCBSD8-STABLE-20100420-x64-DVD.iso of=3D/dev/n= ull=20 bs=3D1m 345+0 records in 345+0 records out 361758720 bytes transferred in 46.191718 secs (7831679 bytes/sec) The first run hits network limits. Both machines are nehalems, intel NICs, I can give details if needed. =2D-=20 Thanks, Josh Paetzel --nextPart3409528.XkxCAgStvQ Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAABAgAGBQJMjVExAAoJEKFq1/n1feG2IxkIALQxDI0wfgMcj72oKxqEI4IQ 01vae+HcO8jO1alBtUR9bJs1e2EepVYfGw+IhBzYV0tZDo5GMw1csoOqHJtmH6hP EeyV3bOO4wTjNxwbahNLv6UHC+OVgjNcDcDZIbUeOqTGEf/cLZmEa4bBYcyx0wIu WIzsjVr0Etjek8GUpkmm0bVmok7huP5LY/I8rfoRjSGNK9PGQM3GL+6RCYcXpdGm Gh7XsIuUa0dSNsCS2egnR3qFLVr+bKFTIe/njTjsCrZ9byqdlbgS7kRsb4FpWDaS vGgJHzurPX7D5dSmIxVIvZygVogwrHJupLUSJjAXJ+CRSMgS6g4NyXfzmanS1Zg= =G8FU -----END PGP SIGNATURE----- --nextPart3409528.XkxCAgStvQ--