Date: Fri, 10 May 2013 20:32:46 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: "Marc G. Fournier" <scrappy@hub.org> Cc: freebsd-fs@freebsd.org Subject: Re: NFS Performance issue against NetApp Message-ID: <968416157.282645.1368232366317.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <030E4A04-D597-49BD-8979-27C3EFB6D276@hub.org>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_282644_1724368082.1368232366315 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Marc G. Fournier wrote: > FYI =E2=80=A6 I just installed Solaris 11 onto the same hardware and ran = the > same test =E2=80=A6 so far, I'm seeing: >=20 > Linux @ ~30s > Solaris @ ~44s >=20 > OpenBSD @ ~200s > FreeBSD @ ~240s >=20 > I've even tried FreeBSD 8.3 just to see if maybe its as 'newish' issue > =E2=80=A6 same as 9.x =E2=80=A6 I could see Linux 'cutting corners', but > Oracle/Solaris too =E2=80=A6 ? >=20 The three client implementations (BSD, Linux, Solaris) were developed independently and, as such, will all implement somewaht different caching algorithms (the RFCs specify what goes on the wire, but say little w.r.t. client side caching). I have a attached a patch that might be useful for determining if the client side buffer cache consistency algorithm in FreeBSD is causing the slow startup of jboss. Do not run this patch on a production system, since it pretty well disables all buffer cache coherency (ie. if another client modifies a file, the patched client won't notice and will continue to cache stale file data). If the patch does speed up startup of jboss significantly, you can use the sysctl: vfs.nfs.noconsist to check for which coherency check is involved by decreasing the value for the sysctl by 1 and then trying a startup again. (When vfs.nfs.noconsist=3D0, normal cache coherency will be applied.) I have no idea if buffer cache coherency is a factor, but trying the attached patch might determine if it is. Note that you have never posted updated "nfsstat -c" values. (Remember that what you posted indicated 88 RPCs, which seemed bogus.) Finding out if FreeBSD does a lot more of certain RPCs that Linux/Solaris might help isolate what is going on. rick > On 2013-05-03, at 04:50 , Mark Felder <feld@feld.me> wrote: >=20 > > On Thu, 02 May 2013 18:43:17 -0500, Marc G. Fournier > > <scrappy@hub.org> wrote: > > > >> Hadn't thought to do so with Linux, but =E2=80=A6 > >> Linux =E2=80=A6=E2=80=A6. 20732ms, 20117ms, 20935ms, 20130ms, 20560ms > >> FreeBSD .. 28996ms, 24794ms, 24702ms, 23311ms, 24153ms > > > > Please make sure both platforms are using similar atime settings. I > > think most distros use ext4 with diratime by default. I'd just do > > noatime on both platforms to be safe. > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to > > "freebsd-fs-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" ------=_Part_282644_1724368082.1368232366315 Content-Type: text/x-patch; name=trynoconsist.patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=trynoconsist.patch LS0tIGZzL25mc2NsaWVudC9uZnNfY2x2bm9wcy5jLnNhdgkyMDEzLTA1LTEwIDE4OjMxOjAxLjAw MDAwMDAwMCAtMDQwMAorKysgZnMvbmZzY2xpZW50L25mc19jbHZub3BzLmMJMjAxMy0wNS0xMCAx OTozODo0NS4wMDAwMDAwMDAgLTA0MDAKQEAgLTI0Niw2ICsyNDYsMTAgQEAgaW50IG5mc19rZWVw X2RpcnR5X29uX2Vycm9yOwogU1lTQ1RMX0lOVChfdmZzX25mcywgT0lEX0FVVE8sIG5mc19rZWVw X2RpcnR5X29uX2Vycm9yLCBDVExGTEFHX1JXLAogICAgICZuZnNfa2VlcF9kaXJ0eV9vbl9lcnJv ciwgMCwgIlJldHJ5IHBhZ2VvdXQgaWYgZXJyb3IgcmV0dXJuZWQiKTsKIAoraW50IG5mc2NsX25v Y29uc2lzdCA9IDM7CitTWVNDVExfSU5UKF92ZnNfbmZzLCBPSURfQVVUTywgbm9jb25zaXN0LCBD VExGTEFHX1JXLAorICAgICZuZnNjbF9ub2NvbnNpc3QsIDAsICJUcnkgZGlzYWJsaW5nIGNhY2hl IGNvbnNpc3RlbmN5Iik7CisKIC8qCiAgKiBUaGlzIHN5c2N0bCBhbGxvd3Mgb3RoZXIgcHJvY2Vz c2VzIHRvIG1tYXAgYSBmaWxlIHRoYXQgaGFzIGJlZW4gb3BlbmVkCiAgKiBPX0RJUkVDVCBieSBh IHByb2Nlc3MuICBJbiBnZW5lcmFsLCBoYXZpbmcgcHJvY2Vzc2VzIG1tYXAgdGhlIGZpbGUgd2hp bGUKQEAgLTUzOCw2ICs1NDIsNyBAQCBuZnNfb3BlbihzdHJ1Y3Qgdm9wX29wZW5fYXJncyAqYXAp CiAJICovCiAJbXR4X2xvY2soJm5wLT5uX210eCk7CiAJaWYgKG5wLT5uX2ZsYWcgJiBOTU9ESUZJ RUQpIHsKKwkgICBpZiAobmZzY2xfbm9jb25zaXN0IDwgMikgewogCQltdHhfdW5sb2NrKCZucC0+ bl9tdHgpOwogCQllcnJvciA9IG5jbF92aW52YWxidWYodnAsIFZfU0FWRSwgYXAtPmFfdGQsIDEp OwogCQlpZiAoZXJyb3IgPT0gRUlOVFIgfHwgZXJyb3IgPT0gRUlPKSB7CkBAIC01NjEsNiArNTY2 LDcgQEAgbmZzX29wZW4oc3RydWN0IHZvcF9vcGVuX2FyZ3MgKmFwKQogCQlucC0+bl9tdGltZSA9 IHZhdHRyLnZhX210aW1lOwogCQlpZiAoTkZTX0lTVjQodnApKQogCQkJbnAtPm5fY2hhbmdlID0g dmF0dHIudmFfZmlsZXJldjsKKwkgICAgfQogCX0gZWxzZSB7CiAJCW10eF91bmxvY2soJm5wLT5u X210eCk7CiAJCWVycm9yID0gVk9QX0dFVEFUVFIodnAsICZ2YXR0ciwgYXAtPmFfY3JlZCk7CkBA IC01NzAsOCArNTc2LDkgQEAgbmZzX29wZW4oc3RydWN0IHZvcF9vcGVuX2FyZ3MgKmFwKQogCQkJ cmV0dXJuIChlcnJvcik7CiAJCX0KIAkJbXR4X2xvY2soJm5wLT5uX210eCk7Ci0JCWlmICgoTkZT X0lTVjQodnApICYmIG5wLT5uX2NoYW5nZSAhPSB2YXR0ci52YV9maWxlcmV2KSB8fAotCQkgICAg TkZTX1RJTUVTUEVDX0NPTVBBUkUoJm5wLT5uX210aW1lLCAmdmF0dHIudmFfbXRpbWUpKSB7CisJ CWlmICgoKE5GU19JU1Y0KHZwKSAmJiBucC0+bl9jaGFuZ2UgIT0gdmF0dHIudmFfZmlsZXJldikg fHwKKwkJICAgIE5GU19USU1FU1BFQ19DT01QQVJFKCZucC0+bl9tdGltZSwgJnZhdHRyLnZhX210 aW1lKSkgJiYKKwkJICAgIG5mc2NsX25vY29uc2lzdCA8IDEpIHsKIAkJCWlmICh2cC0+dl90eXBl ID09IFZESVIpCiAJCQkJbnAtPm5fZGlyZW9mb2Zmc2V0ID0gMDsKIAkJCW10eF91bmxvY2soJm5w LT5uX210eCk7Ci0tLSBmcy9uZnNjbGllbnQvbmZzX2NsYmlvLmMuc2F2CTIwMTMtMDUtMTAgMTg6 MzQ6MjQuMDAwMDAwMDAwIC0wNDAwCisrKyBmcy9uZnNjbGllbnQvbmZzX2NsYmlvLmMJMjAxMy0w NS0xMCAxOTozODo1Ny4wMDAwMDAwMDAgLTA0MDAKQEAgLTY5LDYgKzY5LDcgQEAgZXh0ZXJuIGVu dW0gbmZzaW9kX3N0YXRlIG5jbF9pb2R3YW50W05GUwogZXh0ZXJuIHN0cnVjdCBuZnNtb3VudCAq bmNsX2lvZG1vdW50W05GU19NQVhBU1lOQ0RBRU1PTl07CiBleHRlcm4gaW50IG5ld25mc19kaXJl Y3Rpb19lbmFibGU7CiBleHRlcm4gaW50IG5mc19rZWVwX2RpcnR5X29uX2Vycm9yOworZXh0ZXJu IGludCBuZnNjbF9ub2NvbnNpc3Q7CiAKIGludCBuY2xfcGJ1Zl9mcmVlY250ID0gLTE7CS8qIHN0 YXJ0IG91dCB1bmxpbWl0ZWQgKi8KIApAQCAtNDAyLDcgKzQwMyw4IEBAIG5mc19iaW9yZWFkX2No ZWNrX2NvbnMoc3RydWN0IHZub2RlICp2cCwKIAkJCXJldHVybiAoZXJyb3IpOwogCQltdHhfbG9j aygmbnAtPm5fbXR4KTsKIAkJaWYgKChucC0+bl9mbGFnICYgTlNJWkVDSEFOR0VEKQotCQkgICAg fHwgKE5GU19USU1FU1BFQ19DT01QQVJFKCZucC0+bl9tdGltZSwgJnZhdHRyLnZhX210aW1lKSkp IHsKKwkJICAgIHx8IChORlNfVElNRVNQRUNfQ09NUEFSRSgmbnAtPm5fbXRpbWUsICZ2YXR0ci52 YV9tdGltZSkgJiYKKwkJICAgIG5mc2NsX25vY29uc2lzdCA8IDMpKSB7CiAJCQltdHhfdW5sb2Nr KCZucC0+bl9tdHgpOwogCQkJaWYgKHZwLT52X3R5cGUgPT0gVkRJUikKIAkJCQluY2xfaW52YWxk aXIodnApOwo= ------=_Part_282644_1724368082.1368232366315--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?968416157.282645.1368232366317.JavaMail.root>