Date: Fri, 26 Apr 2013 19:56:37 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: "Marc G. Fournier" <scrappy@hub.org> Cc: freebsd-fs@freebsd.org Subject: Re: Initial NFS Test: Linux vs FreeBSD (769% slower) Message-ID: <1904620729.1184886.1367020597750.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <CB53127E-9D0D-422A-924E-DD1C0D341646@hub.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Marc G. Fournier wrote: > As an appendum =E2=80=A6 I found this post > http://osdir.com/ml/freebsd-fs/2012-10/msg00156.html that talked about > mounting with the oldnfs option, so tried that out =E2=80=A6 same speed = =E2=80=A6 >=20 >=20 > On 2013-04-26, at 09:19 , Marc G. Fournier <scrappy@hub.org> wrote: >=20 > > > > Okay, I just bit the bullet and installed a Linux/Centos 6.4 system > > for testing purposes. > > > > All hardware is *identical*, except that the FreeBSD server has the > > nicer Intel card in it =E2=80=A6 the Linux is just using the onboard > > Broadcom. Everything else is the same =E2=80=A6 same CPUs, same amount = of > > memory, same switch, same netapp =E2=80=A6 the netapp only has one file= r > > head, so its the same head. I created two separate volumes, but both > > are on the same aggregate and the same size. > > > > OSs: > > > > FreeBSD 9.1-STABLE #2: Thu Apr 25 18:18:18 UTC 2013 > > =09- custom kernel, I try to compile everything I need in, what I > > =09missed: > > > > vs > > > > Linux server03.openstudent.ca 2.6.32-358.6.1.el6.x86_64 #1 SMP Tue > > Apr 23 19:29:00 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux > > =09- CentOS 6.4 + 'yum update' > > > > > > Everything is on the local drive *except* that I put > > /usr/local/jboss on the NFS drive, and there is no virtualization > > happening anywhere =E2=80=A6 everything is on the base server OS, and > > nothing else is running on either of the machines. > > > > > > FreeBSD start time for jBoss .. 270731ms (768% *slower*) > > Linux start time for jBoss =E2=80=A6.=E2=80=A6. 35218 ms > > One thing is that, if you write an executable file to a Linux NFS mount point and then run it, without unmounting/remounting, I don't think it will page it in (the write will cache the pages). If you didn't unmount/remount between writing jboss to the server and timing the startup of it, please try it again after doing a dismount/mount. (Doing the dismount/mount on the Linux client resulted in the same # of reads as FreeBSD for a quick test I did, instead of none without the dismount/remount.) A few other things to do: - Time multiple startups after doing a mount, to see if it only the first one that is slow. - Capture the RPC counts for both clients by doing "nfsstat -c" before and after the startup. If the above doesn't give you any good hints w.r.t. why it is slow, you can capture packets during the startup for both clients and look at them in wireshark, to try and figure out what the difference between the Linux and FreeBSD clients are for this case. rick > > Even hitting ctl-c after the test to stop the process is 2x faster: > > > > FreeBSD: 4207ms (169% slower) > > Linux: 2486ms > > > > Note that both systems have an equal lack of tuning =E2=80=A6 I wouldn'= t > > even know *where* to start tuning Linux =E2=80=A6 > > > > Here is nfsstat -m for the two systems =E2=80=A6 both using nfsv3, same > > rsize/wsize values, all default out of the box ... > > > > FreeBSD: > > 192.168.1.1:/vol/freebsd_jboss on /usr/local/jboss-as-7.1.1.Final > > nfsv3,tcp,resvport,hard,cto,lockd,sec=3Dsys,acdirmin=3D3,acdirmax=3D60,= acregmin=3D5,acregmax=3D60,nametimeo=3D60,negnametimeo=3D60,rsize=3D65536,w= size=3D65536,readdirsize=3D32768,readahead=3D1,wcommitsize=3D5175966,timeou= t=3D120,retrans=3D2 > > > > Linux: > > /usr/local/jboss-as-7.1.1.Final from 192.168.1.1:/vol/linux_jboss > > Flags: > > rw,relatime,vers=3D3,rsize=3D65536,wsize=3D65536,namlen=3D255,hard,prot= o=3Dtcp,timeo=3D600,retrans=3D2,sec=3Dsys,mountaddr=3D192.168.1.1,mountvers= =3D3,mountport=3D4046,mountproto=3Dudp,local_lock=3Dnone,addr=3D192.168.1.1 > > > > > > Note, I have time on my side here =E2=80=A6 we are doing a limit pilot = over > > the next month, but it isn't until Sep/Oct that this project goes > > live =E2=80=A6 what can we do to narrow down that gap? > > > > Thx > > > > > > > > > > > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to > > "freebsd-fs-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1904620729.1184886.1367020597750.JavaMail.root>