From owner-freebsd-fs@FreeBSD.ORG Fri Apr 26 16:19:57 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3DA5D60D for ; Fri, 26 Apr 2013 16:19:57 +0000 (UTC) (envelope-from scrappy@hub.org) Received: from hub.org (hub.org [200.46.208.146]) by mx1.freebsd.org (Postfix) with ESMTP id EAAB41125 for ; Fri, 26 Apr 2013 16:19:56 +0000 (UTC) Received: from maia.hub.org (unknown [200.46.151.188]) by hub.org (Postfix) with ESMTP id 9CC083966DD; Fri, 26 Apr 2013 13:19:55 -0300 (ADT) Received: from hub.org ([200.46.208.146]) by maia.hub.org (mx1.hub.org [200.46.151.188]) (amavisd-maia, port 10024) with ESMTP id 93186-10; Fri, 26 Apr 2013 16:19:53 +0000 (UTC) Received: from [10.5.250.150] (remote.ilcs.sd63.bc.ca [142.31.148.2]) by hub.org (Postfix) with ESMTPA id BBF543966DC; Fri, 26 Apr 2013 13:19:51 -0300 (ADT) From: Marc G. Fournier Subject: Initial NFS Test: Linux vs FreeBSD (769% slower) Date: Fri, 26 Apr 2013 09:19:50 -0700 Message-Id: <7F65FF30-7BE5-4D64-9450-84C8E7FB20AD@hub.org> To: "freebsd-fs@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2013 16:19:57 -0000 Okay, I just bit the bullet and installed a Linux/Centos 6.4 system for = testing purposes. =20 All hardware is *identical*, except that the FreeBSD server has the = nicer Intel card in it =85 the Linux is just using the onboard Broadcom. = Everything else is the same =85 same CPUs, same amount of memory, same = switch, same netapp =85 the netapp only has one filer head, so its the = same head. I created two separate volumes, but both are on the same = aggregate and the same size. OSs: FreeBSD 9.1-STABLE #2: Thu Apr 25 18:18:18 UTC 2013 - custom kernel, I try to compile everything I need in, what I = missed: vs Linux server03.openstudent.ca 2.6.32-358.6.1.el6.x86_64 #1 SMP Tue Apr = 23 19:29:00 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux - CentOS 6.4 + 'yum update' Everything is on the local drive *except* that I put /usr/local/jboss on = the NFS drive, and there is no virtualization happening anywhere =85 = everything is on the base server OS, and nothing else is running on = either of the machines. FreeBSD start time for jBoss .. 270731ms (768% *slower*) Linux start time for jBoss =85.=85. 35218 ms=20 Even hitting ctl-c after the test to stop the process is 2x faster: FreeBSD: 4207ms (169% slower) Linux: 2486ms Note that both systems have an equal lack of tuning =85 I wouldn't even = know *where* to start tuning Linux =85 Here is nfsstat -m for the two systems =85 both using nfsv3, same = rsize/wsize values, all default out of the box ... FreeBSD: 192.168.1.1:/vol/freebsd_jboss on /usr/local/jboss-as-7.1.1.Final = nfsv3,tcp,resvport,hard,cto,lockd,sec=3Dsys,acdirmin=3D3,acdirmax=3D60,acr= egmin=3D5,acregmax=3D60,nametimeo=3D60,negnametimeo=3D60,rsize=3D65536,wsi= ze=3D65536,readdirsize=3D32768,readahead=3D1,wcommitsize=3D5175966,timeout= =3D120,retrans=3D2 Linux: /usr/local/jboss-as-7.1.1.Final from 192.168.1.1:/vol/linux_jboss Flags: = rw,relatime,vers=3D3,rsize=3D65536,wsize=3D65536,namlen=3D255,hard,proto=3D= tcp,timeo=3D600,retrans=3D2,sec=3Dsys,mountaddr=3D192.168.1.1,mountvers=3D= 3,mountport=3D4046,mountproto=3Dudp,local_lock=3Dnone,addr=3D192.168.1.1 Note, I have time on my side here =85 we are doing a limit pilot over = the next month, but it isn't until Sep/Oct that this project goes live =85= what can we do to narrow down that gap?=20 Thx