From owner-freebsd-fs@FreeBSD.ORG Tue May 14 17:12:49 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1BB6CBDA for ; Tue, 14 May 2013 17:12:49 +0000 (UTC) (envelope-from scrappy@hub.org) Received: from hub.org (hub.org [200.46.208.146]) by mx1.freebsd.org (Postfix) with ESMTP id DFAC3B55 for ; Tue, 14 May 2013 17:12:48 +0000 (UTC) Received: from maia.hub.org (unknown [200.46.151.188]) by hub.org (Postfix) with ESMTP id 439B5E79DD8; Tue, 14 May 2013 14:12:42 -0300 (ADT) Received: from hub.org ([200.46.208.146]) by maia.hub.org (mx1.hub.org [200.46.151.188]) (amavisd-maia, port 10024) with ESMTP id 11526-06; Tue, 14 May 2013 17:12:41 +0000 (UTC) Received: from [10.5.250.150] (remote.ilcs.sd63.bc.ca [142.31.148.2]) by hub.org (Postfix) with ESMTPA id 0C720E79DD7; Tue, 14 May 2013 14:12:39 -0300 (ADT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: NFS Performance issue against NetApp From: "Marc G. Fournier" In-Reply-To: <20130513005858.GA73875@icarus.home.lan> Date: Tue, 14 May 2013 10:12:37 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <94661399-66AC-4E83-B39B-0426442BB84C@hub.org> References: <1966772823.291493.1368362883964.JavaMail.root@erie.cs.uoguelph.ca> <5190335D.9090105@hub.org> <20130513005858.GA73875@icarus.home.lan> To: Jeremy Chadwick X-Mailer: Apple Mail (2.1503) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2013 17:12:49 -0000 On 2013-05-12, at 17:58 , Jeremy Chadwick wrote: >=20 >> Why does it take 34x as many reads on FreeBSD, where rsize on both >> Linux/FreeBSD are the same ... ? The amount of data to be read is >> the same ... shouldn't the # of reads be within the same ballpark, >> at least ... ? >=20 > Can you provide actual proof of this, re: that at the syscall level, > JBoss is issuing the same nbytes count to read(2) on BSD as it is on > Linux and Solaris? You can accomplish this with strace on Linux and > ktrace (not truss) on FreeBSD. >=20 > For all we know, JBoss could have some "OS optimisation" setting = within > it that says for BSDs use 512 bytes, while for Linux and/or Solaris = use > 16384 bytes. =20 That would explain a 4x difference .. not a 34x difference =85=20 As for strace, if anyone feels this has merit and can tell me what I = need to run, I'm happy to do so =85=20 > Probably off-topic but worth pointing out: I do not know about = Solaris, > but Linux has multiple layers of caching, and is well-known for doing > things like caching (and aggregating!) reads/writes to **block** = devices > (this is why on Linux you have to make sure to avoid caching your > application use O_DIRECT with open(2) or other mechanisms -- the BSDs = do > not do this, block devices are always non-cached). Caching *should* only come into play after the first run of the = application =85 the first run after a reboot of the server shouldn't = have anything in cache yet for caching to come into play =85 unless = Linux/Solaris have some way of anticipating what application I'm going = to run before me =85 ?