From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 17:10:52 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D9AC1065670; Sun, 21 Mar 2010 17:10:52 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id EED398FC1E; Sun, 21 Mar 2010 17:10:51 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2LHAmOJ037217; Sun, 21 Mar 2010 11:10:48 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=iso-8859-1 From: Scott Long In-Reply-To: <20100321165316.GU99813@acme.spoerlein.net> Date: Sun, 21 Mar 2010 11:10:48 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <574BA80D-61C3-4E3A-A5D3-898ABC605AED@samsco.org> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <20100321163051.GT99813@acme.spoerlein.net> <9524C333-F191-4F7A-A5FA-BD52498169C0@samsco.org> <20100321165316.GU99813@acme.spoerlein.net> To: =?iso-8859-1?Q?Ulrich_Sp=F6rlein?= X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 17:10:52 -0000 On Mar 21, 2010, at 10:53 AM, Ulrich Sp=F6rlein wrote: > [CC trimmed] > On Sun, 21.03.2010 at 10:39:10 -0600, Scott Long wrote: >> On Mar 21, 2010, at 10:30 AM, Ulrich Sp=F6rlein wrote: >>> On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: >>>> Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of = an >>>> odd number less than 512k. For the purpose of benchmarking against = these >>>> OS's, having comparable capabilities is essential; Linux easily = beats FreeBSD >>>> in the silly-i/o-test because of the MAXPHYS difference (though = FreeBSD typically >>>> stomps linux in real I/O because of vastly better latency and = caching algorithms). >>>> I'm fine with raising MAXPHYS in production once the problems are = addressed. >>>=20 >>> Hi Scott, >>>=20 >>> while I'm sure that most of the FreeBSD admins are aware of "silly" >>> benchmarks where Linux I/O seems to dwarf FreeBSD, do you have some >>> pointers regarding your statement that FreeBSD triumphs for = real-world >>> I/O loads? Can this be simulated using iozone, bonnie, etc? More >>> importantly, is there a way to do this file system independently? >>>=20 >>=20 >> iozone and bonnie tend to be good at testing serialized I/O latency; = each read and write is serialized without any buffering. My experience = is that they give mixed results, sometimes they favor freebsd, sometime = linux, sometimes it's a wash, all because they are so sensitive to = latency. And that's where is also gets hard to have a "universal" = benchmark; what are you really trying to model, and how does that model = reflect your actual workload? Are you running a single-instance, single = threaded application that is sensitive to latency? Are you running a = multi-instance/multi-threaded app that is sensitive to bandwidth? Are = you operating on a single file, or on a large tree of files, or on a raw = device? Are you sharing a small number of relatively stable file = descriptors, or constantly creating and deleting files and truncating = space? >=20 > All true, that's why I wanted to know from you, which real world > situations you encountered where FreeBSD did/does outperform Linux in > regards to I/O throughput and/or latency (depending on scenario, of > course). I have some tests that spawn N number of threads and then do sequential = and random i/o either into a filesystem or a raw disk. FreeBSD gets = more work done with fewer I/O's than linux when you're operating through = the filesystem, thanks to softupdates and the block layer. Linux has a = predictive cache that often times will generate too much i/o in a vain = attempt to aggressively prefetch blocks. So even then it's hard to = measure in a simple way; linux will do more i/o, but less of it will be = useful to the application, thereby increasing latency and increasing = application runtime. Sorry I can't be more specific, but you're asking = for something that I explicitly say I can't provide. Scott