From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 21:30:30 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28C6D106564A; Sat, 20 Mar 2010 21:30:30 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (outn.internet-mail-service.net [216.240.47.237]) by mx1.freebsd.org (Postfix) with ESMTP id 083708FC16; Sat, 20 Mar 2010 21:30:29 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o2KLUTkH008746; Sat, 20 Mar 2010 14:30:29 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 9EC772D6018; Sat, 20 Mar 2010 14:30:28 -0700 (PDT) Message-ID: <4BA53E77.8010208@elischer.org> Date: Sat, 20 Mar 2010 14:30:31 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: Ivan Voras References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 21:30:30 -0000 Ivan Voras wrote: > Julian Elischer wrote: >> Alexander Motin wrote: >>> Scott Long wrote: >>>> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>>>> Diminishing returns get hit pretty quickly with larger MAXPHYS >>>>> values. >>>>> As long as the I/O can be pipelined the reduced transaction rate >>>>> becomes less interesting when the transaction rate is less than a >>>>> certain level. Off the cuff I'd say 2000 tps is a good basis for >>>>> considering whether it is an issue or not. 256K is actually quite >>>>> a reasonable value. Even 128K is reasonable. >>>> I agree completely. I did quite a bit of testing on this in 2008 >>>> and 2009. >>>> I even added some hooks into CAM to support this, and I thought that >>>> I had >>>> discussed this extensively with Alexander at the time. Guess it was >>>> yet another >>>> wasted conversation with him =-( I'll repeat it here for the record. >> >> In the Fusion-io driver we find that the limiting factor is not the >> size of MAXPHYS, but the fact that we can not push more than >> 170k tps through geom. (in my test machine. I've seen more on some >> beefier machines), but that is only a limit on small transacrtions, > > Do the GEOM threads (g_up, g_down) go into saturation? Effectively all > IO is serialized through them. basically.. You can get better throughput by using TSC for timing because the geom and devstat code does a bit of timing.. Geom can be told to turn off it's timing but devstat can't. The 170 ktps is with TSC as timer, and geom timing turned off. It could just be the shear weight of the work being done. Linux on the same machine using the same driver code (with different wrappers) gets 225k tps. > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"