From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 11 18:44:54 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 86F7258F for ; Thu, 11 Apr 2013 18:44:54 +0000 (UTC) (envelope-from cedric.blancher@gmail.com) Received: from mail-ia0-x236.google.com (mail-ia0-x236.google.com [IPv6:2607:f8b0:4001:c02::236]) by mx1.freebsd.org (Postfix) with ESMTP id 585E6128F for ; Thu, 11 Apr 2013 18:44:54 +0000 (UTC) Received: by mail-ia0-f182.google.com with SMTP id u20so1697593iag.41 for ; Thu, 11 Apr 2013 11:44:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=+Fx83oLcqacrtGXJEeDvTYwX0dl7rxxwPirV7pCksZQ=; b=Xtvp12mapAhevBg8gndjf02k7/IV2vaOIoe/S3rnYepazDrbpcVdoKnJTqdYWCWFIh Z4JZB1sq4mSR57mkO+fGlrC3A9KMkCDFVOVPAb5G5HtwfQ2FbIbU1UDh4gTbXYY+ld0t /OhDgUO3HrfCEg+hKRda6zjsPXnNRGJsZ64eexJ4eVSK47ZBPvEuqvopZj86KHOTrJXY 0LYsHIw/Ql8ypcapQm9B/hJcPJ9cFsMoJnbbAAydOfbz5B24zBzJzE+cACKHhUPlEkLf +IKUAlWng4LQt+hGcp3bB7UUPybOue6DrrGxfQHOoxmb6eRXzy8Vz5c+loHLs8L5iKi7 KF6Q== MIME-Version: 1.0 X-Received: by 10.50.22.3 with SMTP id z3mr5323275ige.80.1365705893903; Thu, 11 Apr 2013 11:44:53 -0700 (PDT) Sender: cedric.blancher@gmail.com Received: by 10.50.126.8 with HTTP; Thu, 11 Apr 2013 11:44:53 -0700 (PDT) In-Reply-To: <5165CB4C.4010608@mu.org> References: <201304101006.13960.jhb@freebsd.org> <5165C4D7.3050308@mu.org> <477C1270D3E5484DA2303CEBE274C9E1250B3DF7@CH1PRD0510MB392.namprd05.prod.outlook.com> <5165CB4C.4010608@mu.org> Date: Thu, 11 Apr 2013 20:44:53 +0200 X-Google-Sender-Auth: UtkCAzHHxkdgAjBh5U01c0n8OvM Message-ID: Subject: Re: Multiple page size support on FreeBSD? From: Cedric Blancher To: Alfred Perlstein Content-Type: text/plain; charset=ISO-8859-1 Cc: Wojciech Puchar , Sebastian Feld , Benjamin Kaduk , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2013 18:44:54 -0000 On 10 April 2013 22:27, Alfred Perlstein wrote: > On 4/10/13 1:09 PM, Andrew Duane wrote: >> >> Like all "performance" items (especially VM), it depends on the hardware >> and the load. On systems with small TLBs it helps more than with large TLBs. >> With software that needs access to lots of different areas the TLB gets more >> traffic so large ones help more. The answer for your firefox browser box >> with i386 is probably different from my compilation engine running MIPS, or >> his web server running AMD. >> >> Back at Digital, we spent a lot of time trying to find the "one true >> answer" to superpages, only to discover there wasn't one. We ended up with a >> combination of automatic use from big allocations, a rarely used API to >> advise for big TLBs, and some background work that coalesced when possible. > > > Thank you Andrew. I agree. A good heuristic is great, but sometimes > exposing the API unlocks some really awesome performance capabilities. > > It seems like both Digital and Sun went this route. > > I'm hoping we can do that as well. > > -Alfred > > >> .................................... >> Andrew L. Duane >> Resident Architect - AT&T Technical Lead >> m +1 603.770.7088 >> o +1 408.933.6944 (2-6944) >> skype: andrewlduane >> aduane@juniper.net >> >> >> >> -----Original Message----- >> From: owner-freebsd-hackers@freebsd.org >> [mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of Alfred Perlstein >> Sent: Wednesday, April 10, 2013 4:00 PM >> To: Benjamin Kaduk >> Cc: Wojciech Puchar; Sebastian Feld; freebsd-hackers@freebsd.org >> Subject: Re: Multiple page size support on FreeBSD? >> >> On 4/10/13 11:42 AM, Benjamin Kaduk wrote: >>> >>> On Wed, 10 Apr 2013, Wojciech Puchar wrote: >>> >>>>> How do your tests work? Do you examine PTEs directly to check for >>>>> superpages or are you relying on the vm.pmap.pde sysctls? >>>> >>>> the later. >>>> >>>> anyway - algorithm described on list - that heuristics detects >>>> consecutive page access doesn't really help the urgent case - RANDOM >>>> access to large amount of memory. >>> >>> The algorithm is not a heuristic based on consecutive accesses, >>> promotion occurs when the entire superpage's worth of memory has >>> actually been accessed. If I remember correctly, the performance gain >>> from superpages was only a few percent, so spending more time trying >>> to decide when to use them would make the algorithm a net wash. >>> >>> You should really watch the talk I linked to if you're interested, it >>> was quite interesting. >>> >>>> sequential access will get minimal improvement. >>>> >>>> IMHO the only way that really make sens is to add options to madvise >>>> to give kernel information about usage. >>> >>> Maybe. >> >> It is cool that FreeBSD got this work via Alan Cox and the others that >> contributed. >> >> I am wondering if it makes sense to have an explicit model. >> >> At one place, for a platform with high performance but a very small TLB, >> we made it possible to explicitly request a large TLB for our process and it >> made a BIG difference for performance. >> >> Sometimes being "general purpose" means that you can expose such low level >> things for the user to tune instead of requiring them to fit the app to a >> heuristic that may change. As Sebastian said it might be worth (well, very worth) to implement the Solaris userland API since there is software which uses it (since Solaris >=9 ). Hey, even ksh93 uses MHA_MAPSIZE_STACK (to force 64k pages when available; http://www.opensource.apple.com/source/ksh/ksh-18/ksh/src/cmd/ksh93/sh/pmain.c?txt) and the X11 xorg server does it, too. Ced -- Cedric Blancher Institute Pasteur