From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 28 23:58:50 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCD69106566B for ; Tue, 28 Apr 2009 23:58:50 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outP.internet-mail-service.net (outp.internet-mail-service.net [216.240.47.239]) by mx1.freebsd.org (Postfix) with ESMTP id 9CB698FC21 for ; Tue, 28 Apr 2009 23:58:50 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 899D514DFB3; Tue, 28 Apr 2009 16:58:51 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id BE74D2D61EC; Tue, 28 Apr 2009 16:58:49 -0700 (PDT) Message-ID: <49F79841.9030702@elischer.org> Date: Tue, 28 Apr 2009 16:58:57 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Robert Noland References: <1240962328.2021.10.camel@wombat.2hip.net> In-Reply-To: <1240962328.2021.10.camel@wombat.2hip.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Julian Bangert , Kevin Day Subject: Re: Question about adding flags to mmap system call / NVIDIA amd64 driver implementation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2009 23:58:51 -0000 Robert Noland wrote: > On Tue, 2009-04-28 at 16:48 -0500, Kevin Day wrote: >> On Apr 28, 2009, at 3:19 PM, Julian Bangert wrote: >> >>> Hello, >>> >>> I am currently trying to work a bit on the remaining "missing >>> feature" that NVIDIA requires ( http://wiki.freebsd.org/NvidiaFeatureRequests >>> or a back post in this ML) - the improved mmap system call. you might check with jhb (john Baldwin) as I think (from his p4 work) that he may be doing something in this area in p4. >>> For now, I am trying to extend the current system call and >>> implementation to add cache control ( the type of memory caching >>> used) . This feature inherently is very architecture specific- but >>> it can lead to enormous performance improvements for memmapped >>> devices ( useful for drivers, etc). I would do this at the user site >>> by adding 3 flags to the mmap system call (MEM_CACHE__ATTR1 to >>> MEM_CACHE__ATTR3 ) which are a single octal digit corresponding to >>> the various caching options ( like Uncacheable,Write Combining, >>> etc... ) with the same numbers as the PAT_* macros from i386/include/ >>> specialreg.h except that the value 0 ( PAT_UNCACHEABLE ) is replaced >>> with value 2 ( undefined), whereas value 0 ( all 3 flags cleared) is >>> assigned the meaning "feature not used, use default cache control". >>> For each cache behaviour there would of course also be a macro >>> expanding to the rigth combination of these flags for enhanced >>> useability. >>> >>> The mmap system call would, if any of these flags are set, decode >>> them and get a corresponding PAT_* value, perform the mapping and >>> then call into the pmap module to modify the cache attributes for >>> every page. >> Have you looked at mem(4) yet? >> >> Several architectures allow attributes to be associated with >> ranges of >> physical memory. These attributes can be manipulated via >> ioctl() calls >> performed on /dev/mem. Declarations and data types are to be >> found in >> . >> >> The specific attributes, and number of programmable ranges may >> vary >> between architectures. The full set of supported attributes is: >> >> MDF_UNCACHEABLE >> The region is not cached. >> >> MDF_WRITECOMBINE >> Writes to the region may be combined or performed out of >> order. >> >> MDF_WRITETHROUGH >> Writes to the region are committed synchronously. >> >> MDF_WRITEBACK >> Writes to the region are committed asynchronously. >> >> MDF_WRITEPROTECT >> The region cannot be written to. >> >> This requires knowledge of the physical addresses, but I believe >> that's probably already necessary for what it sounds like you're >> trying to accomplish. >> >> Back in the FreeBSD-3.0 days, I was writing a custom driver for an AGP >> graphics controller, and setting the MTRR flags for the exposed buffer >> was a definite improvement (200-1200% faster in most cases). > > This is MTRR, which is what we currently do, when we can. The issue is > that often times the BIOS maps ranges in a way that prevents us from > using MTRR. This is generally ideal for things like agp and > framebuffers when it works, since they have a specific physical range > that you want to work with. > > With PCI(E) cards it isn't as cut and dry... In the ATI and Nouveau > cases, we map scatter gather pages into the GART, which generally are > allocated using contigmalloc behind the scenes, so it is also possible > for it to work in that case. Moving forward, we may actually be mapping > random pages into and out of the GART (GEM / TTM). In those cases we > really don't have a large contiguous range that we could set MTRR on. > Intel CPUs are limited to 8 MTRR registers for the entire system also, > so that can become an issue quickly if you are trying to manipulate > several areas of memory. With PAT we can manipulate the caching > properties on a page level. PAT also allows for some overlap conditions > that MTRR won't, such as mapping a page write-combining on top on an > UNCACHEABLE MTRR. > > jhb@ has started some work on this, since I've been badgering him about > this recently as well. > > robert. > >> -- Kevin >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"