From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 28 21:43:19 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60442106566B for ; Tue, 28 Apr 2009 21:43:19 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: from mail-gx0-f176.google.com (mail-gx0-f176.google.com [209.85.217.176]) by mx1.freebsd.org (Postfix) with ESMTP id 274688FC08 for ; Tue, 28 Apr 2009 21:43:18 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: by gxk24 with SMTP id 24so598464gxk.19 for ; Tue, 28 Apr 2009 14:43:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.151.133.5 with SMTP id k5mr11741206ybn.199.1240953795119; Tue, 28 Apr 2009 14:23:15 -0700 (PDT) In-Reply-To: References: Date: Tue, 28 Apr 2009 23:23:15 +0200 Message-ID: From: =?ISO-8859-1?Q?Marius_N=FCnnerich?= To: Julian Bangert Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Question about adding flags to mmap system call / NVIDIA amd64 driver implementation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2009 21:43:19 -0000 On Tue, Apr 28, 2009 at 22:19, Julian Bangert wrote: > Hello, > > I am currently trying to work a bit on the remaining "missing feature" th= at > NVIDIA requires ( http://wiki.freebsd.org/NvidiaFeatureRequests =A0or a b= ack > post in this ML) - =A0the improved mmap system call. > =A0For now, I am trying to extend the current system call and implementat= ion > to add cache control ( the type of memory caching used) . This feature > inherently is very architecture specific- but it can lead to enormous > performance improvements for memmapped devices ( useful for drivers, etc)= . I > would do this at the user site by adding 3 flags to the mmap system call > (MEM_CACHE__ATTR1 to MEM_CACHE__ATTR3 ) which are a single octal digit > corresponding to the various caching options ( like Uncacheable,Write > Combining, etc... ) with the same numbers as the PAT_* macros from > i386/include/specialreg.h except that the value 0 ( PAT_UNCACHEABLE ) is > replaced with value 2 ( undefined), whereas value 0 ( all 3 flags cleared= ) > is assigned the meaning "feature not used, use default cache control". > For each cache behaviour there would of course also be a macro expanding = to > the rigth combination of these flags for enhanced useability. Hmm, I don't like that. What about using something like PAT_WC directly for the userland? Afaik a userland app that uses stuff like this is md anyway. > =A0The mmap system call would, if any of these flags are set, decode them= and > get a corresponding PAT_* value, perform the mapping and then call into t= he > pmap module to modify the cache attributes for every page. > > =A0My first question is if there is a more elegant way of solving that - = the 3 > flags would be architecture specific ( they could be used for other thing= s > on other architectures though if need be ) and I do not know the policy o= n > architecture specific syscall flags, therefore I appreciate any input. > > The second question goes to all those great VM/pmap gurus out there: As f= ar > as I understand, at the moment the pmap_change_attr can only cange the ca= che > flags for kernel pages. Is there a particular reason why this function mi= ght > not be adapted/extended to userspace mappings? If not, I would either add= a > new function to iterate over all pages and set cache flags for a particul= ar > region or add a new member (possibly just add the 3 flags again ? ) to th= e > md part of vm_page_t. Or one could just keep track and return errors as s= oon > as someone tries to map a memory region ( cache-customized mapping is > usually done to device memory ) already mapped with =A0different cache > behaviour. Do you know how other OS handle this stuff? Maybe there is some inspiration there for a clean interface. I'm not sure if I remember correctly but there is something in my mind that we must take care that no virtual pages have different PAT settings for the same physical page. Maybe I read something like this in the AMD's documentation of PAT. Sorry I don't remember exactly but perhaps someone else can explain it better.