From owner-freebsd-current@FreeBSD.ORG Mon Jun 7 17:49:34 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 98C4B16A4CE for ; Mon, 7 Jun 2004 17:49:34 +0000 (GMT) Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E77843D54 for ; Mon, 7 Jun 2004 17:49:34 +0000 (GMT) (envelope-from alc@cs.rice.edu) Received: from localhost (calypso.cs.rice.edu [128.42.1.127]) by cs.rice.edu (Postfix) with ESMTP id B8F604AA85; Mon, 7 Jun 2004 12:49:33 -0500 (CDT) Received: from cs.rice.edu ([128.42.1.30]) by localhost (calypso.cs.rice.edu [128.42.1.127]) (amavisd-new, port 10024) with LMTP id 08248-01-56; Mon, 7 Jun 2004 12:49:33 -0500 (CDT) Received: by cs.rice.edu (Postfix, from userid 19572) id 599754AA50; Mon, 7 Jun 2004 12:49:33 -0500 (CDT) Date: Mon, 7 Jun 2004 12:49:33 -0500 From: Alan Cox To: Alexander Leidinger Message-ID: <20040607174933.GK24461@cs.rice.edu> References: <20040606142446.2900a97e@Magellan.Leidinger.net> <20040606201249.GH24461@cs.rice.edu> <20040607130430.0ebbbb8e@Magellan.Leidinger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040607130430.0ebbbb8e@Magellan.Leidinger.net> User-Agent: Mutt/1.4.2i X-Virus-Scanned: by amavis-20030616-p7 at cs.rice.edu cc: current@freebsd.org Subject: Re: comments in the page coloring options in /sys/conf/NOTES X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2004 17:49:34 -0000 On Mon, Jun 07, 2004 at 01:04:30PM +0200, Alexander Leidinger wrote: > On Sun, 6 Jun 2004 15:12:49 -0500 > Alan Cox wrote: > > > My recent commit fixed a "syntax" error in the comments, specifically, > > a reference to a missing macro. The comments are, however, still > > "semantically" broken: > > > > 1. Cache size alone does not correctly determine the number of colors, > > except for direct map caches. The correct formula is > > > > (cache size in bytes / page size in bytes) / degree of cache associativity > > > > Thus, the comments would lead one to configure his/her system with too > > many colors. (Relative to configuring a system with too few colors, > > this is not so bad.) > > vm_page.h contains: > ---snip--- > #if PQ_CACHESIZE >= 1024 > #define PQ_PRIME1 31 /* Prime number somewhat less than PQ_HASH_SIZE */ > #define PQ_PRIME2 23 /* Prime number somewhat less than PQ_HASH_SIZE */ > #define PQ_L2_SIZE 256 /* A number of colors opt for 1M cache */ > ---snip--- > > The three defines seem to be the tunables for the page coloring, but > neither of them seem to be near cache_size/page_size. So even for the > direct mapped case this doesn't seem to fit your explanation. Actually, it does. Divide 1MB (the cache size) by 4KB (the page size) and you get 256, the appropriate number of colors for a 1MB direct- mapped cache. Despite its name, PQ_L2_SIZE, is not the L2 cache size. As the comment suggests, it is the number of colors. > ... Is it as > easy as using appropriate values for those defines at boot time or is > there more work involved for auto-tuning version? At boot time. > Is there a way to determine the size of the L2 cache and the > associativity at run time in the kernel or are these properties which we > have to obtain from a table (which we have to write and maintain for > automatic tuning)? On modern amd64 and i386 CPUs, yes. I would be surprised if ia64 didn't provide the information. Run a "boot -v" and look carefully at the information. It describes the cache structure. > > 2. The references to L1 should be removed. They are historical > > leftovers. > > cut&paste: > ---snip--- > Index: sys/conf/NOTES > =================================================================== > RCS file: /big/FreeBSD-CVS/src/sys/conf/NOTES,v > retrieving revision 1.1227 > diff -u -r1.1227 NOTES > --- sys/conf/NOTES 1 Jun 2004 06:22:56 -0000 1.1227 > +++ sys/conf/NOTES 7 Jun 2004 10:07:16 -0000 > @@ -103,13 +103,13 @@ > > # Options for the VM subsystem > # L2 cache size (in KB) can be specified in PQ_CACHESIZE > -options PQ_CACHESIZE=512 # color for 512k/16k cache > +options PQ_CACHESIZE=512 # color for 512k cache > # Deprecated options supported for backwards compatibility > #options PQ_NOOPT # No coloring > -#options PQ_LARGECACHE # color for 512k/16k cache > -#options PQ_HUGECACHE # color for 1024k/16k cache > -#options PQ_MEDIUMCACHE # color for 256k/16k cache > -#options PQ_NORMALCACHE # color for 64k/16k cache > +#options PQ_LARGECACHE # color for 512k cache > +#options PQ_HUGECACHE # color for 1024k cache > +#options PQ_MEDIUMCACHE # color for 256k cache > +#options PQ_NORMALCACHE # color for 64k cache > > # This allows you to actually store this configuration file into > # the kernel binary itself, where it may be later read by saying: > Index: sys/vm/vm_pageq.c > =================================================================== > RCS file: /big/FreeBSD-CVS/src/sys/vm/vm_pageq.c,v > retrieving revision 1.13 > diff -u -r1.13 vm_pageq.c > --- sys/vm/vm_pageq.c 30 May 2004 00:42:38 -0000 1.13 > +++ sys/vm/vm_pageq.c 7 Jun 2004 10:38:31 -0000 > @@ -176,8 +176,8 @@ > * > * The page coloring optimization attempts to locate a page > * that does not overload other nearby pages in the object in > - * the cpu's L1 or L2 caches. We need this optimization because > - * cpu caches tend to be physical caches, while object spaces tend > + * the cpu's L2 cache. We need this optimization because cpu > + * caches tend to be physical caches, while object spaces tend > * to be virtual. > * > * This routine must be called at splvm(). > ---snip--- > > I think the comment with the reference to L1 in sys/vm/vm_zeroidle.c is > ok. Correct. (In fact, on newer generation CPUs, we use non-temporal, i.e., non-cached, stores to zero pages in the idle loop. So, there is no inteference with the cache.) > If this is ok, do I have your approval to commit it? Yes, please commit it. Regards, Alan