From owner-freebsd-hackers@FreeBSD.ORG Tue Jun 24 12:36:10 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 93EDE37B401 for ; Tue, 24 Jun 2003 12:36:10 -0700 (PDT) Received: from hysteria.spc.org (hysteria.spc.org [195.206.69.234]) by mx1.FreeBSD.org (Postfix) with SMTP id DC23043FDD for ; Tue, 24 Jun 2003 12:36:08 -0700 (PDT) (envelope-from bms@hysteria.spc.org) Received: (qmail 12569 invoked by uid 5013); 24 Jun 2003 19:34:31 -0000 Date: Tue, 24 Jun 2003 20:34:31 +0100 From: Bruce M Simpson To: freebsd-hackers@freebsd.org Message-ID: <20030624193431.GW9463@spc.org> Mail-Followup-To: Bruce M Simpson , freebsd-hackers@freebsd.org, alc@freebsd.org, jmallett@freebsd.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="4Epv4kl9IRBfg3rk" Content-Disposition: inline User-Agent: Mutt/1.4.1i cc: alc@freebsd.org cc: jmallett@freebsd.org Subject: Optimizing mutex alignment viz cache line access X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2003 19:36:10 -0000 --4Epv4kl9IRBfg3rk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, There's an idea for optimizing cache line access through object alignment tricks in the attached mail. Feedback would be valu[able|ed]. BMS --4Epv4kl9IRBfg3rk Content-Type: message/rfc822 Content-Disposition: inline Date: Tue, 24 Jun 2003 17:45:42 +0100 From: Bruce M Simpson To: Matthew Dillon Cc: David Schultz Subject: Re: Page Coloring Defines in vm_page.h Message-ID: <20030624164542.GU31354@spc.org> References: <20030624111942.GO31354@spc.org> <20030624131748.GB58305@HAL9000.homeunix.com> <20030624133813.GR31354@spc.org> <200306241625.h5OGPhE2094185@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200306241625.h5OGPhE2094185@apollo.backplane.com> User-Agent: Mutt/1.4.1i On Tue, Jun 24, 2003 at 09:25:43AM -0700, Matthew Dillon wrote: [..] > the optimized values are going to be so small (probably not even > measureable) that it isn't worth doing. Ok. Tweaking page coloring viz other architectures is a bikeshed. In the absence of numbers it was necessary to test the idea. Moving swiftly on... > If you want to find something to optimize, then cache-line alignment > of data structures and ensuring that multiple data structures > (such as mutexes) which are potentially used by different cpus do > not share the same cache line is going to reap far greater benefits > then messing around with the page coloring algorithm. Now *that* is an interesting problem, and extremely difficult to crack, at least for the entire kernel. It would require a map of every instance of such a shared structure in terms of its location within each page. In the case of dynamically-allocated structures such as the softc for a character device, it is even harder to deterministically calculate the location of a mutex member of such a softc in memory. Correct me if I'm wrong here? Would the workaround, then, not be to modify the mtx_*() set of functions to use a zone allocator, whose allocation function could then be tuned to allocate the lock structure which is frobbed by the atomic_*() functions on separate cache lines? BMS --4Epv4kl9IRBfg3rk--