From owner-svn-src-all@FreeBSD.ORG Sun Apr 19 23:50:22 2009 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DEB55106564A; Sun, 19 Apr 2009 23:50:22 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id B2B148FC0A; Sun, 19 Apr 2009 23:50:22 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 5798446B23; Sun, 19 Apr 2009 19:50:22 -0400 (EDT) Date: Mon, 20 Apr 2009 00:50:22 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Ivan Voras In-Reply-To: <9bbcef730904191630x4e4f2aeci2d6ac769fc1f73f8@mail.gmail.com> Message-ID: References: <200904192302.n3JN2o6Z023217@svn.freebsd.org> <9bbcef730904191630x4e4f2aeci2d6ac769fc1f73f8@mail.gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="621616949-165048185-1240185022=:71062" Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r191291 - in head: lib/libthr/thread libexec/rtld-elf/amd64 libexec/rtld-elf/arm libexec/rtld-elf/i386 libexec/rtld-elf/ia64 libexec/rtld-elf/mips libexec/rtld-elf/powerpc libexec/rtld-... X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Apr 2009 23:50:23 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --621616949-165048185-1240185022=:71062 Content-Type: TEXT/PLAIN; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT On Mon, 20 Apr 2009, Ivan Voras wrote: > 2009/4/20 Robert Watson : >> On Sun, 19 Apr 2009, Robert Watson wrote: >> >>>  Now that the kernel defines CACHE_LINE_SIZE in machine/param.h, use  that >>> definition in the custom locking code for the run-time linker  rather than >>> local definitions. >> >> This actually changes the line size used by the rtld code for pre-pthreads >> locking for several architectures.  I think this is an improvement, but if >> architecture maintainers could comment on that, that would be helpful. > > Will there be infrastructure for creating per-CPU structures or is using > something like: > > int mycounter[MAXCPU] __attribute__ ((aligned(CACHE_LINE_SIZE))); For now, yes, something along these lines. I have a local prototype I'm using that has an API something like this: * // Definitions * struct foostat *foostatp; * void *foostat_psp; * * // Module load * if (pcpustat_alloc(&foostat_psp, "foostat", sizeof(struct foostat), * sizeof(u_long)) != 0) * panic("foostat_init: pcpustat_alloc failed"); * foostatp = pcpustat_getptr(foostat_psp); * * // Use the pointer for a statistic * foostatp[curcpu].fs_counter1++; * * // Retrieve summary statistics and store in a single instance * struct foostat fs; * pcpustat_fetch(foostat_psp, &fs); * * // Reset summary statistics. * pcpustat_reset(foostat_psp); * * // Module unload * pcpustat_free(foostat_psp); * foostatp = foostat_psp = NULL; The problem with the [curcpu] model is that it embeds the assumption that it's a good idea to have per-CPU fields in adjacent cache lines within a page. As the world rocks gently in the direction of NUMA, there's a legitimate question as to whether that's a good assumption to build in. It's a better assumption than the assumption that it's a good idea to use a single stat across all CPUs in a single cache line, of course. Depending on how we feel about the overhead of accessor interfaces, all this can be hidden easily enough. A facility I'd like to have would be an API to allocate memory on all CPUs at once, with the memory on each CPU at a constant offset from a per-CPU base address. That way you could calculate the location of the per-CPU structure using PCPU_GET(dynbase) + foostat_offset without adding an additional indirection. Robert N M Watson Computer Laboratory University of Cambridge --621616949-165048185-1240185022=:71062--