From owner-svn-src-all@FreeBSD.ORG Sun Apr 12 14:29:38 2009 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D0ABC1065674; Sun, 12 Apr 2009 14:29:38 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id A9D9F8FC18; Sun, 12 Apr 2009 14:29:38 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 6355A46B1A; Sun, 12 Apr 2009 10:29:38 -0400 (EDT) Date: Sun, 12 Apr 2009 15:29:38 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Luigi Rizzo In-Reply-To: Message-ID: References: <200904121406.n3CE6QSH027497@svn.freebsd.org> <20090412143000.GB96365@onelab2.iet.unipi.it> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: per-cpu counters (Re: svn commit: r190967 - head/sys/netinet) X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2009 14:29:39 -0000 On Sun, 12 Apr 2009, Robert Watson wrote: >> As an example, an approach i was considering is replace the existing >> counters with an index in a global_counters[MAXCPU][] array so the counter >> update will become something like this: FYI, one other reason to put it all behind per-subsystem macros is that it allows us to easily prototype another statistics implementation, the no-op, on a per-subsystem basis. Robert N M Watson Computer Laboratory University of Cambridge >> >> - pimstat.pims_rcv_registers_wrongiif++; >> + global_counters[PCPU_GET(cpuid)] >> + [pimstat.pims_rcv_registers_wrongiif]++; >> >> When you allocate an entry you also need to allocate an entry in the >> global_counters, but entries have fixed size, and the modules that you >> build will not depend on the MAXCPU value. >> >> All of this is still compatible with the macros you are adding, but I >> wonder why we should have many PIMSTAT_INC, IPFW_INC, NATD_INC and so on >> instead of more generic COUNTER64_INC() or COUNTER32_INC() > > I have a project along these lines in progress, and will post the proposal to > arch@ once I've finished prototyping it. In particular, it provides common > implementations of "reset" and "report" in order to expose a single userspace > version of the structure via sysctl. > > The reason to use per-subsystem macros is that there's another dimmension > here: virtualization. Some counters are per-CPU, others are per-VIMAGE, and > many are per-CPU-per-VIMAGE. Since virtualization is a work-in-progress, and > they haven't yet virtualized all the counter blocks, I figured it was best to > push all the counter updates behind subsystem-specific macros, and then > virtualization would touch only the macro definitions, not the implementation > in every place. In the sample implementation you show above, changes have to > be made to every instance of the counter when virtualization is > added/removed/changed for PIM, making per-CPU changes hard to MFC. > > Robert N M Watson > Computer Laboratory > University of Cambridge >