From owner-freebsd-arch@FreeBSD.ORG  Tue Apr  2 01:36:47 2013
Return-Path: <owner-freebsd-arch@FreeBSD.ORG>
Delivered-To: arch@FreeBSD.org
Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115])
 by hub.freebsd.org (Postfix) with ESMTP id D8CBFFD8;
 Tue,  2 Apr 2013 01:36:47 +0000 (UTC)
 (envelope-from luigi@onelab2.iet.unipi.it)
Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238])
 by mx1.freebsd.org (Postfix) with ESMTP id 9EA87C05;
 Tue,  2 Apr 2013 01:36:45 +0000 (UTC)
Received: by onelab2.iet.unipi.it (Postfix, from userid 275)
 id A53A273027; Tue,  2 Apr 2013 03:37:58 +0200 (CEST)
Date: Tue, 2 Apr 2013 03:37:58 +0200
From: Luigi Rizzo <rizzo@iet.unipi.it>
To: Gleb Smirnoff <glebius@FreeBSD.org>
Subject: Re: [CFR][CFT] counter(9): new API for faster and raceless counters
Message-ID: <20130402013758.GA96904@onelab2.iet.unipi.it>
References: <20130401115128.GZ76816@FreeBSD.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20130401115128.GZ76816@FreeBSD.org>
User-Agent: Mutt/1.5.20 (2009-06-14)
Cc: arch@FreeBSD.org
X-BeenThere: freebsd-arch@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Discussion related to FreeBSD architecture <freebsd-arch.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-arch>,
 <mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch>
List-Post: <mailto:freebsd-arch@freebsd.org>
List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
 <mailto:freebsd-arch-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 02 Apr 2013 01:36:47 -0000

On Mon, Apr 01, 2013 at 03:51:28PM +0400, Gleb Smirnoff wrote:
>   Hi!
> 
>   Together with Konstantin Belousov (kib@) we developed a new API that is
> initially purposed for (but not limited to) collecting statistical
> data in kernel.
...

really great work, thanks for tacking this.
I have some comments inline:
 
API:

> o MI implementation of counter_u64_add() is:
> 
>      critical_enter();
>      *(uint64_t *)zpcpu_get(c) += inc;
>      critical_exit();

- there are several places which use multiple counters
  (e.g. packet and byte counters, global and per flow/socket),
  so i wonder if it may help to provide a "protected" version of
  counter_u64_add() that requires the critical_enter/exit
  only once. Something like

	PROTECT_COUNTERS(
		safe_counter_u64_add(c, x);
		safe_counter_u64_add(c, x);
		safe_counter_u64_add(c, x);
	);

  where PROTECT_COUNTERS() would translate into the critical_enter/exit
  where required, and nothing on other architectures.

...

BENCHMARK:

> I've got a simple benchmark. A syscall that solely updates a counter is
> implemented. Number of processes is spawned, equal to number of cores,
> each process binds itself to a dedicated CPU and calls the syscall 10^6
> times and exits. Parent wait(2)s for them all and I measure real time of

- I am under the impression that these benchmarks are dominated
  by the syscall time, and the new counters would exhibit a lot
  better relative performance (compared to racy or atomic)
  by doing 10..100 counter ops per syscall. Any chance to retry
  the test in this configuration ?

cheers
luigi