From owner-freebsd-current@FreeBSD.ORG Sun Jul 30 15:33:37 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3A18516A4DE for ; Sun, 30 Jul 2006 15:33:37 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id C6E8F43D45 for ; Sun, 30 Jul 2006 15:33:36 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 4AA8D46BF8; Sun, 30 Jul 2006 11:33:36 -0400 (EDT) Date: Sun, 30 Jul 2006 16:33:36 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Michal Mertl In-Reply-To: <1154217036.23616.28.camel@genius.i.cz> Message-ID: <20060730162756.N16341@fledge.watson.org> References: <200607251254.k6PCsBef092737@lurza.secnetix.de> <200607271058.13055.jhb@freebsd.org> <20060728121525.GA44917@uk.tiscali.com> <200607280928.36573.jhb@freebsd.org> <20060728134701.GA45273@uk.tiscali.com> <20060728210154.GC748@turion.vk2pj.dyndns.org> <1154189612.1565.10.camel@genius.i.cz> <20060729205655.GE748@turion.vk2pj.dyndns.org> <20060729211530.GA50342@uk.tiscali.com> <1154212340.3609.18.camel@genius.i.cz> <20060729230214.GI12597@groat.ugcs.caltech.edu> <1154216319.23616.23.camel@genius.i.cz> <1154217036.23616.28.camel@genius.i.cz> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Peter Jeremy , Paul Allen , Brian Candler , freebsd-current@freebsd.org Subject: Re: vmstat's entries type X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jul 2006 15:33:37 -0000 On Sun, 30 Jul 2006, Michal Mertl wrote: > Michal Mertl wrote: >> Paul Allen wrote: >>> Surely all you need to do is a cheap crit_enter,crit_exit >>> while updating the 64-bit per cpu counters. and on >>> a 64-bit arch you skip the crit_enter,crit_exit. >> >> Critical_enter/exit seem to be quite lightweight (single read/modify/write >> of a variable). > > One more question. Why do you say that crit_* can be avoided on 64-bit arch? > If the reason was that "increment of a 64 bit number is one operation there" > it probably is not true - as somebody already stated, some instruction sets > don't allow atomic increment of a memory location. The UMA zone allocator and malloc allocator both use critical sections to protect their per-cpu statistics and this works quite well in practice. However, critical sections are also significantly not-free compared to a simple integer increment. The argument for using 64-bit arithmetic without synchronization is that a small amount of occasional inaccuracy could be tolerated (a 64-bit increment being lost due to a race, for example), but that large amounts associated with 64-bit arithmetic using 32-bit operations, might not be (a 32-bit wrap being counted 0 times or 2 times). If statistics are going to be protected using critical sections, I suggest we investigate a way to minimize the number of critical sections we enter and exit for statistics purposes, ideally minimizing and amortizing the entry/exit of critical sections. The more time you spend in critical sections, the more likely it is an interrupt will fire during the critical section, and our critical sections are optimized to optimistically assume that interrupts won't fire during critical sections. FWIW, One of the reasons some of the lower level network statistics haven't been modified to occur in critical sections or locked regions is that there are still open design questions about locking for network interfaces and address lists. It's likely that once that work is done, locking of the statistics can be done "for free" by piggy-backing it on other necessary work. I've wanted to see 64-bit statistics on our network interfaces for a long time -- I wrap 32-bit counters very quickly, myself, and it will only happen more quickly in the future! Robert N M Watson Computer Laboratory University of Cambridge