From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 30 00:20:06 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1CA4716A40B for ; Fri, 30 Mar 2007 00:20:06 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.249]) by mx1.freebsd.org (Postfix) with ESMTP id AA8B313C4B8 for ; Fri, 30 Mar 2007 00:20:05 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by an-out-0708.google.com with SMTP id c24so331911ana for ; Thu, 29 Mar 2007 17:20:05 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=kUOIsiEqN+n/DD0841WXspZscMGkcrcUeHBP5WZGFSZEvQDtcK+S/mDAi+cxW5DP/rJvSdIJDCjXGIPKSH2+85lhPScz8d40DrCHn6i9ypTIWnZzO/m9E51tJ9ggma4oFnd/39wnUmyyXrynUYnDfpB5to13t2n65Bgd49W4a08= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=OKnx9WwW/DIuIustsZKqKH9DSO2gwVntKgWpfvI89bCf7PTVuW5UA2MHo1EfWOONHBOnTVrOUyNFZyyJC2liHBqTULiuh8BA4TezyfLXEkrf4Cc8S5MRkExnEm7NX+VfJawi9tRkFqv4VNFWHmvAgGBQCD6x5Yemf2eF2Z7DJos= Received: by 10.100.109.13 with SMTP id h13mr937595anc.1175214005167; Thu, 29 Mar 2007 17:20:05 -0700 (PDT) Received: by 10.100.191.1 with HTTP; Thu, 29 Mar 2007 17:20:05 -0700 (PDT) Message-ID: <3bbf2fe10703291720yd82045ei66ead6e4f251a20@mail.gmail.com> Date: Fri, 30 Mar 2007 02:20:05 +0200 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Yar Tikhiy" In-Reply-To: <20070329112606.GA72497@comp.chem.msu.su> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070328082620.GA1052@dwpc.dwlabs.ca> <20070328102027.T1185@fledge.watson.org> <20070329112606.GA72497@comp.chem.msu.su> X-Google-Sender-Auth: 0b13788422fe330e Cc: freebsd-hackers@freebsd.org, Robert Watson , Duane Whitty Subject: Re: Locking etc. (Long, boring, redundant, newbie questions) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Mar 2007 00:20:06 -0000 2007/3/29, Yar Tikhiy : > On Wed, Mar 28, 2007 at 10:40:58AM +0100, Robert Watson wrote: > > > > Spin locks are, FYI, slower than default mutexes. The reason is that they > > have to do more work: they not only perform an atomic operation/memory > > barrier to set the cross-CPU lock state, but they also have to disable > > interrupts to synchronize with fast interrupt handlers. In general, you > > are right: you should only use a spin mutex if you are running in a fast > > handler, or synchronizing with a fast handler. The one general exception > > is the scheduler itself, which must protect its data structures with spin > > locks in order to implement sleeping primitives. As such, the scheduler > > lock and various other low level locks (such as in turnstiles) are > > implemented with spin locks and not default mutexes. Since default mutexes > > spin adaptively, the reduced overhead of contention experienced with spin > > locks (i.e., no scheduler overhead for simple contention cases) is also > > experienced with default mutexes. > > By the way, could the following observation of mine be related to > the high cost of spin mutexes? > > While doing some measurements of the performance of our vlan driver > in a router, I found that with RELENG_6 the pps rate through the > router degraded considerably (by ~100Kpps) if its physical interfaces > used hardware vlan tagging. I attributed that to the overhead of > allocating and freeing a mbuf tag for each packet as it entered and > then left the router. I used hwpmc and pmcstat to see which kernel > functions took most time and found that critical_exit() made top 5 > in the list of CPU time eaters if the network interfaces were using > hardware vlan tagging. > > The machine was an UP amd64, but it ran FreeBSD/i386, with an UP > kernel. > > As I can see from the code, critical_exit() may grab and later > release a spin mutex. I've got a hazy recollection that our kernel > memory allocator uses critical sections to protect its per-CPU > structures. That's why I suspect that the effect I observed may > have its roots in the behaviour of spin mutexes. Could it be so? This is not entirely true. What happens is that if you enable preemption in your kernel, critical_exit() holds sched_lock just beacause it needs to perform a mi_switch(), so just another thread will be scheduled to run currently (and, please, note that sched_lock will be dropped by internal functions when context switch will be completed). Otherwise you don't pay the penalty of the sched_lock acquisition. Attilio -- Peace can only be achieved by understanding - A. Einstein