From owner-freebsd-arch@FreeBSD.ORG Fri Mar 14 02:19:09 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42C121065671; Fri, 14 Mar 2008 02:19:09 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail14.syd.optusnet.com.au (mail14.syd.optusnet.com.au [211.29.132.195]) by mx1.freebsd.org (Postfix) with ESMTP id CA3268FC14; Fri, 14 Mar 2008 02:19:08 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-252-11.carlnfd3.nsw.optusnet.com.au (c220-239-252-11.carlnfd3.nsw.optusnet.com.au [220.239.252.11]) by mail14.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id m2E2Ii5F014565 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 14 Mar 2008 13:18:49 +1100 Date: Fri, 14 Mar 2008 13:18:44 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Julian Elischer In-Reply-To: <47D9BDF3.80409@elischer.org> Message-ID: <20080314115225.G34431@delplex.bde.org> References: <20080310161115.X1091@desktop> <47D758AC.2020605@freebsd.org> <20080313124213.J31200@delplex.bde.org> <20080312211834.T1091@desktop> <20080313230809.W32527@delplex.bde.org> <20080313132152.Y1091@desktop> <47D9BDF3.80409@elischer.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Peter Wemm , David Xu , arch@freebsd.org Subject: Re: amd64 cpu_switch in C. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2008 02:19:09 -0000 On Thu, 13 Mar 2008, Julian Elischer wrote: > Jeff Roberson wrote: >> I'm not sure why you feel masking interrupts in spinlocks is bogus. It's >> central to our SMP strategy. Unless you think we should do it lazily like >> we do with critical_*. I know jhb had that working at one point but it was >> abandoned. Masking interrupts in spinlocks breaks fast interrupts among other things. Yes, I think it should be done like in critical_*. My version has done this for 6 years or so, but I don't really care about SMP and never made it work right for SMP. Its main impact is on fast interrupt handlers. Interrupt handlers cannot access any data that is not locked, and for non-broken fast interrupt handlers, in practice this means not accessing any global data, since locking global data would be too hard and/or slow. Global data includes all per-CPU-data, and I enforce non-access to this by loading %fs with 0 in fast interrupt handlers. This makes fast interrupt handlers quite difficult to write. An interrupt handler like hardclock(), which stomps around in global data, in some places without even locking the data, is far too large and complicated to be a non-broken fast interrupt handler. I use normal interrupt handlers for hardclock() and statclock() so my lower interrupt latency costs performance. > My memory is that we used to mask interrupts lazily in 4.x Right. Only for i386. The masking is in the PIC so it only affects devices on non-fast interrupts, which should only be slow devices. Lazy masking for critical_*() has the same results (it only affects non-fast interrupts) although its mechanism is different. I implemented this in 386BSD and am unhappy that it was broken in SMPng, though with CPUs hundreds of times faster than they were when 386BSD was new, and with devices not so much faster and/or with larger buffers, the extra latency rarely matters in practice; also, with SMP there is only extra latency if all CPUs happen to hold a spinlock at the same time. Bruce