From owner-freebsd-arch@FreeBSD.ORG Tue May 22 04:43:00 2007 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7F9F616A468 for ; Tue, 22 May 2007 04:43:00 +0000 (UTC) (envelope-from bde@optusnet.com.au) Received: from fallbackmx03.syd.optusnet.com.au (fallbackmx03.syd.optusnet.com.au [211.29.133.136]) by mx1.freebsd.org (Postfix) with ESMTP id 9966613C447 for ; Tue, 22 May 2007 04:42:59 +0000 (UTC) (envelope-from bde@optusnet.com.au) Received: from mail35.syd.optusnet.com.au (mail35.syd.optusnet.com.au [211.29.133.51]) by fallbackmx03.syd.optusnet.com.au (8.12.11.20060308/8.12.11) with ESMTP id l4L38oQ9026226 for ; Mon, 21 May 2007 13:08:50 +1000 Received: from besplex.bde.org (c211-30-216-190.carlnfd3.nsw.optusnet.com.au [211.30.216.190]) by mail35.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id l4L38aEJ030040 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 21 May 2007 13:08:39 +1000 Date: Mon, 21 May 2007 13:08:37 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Jeff Roberson In-Reply-To: <20070520155103.K632@10.0.0.1> Message-ID: <20070521113648.F86217@besplex.bde.org> References: <20070520155103.K632@10.0.0.1> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org Subject: Re: sched_lock && thread_lock() X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2007 04:43:00 -0000 On Sun, 20 May 2007, Jeff Roberson wrote: > Attilio and I have been working on addressing the increasing problem of > sched_lock contention on -CURRENT. Attilio has been addressing the parts of > the kernel which do not need to fall under the scheduler lock and moving them > into seperate locks. For example, the ldt/gdt lock and clock lock which were > committed earlier. Also, using atomics for the vmcnt structure. Using atomics in the vmmeter struct is mostly just a pessimization and and obfuscation, since locks are still needed for accesses to more than one variable at a time. For these cases, locks are needed for correctness, and are also probably more efficient if there are > 2 accesses. Then lock poisoning requires the same locks to be held for updates of the variables (to prevent variables changing while you are using them), and atomic updates of single variables are a pessimization too. The VMCNT*() ineterface encorages this pessimization and has other problems (messes with volatile, and not actually doing atomic accesses for VMCNT_GET()). Some of the cases where locks aren't needed (mainly for userland-only statistics) were already handled better by PCPU_LAZY_INC(). Unfortunately, the method used in PCPU_LAZY_INC() doesn't work for variables that the kernel wants to access as globals. Example of code with multiple accesses: % /* % * Return the number of pages we need to free-up or cache % * A positive number indicates that we do not have enough free pages. % */ % % static __inline % int % vm_paging_target(void) % { % return ( % (VMCNT_GET(free_target) + VMCNT_GET(cache_min)) - % (VMCNT_GET(free_count) + VMCNT_GET(cache_count)) % ); % } Without holding a lock throughout this, this returns a garbage value. Without holding a lock throughout this and the actions taken depending on the return value, garbage actions may be taken. Values that are only slightly wrong might work OK, but this is not clear, and if it does work then volatile variables and [non-]atomic accesses to the variables to get it. Bruce