From owner-freebsd-arch Sun Feb 9 15:29:17 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4BA7037B405 for ; Sun, 9 Feb 2003 15:29:15 -0800 (PST) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 71B0A43F85 for ; Sun, 9 Feb 2003 15:29:14 -0800 (PST) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h19NTDj24866 for ; Sun, 9 Feb 2003 18:29:14 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Sun, 9 Feb 2003 18:29:13 -0500 (EST) From: Jeff Roberson To: arch@freebsd.org Subject: cvs commit: src/sys/kern vfs_bio.c Message-ID: <20030209181740.Q1025-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Modified files: sys/kern vfs_bio.c Log: - Lock down the buffer cache's infrastructure code. This includes locks on buf lists, synchronization variables, and atomic ops for the counters. This change does not remove giant from any code although some pushdown may be possible. - In vfs_bio_awrite() don't access buf fields without the buf lock. I realize I should have brought this up for review before commiting. I'm sending this mail out to give people a chance to comment on my strategy. Here's the scoop. There are many counters for active and free resources in the buf cache that are used to determine behavior at certain points. These, in most cases, are influenced by a min/max value that are setup at boot time and do not change. For these counters I used atomic ops to update them but not locks. The results of reading these atomicly updated values stale will simply be slightly oversteping some boundaries such as maxufmallocspace. At the msleep/wakeup synchronization points the synchronization variables that sometimes contain state have locks to protect them. These locks conveniently protect against sleep/wakeup races as well. I need to add some synchronization to bufwait() etc. I think I may just use one global lock for now since each buf does not need a mutex. Other than that, I put a lock on the buf queues. This is pretty straightforward. To lock the bufs themselves I examined how extensively the buf lock is used. There are only a few ways to get to a buf: getblk(), bread(), etc. These return a locked buf. gbincore(), incore(). These return bufs with unknown lockstates. vnode's dirty/clean buf lists. These return bufs with unknown lockstates. So I have gone through the tree looking for places where we use the unlocked access methods and tried to make these safe. So far I've mostly just cleaned up unlocked accesses in syncer. I've done this by protecting a new flags field in buf with the vnode interlock. This is very convenient for the dirty/clean list cases. In the end the buf lock will protect the majority of the bufs. This is turning into a surprisingly small amount of effort in the individual filesystems. I'm very pleased at this. Comments are, of course, welcome. Thanks, Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message