From owner-freebsd-arch Mon Feb 10 14:25:50 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6490F37B405 for ; Mon, 10 Feb 2003 14:25:48 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1C78543FAF for ; Mon, 10 Feb 2003 14:25:47 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h1AMPkSJ023703; Mon, 10 Feb 2003 14:25:46 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h1AMPkTE023700; Mon, 10 Feb 2003 14:25:46 -0800 (PST) Date: Mon, 10 Feb 2003 14:25:46 -0800 (PST) From: Matthew Dillon Message-Id: <200302102225.h1AMPkTE023700@apollo.backplane.com> To: Alfred Perlstein Cc: Morten Rodal , phk@phk.freebsd.dk, David Schultz , arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. References: <20030210091317.GD5165@HAL9000.homeunix.com> <37473.1044868995@critter.freebsd.dk> <20030210204523.GF12240@slurp.rodal.no> <20030210205443.GA88781@elvis.mu.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :* Morten Rodal [030210 12:45] wrote: :> :> I upgraded my workstation to 5.0-RELEASE shortly after it became :> available, and there is one thing that keeps bugging me. When I :> delete a file that is large, say >600MB, the system performance and :> interactivity will drop remarkebly. : :This happens on my machine as well, a complete shot in the dark fix :that I would try is to find where syncer loops and purposefully have :it drop and pick up Giant between batches of work. : :-Alfred Try reproducing the problem with kernel profiling turned on. The syncer is not likely to be the cause of the large file removal issue, since that involves only bitmaps. The cause is more likely to be softupdate's bitmap dependancies. Try turning off softupdates and see if that improves performance. Softupdates creates some rather nasty buffer cache issues due to the number of buffers it removes from management. Also, dependancies prevent the buffer cache from being able to flush a buffer. The buffer cache was not really designed to deal with flushes which don't undirty a buffer or large numbers of buffers being temporarily removed from management. The syncer is more likely to be the cause of performance issues during large file data operations. Because the syncer holds onto the vnode lock you can get into a situation where lookup()s cause a directory locking chain to reach the root directory which has the effect of stalling just about everything. Either Net or OpenBSD solved this problem by doing partial fsyncs instead of full fsyncs to avoid holding onto the vnode lock for insanely long periods of time. Another solution would be to use the namei cache to lock path names from create/remove/rename ops so the directory vnodes do not have to be locked for the duration of a path lookup. If the directory vnode is not exclusively locked then a race-to-root will not happen due to some random file vnode being locked for a long period of time. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message