From owner-freebsd-hackers Sun Jun 3 11:53:56 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 837FF37B401 for ; Sun, 3 Jun 2001 11:53:54 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.3/8.11.2) id f53Irpe10632; Sun, 3 Jun 2001 11:53:51 -0700 (PDT) (envelope-from dillon) Date: Sun, 3 Jun 2001 11:53:51 -0700 (PDT) From: Matt Dillon Message-Id: <200106031853.f53Irpe10632@earth.backplane.com> To: Ian Dowse Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: UFS large directory performance References: <200106031747.aa54373@salmon.maths.tcd.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Nice idea, but I'm not sure I see the benefit of partially reclaiming :second-level arrays. Because it is a hash array, there isn't really :the concept of a working set; a directory that is `in use' will :rarely see many create/rename/delete operations on a small fixed :set of filenames. The lookup case is already cached elsewhere. I :think an all-or-nothing approach is likely to perform better and :be simpler to implement. Even the lazy allocation of second-level :arrays is unlikely to help a lot if the hash function does its job :well. Well, amoung other things it solves the locking problem you bring up below. Even if partial reclamation doesn't maintain performance, it still definitely allows you to reclaim second level entities on the fly from other inodes for use in a new directory scan when you hit up against the memory limit. You could in fact make the reclamation a simple FIFO rather then an LRU, simplifying the structure. :> If the zone allocator is used for the second level block allocations :> it shouldn't be a problem. You can (had better be able to!) put a mutex :> around zone frees in -current. : :The locking issues I could see were more in the area of finding :inodes to free hashes from. A linked list of dirhash structures :could be maintained (protected by a mutex), but to free the dirhash :belonging to an inode, the inode would probably need to be locked. :That means dereferencing dirhash->dh_inode->i_vnode and trying to :lock it, so things become complex. : :Ian I think you can avoid locking the inode. The dirhash is entirely independant of the inode except when it is accessed to aid in a directory operation. The access point can be locked separately with a mutex in -current, and need not be locked at all in -stable. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message