From owner-freebsd-hackers Sat Jun 2 4: 7: 7 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by hub.freebsd.org (Postfix) with SMTP id 2C3E337B422 for ; Sat, 2 Jun 2001 04:07:04 -0700 (PDT) (envelope-from iedowse@maths.tcd.ie) Received: from walton.maths.tcd.ie by salmon.maths.tcd.ie with SMTP id ; 2 Jun 2001 12:07:03 +0100 (BST) To: Matt Dillon Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: UFS large directory performance In-Reply-To: Your message of "Fri, 01 Jun 2001 10:52:40 PDT." <200106011752.f51HqeS85237@earth.backplane.com> Date: Sat, 02 Jun 2001 12:07:03 +0100 From: Ian Dowse Message-ID: <200106021207.aa78407@salmon.maths.tcd.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200106011752.f51HqeS85237@earth.backplane.com>, Matt Dillon writes: > What are your commit plans? It looks extremely well contained, > it could be committed to -current and then -stable a few days later > without any destabilizing impact at all for when the option isn't > specified. ... > The only potential problem I see here is that you could end up > seriously fragmenting the malloc pool you are using to allocate the > slot arrays. And, of course, the two issues you brought up in > regards to regularing memory use. Thanks for the comments :-) Yes, malloc pool fragmentation is a problem. I think that it can be addressed to some extent by using a 2-level system (an array of pointers to fixed-size arrays) instead of a single large array, but I'm open to any better suggestions. If the second-level array size was fixed at around 4k, that would keep the variable-length first-level array small enough not to cause too many fragmentation issues. The per-DIRBLKSIZ free space summary array is probably relatively okay as it is now. The other main issue, that of discarding old hashes when the memory limit is reached, may be quite tricky to resolve. Any approach based on doing this from ufsdirhash_build() is likely to become a locking nightmare. My original idea was to have ufsdirhash_build() walk a list of other inodes with hashes attached and free them if possible, but that would involve locking the other inode, so things could get messy. Ian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message