Date: Tue, 10 Apr 2001 11:24:35 -0700 (PDT) From: Matt Dillon <dillon@earth.backplane.com> To: Brian Somers <brian@Awfulhak.org> Cc: Jason DiCioccio <Jason.DiCioccio@Epylon.com>, "'current@freebsd.org'" <current@FreeBSD.ORG>, brian@Awfulhak.org Subject: Re: FW: Filesystem gets a huge performance boost Message-ID: <200104101824.f3AIOZ389340@earth.backplane.com> References: <200104101103.f3AB36P49523@hak.lan.Awfulhak.org>
next in thread | previous in thread | raw e-mail | index | archive | help
:> I'm not 100% convinced about the algorithm to avoid clusters filling
:> up with directory-only entries (it looks like a worst-case would fill
:> a cluster with 50% directories and 50% files leaving a bad layout when
:> the directories are populated further), but then the non-dirpref
:> scheme has some far worse worst-case scenarios ;-)
:
:Just to follow up on myself... it seems the dirpref stuff was
:committed to FreeBSD this morning :-]
:
:--
:Brian <brian@Awfulhak.org> <brian@[uk.]FreeBSD.org>
Yup, Kirk committed it. I really like the changes -- in the old days
disk caches were tiny and directories were not well cached on top of that.
It made sense to try to keep directories close to their files.
But today the proximity of a directory to its files is not really that
important. It is far more important for directories to have reasonable
proximity to each other not only to improve directory scans and lookups,
but also to improve caching. Especially for small directories.
Consider that small directories are typically contained in a single
fragment (1K). If directories are spread all over the disk, caching
is non-optimal. But if they are relatively close to each other then
both our VM cache (if vfs.vmiodirenable is set to 1) and the hard
drive's internal cache become extremely effective. With the added
effectiveness of the caches, seeking should wind up being
significantly reduced even for things like 'tar'. Large directories
also benefit, I think.
From looking at the code, I don't think fragmentation will be an issue.
Or, to be more specific, I don't think the fact that files may not wind
up in the same cylinder group as their directory entry is an issue.
Either you have a huge number of directories being accessed and need
the locality of reference within the directory space even if it costs
some additional seeking to access underlying files, or you don't and
the active directories all wind up being cached, removing any additional
seeking from the equation entirely.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200104101824.f3AIOZ389340>