Date: Wed, 02 Sep 1998 18:59:23 +0000 From: Mike Smith <mike@smith.net.au> To: hackers@FreeBSD.ORG Cc: mckusick@mckusick.com Subject: Re: Reading/writing /usr/ports is VERY slow Message-ID: <199809021859.SAA01595@dingo.cdrom.com> In-Reply-To: Your message of "Wed, 02 Sep 1998 18:52:59 -0400." <199809022252.SAA00872@europa.local>
next in thread | previous in thread | raw e-mail | index | archive | help
> Carl Mascott wrote:
> > > In short, I think that there are more long inter-group seeks on FFS
> > > than there need be, due to the directory placement policy. I consider
> > > it undesirable to spread directories out as much as possible all over
> > > the filesystem.
> >
> Mike Smith wrote:
> > One of the substantial contributors to this problem has been the
> > FreeBSD policy of limiting the number of cg's by faking the disk
> > layout. The intention here was to reduce the amount of space wasted in
> > administrative overhead, as well as increase the likelihood of
> > contiguous allocation. Unfortunately one of the less wonderful
> > consequences has been the substantial increase in inter-cg distances.
>
> For a filesystem of any given size the number of cg's is not a
> factor, at least not as long as there are more than a couple of
> them.
The number of cg, or more to the point their layout *is* somewhat
significant, simply because they can still consume moderately
substantial amounts of space. They also seem to be less efficient if
you have zillions of them.
> As long as new directories get distributed pseudo-randomly
> over the filesystem, which is what happens in /usr/ports, it
> doesn't matter how many cg's the filesystem is subdivided into.
> The disk arm will still be covering the same distance.
Effectively, yes, particularly if the creation order and traversal
order were related.
> However,
> if a cg proximity criterion were added to the new-cg-chooser,
> then the size of a cg would become a factor, although I don't
> know how important a factor. For what it's worth, vxfs also
> uses 32 MB cg's ("allocation units" in vxfs-speak).
I don't actually think that the cg size would be an issue; what we're
talking about here is allocating the directory inode, and the data
blocks for the directory will follow. In most cases, the data for
these directories will move around as they're extended anyway, as they
accumulate frags and move into whole blocks.
> Hopefully someone who knows the history of FFS development
> will chime in at this point. It would be nice to know if
> any other directory placement policies were ever tried and
> rejected.
It seems that Kirk is on holidays; at any rate, I've copied him back on
this and hopefully he'll have something enlightening to offer.
Meanwhile, do you have the courage to try the code snippet I included?
I'd be interested to know if it had any affect on your pathalogical
case...
--
\\ Sometimes you're ahead, \\ Mike Smith
\\ sometimes you're behind. \\ mike@smith.net.au
\\ The race is long, and in the \\ msmith@freebsd.org
\\ end it's only with yourself. \\ msmith@cdrom.com
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199809021859.SAA01595>