Date: Fri, 4 Sep 1998 08:38:43 -0400 From: cmascott@world.std.com (Carl Mascott) To: hackers@FreeBSD.ORG Subject: Re: Reading/writing /usr/ports is VERY slow Message-ID: <199809041238.AA00816@world.std.com>
next in thread | raw e-mail | index | archive | help
>Carl Mascott wrote:
>>>In short, I think that there are more long inter-group seeks on FFS
>>>than there need be, due to the directory placement policy. I consider
>>>it undesirable to spread directories out as much as possible all over
>>>the filesystem.
>>
>Mike Smith wrote:
>>One of the substantial contributors to this problem has been the
>>FreeBSD policy of limiting the number of cg's by faking the disk
>>layout. The intention here was to reduce the amount of space wasted in
>>administrative overhead, as well as increase the likelihood of
>>contiguous allocation. Unfortunately one of the less wonderful
>>consequences has been the substantial increase in inter-cg distances.
>
Carl Mascott wrote:
>For a filesystem of any given size the number of cg's is not a
>factor, at least not as long as there are more than a couple of
>them.
Mike Smith wrote:
>The number of cg, or more to the point their layout *is* somewhat
>significant, simply because they can still consume moderately
>substantial amounts of space. They also seem to be less efficient if
>you have zillions of them.
Clarification: For a filesystem of any given size the number of
cg's is not a factor affecting throughput in /usr/ports.
Carl Mascott wrote:
>> However,
>> if a cg proximity criterion were added to the new-cg-chooser,
>> then the size of a cg would become a factor, although I don't
>> know how important a factor. For what it's worth, vxfs also
>> uses 32 MB cg's ("allocation units" in vxfs-speak).
>
Mike Smith wrote:
>I don't actually think that the cg size would be an issue; what we're
>talking about here is allocating the directory inode, and the data
>blocks for the directory will follow. In most cases, the data for
>these directories will move around as they're extended anyway, as they
>accumulate frags and move into whole blocks.
I think it would be an issue. Suppose for the sake of argument
that 50% of the new directories were within 1 cg of their parent.
If cg's were 4 MB instead of 32 MB, then 50% of the seeks would
be approximately 1/8 as long. Even if you grant this, though,
this is not a sufficient reason to change the cg size.
To keep this in perspective I think this is a minor side
issue to the main issue of how or whether to introduce some
lumpiness into the directory allocation policy.
> Meanwhile, do you have the courage to try the code snippet I included?
> I'd be interested to know if it had any affect on your pathalogical
> case...
Maybe, but I want to let the dust settle first, and see if
Kirk McKusick weighs in with anything interesting. I would
also want to become familiar with the FFS source code before
I did anything to it.
One problem with my trying it is that it will never get a good
workout on my box, a single-user workstation with ~50% full
filesystems. I could certainly see any improvement in the
/usr/ports throughput, but behavior with a nearly full
filesystem would never get tested.
--
Carl Mascott
cmascott@world.std.com
uunet!world!cmascott
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199809041238.AA00816>