Date: Thu, 3 Sep 1998 02:35:32 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: mike@smith.net.au (Mike Smith) Cc: tlambert@primenet.com, cmascott@world.std.com, hackers@FreeBSD.ORG Subject: Re: Reading/writing /usr/ports is VERY slow Message-ID: <199809030235.TAA07404@usr07.primenet.com> In-Reply-To: <199809021853.SAA01561@dingo.cdrom.com> from "Mike Smith" at Sep 2, 98 06:53:31 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> This is only useful if the layout has clustered related directories on > the disk. The current code does a good job of keeping them a long way > apart. > > > In fact, directory locality is loosely assureed. > > Almost the opposite is true, especially if you don't make a common > practice of deleting lots of directories. I think we are miscommunicating here. Directory *contents* are loosely assured locality. Directory *heirarchies* are loosely assured near-perfect random non-locality. Per the "find" command argument, this issue is "breadth-first" vs. "depth-first". > > I suspect that you are using a disk that either does not support > > tagged command queues, or supports them, but they are not enabled. > > Multiple device openings are unuseful in this case, as you previously > noted the lookup code doesn't (can't) attempt to read-ahead (it can't > know where /a/b is until it has read /a, etc.) If you are performing a > set of directory recursions, I/O (modulo the cache) will be performed > strictly sequentially. OK. Yes, this breaks down quickly. Either reimplement the directory code as a btree, or, better, "avoid writing code that results in diretory recursions". I think the ports problem in this respect is that the directory entry cache is too small for the traversal being done. Basically, "some idiot is using the FS directory hierarchy as if it were a database index". 8-). The correct thing to do here would probably be to create a "fast find" index, and have find use this (SunOS 4.1 has this; you create the index at 3 am from cron -- 3am because that's when you break to go to Naugles for egg-and-bean burritos 8-)). Alternately, a "portfinder" port would help... Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199809030235.TAA07404>