Date: Tue, 31 Jan 95 14:31:28 MST From: terry@cs.weber.edu (Terry Lambert) To: roberto@blaise.ibp.fr (Ollivier ROBERT) Cc: jkh@freefall.cdrom.com, hackers@freefall.cdrom.com Subject: Re: Optimizing CVS? Message-ID: <9501312131.AA28664@cs.weber.edu> In-Reply-To: <9501310915.AA04063@blaise.ibp.fr> from "Ollivier ROBERT" at Jan 31, 95 10:15:29 am
next in thread | previous in thread | raw e-mail | index | archive | help
> > > I would have to say that by this point, waiting around for CVS operations > > has taken a significant part of my life for the last 2 years. It's > > great, I love it, but damn is it SLOW! :-( Anyone out there closer to > > the CVS project who could say whether or not thoughts have been straying > > to optimization issues now? > > Maybe we should optimize stat(2) before... Ask Remy about the tests > he made between ext2fs and our ffs. Our stat(2) is way slower than the > 1.1.5.1 one. > > When I used CVS under 1.1.5.1 it was very acceptable. Now, when I > do a cvs update on directories like lib/libc, it is slow. Stat needs a lot of work... basically, the new UFS getinfo implementation sucks badly because of an NFS compatability hack that should have been made in such a way as to act as a single penalty instead of a per layer penalty (ie: NFS would still suffer, but UFS would not). There is also some room for VFS stack collapse once a mount has taken place; it's possible to drastically reduce the call overhead by using one of Thomas Rosenthal's tricks and using a top-level call-down vector that's filled out with the first active element in the stack. This works for us because, unlike Rosenthal's work, the stack is assembled on a per mount basis instead of on a per vnode basis. This means there is not near the penalty. Mount, being an infrequent operation, is generally permitted to be slow as necessary, although this shouldn't increas things drastically. Finally, the inode cache needs to be moved over to a vnode cache instead, and generalized at the comsumer level for the file system (ie: moved the hell out of UFS). SVR4 actually has this one right with their DNLC mechanism, although they (incorrectly) do not allow for use of the DNLC as a negative hit cache. Once the vnode->inode cache transition is complete, it should be possible to prefetch the the vnodes for a particular inode during directory enumeration. This fits with the locality-of-reference model which was used to justify storing a single current directory vnode pointer in the proc struct (the process current directory). This won't help with > 15 character file names (the DNLC stly cache will only cache 16 byte of name data inclusive of the NULL), but since that is really a rare occurance in a typical U*** FS, this should *significantly* speed stat operations by avoiding the nami per stat lookup for typical iterative directory enumerations. Cranking the buffer area to greater than 32 bytes total is possible (to push over the 15 character limit), but not recommended. When talking about ext2fs, it should be noted that it does a whole lot of caching of meta-data which is not technically legal, but which it gets away with because the hardware is much more reliable than when UFS was first put together. Most of the synchronus write through is disabled. This isn't a fair comparison because the reliability is not comparable. Linux also has a two stage vnode cache -- effectively the same thing as the SVR4 vnode cache coupled with the VMS second chance cache. I'm not convinced of the utility of the second chance cache in light of the normal POSIX time update semantics, since it seems pretty useless when you go to do a lookup. There's also the fact that the cache is not flushed on a CDROM device on unmount, only on a mount of new media. The same effect could probably be achieved with less code by holding a preallocation freelist of vnodes. The Linux code would seem to have much faster create times if the creates followed an equal or larger number of deletes. There is a finite probability of a terrific explosion in the Linux cache code when swapping removable media (basically, it returning cached data from the previous media). This makes them appear blindingly fast on CDROM remounts, however. I'll have more information as I hack more. Terry Lambert terry@cs.weber.edu --- Any opinions in this posting are my own and not those of my present or previous employers.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9501312131.AA28664>