Date: Wed, 5 Jun 1996 13:46:20 -0700 (MST) From: Terry Lambert <terry@lambert.org> To: dyson@freebsd.org Cc: jehamby@lightside.com, terry@lambert.org, bde@zeta.org.au, dufault@hda, hackers@freebsd.org Subject: Re: Breaking ffs - speed enhancement? Message-ID: <199606052046.NAA29627@phaeton.artisoft.com> In-Reply-To: <199606050648.BAA00451@dyson.iquest.net> from "John S. Dyson" at Jun 5, 96 01:48:15 am
next in thread | previous in thread | raw e-mail | index | archive | help
> > Of course either solution will be a lot more difficult to implement > > initially, but should ultimately provide both faster and safer metadata > > updates, while simultaneously eliminating fsck delays during crash > > recovery! At least that's how I read things. I'm still a newbie kernel > > hacker, after all... Comments? > > Our breakage of the BSD-LFS code isn't all that bad. I had it running > pre-merged VM/Buffer Cache, and it was kind-of unstable, but very fast > to say the least. ANYONE who is going to make a bona-fide attempt to > get it working with FreeBSD will get alot of my hacking time for help. I'm personally now less interested in LFS than I am in soft updates, and more in the direction of a general graph theory soloution to FS's as a set of event nodes, and consistency guarantees as a set of event handling ording rules with soft updates implemented as an inter-node conflict resoloution schema. It's admittedly a bizarre way of looking at FS's, if nothing else. An FFS with soft udates performs within 5% of memory speed, according to Ganger & Patt's paper. Source code is provided in adulterated form in their revised Appendix A of the online version of their paper, but is buggy in places (probably a result of the sanitization process), and is applicable only to FFS, really (definitely a result of not thinking of the FS in terms of events dictated by the FS semantics). > It is missing an LFSCK, and contrary to what alot of people might say, > IMO it is needed. That is likely where most of the work would be. How complex do you view this to be? I believe that most of the LFS single file/directory problems with a catastrophic failure can be handled on mount by rolling transaction back (rolling them forward would require journalling, not just log-structuring). One of the problems I have with LFS in this regard that I *wouldn't* have with an event-based soft updates implementation is implied state tracking across multiple FS objects. One example of this would be a dBase III database file with an index file. When the database changes, the index needs to change as well, iempotently. This is handleable for dBase III by rebuilding the index, but a true relational database implementation could not be so easily fixed. A soft udates implementation would allow you to impose event dependency on the graph for multi-object transactions (assuming multi-object ordering enforcement, like for an LFS log that won't overwrite for two seperate events in the same transaction). So what I'm interested in, eventually, needs both. The LFS is needed for rollback, but isn't really a big speed issue (IMO). > My guess is that just getting LFS running sort-of reliably would be > 2wks+- full time of work, if you understand things to begin with. It > is not a night or two of work, but probably not a couple of months > full time either. This is probably a fairly liberal estimate. I might put it closer th three weeks, but then if you did the work, you'd have an advantage with the VM pieces. 8-). > There are some patches from Margot (I am not sure if we have > integrated them), and also some VM system work (to eliminate alot > of usually unnecessary copies). Margot's patches have *not* been integrated. I think there is some significant work, especially in the cleaner, since the release of the Lite2 code. > If someone will take the project on, and get the code running as-is, > I will do the work to clean up the VM stuff (and help with working > the kernel VM/vfs_bio interface issues.) > > Jeffery Hsu is working on the VFS stuff right now, so I would suggest > waiting until his stuff settles out (or at least track it), but alot > of prelim work could start immediately. > > So, I guess what I am saying is that if someone will take the lead, I'll > help!!! I'm not willing to take the LFS on, right now, but let me echo John's sentiments: I'll help out in anything I can for you. I suspect that there will be some UFS code (which is shared by FFS, MFS, and LFS) that will need to be changed; I'm probably a good guy for that. If anyone was thinking about jumping in, if you can get a commit of John's time like this, then this is probably the best chance you'll get for a nice, compartmentalized kernel hack, where you don't have to worry about related systems being killed by whatever changes you want to make. Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199606052046.NAA29627>