Date: Tue, 20 Apr 1999 11:44:22 -0700 From: John Dyson <dyson@iquest.net> To: Matthew Dillon <dillon@apollo.backplane.com>, hackers@freebsd.org Subject: Re: Directories not VMIO cached at all! Message-ID: <371CCB06.2DDECFA@iquest.net> References: <199904192039.PAA20464@dyson.iquest.net> <199904192051.NAA90282@apollo.backplane.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Because I am under alot of pressure right now, all of my responses to Matt's concerns follow. Note that it seems that he and I are arguing about expediency vs. solution. I prefer solution, and more work is necessary (a quick-fix isn't a solution.) > > My concern is over the fact that the buffer cache is relatively > restrictive. Things get thrown out of it very quickly due to its small > size. You yourself have said that one of the VM system's most important > features was its ability to not throw away things that it may need to > I/O back in soon. > > Well, I submit that whatever space we 'waste' by using the VM system > to back directory information we gain by not having to re-I/O the data. > That begs the question of forcing buffer cache entries to be PAGE_SIZE (4K-16K or so). Yuck!!! There are other ways to deal with the caching of directories, including some approaches that I hadn't developed yet (including explicit mgmt of the sub-page sized data through a different mechanism.) > > VM management devolves down into the art of not throwing away things > you need -- you said it yourself. I think this helps enormously. > That isn't quite accurate. VM mgmt devolves down to avoiding I/O, by using the resources at hand. The minimization goal is minimal I/O, and not to "fit as much into memory as possible." Queueing methods are critical (don't plop as much into queues as they will hold.) Be ready to correct policy mistakes, and more to this point: don't throw away resources that you will likely use. The Namei cache just doesn't cut it for the purposes of directory caching (unless it is changed significantly), because it doesn't utilize the entire directory for it's caching function. By wasting between 3.5K and 15.5K per directory (almost on average), it is apparent that something other than PAGE_SIZE granularity is important. The B_MALLOC was an add-on, and not a design feature in order to get around the wastage. Eventually, a better method was to appear, but that is water under the bridge. One work around that would be satsifactory (until the better scheme arrives) is to allow up to a certain number of buffers, and then spill over into the merged scheme. A reasonable policy needs to be developed, but most of the high and low watermarking mechanisms were in my original code (some of it now broken and/or removed.) I am not claiming that you MUST keep B_MALLOC, but it seems that using the traditional merged VM buffer cache mechanisms isn't correct either. A solution eventually needs to be found, and not just a fix or hack. > :If buffers are not used for sub-page entities, another scheme should be > :used. I fear expediency or sloppiness will prevail. > : > :John > > You are still ignoring the several points I brought up. I wish you would > address them. I put forth several theories as to why there would be > relatively little waste. > You are still throwing a "hack" out as a solution. It isn't an adequate solution to waste so much memory, it is better to solve the problem. The original code had the mechanisms (with a few minor changes) both memory thriftiness for directories and more cache. In a rush to "fix" imaginary problems, the solution for a real one has been significantly complicated. My local codebase, without the broken vfs_bio will have both, with only a few fixes. Alas, you get to invent again, and again and again... If you asked before hacking, things would move along in a more forward path. Various points that beg the issue elided... > > Considering the immensly critical nature of the directory structure, if > there were any single place where I'd be willing to waste some bytes it > would be there. > Here you admit waste, and don't solve the problem... Sigh... The "fix" is a problem statement and not a solution. It is indicative of seeing solutions to problems as coding style or quick work arounds within a given framework, and doesn't show a depth of understanding that supports growth. The issue that you see is a problem statement, and the solution presented is only a lab exercise. Now, a good solution is needed. Support can be gained by asking, rather than the necessary after the fact criticism. A big *sigh*... John Matthew Dillon wrote: > > :> I see an advantage both ways. Not only are we able to use the VM cache > :> to cache directories ( and thus scale directory operations to memory ), > :> but I don't think there is even a downside to mapping whole pages even for > :> small directories. The reason is simple: When you access small > :> directories you tend to access specific files in said directories. When > :> you access specific files, there's a good chance they will be in the namei > :> cache. If they are in the namei cache, the VMIO mapping will not be > :> referenced very often for most small directories which means that the > :> VM cache will throw it away. Hence, no waste. > :> > : > :I cannot believe that you said that: > : The size of the cache buffers are then up to 8X larger when using > : a whole page instead of a 512byte buffer. BTW, VMIO is a misnomer, > : and I named it... VMIO was an earlier incarnation, and some of > : it spilled into the existant code. > : > : Again, do a study to find out if the internal fragmentation makes > : things worse. Don't depend on the VM code to just "throw" things > : away -- if you can make considered decisions instead of deferring > : them to a policy somewhere, make the decision... > : > :John > > My concern is over the fact that the buffer cache is relatively > restrictive. Things get thrown out of it very quickly due to its small > size. You yourself have said that one of the VM system's most important > features was its ability to not throw away things that it may need to > I/O back in soon. > > Well, I submit that whatever space we 'waste' by using the VM system > to back directory information we gain by not having to re-I/O the data. > > But I truely do not believe that we are really wasting all that much > space. How many directories are in-use on a system at any given point? > Now subtract the ones that are fully namei cached... now take into account > the fact that once disassociated from its struct buf, the backing pages > are managed on a page-by-page basis rather then on a buffer-by-buffer > basis. I do not think you are left with much, and what you ARE left with > is easily managed by the VM system. > > Or, let me put it another way: If enough directories are being cached > to make the wasteage theoretically significant, then enough directories > are being cached to blow away the existing malloc buffer space and cause > active data to be thrown away, resulting in unnecessary I/O. Hell, the > processes *accessing* the directories are already an order of magnitude > larger then the storage required for the directories. I just don't think > we lose much by VMIOing directories. > > VM management devolves down into the art of not throwing away things > you need -- you said it yourself. I think this helps enormously. > > -Matt > Matthew Dillon > <dillon@backplane.com> > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?371CCB06.2DDECFA>