Date: Tue, 21 Mar 2000 00:01:27 +0000 From: Paul Richards <paul@originative.co.uk> To: Alfred Perlstein <bright@wintelcom.net> Cc: Poul-Henning Kamp <phk@critter.freebsd.dk>, Matthew Dillon <dillon@apollo.backplane.com>, current@FreeBSD.ORG, richard@starburst.demon.co.uk, richard@netcraft.com Subject: Re: patches for test / review Message-ID: <38D6BBD7.DA4B950B@originative.co.uk> References: <20000320111544.A14789@fw.wintelcom.net> <20102.953580112@critter.freebsd.dk> <20000320115902.C14789@fw.wintelcom.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Alfred Perlstein wrote: > > * Poul-Henning Kamp <phk@critter.freebsd.dk> [000320 11:45] wrote: > > In message <20000320111544.A14789@fw.wintelcom.net>, Alfred Perlstein writes: > > > > >Keeping the currect cluster code is a bad idea, if the drivers were > > >taught how to traverse the linked list in the buf struct rather > > >than just notice "a big buffer" we could avoid a lot of page > > >twiddling and also allow for massive IO clustering ( > 64k ) > > > > Before we redesign the clustering, I would like to know if we > > actually have any recent benchmarks which prove that clustering > > is overall beneficial ? > > Yes it is really benificial. Yes, I've seen stats that show the degradation when clustering is switched off. Richard Wendlake (who wrote the OS detection code for the Netcraft web server survey) did a lot of testing in this area because of some pathological behavior he was seeing using Gnu's dbm package. Richard, do you want to post a summary of your tests? > > I'm not talking about a redesign of the clustering code as much as > making the drivers that take a callback from it actually traverse > the 'union cluster_info' rather than relying on the system to fake > the pages being contiguous via remapping. > > There's nothing wrong with the clustering algorithms, it's just the > steps it has to take to work with the drivers. Well, there is something wrong with our clustering algorithm. It always starts a new cluster when the first block of a file is written to. I found this when trying to explain some of the pathological behavior that Richard was seeing. Imagine an algorithm that will write blocks 0,5,2,7,4,1,6,3,0,... The clustering algorithm starts a new cluster if the block is at the beginning of the file, so writing block 0 will always start a new cluster. When block 5 is written out, the clustering code will try and add it to the existing cluster, will fail and so will flush the existing cluster which only has block 0 in it and then start another cluster, with block 5 in it. This continues, with the previous cluster being flushed and a new cluster being created with the current block in it. Eventually, we get to the point where 7 blocks have been flushed and the current cluster contains block 3. When it comes to write out the next block 0 the clustering algorithm doesn't bother trying to add the block to the existing cluster but immediately starts a new one so the cluster with block 3 in it *never gets flushed*. Paul. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?38D6BBD7.DA4B950B>