Date: Mon, 14 May 2007 00:33:23 +0100 From: "Thomas Sparrevohn" <Thomas.Sparrevohn@btinternet.com> To: "'Benjamin Lutz'" <mail@maxlor.com>, <freebsd-hackers@freebsd.org> Cc: 'Michel Talon' <talon@lpthe.jussieu.fr>, 'Kris Kennaway' <kris@obsecurity.org> Subject: RE: DPS Initial Ideas Message-ID: <008101c795b7$19946d60$4cbd4820$@Sparrevohn@btinternet.com> In-Reply-To: <200705132355.05941.mail@maxlor.com> References: <20070512004209.GA12218@lpthe.jussieu.fr> <20070513202737.GA63102@xor.obsecurity.org> <004301c795a1$c7e89410$57b9bc30$@Sparrevohn@btinternet.com> <200705132355.05941.mail@maxlor.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Sunday 13 May 2007 23:00, Thomas Sparrevohn wrote: > > The on-disk format seems to be the wrong angle on the issue - The > > current structure Works well - but it has a number of drawbacks - > > however it no way clear whether that The answer is another > > INDEX/storage structure > > When coming up with ideas what to change INDEX's storage method to, > just keep in mind that > > - There is very little flexibility whatsoever in the way data is stored > in the file. Each entry in the INDEX has its 13 or so fields, and > that's it. One of the strengths of XML, self-descriptiveness for very > dynamic data structures, doesn't matter for INDEX. Basically, imho, > using XML for tabular data = bad. Agree and Disagree ;-) For an on disk storage structure absolutely yes - A file that was native XML does not make sense - let's forget XML/SQL etc For a second The issue at hand is two/many fold - What I am trying to convey is 1) There is one discussion about "Internal Ports" and FreeBSD on disk format 2) There is one discussion about what are the best ABI for tools using that data 3) There is a discussion about Dependency handling in general The issue of looking up an installed port can be handled separately - and indeed most Of the port management tools does exactly that e.g. portupgrade The challenge with ports are the "static dependencies" and the "ad-hoc dependencies". > > - INDEX exists for speed. Accessing the information in it should be as > fast as possible. I object to any change that increases the time > needed to search for and parse INDEX entries. I've written a little > searching tool (it can be found ports-mgmt/psearch). If INDEX were to > be converted to XML, just because of that it would be considerably > slower. If psearch then were to use standard XML parsing libs, the > slowdown would probably be at least an order of magnitude. > Absolutely - that is exactly the problem - all the port maintenance tools are Depending on a intimate understanding of directory structures etc making it Very tricky to change without breaking the tools. > > The second point is most important here. This whole thread exists > because people consider the existing ports system to be too slow. How > is using XML going to help with that at all? > But exactly what is to slow? - The fact that be default the system is single threaded or "make install". IMO Ports are still far better that most other approaches- I converted INDEX into postgresSQL because I was playing around with making a message queue based approach - and it becomes BIG - The only table structure difference from the current format was that I was able to track "who is depending on" a port - which I am pretty sure could be handled in the current framework - e.g. we could add a file having the depending port names or so I would think a better approach would be to use a static matrix or an information tree structure. In another life I solved a problem like this that way e.g. each entry was assigned a unique prefix sequence etc. I am not sure however if that would be overkill for a typical install base of 500-600 ports. I suggest that people play around with INDEX and SQL - it quite easy to make a schema and a data loader - On a day to day basis I doubt that it will be much faster because it has to scan the ports directory as well. Regards T. PS. Another fun thing to play with is to load all files into a database - Makefile the lot
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?008101c795b7$19946d60$4cbd4820$>