Date: Sat, 12 May 2007 18:24:35 -0400 From: Kris Kennaway <kris@obsecurity.org> To: Michel Talon <talon@lpthe.jussieu.fr> Cc: freebsd-hackers@freebsd.org, Mike Meyer <mwm@mired.org>, Kris Kennaway <kris@obsecurity.org> Subject: Re: DPS Initial Ideas Message-ID: <20070512222435.GA28981@xor.obsecurity.org> In-Reply-To: <20070512214422.GA88480@lpthe.jussieu.fr> References: <20070512004209.GA12218@lpthe.jussieu.fr> <17989.8202.624522.136573@bhuda.mired.org> <20070512090935.GA13929@lpthe.jussieu.fr> <20070512193302.GA24673@xor.obsecurity.org> <20070512214422.GA88480@lpthe.jussieu.fr>
next in thread | previous in thread | raw e-mail | index | archive | help
--k+w/mQv8wyuph6w0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, May 12, 2007 at 11:44:22PM +0200, Michel Talon wrote: > On Sat, May 12, 2007 at 03:33:02PM -0400, Kris Kennaway wrote: > > On Sat, May 12, 2007 at 11:09:35AM +0200, Michel Talon wrote: > >=20 > > > Seriously, the FreeBSD package > > > system is in great need of a profound overhaul, pretending it works w= ell > > > is complete denial of reality. I hope that young people working on=20 > > > summer code projects will infuse *new* ideas, and not spend their > > > vacations polishing inadequate tools. > >=20 > > I know that this is your belief, but please try to avoid grasping at > > straws: there are elements in your argument that are along the lines > > of "The FreeBSD package system is broken and needs to be fundamentally > > changed. Rewriting it to use SQLite is a fundamental change. > > Therefore rewriting it to use SQLite will fix the problems." > >=20 >=20 > Really i don't think at all this way. I think that *perhaps* SQLite > may marginally better than a Berkeley database for solving part of the > problem, not much more. What i reacted to, was the conservatism which=20 > pervades the community as soon as someone emits the idea of using a new t= ool.=20 It seems to me that you do not appreciate the reasons behind this conservatism. A very important one is that we have two students who have committed to spending their summer working on improving the existing pkg_tools in ways that will solve some of the real problems we are facing, and the project we have agreed upon is that they will be using existing tools rather than rewriting from scratch as part of a not-yet-defined larger project. To some extent it is the timing here that is most unfortunate. If SQLite has been raised as a viable alternative a few months ago it would have made a great project idea, but instead we have committed to improving our existing tools, and the barrier for throwing out these plans is therefore very high. The burden of proof is therefore set much higher than "SQL is awesome and buzzword-compliant and might be better". > It may be that borrowing from Debian the idea of "abstract" dependencies > which can be fulfilled by several concrete packages may also simplify > the dependency problem. For example tomcat may depend on "java" and java > my be fulfilled either by diablo-jdk15 or jdk15. This way when you change > from diablo-jdk15 to jdk15 you don't need to change anything to tomcat. >=20 > Another feature that Debian has, and which may happily complete the previ= ous > one, is the specification of necessary dependencies with a version number > in a certain range (this obviously requires a reasonable standardisation = of > version numbers, so that comparison of <some package>-0.99 to=20 > <some package>-1.0-rc doesn't depend on arcane rules). This way you don't= need > to change dependencies which are in the correct range, even if a more rec= ent > version exists. This mechanism has been imported in NetBSD pkgsrc. We actually have both of these features in ports, but they have not been pushed down into packages. I think it will be relatively simple to do so, without requiring a rewrite from scratch. > And a problem which has proven useful in Debian is keeping track of the > packages which have been required by the end user and those which have be= en > installed as dependencies. This is the difference between apt-get and > aptitude. Apparently people are very happy to be able to remove not only > a package they have required, but also all its dependencies (which are > not required by another program) at one stroke. This also helps in case > some big package requires dependency A, but after upgrade, they have chan= ged > their mind and require alternative dependency B. With this mechanism, aft= er > upgrade A disappears, while without it you will have both an upgraded ver= sion > of A and B. I have observed on my machine this is an important cause=20 > of time monotonic bloat of the package tree. This one could also be added to the existing tools. > To answer the slowness problem in registering installed packages, one may > think about making use of the INDEX file. In fact all the information that > is necessary to fill the dependency entries is contained in INDEX, and > accessible here in milliseconds with any tool such as awk. It so happens = that > the ports system doesn't make any use of the INDEX file and systematically > recomputes the dependencies through recursive make invocations which are = very > time consuming. Of course this requires up to date INDEX, or a mechanism = to > keep INDEX continually up to date. The problem is that maintaining the INDEX is expensive and/or tricky. p5-FreeBSD-Portindex comes close but seems to have some wrinkles. > Part of the registration is also filling the +REQUIRED_BY files of the > dependencies of a package when one installs a package. If this package h= as a > lot of dependencies this means opening, editing and closing a large numbe= r of > files. This is expensive. One may imagine using a database containing the > global dependency information, then +REQUIRED_BY files are no more necess= ary, > since the information can be recomputed in very little time. Yes, this is one thing that is addressed in the current plan. > > Given that this work is happening (or at least will be happening, I am > > not sure when the SoC officially starts), the best thing is for > > interested people to work with Garrett to help him achieve the goals > > of his project. >=20 > Sure. I am convinced this is the reason why several people, including mys= elf > present some ideas in the mailing list now, before Garrett begins working= on > his project. I think this is a retcon; the original poster did not appear to have this motivation and neither do many others who have posted. Kris --k+w/mQv8wyuph6w0 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD4DBQFGRj6jWry0BWjoQKURAmUpAJMEv/USguMTVaa80MdBUCm1kjzKAJ4iuNsw rHV3bYRw4I013bH6n/Db8Q== =cIQm -----END PGP SIGNATURE----- --k+w/mQv8wyuph6w0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070512222435.GA28981>