Date: Tue, 31 Oct 2000 23:59:47 -0800 From: Tim Kientzle <kientzle@acm.org> To: "Daniel C. Sobral" <dcs@newsguy.com> Cc: Patrick Bihan-Faou <patrick@rageagainst.com>, libh@FreeBSD.ORG Subject: Making the Packages System Better Message-ID: <39FFCD73.7364C2BF@acm.org> References: <39DCC860.B04F7D50@acm.org> <20001006155542.A29218@cichlids.cichlids.com> <39F3CDD7.15B889E7@acm.org> <20001023190412.B507@cichlids.cichlids.com> <39F47E98.4BB647AA@acm.org> <20001023202244.B10374@cichlids.cichlids.com> <39F48F4A.38D458C2@acm.org> <39FCF244.5A8C8E59@newsguy.com> <39FDC12E.304B0011@acm.org> <39FE2406.150CA3B1@newsguy.com> <00cb01c042f1$1a347190$040aa8c0@local.mindstep.com> <39FE562C.714DBE7C@newsguy.com>
next in thread | previous in thread | raw e-mail | index | archive | help
"Daniel C. Sobral" wrote:
> Somehow I dislike PATHs, MANPATHs and LIBPATHs with 40 or 50 entries, I
> dislike a /usr that won't fit in one screen, I dislike having programs
> all over the place, I dislike having to edit /etc every time I want to
> make a new program available, and I specially dislike having to instruct
> users in setting up their accounts to be able to use a new program .
The following is an outline of an approach to package management
that addresses all of these concerns, does not require a package
database, and involves minimal automation. Just to make things clear,
I'm specifically talking about pre-compiled packages such as would
be downloaded from the FreeBSD website or installed from a CD-ROM.
1) Each package resides in it's own private directory.
E.g., "foo" version 3.4 resides in /usr/packages/foo-3.4/
From here on, I'll refer to /usr/packages/foo-3.4/ as PREFIX.
2) Certain directories within PREFIX have special meaning.
PREFIX/bin - user binaries
PREFIX/lib - programming libraries
PREFIX/man - man pages
PREFIX/include - C/C++ include files
PREFIX/rc.d - boot-time startup scripts
<others may be required>
Directories not on this list have no special meaning and
can be used however the package wants to use them.
3) Special link directories are created for those recognized
directories:
/usr/packages/bin contains symlinks to /usr/packages/*/bin/*
/usr/packages/lib contains symlinks to /usr/packages/*/lib/*
/usr/pacakges/man/... tree contains symlinks to corresponding files
in /usr/packages/*/man/...
/usr/packages/include contains symlinks to
/usr/packages/*/include/*
... etc ...
Only recognized directories have their contents symlinked.
For example, PREFIX/private is never symlinked, nor is
PREFIX/libexec or PREFIX/libdata or any other directory not on
a particular list. This is an important point.
Several important things to note here:
* User paths, MANPATH, LIBPATH, etc, do not ever need to be augmented
when new directories are added, since those paths will all use the
link directories.
* Package installation/deinstallation/upgrading are pretty
trivial. The only significant automation needed is to
build/maintain the link dirs.
* Tentative upgrades are fairly simple: /usr/packages/foo-3.5/ can
be installed without first removing /usr/packages/foo-3.4/
(A little intelligence in building the link directories
obviously helps here, but nothing unreasonable.) In comparison,
supporting tentative upgrades with the current approach requires
the ability to archive all updated files individually and restore
them for a rollback. (Question: do you correctly handle files
that existed in the previous version but not the new version?)
* Rolling back an incomplete install is trivial: just delete the
directory.
With the current system, rolling back an incomplete install
requires that you make copies of any overwritten or altered files
and keep track of each file that does get installed. A crashed
incomplete install is an ugly mess with the current defaults.
* No temporary directory is needed, since a new directory
is created for each package. If the package name isn't known in
advance, you can create a temporary directory in /usr/packages
and just rename it when the final name is known. Similarly,
package archive creation is much simplified (no need to fake
a directory heirarchy).
* Some packages will obviously have to touch files outside of
their directory; this could be handled in several ways, the easiest
is to have an "install" script in the PREFIX dir and execute it
after unpacking.
* Dependency handling is very efficient, even with networked installs.
You can pull down a package and unpack it directly into it's PREFIX
dir, then test dependencies and recursively install those
dependencies.
By executing the "install" script and updating link directories as
the recursion unwinds, you ensure that the visible parts of the
installation are done in the correct order without requiring any
temporary storage and without ever having to pause a download in
progress. In contrast, the current system requires you to either
stop the current download to handle the dependency or have multiple
pending installs sitting in temporary directories requiring additional
disk space. (This is admittedly a subtle point. ;-)
* The symlink directories should only contain symlinks and
never contain actual files. Otherwise, you run into some
confusing maintenance issues. (Learn from database design:
don't mix derived data and primary data.)
* Packages should be compiled directly for their specific PREFIX.
Files outside of the "special" directories do not get symlinked.
Among other things, this avoids a lot of filename conflicts.
(Two different spelling packages that have a "dictionary" file
in PREFIX/private/dictionary won't conflict since that file
doesn't need to be symlinked.)
The basic principle: packages should keep their guts to themselves.
In essence, the symlink directories are just how a package
publicizes it's public interfaces.
* This does not require any changes to the existing "ports"
collection as long as all ports are PREFIX-aware. They
can just be compiled with package-specific PREFIXes.
* The symlink dirs should probably build shadows of any
directories. E.g., /usr/packages/tcl-8.0/lib/tcl/xxxx
should result in a real dir /usr/packages/lib/tcl/
with symlinks to corresponding files. That allows several
tcl packages to have files within .../lib/tcl/
* Often, you need no special utilities at all with this scheme.
For example, a list of installed packages: ls /usr/packages
Building/maintaining the link directories is a pretty simple
exercise in walking a directory heirarchy that can be cribbed
from any number of sources. As a crude first cut, you can simply:
cd /usr/packages/bin && ln -s ../*/bin/* .
But that rapidly runs into argument-length limits. Besides,
it's much faster to compare the existing symlinks to the
desired symlinks and only alter the ones that have changed
(delete symlinks that point to deleted packages or add new
symlinks). Updating all of the symlink directories shouldn't
take more than a second or two. I have Perl code that does
most of this if you're interested (although it could be improved
by adding full shadow-dir support).
In short, I'd argue that this approach is significantly simpler
than the current package tools: it doesn't require a database;
it makes it easy to provide such advanced features as
rollback, recovering failed installs and tentative upgrades.
I'm fairly confident that tools to support this approach would
be much simpler than overhauling the existing tools to provide
such features. However, note that such tools would not work
with the current /usr/local tarpit, since they rely heavily on
the assumption that each package goes into its very own directory.
If you must support the /usr/local tarpit, then you've got some
ugly coding ahead.
Comments?
- Tim Kientzle
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-libh" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?39FFCD73.7364C2BF>